Mach Traps (system calls)

In order to combine Mach and BSD into one kernel (xnu), syscall numbers are divided into different tables. On a PowerPC system, when the ``sc'' instruction is executed, the syscall number is stored in r0 and used to determine which syscall to execute. Positive syscall numbers (smaller than 0x6000) are treated as FreeBSD syscalls. In this case the sysent table is offset and the appropriate function pointer is used.

In cases where the syscall number is greater than 0x6000, PPC specific syscalls are used and the ``PPCcalls'' table is offset. However, in the case of a negative syscall number, the mach_trap_table is indexed and used.

The code below is taken from the xnu source and shows this process.

   oris    r15,r15,SAVsyscall >> 16     ; Mark that it this is a 
                                        ; syscall

   cmplwi  r10,0x6000                   ; Is it the special ppc-only 
                                        ; guy?
   stw     r15,SAVflags(r30)            ; Save syscall marker
   beq--   cr6,exitFromVM               ; It is time to exit from
                                        ; alternate context...

   beq--   ppcscall                     ; Call the ppc-only system 
                                        ; call handler...

   mr.     r0,r0                        ; What kind is it?
   mtmsr   r11                          ; Enable interruptions

   blt--   .L_kernel_syscall            ; System call number if
                                        ; negative, this is a mach call...

   lwz     r8,ACT_TASK(r13)             ; Get our task
   cmpwi   cr0,r0,0x7FFA                ; Special blue box call?
   beq--   .L_notify_interrupt_syscall  ; Yeah, call it...

On an Intel system, things are a little different. The ``int 0x81'' (cd 81) instruction is used to call Mach traps. The ``sysenter'' instruction is used for the BSD syscalls. However, the syscall number convention remains the same. The eax register is used to store the syscall number in either case.

It seems that most people developing shellcode on Mac OS X stick to using the FreeBSD syscalls. This may be due to lack of familiarity with Mach traps, so hopefully this paper is useful in re-mediating that. I have extracted a list of the Mach traps in the mach_trap_table from the xnu kernel. (792.6.22).

• List of mach traps in xnu-792.6.22