Implementation

The implementation for this approach involves registering a vectored exception handler that is able to handle the access violation and single step exceptions that are generated. Since this approach relies on setting the segment registers DS and ES to the null selector, an implementation must take steps to update the segment register state for each running thread in a process and for all new threads as they are created. Updating the segment register state for running threads involves enumerating running threads in the calling process using the toolhelp library. For each thread that is not the calling thread, the SetThreadContext routine can be used to update segment registers. The calling thread can update the segment registers using native instructions. To alter the segment registers for new threads, the DLL_THREAD_ATTACH notification can be used. Once all threads have had their DS and ES segment registers updated, memory references will immediately begin causing access violation exceptions.

When these access violation exceptions are passed to the vectored exception handler, appropriate steps must be taken to restore the DS and ES segment registers to a valid segment selector, such as 0x23. This is accomplished by updating the SegDs and SegEs segment registers in the CONTEXT structure that is passed in association with an exception. In addition to updating these segment registers, the trap flag (0x100) must also be set in the EFlags register so that the DS and ES segment registers can be restored to the null selector in order to trap subsequent memory accesses. Setting the trap flag will lead to a single step exception after the instruction that generated the access violation executes. When the single step exception is received, the SegDs and SegEs segment registers can be restored to the null selector.

These few steps capture the majority of the implementation, but there is a specific Windows nuance that must be handled in order for this to work right. When the Windows kernel returns to a user-mode process after a system call has completed, it restores the DS and ES segment selectors to their normal value of 0x23. The problem with this is that without some way to reset the segment registers to the null selector after a system call returns, there is no way to continue to track memory accesses after a system call. Fortunately, there is a relatively painless way to reset the segment registers after a system call returns. On Windows XP SP2 and more recent versions of Windows, the kernel determines where to transfer control to after a system call returns by looking at the function pointer stored in the shared user data memory mapping. Specifically, the SystemCallReturn attribute at 0x7ffe0304 holds a pointer to a location in ntdll that typically contains just a ret instruction as shown below:

0:001> u poi(0x7ffe0304)
ntdll!KiFastSystemCallRet:
7c90eb94 c3              ret
7c90eb95 8da42400000000  lea     esp,[esp]
7c90eb9c 8d642400        lea     esp,[esp]

Replacing this single ret instruction with code that resets the DS and ES registers to the null selector followed by a ret instruction is enough to make it possible to continue to trap memory accesses after a system call returns. However, this replacement code should not take these steps if a system call occurs in the context of the exception dispatcher, as this could lead to a nesting issue if anything in the exception dispatcher references memory, which is very likely.

An implementation of this approach is included with the source code provided along with this paper.