Absence of GuardStack

The GuardStack (GS) support included with versions of the Microsoft Visual Studio compiler since 2002 offers a compile-time mitigation to traditional stack-based buffer overflows[23]. It supports this through a combination of a random canary inserted into a stack frame at runtime and an intelligent stack frame layout algorithm. The random canary is pushed onto the stack when a function is called and then popped off the stack and validated prior to function return. If the canary does not match the expected value, it is assumed that a stack-based buffer overflow occurred and that the process should be terminated.

Since the initial release of GS support a number of techniques have been described that could be used to bypass or weaken it[5,16,20]. While these techniques were at one time useful or have not yet been fully realized, the author assumes that most would agree that the GS implementation provided by the most recent compiler is robust (with the exception of SEH). There is currently no publicly known universal bypass technique for GS that the author is aware of. Given this assumption, functions that are protected by GS become less interesting from the standpoint of identifying stack-based buffer overflows. On the other hand, functions that are not protected by GS can instantly be qualified as interesting targets for review. This is especially true with binaries that have been compiled with GS support but contain a number of functions that the compiler has chosen not to compile with GS protections².

As previous research has illustrated[27], it is possible to identify functions that have not been compiled to use GS through the use of simple static analysis tools. It is also possible to further refine the approaches described in previous research if one has symbols and one assumes that the most recent compiler was used. This can be accomplished by analyzing the call graph of an executable and noting the set of functions that do not call __security_check_cookie. Considered another way, the same set of functions can be identified by taking the set of all functions contained within a binary less the subset that call __security_check_cookie. The set of functions that is identified by either approach can be annotated with an exploitation property that indicates that they may contain stack-based buffer overflows that would not be hindered by GS.

It may also be prudent to take the compiler version that was used into consideration when analyzing binaries. This is important due to the fact that older versions of the compiler used a GS implementation that could be trivially defeated in certain circumstances[16]. For example, previous versions of GS did not layout the stack frame in a manner that would prevent an attacker from overwriting other local variables and function arguments. In scenarios where this occurred and an overwritten local variable or parameter was dereferenced (such as by invoking a function pointer), the mitigation offered by GS would be meaningless. Thus, a secondary exploitation property could involve identifying functions where attacks such as the one described above could be possible.