|Informative Information for the Uninformed|
Internally, Protected Mode is implemented by running IExplore.exe as a low integrity process. With the default security descriptor that is applied to most securable objects, low integrity processes may not generally request access rights that map to GENERIC_WRITE for a particular object. As Internet Explorer does need to be able to persist some files and settings, exceptions can (and are) carved out for low integrity processes in the form of registry keys and directories with special security descriptors that grant the ability for low integrity processes to request write access. Because the IExplore process cannot write files to a location that would be automatically used by a higher integrity process, and it cannot request dangerous access rights to other running processes (such as the ability to inject code via requesting PROCESS_VM_WRITE or the like), malware that runs in the context of a compromised IExplore process is (theoretically) fairly contained from the rest of the system.
However, this containment only holds as long as the system happens to be free of implementation errors. Alas, but perhaps not unexpectedly, there are in fact implementation problems in the way the system manages processes running at differing integrity levels that can be leveraged to break out of the Protected Mode (or low integrity) jail. To understand these implementation errors, it is first necessary to gain a basic working understanding of how the new integrity-based security model works in Windows. The integrity model is key to a number of Windows Vista features, including UAC (User Account Control).
When a user logs on to a computer in Windows Vista with UAC enabled, their shell is normally started as a ``medium'' integrity process. Integrity levels are integers and symbolic designations such as ``low'', ``medium'', ``high'', or ``system'' are simply used to indicate certain well-known intermediate values. Medium integrity is the default integrity level even for built-in administrators (except the default ``Administrator'' account, which is a special case and is exempted from UAC). Most day to day activity is intended to be performed at medium integrity; for instance, a word processor program would be expected to operate at medium integrity, and (theoretically) games would generally run at medium integrity as well. Games tend to be rather poorly written in terms of awareness of the security system, however, so this tends to not really be the case, at least not without added help from the operating system. Medium integrity roughly corresponds to the environment that a limited user would run as under previous versions of Windows. That is to say, the user has read and write access to their own user profile and their own registry hive, but not write access to the system as a whole.
Now, when a user launches Internet Explorer, an IExplore.exe process is launched as low integrity. The default security descriptor for most objects on Windows prevents low integrity processes from gaining write access to medium integrity securable objects, as previously mentioned. In reality, the default security descriptor denies write access to higher integrities, not just to medium integrity, though in this case the effect is similar in terms of Internet Explorer. As a result, the IExplore.exe process cannot write directly to most locations on the system.
However, Internet Explorer does, in certain cases, need to gain write to locations outside of the low integrity (Protected Mode) sandbox. For this task, Internet Explorer relies on a helper process, known as ieuser.exe, which runs at medium integrity level. There is a tightly controlled RPC interface between ieuser.exe and IExplore.exe that allows Internet Explorer, running at low integrity, to request that ieuser.exe display a dialog box asking the user to, say, choose a save location for a file and then save said file to disk. This is the mechanism by which one can save files in their home directory even under Protected Mode. Because the RPC interface only allows IExplore.exe to use the RPC interface to request that a file to be saved, a program cannot directly abuse the RPC interface to write to arbitrary locations, at least not without user interaction.
Part of the reason why the RPC interface cannot be trivially abused is that there also exists some protection baked into the window manager that prevents a thread at a lower integrity level from sending certain, potentially dangerous, messages to threads at a higher integrity level. This allows ieuser.exe to safely display user interface on the same desktop as the IExplore.exe process without malicious code in the Internet Explorer process simply being able to simulate fake keystrokes in order to cause it to save a dangerous file to a dangerous location without user interaction.
Most programs that are integrity-level aware operate with the same sort of paradigm that Internet Explorer does. In such programs, there is typically a higher integrity broker process that provides a tightly controlled interface to request that certain actions be taken, with the consent of the user. For example, UAC has a broker process (a privileged service) that is responsible for displaying the consent user interface when the user tries to perform an administrative task. This operates similar in principal to how Internet Explorer can provide a security barrier through Protected Mode because the lower privileged process (the user program) cannot magically elevate itself to full administrative rights in the UAC case (which runs a program at high integrity level, as opposed to the default medium integrity level). Instead, it could only ask the service to display the consent UI, which is protected from interference by the program requesting elevation due to the window manager restrictions on sending dangerous messages to a higher integrity level window.