NTLM

Microsoft attempted to address the shortcomings of LM with NTLM. Windows NT introduced the NTLM(NT LanManager) authentication method to provide stronger authentication. The NTLM protocol was originally released in version 1.0(NTLM), and was changed and fortified in NT SP6 as NTLMv2. When exchanging files between hosts in a local area network, printing documents on a networked printer or sending commands to a remote system, Windows uses a protocol called CIFS - the Common Internet File System. CIFS uses NTLM for authentication.

In NTLM, the protocol covered in this document, the authentication works in the following manner. When the client connects to the server and requests a new session, the server replies with a positive session response. Next, the client sends a request to negotiate a protocol for one of the many dialects of the SMB/CIFS family by providing a list of dialects that it understands. The server picks the best out of those and sends the client a response that names the protocol to use, and includes a randomly generated 8 byte challenge.

In order to log in now, the client sends the username in plaintext(!), and also the password, hashed NTLM style. The NTLM hash is generated in the following manner:

[UsersPassword]->[LMHASH]->[NTLM Hash]

The NTLM hash is produced by the following algorithm. The client takes the 16 byte LM hash, and appends 5 null bytes, so that the result is a string of 21 bytes length. Then it splits those 21 bytes into 3 groups of 7 bytes. Each 7 byte string is turned into an 8 byte odd parity DES key once again. Now the first key is used to encrypt the challenge with the DES algorithm, producing an 8 byte hash. The same is done with keys 2 and 3, so that there are two additional 8 byte hashes. These 3 hashes are simply concatenated, resulting in a single 24 byte hash, which is the one being sent by the client as the encrypted password.

Mudge already pointed out why this is really stupid, and I'll just recapitulate his reasons here. An attacker capable of sniffing traffic can see the username, the challenge and the 24 byte hash.

First of all, as stated earlier, if the password is less than 8 bytes, the second half of the LM hash always is 0xAAD3B435B51404EE. For the purpose of illustration, let's assume the first part of the hash is 0x1122AABBCCDDEEFF. So the entire LM hash looks like:

-------------------------------------------
| 0x1122AABBCCDDEEFF | 0xAAD3B435B51404EE |
-------------------------------------------

When transforming this into an NTLM hash, the first 8 bytes of the new hash are based solely on the first 7(!) bytes of the LM hash. The second 8 byte chunk of the NTLM hash is based on the last byte of the first LM hash, and first 6 bytes of the second LM hash. Now there are 2 bytes of the second LM hash left. Those two, padded with 5 null bytes and used to encrypt the challenge, form the third 8 byte chunk of the NTLM hash. That means in the example this padded LM hash

------------------------------------------------------
| 0x1122AABBCCDDEE | FFAAD3B435B514 | 04EE0000000000 |
------------------------------------------------------

is being turned into the 24 byte NTLM hash. If the password is smaller than 8 characters in size, the third part, before being hashed with the challenge to form the NTLM hash, will always look like this. So in order to test wether the password is smaller than 8 bytes, it's enough to take this value, the 0x04EE0000000000, and use it to encrypt the challenge that got sniffed from the wire. If the result equals the third part of the NTLM hash which the client sent to the server, it's a pretty safe bet to say the password is no longer than 7 chars. It's even possible to make sure it is. Assuming from the previous result that the second LM hash looks like 0xAAD3B435B51404EE, the second chunk of the 24 byte NTLM hash is based on 0x??AAD3B435B514. The only part unknown is the first byte, as this one is based on the first LM hash. One byte, thats 256 permutations. By brute forcing those up to 256 possibilities as the value of the first byte, and using the resulting key to encrypt the known challenge once again, one should eventually stumble over a result that's the same as the second 8 bytes of the NTLM hash. Now one can rest assured, that the password really is smaller than 8 bytes.
Even if the password is bigger than 7 bytes, and the second LM hash does not end with 0x04EE thus, creating all possible 2 byte combinations, padding them with 5 null bytes and hashing those with the challenge until the final 8 byte chunk of the NTLM hash matches will easily reveal the final 2 byte of the LM hash, with no more than up to 64k permutations.