Scrutinizing Windows Authentication

The protocols behind the process

May 15, 2005

14 Min Read
ITPro Today logo

Even though authentication is an essential procedure for every Windows user, computer, and service account, it's perhaps one of the least understood mechanisms among systems administrators. Everyone knows you need to have the right password to log on to a computer, but do you understand what's going on behind the scenes? Windows authentication and its associated protocols come into play every time a user, computer, or service logs on locally or to a domain controller (DC). In this article, I'll talk about the basics of Windows authentication, then dive into its related protocols. I'll also offer some quick recommendations for increasing the authentication security of your Windows network.

Authentication in General
Authentication is just one component of any computer-based access-control system. As you see in Figure 1, page 12, access-control systems comprise identification, authentication, authorization, and accounting.

Identification. Identification is a piece of information that uniquely identifies a security principal (e.g., user, group, computer, service account) within a common directory service. A directory service such as Active Directory (AD) provides a way to uniquely identify objects, similar to the way DNS makes sure no two people share the same Internet email address. Behind the scenes, Windows is using SIDs, globally unique identifiers (GUIDs), and other unique tags. In most cases, for the purposes of identification, you can simply type in your unique account's friendly name, such as Rgrimes. In a large AD forest, you might need to use user principal names (UPNs), such as [email protected]. If you use smart cards in your environment, the security principal can instead present his or her digital certificate or key information.

Authentication. After a security principal types in or otherwise presents the necessary identification information (e.g., username, security token), he or she must type in or present privately known authentication information (e.g., password, PIN). In Windows, the security principal provides this information at the logon screen with the help of the Microsoft Graphical Identification and Authentication DLL (msgina.dll) and Winlogon.exe programs. The system's authentication protocol and mechanism encodes the presented authentication information at the desktop and sends an authentication request. The Windows authentication service can be either a SAM database or AD. The SAM database handles local logons or logons to Windows NT 4.0 DCs. AD authenticates requests in Windows 2000 or later domains. An authentication protocol (e.g., LAN Manager, NT LAN Manager--NTLM, NTLMv2, Kerberos) is used to transport the authentication request and the resulting authentication transactions to and from the logon screen and the authentication service. In a moment, I'll discuss each authentication protocol separately.

Authorization. If the authentication service verifies the combination of the identity and authentication "secret," the system successfully authenticates the security principal. Next, the system collects the security principal's (i.e., the user's) group membership. The user often belongs to a number of explicit groups--local, domain local, global, or universal--through the normal membership processes. The system verifies local groups in local SAM databases, and verifies domain local and global groups on the DCs in the user's home domain, as well as universal groups on the DC that hosts the Global Catalog. The system gathers all group memberships, directly or indirectly, to collect security permissions.

Just after authentication, the system collects SIDs of the user's account and group memberships into an object called the access token. The user might need to log out and log back on to effect new security permissions. When the user needs to access an object (e.g., file, folder, printer, registry key) that's protected by NTFS permissions, the process (e.g., Windows Explorer) that's acting on behalf of the user presents its access token. Each NTFS object has a list of access control entries (ACEs), which are essentially the familiar NTFS permissions (e.g., Allow Read, Allow Write). The matrix of ACEs assigned to users and groups makes up that object's ACL. Note that an object's ACL is represented by the security permissions you can view in Windows Explorer.

The user's access token, which includes the user account and groups with which the user is associated, determines the effective permissions the user has. Authorization is the process of allowing or denying a user access to a particular object based on the presented access token as compared to the object's ACL. Windows' Security Reference Monitor, which Figure 2 shows, handles authorization. In Figure 2's example, the User has Read, Write, and Modify permissions. However, the Everyone group, to which the user belongs, is denied the Modify permission. Other group memberships give the user Read and Modify permissions, but the Everyone group's Deny permission overrides the Modify permission. The object also has ACLs that deny Full Control to the HR group, but the user isn't a member. Thus, the user's effective permissions on the object in Figure 2 are Read and Write.

Accounting. If auditing is enabled in Windows, the system saves the authentication event to the Security log file, and therein lies the final component of an access control system: accounting. Most of the complex initial-logon and subsequent authorization events happen in seconds and are hidden from the user. The authentication protocol handles all the hard work.

Protocol Function
An authentication protocol must do at least two things. First, it must securely transfer authentication transactions from requester to authentication database, and to any other computers at which a resource is being accessed. Second, it must securely store the authentication password or token. Password crackers are most interested in the second point. An authentication protocol must securely transfer the user's inputted information to the authentication database (i.e., the SAM or AD). To do so, the protocol signs, obscures, or encrypts the transaction, along with time-stamping it so that a malicious hacker can't replay the passed credentials at a later date. Also, to prevent an authentication-database compromise from immediately revealing the user's password, the protocol should ensure that passwords are stored in the authentication database in obscured form.

For more than a decade, most authentication protocols have provided protection by storing the password in an obscured form (typically hashed) in the authentication database and never transmitting the plain-text password (or even the obscured form) between the requester and the authentication database. This challenge-response process occurs as follows:

  • 1.The computer gathers identity and authentication information from the user and requests authentication from the authentication server.

  • 2.The authentication server generates a random one-time data value (called the challenge) and sends it to the requester.

  • 3.The requester receives the challenge and mathematically manipulates it with the obscured form of the password, then sends the result (called the response) back to the authentication server.

  • 4.The authentication server also mathematically manipulates the challenge in a way that should be identical to what the workstation performed, and compares its result with the received response. If the two results agree, the requester is successfully authenticated.

Authentication protocols use the challenge-response process so that the user's password never travels over the network. Some authentication protocols perform this process better than others.

Local vs. Domain Logon
When a user logs on, one of the first things Windows must determine is whether the logon is for only the local machine or for a domain account. Users who are using a local account to log on can access resources only on the computer they're accessing and only if their user-account information resides in the local SAM database. Should users need to access resources on remote computers without domain authentication, the user accounts would need to be duplicated in the local SAM database on every computer to which they need access. The accounts on each participating computer would need to be synchronized--with logon names, passwords, and expiration dates matching across all computers--or things would really get ugly. A moderate-sized peer-to-peer (P2P) network with only local logons is cumbersome to maintain.

DCs avoid the requirement of maintaining several synchronized user accounts across multiple computers. With domain authentication, computers logged on to the domain look to DCs to validate domain user-account credentials during authentication requests. Thus, when a remote user attempts to access a resource that a particular computer maintains locally, the computer can query a DC to verify the requesting user's identity. Domain user accounts reside only on DCs and have to be created only once. Any participating computer that needs to verify any account in the domain can access them at any time. The problem of synchronizing logon names, passwords, and expiration periods across multiple computers doesn't exist because the account's credentials and management exist in only one location--the domain DC. Regardless of whether the logon is local or to the domain, Windows must authenticate the user's logon request.

Windows Authentication Protocols
As I mentioned previously, Windows has four primary authentication protocols: LAN Manager, NTLM, NTLMv2, and Kerberos. LAN Manager debuted in the days of DOS and continued through early versions of Windows. NTLM appeared in NT. NT Server 4.0 Service Pack 4 (SP4) introduced NTLMv2, and Win2K introduced Kerberos. By default, all Win2K and later machines support all four authentication protocols, and other workstations and servers can direct these machines to use a certain protocol for a given authentication request. Fully patched Windows 9x and later machines can use LM, NTLM, and NTLMv2. In the Microsoft environment, only Win2K and later clients logging into a Win2K and later domain can use Kerberos. A Win2K or later computer must have Kerberos and at least one of the other authentication protocols available for use.

Security researchers have shown that the older protocols--LM and NTLM--are susceptible to eavesdropping and password-cracking attacks. Consequently, you should use only Kerberos and NTLMv2 in your environment, if possible. To understand this advice, you need to know a little more about each protocol.

LAN Manager
IBM created the LAN Manager protocol, implementing it in early versions of Windows and Windows networking. Like all Microsoft authentication protocols, LAN Manager makes a password hash--aka the LM hash--that is stored and utilized between sender and receiver during the authentication process. LAN Manager creates LM hashes by changing all of a password's alphabetic characters to uppercase, creating two 7-character halves, then enciphering. The LAN Manager authentication protocol then uses the LM hash in a series of steps similar to those of the challenge-response process I described earlier.

Although LAN Manager was acceptable during its early stages, today it's considered an extremely weak protocol. Password-cracking tools can convert passwords encoded with LAN Manager hashing back to the plain-text representations in seconds. LM hashes are fundamentally frail, with a few glaring weaknesses:

  • Passwords can comprise only a limited series of 128 ASCII characters.

  • Passwords are truncated at 14 characters.

  • If a password contains fewer than 14 characters, the extra spaces are padded in the easy-to-identify hashed form, revealing the exact password length.

  • LAN Manager converts all alphabet characters in the password to uppercase before hashing.

  • Why would anyone still use LAN Manager hashing? It's enabled by default on all Windows computers, including Windows Server 2003 for backward compatibility. Windows' latest authentication databases store the weaker LM hash alongside its more secure counterparts, just in case a LAN Manager transaction is necessary. You can--and should--disable LAN Manager if you aren't using legacy or other applications that depend on LAN Manager authentication.

When NT arrived on the scene, Microsoft created and deployed the more secure NTLM authentication protocol. NTLM uses a more sophisticated authentication algorithm and creates a more secure stored password hash--aka the NTLM hash. NTLM passwords support as many as 128 characters. Unlike LAN Manager hashing, which is limited by the use of ASCII characters, NTLM hashing supports the full Unicode character set, thereby adding complexity. NTLM truncates the NTLM hash at the 128th character, converts the hash into a 16-bit Unicode value, runs it through an MD4 digest function, and stores it as a 32-character hexadecimal string. The NTLM-authentication challenge-response series of steps uses the NTLM hash to provide an authentication sequence significantly more complex than the one that LAN Manager provides.

Eventually, security researchers proved that NTLM is unsecure. Microsoft responded by creating NTLMv2, which is still reliably secure for most environments--although Kerberos has surpassed it as today's default Windows protocol--and still in use for local logons (and other instances). The NTLMv2 protocol is similar to NTLM, except that the NTLMv2 password hash uses HMAC-MD5 message authentication, and the challenge-response sequence is timestamped to prevent replay attacks.

Overall, NTLMv2 is more difficult to compromise with brute-force attacks than NTLM because it uses a 128-bit encryption key. Only two password crackers (one of them is Symantec's LC5) claim to be able to crack NTLMv2 password hashes.

Microsoft adopted Kerberos as the default domain-authentication protocol for Win2K and later AD domains. Kerberos is an open standard, and its interoperability is supported with foreign Kerberos domains (called realms in UNIX and Linux). In AD domains, every DC is a Kerberos Distribution Server (KDC) and is capable of participating in Kerberos authentication. Kerberos gives you the following features for increased protection:

  • Mutual authentication of client and server to each other

  • Increased password protection because Windows sends the password credentials for only the initial logon rather than every authentication event, and all communication sessions are encrypted

  • Timestamped challenge-response sequence to prevent replay attacks

  • Delegation of identity (i.e., allowing a server process to access a remote resource on behalf of the user)

  • Interoperability

Here's Kerberos in a nutshell:

1.After successfully authenticating normally, the user's computer requests a security ticket from the Kerberos server (a DC) for future authentication requests.

2.The Kerberos server hands the requester a ticket that lets it participate in future authentication and authorization events without the necessity to resubmit the original authentication secret.

3.When the requestor needs to access a participating resource server, the requester gets another access ticket from the Kerberos server and submits it to the resource server for approval.

4.The requester's original authentication credentials don't pass over the network for any of the subsequent authentication sessions (until the ticket handed out in Step 2 expires).

Note that although Kerberos somewhat mimics a private public key infrastructure (PKI), all information is secured by the use of symmetric keys--as opposed to the asymmetric keys that most authentication services use.

Smart Cards
Passwords and other single-factor authentication methods are quickly becoming unreliable. E-commerce is becoming a way of life, and identity-stealing malware (e.g., spam, phishing attacks) are making passwords too easy to exploit. Many experts feel that two-factor authentication--in the form of smart cards, USB tokens, or some other sort of cryptographic device--will become the norm for Internet-related transactions. Microsoft natively supports the use of digital certificates and smart card authentication. To support a smart card infrastructure, you must install Certificate Services and distribute smart card certificates. Of course, you must also have the physical smart cards, the readers, and the vendor's software. Then, when necessary, users must insert their smart cards into the local reader to access a Windows computer interactively. Appropriately deployed, smart cards can significantly enhance authentication.

Authentication Protocol Defenses
Some hacking journals falsely suggest that hacking Microsoft authentication is still easy. In fact, of the more than 20 password crackers available, only 2 work with NTLMv2 and only 1 works with Kerberos. With a handful of simple steps, you can deflect even those. Here are the steps you should take to defeat password crackers and resetters. (You can configure most of these options locally or by using Group Policy.)

  • Disable the storage of LM hashes, as the Microsoft article "How to prevent Windows from storing a LAN manager hash of your password in Active Directory and local SAM databases" (;en-us;299656) describes. Doing so is essential to preventing password crackers from easily recovering the original password.

  • Disable all authentication protocols except NTLMv2 and Kerberos, after testing extensively. For instructions on how to do so, see the Microsoft TechNet article "Network security: LAN Manager authentication level" (

  • Disable booting from removeable media to prevent password-cracking tools from booting around the OS. Disabling booting from all but the default drive prevents offline password crackers from gaining access to authentication databases that store the password hashes.

  • Require that users use complex passwords that contain eight characters or more.

  • Require that users change their passwords at least once per quarter, if not more frequently.

  • Enable account lockouts, if only for 1-minute lockouts, with automatic resets. Doing so prevents successful online password guessing.

Secure Users
With NTLMv2, Kerberos, and smart cards, Windows has reliable and secure authentication mechanisms that are significantly resistant to eavesdropping and brute-force password cracking. But remember that all the good advice and strong authentication protocols in the world are for naught if your users create weak passwords. Ensure that password complexity is enabled, and educate users about how to choose strong, secure passwords.

Sign up for the ITPro Today newsletter
Stay on top of the IT universe with commentary, news analysis, how-to's, and tips delivered to your inbox daily.