400 error is seen by the end user when trying to access a IIS web site that has Windows Authentication enabled. Not all users see this behavior. The user might be a member of several Active Directory groups.
When a user logs into a workstation on the domain, a kerberos authentication ticket is created which contains the user’s Active Directory group information. When the browser (i.e. IE) is performing pass through authentication (i.e. Windows Authentication aka IWA), it sends this kerberos ticket in the header of the request so that IIS can consume the user information. If the user is a member of many AD groups, their kerberos ticket may exceed that allowable limit specified on the IIS server side (in this case, the ADFS signin page).
Company mergers and acquisitions often result in Active Directory domain migrations. My experience has been that in many cases, these migrations are done with limited time set aside for any type of cleanup of group memberships. Often you’re dealing with years of accumulation of groups and unfortunately it seems with little documentation on what groups may not be necessary anymore. Post-migration, you have Active Directory objects stamped with SID History and possibly hundreds of groups that may or may not be necessary but no one has the time to research them to find out.
The downside to ending your migration here is that the SID History and stale group memberships all roll up into the client’s Kerberos token size. When the Kerberos token becomes excessively large, it can start to produce one issue after another.
Issues start to arise once the user’s Kerberos token exceeds 12,000 bytes; the user will start to run into odd issues with authentication and possibly Group Policy. You’ll find some organizations work around this by setting the “MaxTokenSize” setting in the workstation’s registry to the maximum value of 48,000 bytes (the actual maximum is 65,535 but the recommended is 48,000 for reasons beyond the scope of this post).
The relevance to AD FS is that during the AD FS authentication, the HTTP request sent to IIS contains the Kerberos token in the HTTP header. IIS has a HTTP header size limit of 16,384 bytes by default; after you account for base64 conversion and overhead, you’re really looking at around 12,000 bytes available for your Kerberos token.
During AD FS authentication, users with tokens in the 12,000 bytes range will fail to authenticate. What the users will see, if they look closely enough, is an “HTTP 400 – Bad Request” response from the AD FS server.
Solution / Workarounds
Since the restriction here is the IIS header size, fixing just the AD FS servers may not be enough. You could certainly have other IIS-based applications in your environment that would experience the same error. The ideal solution would be to reduce the group memberships to the point that the Kerberos token size is in the allowable range. Unfortunately, this cleanup can’t always happen so the alternative is to increase the limits.
The keys “MaxFieldLength” and “MaxRequestBytes” can be added to the registry on the AD FS server and this will allow for a larger HTTP header to accommodate the large Kerberos token. Additional information on these values and their recommended maximums is available in Microsoft KB2020943. A reboot of the AD FS server after applying this change and the users with large Kerberos tokens should be able to authenticate successfully.
Calculating the user’s token size is a somewhat complicated process. The token size will depend on the type of groups the user is a member of, the group nestings and the SID history in the environment. The Microsoft utility “Tokensz.exe” is probably your best estimate for the token size.