The importance of knowing about certificates

Deploying and managing a Lync/Skype for Business environment demands you to know a lot more about technologies and protocols. Their communications are encryption which means that you need to deploy certificates and specially to maintain them over time. One wrong, forgotten or misplaced certificate can give you lot of headaches.

The following issue I faced recently, on a Lync 2013 environment is a good example of how a simple misplaced CA certificate can cause unexpected behaviours.

ISSUE and symptoms

After restarting the servers, the Lync services start reporting connection errors and denials due to certificate validation.

As a consequence, the users experience several issues:

  • Contacts presence status unknown
  • Address book unavailable
  • Unable to schedule, start or join meetings
  • External users unable to join/dial-in meetings

They are still able to sign-in, send IM’s and perform peer-to-peer calls (including video and desktop sharing) and PSTN inbound/outbound calls.

On the servers you will find from several others, eventID 32042 errors ‘Invalid incoming HTTPS certificate’ and eventID 30998 ‘Sending HTTP request failed’

Cause

The clue came from some informational events of services receiving invalid client certificates.EventID-61029

The last description line ‘Certificate error: 21482049809.’  translates to error code 0x800b0109, which is defined as CERT_E_UNTRUSTEDROOT. Lync server could not trust the subordinate CA that was installed on the local machine store ?!

Turns out that a new PKI has been deployed, and I found a subordinate CA incorrectly installed on the ‘Trusted Root Certificate Authorities’ of the servers.
(a subordinate CA can be easily identified because it’s not self-signed (‘Issue To’ name doesn’t match the ‘Issued By’)

subCA-on-TrustedCAstore

Windows Server 2012 (and higher) implements checks for a higher level of trust for certificate authentication. By finding the invalid certificate, doesn’t provide any Trust Root CA list and therefore the services cannot to validate the certificates presented to them.

SOLUTION

  1. Delete the subCA from the Trusted Root CA store of the server
  2. Reboot the server so it can load correctly the Trusted Root CA list.

Final notes

The issue will only start after a server reboot, so it can take quite some time to associate the cause/effect…. especially if you have just install a OS update !! (and blame it, uninstall, …)

Only after knowing exactly the issue, I manage to ‘google-fu’ a 4 years-old KB2795828 with a similar situation.
From that one I got a very usefull powershell script that help us to find any non self-signed CA on the Trusted Root CA store of the machine

Get-Childitem cert:\LocalMachine\root -Recurse | Where-Object {$_.Issuer -ne $_.Subject} | Format-List *

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.