Mac users started experiencing unexpected problems today, including apps taking minutes to launch, stuttering and unresponsive in macOS, and other issues. The problems seemed to start around the time Apple started rolling out the new version of macOS, Big Sur, but it affected users of other versions of macOS, such as Catalina and Mojave.
Other Apple services also experienced slowdowns, outages, and strange behavior, including Apple Pay, Messages, and even Apple TV devices.
It didn’t take long for some Mac users to take notice
trustd– a macOS process responsible for checking with Apple’s servers to confirm that an app is notarized – tried to contact a host named
ocsp.apple.com but failed repeatedly. Among other things, this resulted in system-wide slowdowns as apps tried to launch.
Users who opened Console and filtered to find the error encountered numerous successive errors related to
trustdas shown below.
The affected hostname (which is really just a pointer to a bunch of servers on Apple’s CDN) is responsible for validating all sorts of Apple-related cryptographic certificates, including those used by app notarization. First introduced in Mojave and made mandatory in Catalina, notarization is an automated process Apple performs on developer-signed software:
Apple’s Notary Service is an automated system that scans your software for malicious content, checks for code signing issues, and returns the results to you quickly. If there are no issues, the notary service generates a ticket that you can staple to your software; the notary service also publishes that ticket online where Poortwachter can find it.
The “OCSP” part of the host name refers to Online Certificate Status Protocol stapling, or simply “certificate stapling”. Apple uses certificate stapling to streamline the process by which millions of Apple devices check the validity of millions and millions of certificates every day.
When an Apple device cannot connect to the network, but you still want to launch an app, the notarized validation is considered to be “soft fail” – that is, your Apple device is supposed to recognize that you are not online and allow the app to start anyway. However, due to the nature of what happened today, calls to the server just seemed to hang instead of dropping out softly. This may be because everyone’s device can still perform a DNS lookup
ocsp.apple.com without any issues, leading the devices to believe that if they could do a DNS lookup, they should be able to connect to the OCSP service. So they tried – and timed out.
The situation lasted for several minutes, and while workarounds circulated on forums, chat rooms, and Twitter, the problem behavior eventually subsided as Apple supposedly fixed the underlying problem.
Apple had previously announced that Big Sur would be launching today, and the issues started rolling out almost right on time. We’ve reached out to Apple for comment and will share any statement if we receive one.