Professional Documents
Culture Documents
At 20:35 (UTC) on the 05th of May 2021, VUALTO received notifications alerting performance issues with all
VUDRM services (Fairplay, Widevine, & Playready). All devices using the VUDRM service were affected and unable
to play Live and VOD content with DRM.
Causes of issue:
A third party provider which the VUDRM platform utilises had a change of their Trusted Root. This new Trusted Root
was not recognised by the underlying service.
This led to failures on the 3 way handshake that in turn prevented normal functioning of other services within the
DRM platform, including Widevine and Fairplay.
Timeline/Actions
20:35 UTC VUALTO was alerted of issues with playback related to the VUDRM platform on devices using
Widevine, Fairplay, & Playready
20:40 UTC First line teams are unable to identify a root cause and full escalation is communicated to the
relevant teams.
21:00 UTC An internal meeting is held with the Department heads for DRM and Infrastructure.
21:30 UTC Investigations from the internal meeting identify the cause of the issue, which is detailed within
21:45 UTC Resolution steps are agreed upon and deployed to the underlying services for all regions.
22:06 UTC Steps are completed and DRM service is fully restored to all regions.
23:00 UTC Secondary checks are made confirming service is still fully operational without degradation
following the resolution steps an hour previously.
The Trusted Root issue between the underlying VUDRM service and the third party has been resolved by deploying
the relevant amended files. In order to prevent a recurrence of this issue, Vualto have introduced necessary steps
into the VUDRM Continuous Delivery Pipeline that mitigate the risk of Trusted Root modifications carried out by third
party services.