Single Blog

2020: 100% content uptime

23 December 2020, Written by 0 comment

Wow! In 2020 your media with Jet-Stream were 100% available. CDNs often promise 100% availability, but this is marketing speak. At Jet-Stream, your content was available every minute of the entire year.

CTO Joshua and CEO Stef made a (Dutch spoken) podcast on this topic. They discuss some examples of customers who experienced issues and how Jet-Stream resolved these. Listen in!

Uptime SLAs often concern the availability of a platform, but not necessarily the streams. Every CDN has outages: unplanned and planned. A redundant setup allows them to send traffic from one location to another.

This switch is usually done with a DNS change. They assume that all new requests will be transferred within 1 minute. But that assumption is incorrect: after all, DNS is not enforced. DNS records can be cached: at providers, at companies, at your home, in your device and in your browsers. The result is that large groups of people are sent to unreachable or faltering servers for hours.

The SLA is formally met, but it is of no use to you and your viewers.

Jet-Stream also has a redundant setup, of course. The core is 4 times redundant. Video content is replicated 12 times. Despite the occasional network outage or power failure, and despite the occasional maintenance of Jet-Stream on servers, your videos and live streams were always available, 100%.

We achieve this high availability because we do not use DNS but active load balancing. An own invention.

Every viewer requesting your streams goes through an active load balancing layer. This global service determines within a millisecond whether the viewer has rights to watch, and which server or which CDN is most suitable to serve that specific viewer.

The Jet-Stream load balancing algorithm uses more than a dozen parameters, including real-time performance, load, availability, cost and proximity to each server and CDN, and the topological location of each viewer. As soon as a server or CDN in a particular region underperforms, it is immediately pulled from the active pool. Viewers are thus individually sent to the environment that offers the most perfect stream for them. This not only achieves a much higher uptime, and also gives a minimal performance guarantee.

We call this broadcast grade availability, opposed to DNS based best effort availability.

Jet-Stream has occasionally taken servers into maintenance, for security updates or to roll out new features. This has had no effect on the availability of your streams: these servers were temporarily removed from the active pool in advance, and only serviced when there was no more traffic. Customers were informed in advance and were advised to deliver their live streams redundantly to us.

In 2020 there was 1 unexpected serious outage at Jet-Stream: 1 data center lost its power. This had no effect on uptime because our monitors immediately detected this issue and instant requests were diverted to another location.

All in all, there have been about 3 of the more than 525,000 minutes in the year in which non-redundant live streams were temporarily unavailable. (Our advice is to always deliver live streams redundant.) Redundant live streams and videos were 100% available.

It is very difficult to achieve such figures year after year. Unprecedented compared to the failures that other CDNs experience. A great achievement by our technical team! Kudos to them. It couldn’t be better. It is therefore quite possible that we also will have longer outages in the future than we would like. That’s the honest story, instead of cheating with fake 100% uptime marketing blah.

What also helps is that we have a MultiCDN environment. If one of the CDN partners has a malfunction, we can immediately switch your account to another CDN. You see many parties deeply integrating with 1 supplier. If a problem arises, then you are really stuck. You will not be bothered by this through us.

We do see one point of improvement: existing viewers. Streams are really different from keeping a website up and running. A stream doesn’t load any static content. It is a continuous process. You are stuck with sessions and tokens. Sometimes with sockets. Suppose a server goes down and existing viewers or listeners are also lost? You don’t want it.

Fortunately, we have our own professional video player for a while. And our load balancers already contain technology to not send your viewer to 1 server or CDN, but to provide a list of recommendations to the player. We will investigate whether we can add mid-stream intelligence to our player, so that your viewer – even in a secure session – can automatically switch to the always best performing CDN while watching.

Maybe it’s a bit over the top engineering, to resolve those last few minutes. But you know, our tagline is ‘we deliver’. Our engineering heart tells us to invest time in this 🙂

We hope you enjoy the podcast. We discuss some rare issues that we encountered in a number of use cases, for instance when Apple updated Safari and suddenly progressive downloads didn’t work with one specific CDN anymore. Or where a customers’ DRM encrypted streams became popular, so they were automatically offloaded to CDNs, and we found out that some CDNs block DRM headers. Let us know your feedback. We’re also curious: what is your streaming vendors actual content availability? Are they transparent about it?