[foms] Chunked/adaptative streaming at Dailymotion

[Pierre-Yves KEREMBELLEC]

Thank you for the excellent summary of your work.
On Tue, 2 Nov 2010 18:49:28, Pierre-Yves Kerembellec wrote:
> it would just be a tremendous task to move those peta-bytes around, not to mention splitting them into billions of small files!
And the petabytes of data in billions of files would overwhelm any CDN provider, and make caching near impossible.

Well, not really. Akamai is sized to support such load for instance.

This is why we use virtual chunking on unmuxed A/V streams.  We minimize the asset footprint, and use simple HTTP range requests.  Our streaming model scales well on any CDN.

As mentioned in my post, the main benefit for us is to leverage the existing internet caching infrastructure to lower the egress bandwidth cost from our origin servers:
I'm not so sure regular caches will actually cache bytes-ranges properly for instance (they may probably consider such requests as transient and not bother caching
a partial resource). Also, HTTP range requests are ok as long as the underlying origin files are using a streamable container, with no need for headers manipulation
when only sending a portion of them. My guess is that you're using MPEG2-TS files, which fits this purpose quite fine. But we need to address several different devices
(and we do not control the client), so remuxing everything in MPEG2-TS (or equivalent) is not really an option.

> This is why the origin streaming server is taking care of re-muxing original video files on the fly
It is an elegant solution, but I am wondering about cost and scalability.  What does your peak load look like?  Our streaming video (mostly adaptive, 1 hr average duration), drives ~20%
of peak US internet traffic, and I can not imagine that this model could scale to meet our load.

Our origin servers are commodity hardware equipped with one (or two aggregated) 1Gbps interface(s), and about 2TB of disk space for caching. Each server is capable
of saturating the interface(s) across thousands of simultaneous connections with this design (on a 2Gbps server, we may serve about 4000 different clients simultaneously
when streaming low-bandwidth content encoder at 500kbps). We have about 80 of these internally, generating an aggregate output bandwidth of 100Gbps. The point is this
is completely scalable by just adding new servers if needed (the load balancing and cache stickiness is performed by an applicative layer on top of the video cluster).

Pierre-Yves

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.annodex.net/cgi-bin/mailman/private/foms/attachments/20101104/54f8342a/attachment.htm