Efficient Content Distribution with Managed Swarms

Peer-assisted content distribution has become more prevalent in recent years to match the increase of high-quality media on the Internet. Peer-assisted systems promise to improve download performance and scalability by harnessing bandwidth at peers in addition to the resources of server-class hosts. However, existing systems are ill-suited for high-performance distribution: today's systems focus on distribution of only a single file, and are inefficient for distributing large content libraries.

In this talk I will introduce a new approach to content distribution that achieves high performance based on managed swarms. Managed swarms address download performance across multiple concurrent swarms by viewing content distribution as a global optimization problem. A logically centralized coordinator orchestrates downloads by measuring the behavior of swarms and calculating an efficient allocation of resources to achieve a global performance goal. I will discuss the role of the coordinator for managing swarms, then I will describe two new algorithms based on managed swarms that maximize system-wide throughput in realistic deployment scenarios. Extensive simulations and deployments show that managed swarms provide a scalable, efficient solution for distributing content libraries, and that they significantly outperform centralized distribution services and existing swarming protocols.

Ryan S. Peterson is completing his Ph.D. at Cornell University, where he has been working on content distribution under the advisement of Emin Gun Sirer. He has also worked on distributed systems for publish-subscribe, reliable storage, and content discovery. Prior to Cornell, Ryan earned a B.S.E. degree from Princeton University. Ryan is the 2009 recipient of the Google Fellowship in Distributed Systems, and he will be joining Google next month.