Dragonfly

Join the chat at https://gitter.im/alibaba/Dragonfly License FOSSA Status Build Status

Dragonfly

Contents

Introduction

Dragonfly is an intelligent P2P based image and file distribution system. It aims to resolve issues related to low-efficiency, low-success rate and waste of network bandwidth in file transferring process. Especially in large-scale file distribution scenarios such as application distribution, cache distribution, log distribution, image distribution, etc. In Alibaba, Dragonfly is invoiked 2 Billion times and the data distributed is 3.4PB every month. Dragonfly has become one of the most important pieces of infrastructure at Alibaba. The reliability is up to 99.9999%.

While container technologies makes devops life easier most of the time, it sure brings a some challenges: the efficiency of image distribution, especially when you have to replicate image distribution on several hosts. Dragonfly works extremely well with both Docker and PouchContainer for this scenario. It also is compatible with any other container formats.

It delivers up to 57 times the throughput of native docker and saves up to 99.5% the out bandwidth of registry.

Dragonfly makes it simple and cost-effective to set up, operate, and scale any kind of files/images/data distribution.

Features

The project is an open source version of the dragonfly and more internal features will be gradually opened.

Comparison

Test Environment  
Dragonfly server 2 * (24core 64GB 2000Mb/s)
File Source server 2 * (24core 64GB 2000Mb/s)
Client 4core 8GB 200Mb/s
Target file size 200MB
Executed Date 2016-04-20

For Dragonfly, no matter how many clients issue the file downloading, the average downloading time is always around 12 seconds. And for wget, the downloading time keeps increasing when you have more clients, and as the amount of wget clients reaches 1200, the file source will crash, then it can not serve any client.

License

Dragonfly is available under the Apache 2.0 License.

Commercial Support

If you need commercial support of Dragonfly, please contact us for more information: 云效.

Dragonfly is already integrated with AliCloud Container Services If you need commercial support of AliCloud Container Service, please contact us for more information: Container Service