OpenStack Summit May 2015 Vancouver has ended
Back To Schedule
Tuesday, May 19 • 11:15am - 11:55am
Ceph at CERN: A Year in the Life of a Petabyte-Scale Block Storage Service

Sign up or log in to save this to your schedule, view media, leave feedback and see who's attending!

The CERN IT Department operates a large distributed OpenStack infrastructure with around 10,000 VMs and 100,000 CPU cores. This cloud is predominantly used for physics data analysis, but there remains a long tail of conventional IT services and user-managed application VMs. These varied use-cases created a new requirement in our department, one to build a scalable, reliable, performant and future-proof block storage backend. To solve this problem, in mid-2013 we deployed 3 petabytes of Ceph software-defined storage.

This talk will present a year in the life of our Ceph cluster, focusing on the operations issues related to the RBD use-cases. Based on our experience hosting around 1000 Cinder volumes and 1500 Glance images, we’ll describe how the cluster is used, its access patterns and growth, mistakes we've made, problems overcome, upgrades successfully performed, and highlight some tips and tricks we’ve learned to tune both the servers (Ceph) and clients (qemu-kvm). Lastly, we’ll conclude by outlining our future plans for cloud-based block storage across our multiple datacentres and describe where we believe cloud storage may have further potential at CERN.

avatar for Dan van der Ster

Dan van der Ster

Computing Engineer, CERN
Dan is the Ceph Service Manager at CERN, where he has worked the past 11 years as a Distributed Systems and now Storage Engineer. Dan is an experienced Ceph operator, and founding member of the Ceph Board, representing the Associate Members. Dan holds a PhD in Computer Engineering... Read More →

Tuesday May 19, 2015 11:15am - 11:55am PDT
Room 109

Attendees (2)