This event has ended. Create your own event → Check it out
This event has ended. Create your own
View analytic
Tuesday, May 19 • 11:15am - 11:55am
Ceph at CERN: A Year in the Life of a Petabyte-Scale Block Storage Service

Sign up or log in to save this to your schedule and see who's attending!

The CERN IT Department operates a large distributed OpenStack infrastructure with around 10,000 VMs and 100,000 CPU cores. This cloud is predominantly used for physics data analysis, but there remains a long tail of conventional IT services and user-managed application VMs. These varied use-cases created a new requirement in our department, one to build a scalable, reliable, performant and future-proof block storage backend. To solve this problem, in mid-2013 we deployed 3 petabytes of Ceph software-defined storage.

This talk will present a year in the life of our Ceph cluster, focusing on the operations issues related to the RBD use-cases. Based on our experience hosting around 1000 Cinder volumes and 1500 Glance images, we’ll describe how the cluster is used, its access patterns and growth, mistakes we've made, problems overcome, upgrades successfully performed, and highlight some tips and tricks we’ve learned to tune both the servers (Ceph) and clients (qemu-kvm). Lastly, we’ll conclude by outlining our future plans for cloud-based block storage across our multiple datacentres and describe where we believe cloud storage may have further potential at CERN.

avatar for Dan van der Ster

Dan van der Ster

Storage Engineer
Dan is storage engineer and Ceph service manager in the Data and Storage Services group at CERN IT. Prior to working on storage, Dan worked in the field of Grid Computing, including work on the Ganga and PanDA systems for the ATLAS experiment at CERN. Dan earned a PhD in Computer Engineering at the University of Victoria in 2008.

Tuesday May 19, 2015 11:15am - 11:55am
Room 109

Attendees (492)