OpenStack Summit May 2015 Vancouver has ended
Thursday, May 21 • 11:00am - 11:40am
Federating clouds for High Energy Physics

Sign up or log in to save this to your schedule and see who's attending!

Federating clouds for High Energy Physics

High Energy Physics (HEP) experiments, such as those of the Large Hadron Collider (LHC) at the CERN Laboratory, have large demands in storage, compute and networking. Although a large fraction of the resources are provided with high throughput compute clusters, many HEP computing resource providers are transitioning to providing OpenStack clouds.

We have developed a system for federating distributed and independently operated clouds on three continents into a single infrastructure.   The system is used to run HEP applications from the ATLAS experiment at CERN and the Belle-II experiment at the KEK laboratory in Japan.  The system, incorporating over 20 clouds, has been in production for approximately three years, and has executed many millions of HEP jobs with peak workloads over 3000 cores.

In the session we review how we have assembled this system from existing open source products and developed a few new products to close functional gaps. We use Glint, an OpenStack incubator project of our design, to manage image distribution to clouds and HTCondor with CloudScheduler to distribute compute jobs.  We describe our operational experience and challenges over the past years. Our current effort is to expand the functionality so that it can run applications with large input data requirements.

avatar for Ian Gable

Ian Gable

Technical Manager, HEPnet Canada/University of Victoria
Ian Gable has worked in diverse areas of HPC from High Energy Physics collision simulation to telescope data analysis. Ian specializes distributed systems in relation to large science projects like the ATLAS Experiment at the LHC, with special emphasis on WAN applications and clouds... Read More →

Thursday May 21, 2015 11:00am - 11:40am
Room 116/117

Attendees (0)