How did European researchers working on the Higgs boson recently make one of the most revolutionary physics discoveries in recent decades? From an IT perspective, they relied on a good old-fashioned grid computing infrastructure, though a new cloud-based one may be in the offing.
The European Nuclear Energy Association’s (CERN) decade-old grid computing infrastructure has been used extensively during the past few years for research that culminated with discovery of the Higgs boson, the sub-atomic particle that imparts mass.
Unlike a public cloud, where data and compute resources are typically housed in one or more centrally managed data centres with users connecting to those resources, CERN interconnected grid network relies on more than 150 computing sites across the world sharing information with one another.
For the first couple of years after the grid computing infrastructure was created, it handled 15 petabytes to 20 petabytes of data annually. This year, CERN is on track to produce up to 30 PB of data. "There was no way CERN could provide all that on our own," says Ian Bird, CERN’s computing grid project leader. Grid computing was once a buzz phrase similar to that of what cloud computing is now. "In a certain sense, we’ve been here already," he says.
CERN, where lives the focal point of the Higgs boson research in the Large Hadron Collider (LHC), is considered Tier 0 within the grid. That is where scientific data is produced by smashing particles together in the 24 km LHC tunnel. Data from those experiments is then sent out through the grid to 11 Tier 1 sites, which are major laboratories with large-scale data centres that process much of the scientific data. Those sites then produce datasets that are distributed to more than 120 academic institutions around the world, where further testing and research is conducted.
The entire grid has a capacity of 200 PB of disk and 300,000 cores, with most of the 150 computing centres connected via 10Gbps links. "The grid is a way of tying it all together to make it look like a single system." Each site is mostly standardised on Red Hat Linux distributions, as well as a custom-built storage and compute interfaces, which also provide information services describing what data is at each site.
Research that contributes to a ground-breaking discovery like the Higgs announcement, though, is not always centrally organised. Bird says in fact it is quite a chaotic process and one that makes it difficult to plan for the correct amount of compute resources that will be needed for testing at the various sites. For example, when there is a collision in the LHC, impacted particles leave traces throughout the detector. A first level of analysis is to reconstruct the collision and track the paths of the various particles, which is mostly done at the Tier 0 (CERN) and Tier 1 sites. Other levels of analysis are broken into smaller datasets and distributed to the partnering academic institutions for analysis. From there, a variety of statistical analysis, histograms and data mining is conducted. If a certain discovery is made, an analysis might be refined and another test may be run. "You really can’t predict the workflows," he says.
That is why Bird and CERN are excited about the potential for using some cloud-based services. "We’re interested in exactly what it would take to use cloud storage," he says. "But at this point, we’re just not sure of the costs and how it would impact our funding structure." CERN receives money from various academic institutions that have access to the data CERN creates to analyse it. Many of those partnering academic groups have compute resources in place and want the CERN data on their own sites to run experiments on and make that resource available to their academic communities. "From a technical point of view, it could probably work," he says. "I just don’t know how you’d fund it."
CERN has made some initial forays into the cloud. Internally, CERN is running a private cloud based on OpenStack open source code. Many of the partnering organisations have private clouds on their own premises as well.
In March, CERN and two other major European research organisations took steps to create a public cloud resource called Helix Nebula – The Science Cloud. It is a partnership of research organisations, cloud vendors and IT support companies that are powering a community cloud for the scientific and research community. The two-year pilot program CERN has recently kicked off will begin by running simulations from the LHC in the Helix Nebula cloud.
Bird is hopeful about the cloud, figuring that within another decade the cloud will be where grid computing is now. "It’s just not obvious how we’ll get to that point," he says. But even if the cloud has its challenges, Bird is confident that the scientists who made one of the most important scientific discoveries in decades should be able to figure out the cloud.
IDG News Service
Subscribers 0
Fans 0
Followers 0
Followers