XSEDE’13 in San Diego – when super heros met supercomputers

There aren’t too many conferences where you meet Batman, Dr Who and the Wicked Witch of the West while still checking into the hotel. XSEDE’13 in San Diego this year overlapped with the famous Comic Con event right next door – so caped super heros marching past in the lobby was apparently part of the deal. Comic Con attracts over 120,000 participants every year, XSEDE slightly fewer at 700. But this is an ever rising number year on year, as project leader John Towns was pleased to point out. And I have a suspicion that the categories of ‘comic book nerds’ and ‘IT geeks’ are perhaps not entirely mutually exclusive sets…

XSEDE, the Extreme Science and Engineering Discovery Environment, is a National Science Foundation-supported project that brings together supercomputers, data collections, and computational tools and services to support science and engineering research and education. The annual XSEDE conference focuses on the science, education, outreach, software, and technology used by the XSEDE and other related communities worldwide.

The programme kicked off with a day of well attended tutorials – with a 200-strong host of students at the event, the training opportunities were well appreciated, as was the opportunity to showcase their work in poster sessions and competitions. Even younger participants were catered for by a morning robotics class, “Ready, Set, Robotical” which I was more than tempted to join.

Training is always a strong theme at XSEDE, and the challenges of providing online courses in parallel computing were discussed, as well as developing undergraduate and graduate programmes in computational science. Campus champions are the backbone of outreach on a regional basis, and XSEDE is now looking to expand the scheme into subject specific champions. This emerged as a theme for future collaboration between XSEDE and the European Grid Infrastructure in the Birds of a Feather session. EGI recently set up an EGI Champions scheme, including representatives from fields as disparate as life sciences, environmental sciences and maths. Other areas where EGI and XSEDE expect to work together include federated high throughput data analysis, federated clouds and user support. One use case already in progress covers HPC-HTC workflows for the computational chemistry community. This was one of the joint use cases that emerged in the call issued at the beginning of the year. So there's lots to watch out for, even now the caped crusaders have (mostly) left town.

Riding the data deluge on the shoulders of giants

The grid and the cloud are dead! Long live big open data! At least that seems to be the statement coming from a number of recent meetings.

Last week saw an exploratory meeting of the Research Data Alliance in Washington DC. Around 120 people from around the world had gathered to try and understand the blocking issues in allowing researchers to access and share data between disciplines, institutions and across countries. The issue here is not just around the pure issues of technical interoperability, but more broadly about building a community that can propose refine their technical work to reduce the barriers to a collaborative global data infrastructure.

The driving force behind this initiative (projects and funding agencies in USA and Europe) is not just about achieving technical operability, but about ensuring open access and exploitation of the big data sets being generated by researchers by others outside the generating community. An interesting example was presented by Chris Greer from NIST. He cited the how the release of NASA Landsat images in 2008 for unrestricted use had now created an estimated value of $935M/year to the environmental management industry. While it is not always necessary for the investments made by the public to yield economic returns – it cannot hurt!

However, the challenge in building any infrastructure is to balance the common needs against those that are specific to a particular science domain. Each research community will have developed its own vocabulary, its own metadata descriptions, its own data access services to expose the underlying data models. Where should the line be drawn between common and domain specific?

What are the common mechanisms that are needed to allow different research communities to collaborate and share data? While this is still work in progress, some consensus is emerging. For instance the need for persistent data identifiers that enable individual data sets and data objects to be described, discovered and located. Authentication and authorization is still needed when discussing open science data as funders like to know how the generated data is being used and it is possible that some data will be restricted to members of a collaboration for some of the time.

Leif Laaksonen from CSC described how within Europe the EUDAT (http://www.eudat.eu/) project is examining some of these technical issues with the recently started ICORDI (http://www.icordi.eu/) project now providing coordination and input into international activities such as the RDA. Andrew Treloar related how activities by the Australian National Data Services is working on helping scientists transforming data (generally unmanaged, disconnected, invisible and has a single user) to structured collections (that are managed, connected, findable and reusable) that can provide more value.

At this week’s Microsoft e-Science Workshop (that is co-located with IEEE e-Science and the Open Grid Forum 36 in Chicago this week) the focus on big data continued with sessions dedicated to Open Data for Open Science. Using environmental science as an example, with many examples drawn from NSF’s Earth Cube (http://www.nsf.gov/geo/earthcube/) initiative, the issues of data interoperability and reuse were again prominent.

The environment is an excellent example of how data reuse is needed across different domains in order to maximize knowledge discovery due to the inherent coupling between functions within the natural ecosystem and their impact within society. For instance, how do you ensure that satellite data can be coupled to land/sea/air observations collected independently over many years? How should the data coming out of the many instruments that make up an ocean observatory be integrated given the different manufacturers and their data formats?

The focus in this work is not so much on standard service interfaces but on standard data languages. Data markup languages encompassing the tracks of research vessels across the ocean, of the output from instruments, capturing the semantics of hydrologic observations are examples of many community driven initiatives. Organisations such as the Open Geospatial Consortium (OGC) – composed of representatives from academia and industry – play an important role for the environmental community due to the geospatial nature of many of their datasets and form a basis for much of the work that now takes place. The issue is about opening up your data for access not opening up your database!

Given the size and number of the environmental data sets generated from instruments and simulations, converting this data to information and knowledge provides many challenges. High Performance Computing can provide the raw simulation output and High Throughput Computing can help support ensemble studies to explore the sensitivity of the simulation. These local resources can be supplemented by capabilities accessed through grids (PRACE, EGI, OSG and XSEDE) and through commercial and publicly funded clouds.

While standards form one aspect of this discussion it is not the only , any standards need to encompass the variety of different use cases and users.

So has hype around big data grown to the point where it has now swallowed up the cloud which is still bloated on gobbling up the grid?

One of the challenges of big data is finding the infrastructure to analyse it! Here the cloud’s business model of the flexible and rapid provisioning of resources demonstrates its strengths. Creating the storage to handle the intermediate and final results generated from a HPC cluster provisioned on demand, demonstrates the need for a flexible model. As the data used in the analysis will need to be retrieved or placed in persistent data stores, issue such as authenticated and authorized access to these distributed resources becomes critical – a typical grid scenario.

In moving from one paradigm to another its important to not discard the experience and technology gained previously.

Speaking to Steve Brewer about EGI Champions

In a whirlwind week of exciting talks, illuminating conversations and great food and company, one of the most exciting announcements of the week at EGI Technical Forum 2012 was EGI Champions, which I heard about in the communications session on Tuesday in an excellent presentation from EGI's User Community Support Officer, Richard McLennan. EGI Champions really formalises a practice that has been going on throughout the e-science community, including at EGI, whereby certain star users of EGI's e-infrastructure act as beacons to attract new users in sciences currently underrepresented in the user-base (such as life and molecular sciences).

I had a chance to talk to EGI Chief Community Officer, Steve Brewer, about the programme:

Strong hints of the Higgs - live from CERN

Good morning from the restaurant at CERN where I've been watching the announcement of the results from the CMS and ATLAS experiments at the Large Hadron Collider on the screens. These two experiments have been competing in the hunt for the Higgs boson particle, the missing piece from the standard model jigsaw. The announcement happened simultaneously at CERN, a bit further away on the site from where I was sitting, and at ICHEP in Melbourne (a loooong way from where I was sitting).

As an ex-CERNoise (as they call themselves) I was particularly interested in the announcement. I joined CERN in 2008 to run communications for the Enabling Grids for E-sciencE project, the grid computing infrastructure project that is now the European Grid Infrastructure (www.egi.eu). EGI, in collaboration with the World LHC Computing Grid (WLCG) provides the distributing computing infrastructure that has processed the petabytes of data streaming from the four LHC main experiments. It was good to hear a mention of the contribution that computing has made to this momentous global achievement during the talk from Joe Incandela of CMS and Fabiola Gianotti of ATLAS - by reducing the time taken to process the individual events, the results can be announced quicker. No small achievement when the US Tevatron is snapping at their heels http://www.bbc.co.uk/news/science-environment-17269647

While I was at CERN, the LHC started up in September 2008 to astonishing media and global interest... and then two short weeks later experienced a dramatic quench in one of the magnets deep underground, causing serious damage to the machine and a shut down in running of over a year. The mood at CERN switched from what probably counts as wild excitement in physicist terms (well, we had a staff party, and there was dancing of a sort) to one of pragmatic problem-solving and a collective rolling up of sleeves. The LHC has been a long time in the building, with many many problems to solve along the way  - here was just another phase to plan through. Just over a year later in November 2009, the LHC came back more stable and reliable than before and has since run pretty much flawlessly, ramping up the energy to where it is now at a record breaking 8 TeV.

This morning I was back at CERN for a different meeting but happened to be in residence on site in time for the joint announcement from CMS and ATLAS. I watched the auditorium fill up with students, physicists and a good selection of the world's media. As Peter Higgs arrived to take his seat, a round of gentle applause broke out. After a bit of milling about, a clearly rather nervous Joe Incandela from the CMS experiment took the podium to run through his slides. Following the Twitter feed online, these caused a certain level of bemusement among the Twitterati - the general thrust was I don't really understand this, but it sounds good. Despite working at CERN in the past, I was kind of with them, but the punchline came out pretty clearly at the end of Incandela's talk: We have found a new boson with mass of 125.3  GeV +/- 0.6 at 4.9 standard deviations or "sigma". A loud and extended round of applause in the auditorium followed. To put it in context, one sigma means the results could be random fluctuations in the data, 3 sigma counts as an observation and a 5-sigma result is a discovery. It is about a one-in-3.5 million chance that the signal they see would appear if there were no Higgs particle - so extremely unlikely. However, the CERN press release (http://press.web.cern.ch/press/PressReleases/Releases2012/PR17.12E.html) reminded us that the results presented today are preliminary, as the data from 2012 is still under analysis. The complete analysis is expected to be published around the end of July.

Fabiola Gianotti of the ATLAS experiment followed next. "It's tough to talk second," said Fabiola, "because all the clever things have been said." She then went on to very much prove that wasn't the case with a highly technical presentation (despite the comic sans font, which did not go down well online). However, when she got to the part about zooming in on the data... more spontaneous applause starts up (with a slightly ironic echo here in the restaurant). The ATLAS experiment results echo the CMS results - 126.5 GeV for a boson at 5.0 sigma, in agreement with the standard model of physics. In a personal statement, Gianotti said "It is very nice for it to be at this energy because we can measure it here at CERN in abundance – thanks nature!"

Rolf Heuer, the Director General of CERN had a final say. "It's been a global effort, a global success. It has only been possible because of the extraordinary achievements of the experiments, infrastructure and the grid computing." He continued, "We have a discovery, we have a new particle consistent with the Higgs boson. It’s a historic milestone today but we are only at the beginning. We can all be proud and it has global implications for the future. I think we can be very optimistic." Peter Higgs added his congratulations and announced himself "extraordinarily impressed". I suspect he isn't alone.

So a highly significant day for CERN, for the hunt for the Higgs and for me, as its pretty satisfying to be at CERN both for the LHC start up, and the announcement of the strongest hints of the famous particle so far. It's 28 years since the idea of the LHC was first suggested at a physics meeting, more than half my life time ago - I feel very lucky to have been here at this time to see the results of the work of over 20 years, thousands of people worldwide, not to mention six Director Generals. Congratulations!

Matching the needs of users: a life sciences perspective

Whether it’s for a bioinformatics application or for visualising cells at a molecular level, it is clear life science communities are increasingly calling upon exascale computational capabilities, simulation and modelling applications and e-infrastructures to accelerate their research. Although there is enormous potential for integration and modelling on a multi-scale level, it is surprising that life scientists are still not major users of the grid (approximately 11% of life scientists use PRACE and 15% using national computing centres, and less are using the Grid itself). So it's great to see that many projects presented at the EGI Community Forum this week, are helping non-experts to adopt the technology.

On Wednesday morning, we heard from Alexander Bovin about the WeNMR project, who are successfully growing their life science user base (http://www.wenmr.eu/). WeNMR is the first and largest Virtual Research Community to be supported by EGI within the life science area. WeNMR provide simple user-friendly access to Grid via web portals. Two dozen computational NMR services are available on the platform, and a number of applications for modelling macromolecules e.g. 3D Dart, Haddock. Users have access to various training resources (including wikis, online tutorials, a blog, a help centre, NMR and SAXS services).

The average life science researchers often doesn't come from a computing background, and hence choosing the most appropriate application software/architecture to fit their problem can prove difficult.  The ScalaLife  project (presented by Professor Erwin Laure from the Royal Institute of Technology (KTH), Stockholm in Wednesday) is an ambitious initiative that hopes to provide a one-stop-shop knowledge-base and advice centre for the computational Life Science community. You can read more on the project in a blog entry yesterday from iSGTW journalist, Adrian Giordani. ScalaLife aims to help researchers by providing best practice, support and experience in order to help them develop more accurate structural and functional models, and in the longer-term could potentially have some impact in reducing the cost of generating new drugs.

Another project that resonated with me was ScienceSoft (http://sciencesoft.web.cern.ch/), which again is a new initiative to assist scientific communities in finding the software they need. The project is currently in a early scoping phase, but hopes to promote the development and use of open source software for scientific research. The project will provide a marketplace to match user needs to software products and services. Possible content could include catalogues, software services, a skills database, information on who likes what and why (a rating system), a peer review system as well as a citation system to allow software to be references in papers. At the moment, ScienceSoft is looking for ideas from community members, editors and developers to offer advice on where functionality is needed. GridCast interviewed Alberto Di Meglio about the initiative in a video featured yesterday afternoon. 

Investigating molecular machines with Alexandre Bonvin at ISGC2012

This morning at the ISGC2012 event in Taipei, Alexandre Bonvin of the University of Utrecht introduced us to how he is using the grid to investigate the molecular machines and the social life of proteins. In many diseases, for example, it is the breakdown in communication between proteins that leads to problems. “I’m coming at this as from the user perspective,” he told us. “A key part of my role is to hide the complexity of the grid from the users.”

Structural biology uses techniques such as NMR and synchrotron X-ray analysis to delve into the structure of complex molecules, such as proteins and peptides. The big machines acquire the data from samples, and this is processed to determine what the molecule contains and the constraints on its shape. The number crunching on its structure starts there – and this is where the grid comes in. The grid is just right for this process, which is CPU intensive, consists of lots of parallel jobs but needs limited data transfer and storage.

The WeNMR project leads the largest EGI Virtual Organisation in the life sciences. It has nearly 400 users across 41 countries, including 4 outside Europe - in Taiwan in collaboration with ASGC, Latin America through GISELA and in SAgrid’s South Africa. This represents around 30,000 cores and 1.6 million jobs in the last year. Just over half of the jobs run on the Dutch National Grid Infrastructure, BigGrid, which provides 75% of the CPU hours.

As Alexandre said, grid is a complex business, and WeNMR provides a web portal offering ‘protocolised’ access to the grid for protocols HADDOCK, CS-ROSETTA and GROMACS. This is both a user friendly gateway for the NRM and SAXS communities and a virtual research platform. Linked to the ESFRI-INSTRUCT project, WeNMR works to foster the adoption and use of e-infrastructure and to get over the entry barrier reported by many users, while at the same time offering the top of the line software and services that attract expert users. The secret to sustainability is of course to get scientists, and not just the computer experts but the average user as well.

There was also a brief word on scientific cloud computing. Alexandre’s team is using cloud computing, supported by the supercomputing centre SARA in Amsterdam, to process libraries of proteins. Because this needs several sets of software and libraries, it is much simpler to set up a virtual machine once with all the software installed, then you don’t have to deal with clashes with scientific linux on the grid. “For the other portals,” said Bonvin, “grid is the perfect solution- cloud will not gain us anything.”

You can hear more from Alexandre, his team at Utrecht University and the CONCO project on the newly released YouTube video “Stories from the Grid” by EGI (European Grid Infrastructure). The first episode shows how a component of the cone snail's toxic venom is being modified using the aid of computer models, to help produce new anaesthetics and alleviate the muscle spasms caused by the condition dystonia. Episodes 2 and 3 coming soon!

The EGI Training Marketplace – Time to Spread the Word

By Richard McLennan, EGI User Community Support Officer

A friend of mine lives in one of those luxurious third floor Amsterdam apartments overlooking a picturesque canal in a smart part of town. For many months the street light outside the apartment failed to work but it was not until her gleaming new Yamaha Supersport motorbike needed to be parked securely on a well illuminated spot outside that she took a personal interest in getting the lamp fixed. A little research and a telephone call or two put her in touch with the right City Council maintenance department and within 24 hours, that Amsterdam street lamp was repaired and the Yamaha was securely parked under a bright spotlight. What has any of this got to do with the EGI Training Marketplace? Well, like so much in life, this story is all about the importance of effective communication - the case of repairing a street light is really very simple but nothing can happen until the right ‘someone’ is told that the job needs doing. The same is true for the EGI Training Marketplace.

A short while back I wrote an article for the EGI Blog page in which I compared EGI’s new Training Marketplace to a form of ‘ebay for grid training’. The portal had gone live on 7 June after a good deal of development work by a small team at STFC, UK. Having been personally involved in proposing, testing and fine tuning the product, I was convinced that all that remained for it to become a huge success was for me to tell people how good it was and then to sit back and watch the portal hit rate grow. After all, ebay itself already is a huge success story. Just as it provides a web based framework through which a constantly growing community of buyers and sellers get in touch with each other to exchange something, so the Training Market place provides a web based framework through which ‘grid computing’ training and training needs can be advertised so that those who seek specialist training can find what they need ….. in theory.

So, back to fixing street lights. The way things get fixed – lights on streets, in offices or even in our homes - is by waiting a long time for ‘a miracle to happen’ or alternatively, by playing an active part in the process and thereby making it happen quickly. More often than not we just need to ‘communicate effectively’ for someone to start whatever process we need to happen. The Training Marketplace seems to have remained largely untouched for a couple of months now and while it stands a chance of becoming successful if we all wait for long enough, I am sure that there is much better way. With around 18,000 registered grid users who either need or can provide specialist training, our Marketplace needs little more than to become well known. So here is the reason for this article: did you know that the Training Marketplace existed and that it is accessible through the EGI UCST Webpage? Have you set up any training events which could be promoted via the Training Marketplace? Even if the training event is closed to your own community, it may still help others to know about your training event. Go on, play an active part in putting the spotlight on your training, your training needs and our Training Marketplace.

e-ScienceTalk signs an MoU with EGI

I’m very pleased to announce that on 23 June e-ScienceTalk signed a Memorandum of Understanding with EGI-InSPIRE. This formalises the fruitful working relationship that already exists between the two projects.

The agreement was signed by Steven Newhouse, Director of EGI.eu and myself as project coordinator for e-ScienceTalk. EGI.eu is the coordinating partner for projects.

“One of the key aims of EGI is to enable access to computing resources for all scientists in Europe and we are working actively to increase the scientific diversity of our users,” said Dr Newhouse. “But for that we need to communicate the benefits of grid computing and this is where the e-ScienceTalk project can help.”

Our goal in e-ScienceTalk of course is to increase the visibility of European e-infrastructures in the wider world. To achieve this, we work to raise awareness of the contributions that distributed computing has made and continues to make to solving the scientific and societal challenges facing us today. We aim to share success stories and case studies with a wide variety of audiences, from the general public to policy-makers.

Our collaboration with EGI.eu will give us access to Europe’s leading scientific grid infrastructure community and we will do our best to let everyone know about their achievements.

All about EGI

Want to get clued up on EGI and EGI.eu before the Technical Forum next week? All is explained below in this great video from the BELIEF-II project.



And for anybody who's especially keen on the subject, you can find out even more in our 'European Grid Infrastructure' GridBriefing at the GridTalk website.

EGI's first event

The EGI Technical Forum 2010 will be the first major event within the EGI community and will bring together European distributed computing projects and their collaborators in academia and businesses, from around Europe and around the world. In doing so it will provide an ideal venue for, the developers, end-users, resource providers and decision makers involved in these projects to meet and discuss the continued transition to a sustainable production grid infrastructure for Europe.

The EGI Technical Forum 2010 will be held in central Amsterdam at Beurs van Berlage between 14-17th September 2010 in partnership with the BiG Grid (Dutch NGI) project. And what's more GridCast will be right on hand to bring you all the news live.

We're currently putting togehter our team of bloggers to cover the Technical Forum so if you're going to be attending the conference next week and fancy helping out get in touch. We'd love to hear from you.

Steven NEWHOUSE introduction to EGEE 5th User Forum

Steven Newhouse is the actual EGEE technical director, and the future EGI Director. He is talking about the end of EGEE, and its future: EGI.

Steven Newhouse on EGI and middleware

I recently interviewed my fellow blogger Steven Newhouse, the interim EGI director, about how EGI has been coming along.

You can read his interview in iSGTW, but if you want to hear steven chat about the unversal middleware distribution within EGI have a listen to the clip below.

Hot off the presses: Amsterdam thrilled to host EGI.org

Speaking with David Groep, NIKHEF, member of the successful bidding team from Amsterdam:

“When I heard the decision I thought, 'Yes! We did it.' This will be a great opportunity for Amsterdam, hosting such an important international organization and we especially hope that the staff of EGI.org will benefit from our beautiful city.


I'm very grateful to the people who voted for Amsterdam, I know how hard the selection process was. There were other strong contenders which were also excellent options. Of course,” he says “I think Amsterdam was the best choice.”

Read yesterday's press release from the European Grid Initiative.

“I hope the location in Amsterdam will help EGI be a great success. We'll do our best to ensure the success of EGI, to keep the users happy so that we get great science.”