Our heads in the Clouds

If it's Sunday it must be Taipei!

Although I would have loved to have written this from the top of Taipei 101, I was not allowed to bring up my backpack with laptop to the observation deck - so this has had to wait 24 hours to be written. But wandering about the observation deck and looking down to the traffic on the ground - even if the view was occasionally obscured by the low clouds - it provided a perfect opportunity to reflect on some of the meetings over the last few weeks and the some more meetings coming up during the next week while a guest at ISGC2010 in Taipei.

Given the lofty perspective from the observation deck my thoughts inevitably drifted between clouds and the ongoing transition from EGEE to EGI.

A few weeks ago in Brussels there was a meeting primarily of European cloud computing experts at CloudScape II. It provided an opportunity to see how the academic community's approach to cloud had evolved in the year since the last meeting. The attitude at the meeting was positive - shifting from 'if' to the 'when' of cloud adoption. However, as cloud encompasses many technologies and adoption scenarios it still seemed to be a matter of waiting for people to jump, but it was encouraging to see many small scale proof of concepts emerging from resource providers in the research community.

Cloud provision is very advanced for many office tasks. In the establishment of EGI.eu we are exploiting external infrastructure providers where ever possible. The office email and intranet is provided by Google. There are also a number of file storage providers and audio conferencing providers that offer internet enabled services that we will aim to use. As we start to look at the infrastructure we need to support the EGI-InSPIRE project (integrated single-sign on across web sites, wikis, email lists, time sheets, etc.) the solutions look harder to find. This perhaps illustrates one of the issues in the cloud computing world at the moment... the lack of standards make the integration across different providers very challenging.

So what does this mean for an e-infrastructure resource provider? Across the European e-infrastructure a phenomenal amount of effort is invested in keeping the infrastructure based around a single middleware distribution up and running. As we move to EGI the pressure to support communities in Humanities, Life Sciences, and others where they each have their own middleware distribution that they wish to deploy and maintain increases. Virtualising our infrastructure (i.e. using the techniques and operating models that have produced such a paradigm shift in commerce) seems the only way forward.

A positive consequence of this is that resource providers focus on providing a network of virtualised compute resources, linked through GEANT at sites around Europe. The managers of the various Virtual Research Communities (VRC) use these virtualised resources to deploy their virtual machines at the sites needed to support their own research communities and they become responsible for ensuring their software infrastructure works as planned. These VRC managers now own the issue of software development, deployment, upgrade and operation on behalf of their own communities. The ability of communities to deploy their compute services dynamically to resources that are near their primary data sources enables them to optimise the placement of their analysis workflow with respect to their data.

Within EGEE at the moment our focus is on making sure the individual National Grid Initiatives (NGIs) are up and running and ready to start operations in May. A series of meetings in Amsterdam last week explored how the current EGEE activities for dissemination, training, operations and the user communities are moving towards the EGI model through the support of their NGIs. Some NGIs are already very advanced - others are still coming up to speed and understanding what these new operational responsibilities mean. Certainly, the initial year of EGI will be focused on ensuring that the core operational infrastructure remains running as it is.

Once the NGIs are operating as a coherent whole consideration as to when to move to towards a European Infrastructure as a Service model, with the empowerment that this would bring to the user community, can begin. In doing so they may loose many of the advantages that the current managed platform that they have through projects such as EGEE.

Remember - always be careful what you wish for... it may just come true!