Imagine, just for a moment, the following scenario. A manager in a production company is planning for the future. She sits at her desktop computer and requests information on production capacity and existing orders. The system reacts by asking if she wants the information tabulated against time and if she wants to visualise it using bar charts, to which she replies yes. The system starts to execute this request but also asks if, in addition, she requires predicted sales (and which prediction method), workforce dynamics (known and predicted), current and predicted exchange rates in the countries where the company is operating and current and predicted costs of raw materials. She replies that she would like all this. The system then starts to execute this request and also suggests that perhaps similar analyses using available data on the five major competitors to the company would also be useful. Again she says she would like to have this information.
Mobility
The system knows from her calendar that she is in meetings for an hour and is then going to the airport. It therefore configures the output information ready to be viewed on a PDA (personal digital assistant) so she can preview it in the taxi on her way to the airport but also prepares her a presentation format on a laptop ready for her to collect by Wi-Fi access at the airport so she can add appropriate text on the plane and be ready for the meeting to which she is going.
IT professionals know that the system to satisfy such a request has several interesting properties. Among them, it must have knowledge-assisted access to heterogeneous information sources and an ability to integrate into a homogeneous output. It must have appropriate software to do the search/retrievals, analyses, aggregations and visualisations required. It must have appropriate computation facilities. It must have output (and input) human-computer interaction capabilities that are flexible and knowledge-assisted. Of course privacy, security and performance are required, together with reliability, sustainability and availability of the system.
This is one scenario. It is relatively simple and in real life would be more complex with other people involved. It is easy to imagine similar scenarios in the medical domain, environment, extraction industries, agriculture, transport, leisure industry, academia – wherever.
Impossible dream
An impossible dream? I think not. Many of the components have – usually separately – been demonstrated. It may be argued that this vision is the future for IT. The rest of this article traces the development of grids and outlines the potential of the concept.
Early history
In 1999, the UK academic research funding agencies were proposing their plans for the next five years. It was felt that these had insufficient IT and that the IT proposed was incoherent. I was asked to produce a ‘thought piece’ on IT support for R&D for the next five to 10 years. This paper [Je00] led, via various committees, to the UK E-science programme and, later, to the grid programme of the EC (European Commission) in Framework Programme 6.
Three layer model
The paper proposed a three-layer model. The lowest ‘computation’ layer is analogous to that of the US grid environment providing computation power, data stores and communications capability. It also includes detectors, so that fresh data (e.g. meteorological, road traffic, production units produced, environmental pollution) is available if required.
The middle ‘information’ layer provides homogeneous access over heterogeneous information sources and appropriate software for analysis, data reduction and visualisation.
The top ‘knowledge’ layer includes human generated knowledge but also machine-generated knowledge (by data mining).
The three layers are connected in an upward direction by the data to knowledge abstraction (i.e. creation of theories whether in business or academia) and in the downward direction by control (i.e. once we know (knowledge) that we can detect earth atmosphere clouds from spacecraft at certain wavelengths we can set the detectors only to those wavelengths if we are searching for clouds).
Simple idea
The idea behind it is simple. The researcher (or more generally the end-user whether for business or leisure purposes) wishes to have appropriate information provided with the lowest possible threshold of effort and the highest possible quality (recall, relevance, presentation form). In short, “the right information in the right place at the right time in the right form”. The end user ‘plugs into’ the grid environment and does not care (subject to quality, security, cost, ethical considerations) from where the resources (computation, information, software) are provided. A simple analogy is the early morning cup of tea: the end-user does not care from where the electricity comes as long as it boils the water in the kettle.
The concept
In 1998-99 in the USA, the concept of grid had emerged emphasising meta-computing (linked supercomputers scheduled as one) [FoKe98]. However, in the UK it was felt that the real requirement was for virtualisation of all resources below a service-oriented middleware layer. Put simply, this means that the end-user interacts with the grids environment intelligently to determine the request to the satisfaction of each and the environment then executes it, interacting with the end-user as necessary. This execution involves obtaining suitable hardware resources, data/information resources and software components. The execution also involves interaction with the user through whatever device(s) the user utilises during the execution – from desktop through laptop to palmtop – and allowing for differently-abled access – covering disabled users and users temporarily disabled (e.g. when driving). Finally it involves users interacting with each other cooperatively, even if remotely.
Progress
In the UK’s E-science programme [eSC] there are four main strands. (See the inset panel, E-Science Strands.)
In Europe other countries initiated similar programmes. The EC set up a grids unit and there is a very active set of projects ongoing to develop various aspects of grids in a European context. Furthermore, the EC set up an expert group, (known as the Next Generation grids (NGG) expert group) which has produced three visionary reports available on the grids unit webpage [NGG]. The main conclusions from NGG1 provided a specification for a virtualisation environment. Those from NGG2 stressed autonomics – the self-management and scalability aspects required for reliable transparent operation – and requirements of the operating system. NGG3 concentrated on a service-oriented architecture of software components that interact intelligently to achieve the end-user requirements.
In the USA
Meanwhile in the USA, the emphasis remained on the original grid concept of meta-computing although with some movement towards service-oriented architectures.
Industry took up the challenge and has products, following the concepts and ideas in the NGG expert group. Examples include IBM, which has produced autonomic (self-managing) systems while ORACLE has produced performant cluster-based database systems. Many companies are working on service-oriented architectures.
In the period of 2000-2002, people from Europe were concerned that the US approach was insufficiently general to allow deployment beyond (certain areas of) academic R&D. The European vision of grids was wider than the US vision of grid. In particular the grids vision emphasised data, information and knowledge rather than supercomputing and emphasised the need for a software development in a service-oriented (component) approach to development of the middleware and the applications instead of the rather monolithic approach to scheduling utilised in the US. Finally, the Europeans wanted a convergence to indistinguishability of WWW and grids. This led to the adoption through GGF (Global Grid Forum) of OGSA (Open Grid Services Architecture), which essentially integrates grid concepts with the web services concepts.
Meaning
For the end-user, grids provide an easy-to-use environment that interacts intelligently to deliver what is required. It can be compared with the user experience with a very helpful travel agent, lawyer, doctor, estate agent or similar.
For the system designer grids provides an architecture within which many required components (hardware, software, information) exist already and the skill is in the intelligently-automated assembly of these components to satisfy the end-user request. Common requests could be satisfied by pre-assembled (composed) components. Initially many non-atomic components will be composed manually by experts but progressively the composition itself will be automated.
For the programmer, grids provide an environment within which software components, viewed as services, are constructed and executed.
The major objective is to increase reliability and scalability giving appropriate performance and security during execution for the end-user. A second objective is to provide an environment which encourages the development (programming) of atomic components which are fit for purpose, of high quality (performance, security, reliability) and multiply used.
Where next?
The end-goal is that (subject to privacy, security, rights and cost constraints) the whole world of information and computation is available to the end-user. Clearly this is more applicable in the academic research world than the commercial world, but the ‘world of information’ for a commercial company will involve its partners in virtual organisations (e.g. a group of companies bidding for a contract). This has interesting implications for dynamic reconfiguration of security parameters including authentication of users and authorisation of access to information, software or hardware (unseen by the end-user behind the virtualisation layer).
Furthermore, the execution of end-user requests is reliable because the grids environment is self-managing, self-sustaining and self-tuning giving performance and availability (e.g. by replacement of failed or failing components whether hardware, software or information sources).
The self-composing of intelligent software component services (now becoming known as service oriented knowledge utilities) with dynamic reconfiguration of the execution graph of components to assure resilience and performance, is the preferred mechanism to makes grids a reality.
It is clear there is a very large set of R&D challenges to be met to achieve the dream. However, it is believed that there will be worthwhile delivered products along the way so that the final objective is approached incrementally with benefits at each step.
More Information
There are many sources of information. Some introductory material in readable form is found in issues of ERCIM News, particularly issues 45 and 59 [E45] [EN59]. The UK E-Science website (referenced below under [eSc]) has a wealth of information. [Je04] provides a broad overview and emphasises the R&D challenges.
************************************
E-Science Strands
E-Science programme four main strands:
(a) the provision of a national GRID service providing shared resources;
(b) the provision of middleware to virtualise the GRID service (initially GLOBUS (middleware to schedule servers) with SRB (Storage Resource Broker from San Diego), a fast FTP (File Transfer protocol); LDAP (Directory Service) and Access GRID (a kind of super videoconferencing));
(c) advanced research applications utilising the grid service that could not be done in other ways – these provide ‘new science’ but also demonstrate the requirement for and capabilities of the grid service;
(d) R&D in IT to develop better middleware for the purpose: one example is the development of OGSA (Open GRID Service Architecture) components which make the WWW and GRIDs environments indistinguishable and particularly OGSA/DAI (Data Access Interface) which overcomes some problems of SRB in database access).
References
[EN45] http://www.ercim.org/publication/Ercim_News/enw45/
[EN59] http://www.ercim.org/publication/Ercim_News/enw59/
[eSc] http://www.rcuk.ac.uk/escience/
[FoKe98] I Foster and C Kesselman (Eds). The Grid: Blueprint for a New Computing Infrastructure. Morgan-Kauffman 1998
[Je04] Jeffery, K.G.; ‘grids, Databases and Information Systems Engineering Research’ in Bertino,E; Christodoulakis,S; Plexousakis,D; Christophies,V; Koubarakis,M; Bohm,K; Ferrari,E (Eds) Advances in Database Technology – EDBT 2004 Springer LNCS2992 pp3-16 ISBN 3-540-21200-0 March 2004
[NGG] http:// www.cordis.lu//ist/grids
Prof Keith G Jeffery is an ICS Fellow and director of information technology and head of the business and information technology department, CCLRC Rutherford Appleton Laboratory.








Ltd