Connecting computers over the internet offers the possibility of creatingcommunities of interest that can combine IT and computing power to solveproblems, writes Jamie Smyth

Imagine a world where companies, governments and organisations can pool their collective computing power to complete very complex tasks and offer information technology (IT) on demand in a way similar to how electricity is supplied to consumers.

It may sound like an aspiration to the average computer user and most corporations, but for Dr Irving Wladawsky-Berger, vice-president of IBM's next generation internet strategy, and a growing band of researchers, this represents the next big thing in IT.

Grid computing, the term used to describe a method of connecting computers using the internet, offers companies and researchers the possibility of creating communities of interest that can combine their IT and computing power to solve problems.

High-profile projects, such as the recent SETI@home experiment - a search for extraterrestrial intelligence co-ordinated by the University of Berkeley at California - have benefited by adopting grid techniques.

By linking thousands of computers together over the internet, researchers drew on massive amounts of processing power to conduct complex tasks previously possible only by using supercomputers.

The SETI project, initiated in May 1999, encouraged thousands of ordinary computer users to download software and share their computing resources while they were not using their machines.

This contribution from the public has helped the researchers begin analysing up to 50 terabytes of information. "Instead of only being able to deal with specific computers in their localisation, researchers can deal with the whole community that is sharing a grid and has access to a shared set of computing capabilities," says Dr Wladawsky-Berger.

Geographic barriers do not apply as applications and data are shared and can be spread around the world, he says.

"For example, if there is a researcher in high-energy physics, he or she will normally be dealing with colleagues, applications and high-energy physics data that are spread in major centres throughout the world.

"They [the researchers] will not know where the applications are but simply access them by letting the grid software find it for them, allocate their time, authenticate their ability to use it, and then they can go ahead and do that," he says.

A number of grids are being used in the field of life sciences, with researchers sharing computer applications and data and leveraging off each other's expertise.

For example, the University of Pennsylvania in the US recently began working with IBM to establish a grid for breast cancer research.

This will be accessible to hospitals, doctors and researchers, allowing them to share digital mammograms to enable earlier detection and diagnosis.

But these research projects may just be the tip of the iceberg as far as grid is concerned, as commercialisation of the concept starts and private firms take an interest in the new technology, according to Dr Wladawsky-Berger.

The capability of hiding the physical machines from the users and enabling the equipment to find the right resources automatically to make users' lives easier is known as "virtualisation".

It is this ability to hide the technology from users while giving them access to powerful resources on demand which IBM and Dr Wladawsky-Berger believe is the most exciting aspect of the grid. It opens up the possibility of using grid technology to supply IT on demand and use it as a cost-efficient means of managing the vast amounts of data that are created daily.

"If you look at every single type of new technology that has made it big in society, like electricity, telephony or automobiles, at some point you need to hide the technology and concentrate on the usage because you want to reach as many people as possible and make it simpler for them to use.

"The number-one advantage offered by grid is flexibility and quality of service. For example, if you don't have to buy electricity from the nearest power station to you but can get it from anywhere in Ireland, or from other countries, the probability is that you will get a better level of service."

Utilising spare computing capacity within organisations will also improve the efficiency of IT. Although, with computing technologies becoming more inexpensive by the day, this will not be as important a consideration, says Dr Wladawsky-Berger.

IBM plans to invest millions of dollars to grid-enable all its software, service and middleware products over the next several years to place it at the forefront of grid-computing technologies.

"It will be a pretty big companywide investment," says Dr Wladawsky-Berger. "What we are doing is taking a business to the next level by providing more sophisticated functionality over the internet in an easier-to-use manner."

IBM is already using grid in the commercial arena to boost its revenues and earlier this year signed its first major deal with a games company called Butterfly.net.

"It's a company that provides a platform for interactive gameplay for communities. In other words, instead of a young person just doing an online game by themselves, there are lots of communities participating. To do this you need to have a very flexible infrastructure so if any particular game has a lot of people signing online to play, they need to provide the extra capacity on demand," says Dr Wladawsky-Berger.

There will be other examples of using grid shortly, he says. IBM has begun pilot projects with a number of governments and some financial and telecoms companies, although no names can be disclosed yet to the press due to commercial sensitivities.

But there are some major challenges that need to be overcome before grid can become pervasive.

"From a technology point of view, the number-one challenge is to develop the software protocols and develop lots of middleware around those protocols to be able to automate the establishment and management of the grid," says Mr Wladawsky-Berger.

"But it takes time to develop software on an internet scale which is what the grid is about."

Another significant challenge is creating the expertise and skills needed to develop grid and the software required to make it happen, according to Dr Wladawsky-Berger.

These skills are mostly found in research organisations but are still relatively new even there. It will take time for these to drip down into the corporate world, he says.

But the opensource nature of grid-computing technologies should also help the spread of these skills as young people are trained at universities, says Mr Wladawsky-Berger, the man responsible for IBM's decision to place the Linux operating system at the core of the firm's strategy.

"It will also take time for companies to change their business models and get comfortable with doing business in a different way. Rather than controlling the technology, it is about the applications and running the business."

But businesses and organisations that doubt Dr Wladawsky-Berger's next big thing may find themselves striving to catch their competition in a few years.