Grid computing has the potential to transform everything
from drug discovery to aerospace design to analyzing stock
portfolios but, until recently, its benefits were confined
to regions where computing infrastructure is plentiful.
HP is working with researchers in Brazil to change that
--- exploring new technologies that could bring the advantages
of grid computing to some of computing's have-nots.
By hooking together individual computers around the world,
scientists have created virtual supercomputers or grids that
can quickly process vast amounts of information, helping
to produce breakthroughs in meteorology, physics, medicine
and other fields.
The Brazilian effort, relying on a software program called
OurGrid, represents a lightweight approach to grid computing.
Most grids today are composed of large organizations and
new users must negotiate entry. To join the new network,
users need only download the open source software (the latest
release is OurGrid 3.0.2). By installing and using this software,
users can make their own resources available to others and
also become able to access resources from peers.
Unlike a classic grid approach, the Brazillian effort's
uses are limited to applications that can be decomposed in
smaller tasks running independently of each other such as
data mining, massive searches, computer imaging and computational
biology.
Regardless, scientists have found OurGrid to be plenty useful:
The software has possible applications to projects ranging
from drought forecasting to medical research.
"Despite the great progress made in the last years,
grid computing is still far from reality to most users," says
Walfredo Cirne of Universidade Federal de Campina Grande
(UFCG), which initiated the effort. "OurGrid is an attempt
to change this, delivering grid power to whoever needs it.”
HP began funding the UFCG effort in 2003, and researchers
from HP Brazil and HP Labs Bristol (UK) then also began contributing
to the collaborative effort. (The collaboration is one of
numerous grid projects at HP. To learn more, go here.)
"There's a well-known slogan about grid that says it'll
do for computing resources what the Web did for data," says
Miranda Mowbray, a researcher at HP Labs Bristol, who worked
on the system's resource-allocation mechanism and advised
on ways to strengthen the system's online community.
"Grids use the same insight -- that a simple, decentralized,
open system may be more effective than a proprietary one," she
adds. "Our implementation goes a step further than the
standard approach in its simplicity and the autonomy of its
distributed parts."
Along the way to developing the new technology, researchers
faced (and continue to face) a series of challenges, including
skepticism from the broader grid community about their unorthodox
approach. In addition, key characteristics of the technology
-- that it's lightweight and decentralized -- pose further
problems. Without centralized information on the state of
the network, there's no way of knowing whether a specific
machine is available, or to provide a global audit trail
or shared cryptographic infrastructure. As a result, researchers
have to find new ways to do scheduling, resource allocation
and security while using limited information.
Members of the new grid community are already producing
results in medical and scientific research.
A medical research project in Rio de Janeiro used the system's
computing power to successfully screen drugs for effectiveness
with an HIV variant that is more common in Brazil and parts
of Africa but rare in US and the EU -- the two markets that
sway new drug development.
“The results of this type of project could have a
huge impact on the availability of the proper medications
in Brazil and Africa,” says Darlei Abreu of HP Brazil
R&D.
Other scientists are experimenting with system resources
to create a better model for predicting drought cycles in
the Sertão area of Northeast Brazil. This vast region is
the poorest in Brazil and suffers from severe and recurring
drought. More accurate predictions could lead to better water
allocation and more effective relief programs.
"This kind of research could save lives and prevent unnecessary
suffering," says UFCG's Cirne.
A key feature of the new technology is that it enables users
to exchange resources, a simple peer-to-peer protocol known
as network of favors. Users are encouraged to donate their
idle resources to execute applications for peers (doing them
a "favor") because doing so increases their chances
of receiving an equivalent favor back from the community.
"In some sense, users 'store' their idle resources
(which would be being totally wasted) on the community for
later use," says Roque Scheer, an HP Brazil R&D
manager.
Instead of requiring negotiation as a standard grid does,
the new grid technology enables a much simpler automatic
entry.
"Network of Favors ensures that, on average, you get back
from the community the same amount of resources (CPU time)
you donated," Cirne adds.
Some of HP's technical contributions to the collaborative
effort are finding their way back into HP research projects.
In Bristol, researchers are testing to determine whether
the grid's resource allocation mechanism can be used in a
trial of a service utility for film animators. (See related
story, "Lights!
Cameras! Compute!)
At the same time, researchers at HP, UFCG and other institutions
are working to make the technology more versatile and more
useful, exploring ways to use grid technology in commercial
settings, creating large-scale community grids and looking
into grid scheduling and dynamic resource allocation.
They also hope to make it possible for the system to run
more types of applications, and they are working to apply
grid standards (set by the Global Grid Forum) so that the
new system can interoperate with other grids around the world.
Mowbray, who hopes to return to UFCG this summer to explore
whether OurGrid can be extended to support sharing of data
as well as sharing of computing power, says the work is both
exciting and rewarding.
"It's just great to work on a project that supports applications
like screening pharmaceuticals to treat a Brazilian variant
of HIV/AIDS, and planning for water security in drought-prone
Northeast Brazil," she says. "I find that very motivating."
|
Printable version
HP
Brazil-sponsored grid