 |
|
|
|
|
| An important function provided by the management system for Computation Grids is to monitor the performance and status of jobs executing in the Grid. The heterogeneity, complexity, dynamism and scale of modern Grids present challenges to delivering this functionality. Monitoring data is often collected by a heterogeneous mix of monitoring tools with different access interfaces and managed by disparate organizations, while the tasks of a job are scheduled dynamically on one or more nodes. Traditional monitoring solutions provide few mechanisms for assimilating data about such a distributed job and its component tasks, while manual collection is error-prone, slow, fragile, and prevents using the data to drive other automation tasks.
We collaborate with several key Chinese universities responsible for ChinaGrid development to develop monitoring solutions to address these issues. ChinaGrid, an initiative sponsored by the Chinese Ministry of Education, is one of the largest Grid computing projects in the world. Our solution works with the job schedulers in a Grid and existing data collectors to automatically track and monitor job execution. It automatically generates CIM models that describe the ChinaGrid infrastructure and a job's use of the infrastructure, which are then used to automatically establish a link between a job or its subcomponents and the monitoring data for it. The Web service standard is used to present the data consumer with a common interface to the data. Our solution scales well and effectively eliminates the barriers to accessing the monitoring data.
» Top |
|
|
|
|
|
|
|
| Digital content management and preservation has been a key research effort at HP Labs. One result of the effort is the development of the DSpace system, an open source digital repository system initially developed by HP Labs in collaboration with MIT libraries. The DSpace software platform enables organizations to capture, store, index, preserve and distribute their digital assets. Over 330 organizations worldwide have used DSpace to build digital library systems since 2001.
At HP Labs China, we examine the issue of federating multiple digital content management systems and present a common portal to browse and search their contents. Federation allows the contents distributed in multiple organizations to be aggregated using a consistent vocabulary for centralized searching and browsing. We develop technologies for building a large-scale, distributed digital content management infrastructure based on DSpace.
Our research targets university digital museums in China, a project initiated by the Chinese Ministry of Education. While each university is responsible for digitizing and preserving its own collection of museum objects, sharing of these contents across multiple universities requires federated content management. We collaborate with Beihang University in an effort to build a federated China university digital museum. We have developed a federated DSpace, DM-DSpace, which can be set up as either the federated system (data center) or local museum using different configurations. DM-DSpace is also being leveraged in other China digital museum projects, such as China Digital Science and Technology Museum.
» Top |
|
|
|
|
|
|
|
| Data centralization/consolidation has become an industry trend. While centralized data management is significant for some enterprises, such as banks, it may not be the best choice for certain area where data are naturally distributed. In some case, such as population information management, the basic information can be centralized but next level details could be kept locally thus distributed.
Further, the essence of data centralization is not in centralized storing, but in centralized control, which therefore allows us to consider an alternative: distributed data centralization, referred to as virtual data centralization, or virtual data management.
The idea of virtual data management is consistent with grid computing, as virtual organization (VO) is the soul in this context. The main goal of this project is to build a Virtual Database (VDB) over multiple data sources connected through ChinaGrid, and provide Virtual Database Service (VDBS).
VDB is based on a middleware with database capability; it maintains the information, or meta-data, on the underlying data sources. To acquire and update such information is one system requirement, while sharing such information among VDBs is another. A VDB is accessed through SQL statements extended with specific types (e.g. video short) and operators on those types. Tables in a VDB are virtual tables (VTs) with virtual schemas. A VT corresponds to the data maintained in one or more tables in one or more remote databases, chunk servers, or other VDBs. For example, a possible query may require joining two tables from different databases.
The idea of virtual data management is in-line with ChinaGrid. However, ChinaGrid Support Platform which is the middleware for the whole ChinaGrid divided data management stored in ChinaGrid backbone into two categories: file data management and heterogeneous database management. ChinaGrid also needs virtual data management to manage various data source using one uniform entrance. It's one of the projects in collaboration with Tsinghua University.
» Top |
|
|
|
|
|
|
|
| This program examines innovative parallel architectures and algorithms for managing and processing massive amounts of data, and for integrating information from a large number of sources. Combining parallel computing with large scale data management, we examine technologies that help scale compute-intensive applications to very large data sets, and to harness parallelism in achieving real-time responses. Instead of bringing data to where the programs execute in a parallel computing environment, we explore approaches that bring programs to where the data resides.
We collaborate with research partners who build simulation models or predictive models in scientific disciplines. One example of our effort is a project in the area of hydro-informatics, in partnership with China National Lab of Hydraulic Engineering at Tsinghua University. As scientists need to draw upon a very large set of temporal and spatial data to build predictive models for water resources around large river systems, we explore methods to partition the large data set among nodes in a computing cluster, and distribute computation among the nodes while optimizing for temporal and spatial dependencies inherent in the computation requirement. We study methods to balance computation on-demand with pre-materialized and stored results.
We also examine emerging technologies that enable multiple sources of information to be integrated. Multiple sources of geo-spatial information, including satellite images, maps, etc., can be mashed up with results of scientific models that predict environmental conditions, such as water levels in major rivers. We utilize emerging standards and new technologies to perform information integration as well as 3D visualization.
» Top |
|
|
|
|
|
| |
|
Worldwide sites |
|
|
|
|
Printable version
