Technical Reports


Click here for full text: PDF

Clusterken: A Reliable Object-Based Messaging Framework to Support Data Center Processing

Stiegler, Marc; Li, Jun; Kambatla, Karthik; Karp, Alan
HP Laboratories


Keyword(s): data center, publication/subscription, reliable object-based messaging

Abstract: Hadoop enables high productivity in the development of MapReduce applications. However, Hadoop becomes less effective the further an application's natural pattern of computation is from MapReduce. This paper presents Clusterken, a reliable, object-based messaging framework to support data-center-based applications. Clusterken provides object-level virtual cluster management, exactly-once message processing, composable reliability, output validity, and authorization based access control. Together, these features simplify programming and improve productivity on distributed applications by enabling arbitrary interaction patterns. We compare two implementations of a specification for a publication/subscription system, one in Hadoop, the other in Clusterken. The comparison suggests that, for at least this one application, Clusterken can yield a fourfold increase

5 Pages

External Posting Date: April 6, 2011 [Fulltext]. Approved for External Publication
Internal Posting Date: April 6, 2011 [Fulltext]

Back to Index