publications

This page lists papers and other items available for external access.

Within each section the papers are in reverse chrological order, with the most recent at the top. Files are generally stored in Adobe PDF, and sometimes in plain or compressed PostScript. Document sizes quoted should be treated as a rough guide, rather than definitive.

This symbol flags both new papers and old papers recently made available.
This symbol flags our "top" papers - the ones we consider the most important.

Please send comments and feedback to alistair.veitch@hp.com. Although I maintain this page, not all of the papers are mine -- Alistair Veitch.

Copyright notice

The documents contained in these directories are intended by the contributing authors as a means to ensure timely dissemination of scholarly and technical work on a non-commercial basis. Copyright and all rights therein are maintained by the authors or by other copyright holders, notwithstanding that they have offered their work here electronically. It is understood that all persons copying this information will adhere to the terms and constraints invoked by each author's copyright. These works may not be reposted without the explicit permission of the copyright holder.

Except where otherwise noted, the copyright holder is Hewlett-Packard Company, in year of publication. All rights reserved.

Table of contents

In addition to acting as a guide to the various sections on this page, this list also highlights recent additions (marked with ), and our top papers (marked with ) from each category.

• Storage management
  • Algorithms for data migration
  • Traveling to Rome: a retrospective on the journey
  • Storage workload estimation for database management systems
  • Improving recoverability in multi-tier storage systems
  • An arrival curve based approach for QoS guarantees in shared storage systems
  • Adaptive control of virtualized resources in utility computing environments
  • Proportional share scheduling for distributed storage systems
  • Designing dependable storage solutions for shared application environments
  • On the road to recovery: restoring data after disasters
  • Triage: performance differentiation for storage systems using adaptive control
  • Designing for disasters
  • FAST 2003 keynote address
  • Appia: automatic storage area network design
  • Hippodrome: running circles around storage administration
  • Ergastulum: Quickly finding near-optimal storage system designs
  • Minerva: an automated resource provisioning tool for large-scale storage systems
  • Traveling to Rome: QoS specifications for automated storage system management
  • Automatic design of storage systems to meet availability requirements
• Storage systems
  • Storage, data, and information systems
  • FAB: building distributed enterprise disk arrays from commodity components
  • Buttress: a toolkit for flexible and high fidelity I/O benchmarking
  • Seneca: remote mirroring done write
  • Using MEMS-based storage in disk arrays
  • Façade: virtual storage devices with performance guarantees
  • My cache or yours? Making storage more exclusive
  • HP AutoRAID
  • An introduction to disk drive modelling
  • The TickerTAIP parallel RAID architecture
• Tuscany: economy-based service-oriented architectures
  • Utility functions, prices, and negotiation
  • Keynote and panel talks at SMDB'08.
  • Two talks on business-driven IT management at BDIM'07 and IM 2007
  • Three talks related to Middleware 2006
  • Service contracts and aggregate utility functions
  • Profitable services in an uncertain world
• Global data placement
  • The Pangaea wide-area file system
• Security
  • DDoS-resilient scheduling to counter application layer attacks under imperfect detection
  • Plutus: Scalable secure file sharing on untrusted storage
• Digital preservation
  • A fresh look at the reliability of long-term digital storage
  • The LOCKSS peer-to-peer digital preservation system
• Miscellaneous papers
  • Software: Pantheon simulator and I/O traces
  • Hamlyn: sender-directed communication
  • Idleness and its exploitation

Storage management

Algorithms for data migration [PDF 613KB] E. Anderson, J. Hall, J. Hartline, M. Hobbes, A. Karlin, J. Saia, R. Swaminithan, and J. Wilkes. Algorithmica 51, August 2008. DOI 10.1007/s00453-008-9214-y.

Traveling to Rome: a retrospective on the journey [paper, PDF 194KB and presentation, PDF 2.9MB]. John Wilkes. Proceedings of The Rise and Rise of the Declarative Datacentre (R2D2), a joint Microsoft/HP research meeting, Cambridge, UK, 12-13 May 2008. Microsoft Research technical report MSR-TR-2008-61, pages 49-52.

Storage worklaod estimation for database management systems [PDF 468KB]. Oguzhan Ozmen, Kenneth Salem, Mustafa Uysal., and Hossein Sheikh Attar. Proceedings of the 2007 ACM SIGMOD International Conference on Management of Data (SIGMOD '07), Beijing, China, June 2007.

Improving recoverability in multi-tier storage systems [PDF 246KB]. Marcos K. Aguilera, Kimberly Keeton, Arif Merchant, Kiran-Kumar Muniswamy-Reddy, and Mustafa Uysal. Proc. International Conference on Dependable Systems and Networks (DSN-2007), Edinburgh, Scotland, June 2007.

An arrival curve based approach for QoS guarantees in shared storage systems [PDF 281KB]. Ajay Gulati, Arif Merchant, and Peter Varman. Proc. SIGMETRICS, San Diego, CA, June 2007.

Adaptive control of virtualized resources in utility computing environments [PDF 440KB]. Pradeep Padala, Xiaoyun Zhu, Mustafa Uysal,Zhikui Wang, Sharad Singhal, Arif Merchant, Kenneth Salem, and Kang Shin. Proc. European Systems Conference (EuroSys), Lisbon, Portugal, March 2007.

Proportional share scheduling for distributed storage systems [PDF 664KB]. Yin Wang and Arif Merchant. Proc. 5th USENIX Conference on File and Storage Technologies (FAST '07), San Jose, CA, February 2007.

Designing dependable storage solutions for shared application environments [PDF 106KB]. Shravan Gaonkar, Kimberly Keeton, Arif Merchant and William H. Sanders. Proc. International Conference on Dependable Systems and Networks (DSN-2006), Philadelphia, PA, pp. 371-382, June 2006.

On the road to recovery: restoring data after disasters [PDF 312KB]. Kimberly Keeton, Dirk Beyer, Ernesto Brau, Arif Merchant, Cipriano Santos and Alex Zhang. Proc. European Systems Conference (EuroSys), Leuven, Belgium, pp. 235-248, April 2006.

Triage: performance differentiation for storage systems using adaptive control [PDF 442KB]. Magnus Karlsson, Christos Karamanolis and Xiaoyun Zhu. ACM Transactions on Storage, Vol 1, No 4, pp. 458-480, November 2005.

Controllable fair queuing for meeting performance goals [PDF 508KB]. Magnus Karlsson, Christos Karamanolis and Jeff Chase. IFIP International Symposium on Computer Performance Modeling, Measurement and Evaluation (PERFORMANCE), pp. 278-294, October 2005, Juan-les-Pins, France.

Designing controllable computer systems [PDF 71KB]. Christos Karamanolis, Magnus Karlsson and Xiaoyun Zhu. USENIX Workshop on Hot Topics in Operating Systems (HotOS), June 2005, pp. 49-54, Santa Fe, NM.

Dynamic black-box performance model estimation for self-tuning regulators [PDF 378KB]. Magnus Karlsson and Michele Covell. International Conference on Autonomic Computing (ICAC), pp. 172-182, June 2005, Seattle, WA.

An adaptive optimal controller for non-intrusive performance differentiation in computing services [PDF 92KB]. Magnus Karlsson, Christos Karamanolis and Xiaoyun Zhu. International Conference on Control and Automation (ICCA), June 2005, Budapest, Hungary.

Quickly finding near-optimal storage system designs [PDF 219KB]. Eric Anderson, Mahesh Kallahalla, Susan Spence, Ram Swaminathan, and Qian Wang. ACM Transactions on Computer Systems (TOCS) 23(4):337-374, November 2005.
This is a revision of an earlier SSP tech report which included more graphs and an earlier version of the description. The software is known internally as Ergastulum.

Lessons and challenges in automating data dependability [PDF 151KB]. Kimberly Keeton, Dirk Beyer, Jeff Chase, Arif Merchant, Cipriano Santos and John Wilkes. Proc. ACM SIGOPS European Workshop, Leuven, Belgium, September 2004.

Triage: performance isolation and differentiation for storage systems [PDF 116KB]. Magnus Karlsson, Christos Karamanolis and Xiaoyun Zhu. Proc. of International Workshop of Quality of Service (IWQoS'04), pp. 67-74, June 2004, Montreal, Canada. A longer version [PDF 146KB] is also available, that includes the stability proof.

A framework for evaluating storage system dependability [PDF 158KB]. Kimberly Keeton and Arif Merchant. Proc. of International Conference on Dependable Systems and Networks, Florence, Italy, June-July 2004.

Issues and challenges in the performance analysis of real disk arrays [PDF 972KB]. Elizabeth Varki, Arif Merchant, Jianzhang Xu and Xiaozhou Qiu. IEEE Transactions on Parallel and Distributed Systems (TPDS) 15(6):559-574, June 2004. Copyright © 2004 IEEE.

Designing for disasters [PDF 313KB]. Kimberly Keeton, Cipriano Santos, Dirk Beyer, Jeffrey Chase and John Wilkes. Proc. of File and Storage Technologies (FAST'04) San Francisco, CA, March-April 2004.

Automatic design of dependable data storage systems [PDF 66KB]. Kimberly Keeton and John Wilkes. Proc. of Workshop on Algorithms and Architectures for Self-managing Systems, San Diego, CA, June 2003, pp. 7-12.

Data services -- from data to containers [presentation, PDF 7.4MB]. John Wilkes. File and Storage Technologies (FAST'03) San Francisco, CA, March-April 2003.

Automating data dependability [PDF 50KB]. Kimberly Keeton and John Wilkes. Proceedings of the 10th ACM-SIGOPS European Workshop, Saint-Emilion, France, September 2002, pp. 93-100.

Ergastulum: quickly finding near-optimal storage system designs [PDF 485KB]. Eric Anderson, Mahesh Kallahalla, Susan Spence, Ram Swaminathan, and Qian Wang. HP Laboratories SSP technical report HPL-SSP-2001-05, June 2002.
There is also an expanded, technical report version [PDF 643KB], which adds a full set of graphs, but is otherwise similar.

The Rubicon workload characterization tool [PDF 162KB]. Alistair Veitch and Kim Keeton HP Laboratories SSP technical report HPL-SSP-2003-13, March 2003.

Hippodrome: running circles around storage administration [PDF 166KB]. Eric Anderson, Michael Hobbs, Kimberly Keeton, Susan Spence, Mustafa Uysal, and Alistair Veitch. Conference on File and Storage Technology (FAST'02) pp. 175-188 (28-30 January 2002, Monterey, CA). Published by USENIX, Berkeley, CA.
Slides from a Nov.~2002 version of this presentation [PDF 0.4MB, PowerPoint 2.5MB].

Selecting RAID levels for disk arrays [PDF 733KB]. Eric Anderson, Ram Swaminathan, Alistair Veitch, Guillermo A. Alvarez and John Wilkes. Conference on File and Storage Technology (FAST'02) pp. 189-201 (28-30 January 2002, Monterey, CA). Published by USENIX, Berkeley, CA.

Aqueduct: online data migration with performance guarantees [PDF 450KB]. Chenyang Lu, Guillermo A. Alvarez, and John Wilkes. Conference on File and Storage Technology (FAST'02) pp. 219-230 (28-30 January 2002, Monterey, CA). Published by USENIX, Berkeley, CA.

Efficient verification of performability guarantees [PostScript 245KB, compressed PostScript 98KB, PDF 74KB]. Guillermo A. Alvarez, Mustafa Uysal, Arif Merchant. Fifth International Workshop on Performability Modeling of Computer and Communication Systems (PMCCS 5) (15-16 September 2001, Erlangen, Germany), pages 95-99.

An experimental study of data migration algorithms [published PDF 210KB, original PDF 270KB]. Jared Saia, Eric Anderson, Joe Hall, Jason Hartline, Michael Hobbes, Anna Karlin, Ram Swaminathan, and John Wilkes.
Published in G.S. Brodal, D. Frigioni, A. Marchetti-Spaccamela (Editors), Algorithm Engineering, the Proceedings of WAE 2001: 5th Workshop on Algorithm Engineering (BRICS, University of Aarhus, Denmark, August 28-30, 2001). Springer-Verlag Lecture Notes in Computer Science 2141, pp. 145-158, August 2001. DOI 10.1007/3-540-44688-5.

The Java code used for the experiments described in the paper is available as a gzipped tar file [4.8MB]. Note: this is a cached copy. The original can be found at Jared Saia's migration web page.

Minerva: an automated resource provisioning tool for large-scale storage systems [PDF 232KB, compressed ps 214KB, PostScript 848KB]. Guillermo A. Alvarez, Elizabeth Borowsky, Susie Go, Theodore H. Romer, Ralph Becker-Szendy, Richard Golding, Arif Merchant, Mirjana Spasojevic, Alistair Veitch, and John Wilkes. ACM Transactions on Computer Systems 19(4):483-518, Nov. 2001, Copyright © 2001 ACM.

A modular, analytical throughput model for modern disk arrays [PDF 110KB, compressed ps 188KB, PostScript 848KB]. Mustafa Uysal, Guillermo A. Alvarez, and Arif Merchant. Proceedings of the Ninth International Symposium on Modeling, Analysis and Simulation of Computer and Telecommunications Systems (MASCOTS-2001), pages 183-192, 15-18 August 2001, Cincinnati, Ohio, (C) Copyright 2001 IEEE.

Simple table-based modeling of storage devices [PDF 65KB, gzipped ps 31KB, PostScript 111KB]. Eric Anderson. HP Laboratories SSP technical report HPL-SSP-2001-4 July 2001.

Traveling to Rome: QoS specifications for automated storage system management [PDF 166KB]. John Wilkes. Proc. Intl. Workshop on Quality of Service (IWQoS'2001) pages 75-91 (6-8 June 2001, Karlsruhe, Germany). Springer-Verlag Lecure Notes in Computer Science 2092, Copyright © 2001 Springer-Verlag.

On algorithms for efficient data migration [PostScript 280KB, compressed ps 130KB, PDF 226KB]. Joseph Hall, Jason Hartline, Anna R. Karlin, Jared Saia (University of Washington, Seattle), John Wilkes (Hewlett-Packard Laboratories). Presented at the 12th Annual ACM-SIAM Symposium on Discrete Algorithms (SODA) (7-9 January 2001, Washington, DC).

Storage Systems Management [PDF 1410KB, handouts in compressed PostScript 6.0MB]. Guillermo Alvarez, Kim Keeton, Arif Merchant, Erik Riedel, John Wilkes. Presented at the International Conference on Measurement and Modeling of Computer Systems (SIGMETRICS 2000) (Santa Clara, CA), June 2000.

Capacity planning with phased workloads [PostScript 646KB, compressed PostScript 244KB, PDF 410KB]. Elizabeth Borowsky, Richard Golding, Patricia Jacobson, Arif Merchant, Louis Schreier, Mirjana Spasojevic and John Wilkes. Proc. WOSP'98 (Santa Fe, NM), October 1998. (C) Copyright 1998 ACM.
Plus: Slides for the presentation given at WOSP [PostScript 1902KB, compressed PostScript 644KB, PDF 338KB].

Using attribute-managed storage to achieve QoS [PDF 37KB]. E. Borowsky, R. Golding, A. Merchant, L. Schreier, E.Shriver, M.Spasojevic, and J. Wilkes. Proc. 5th Intl. Workshop on Quality of Service (Columbia Univ., New York, NY), June 1997, pp. 199-202.

Eliminating storage headaches through self-management [PDF 19KB]. Elizabeth Borowsky, Richard Golding, Arif Merchant, Elizabeth Shriver, Mirjana Spasojevic, and John Wilkes. Abstract for work-in-progress talk. 1996 OSDI Symposium (Seattle, WA, 28-31 October 1996).

Automatic design of storage systems to meet availability requirements [PDF 95KB]. Khalil Amiri and John Wilkes. HP Laboratories technical report HPL-SSP-96-17 (August 1996). Released externally in June 2008.

A formalization of the attribute mapping problem [PDF 287KB]. compressed PostScript 149KB, PostScript 407KB]. Elizabeth Shriver. HP Laboratories technical report HPL-SSP-95-10 revision D (July 1996). Released externally in May 2001.

Attribute-managed storage [PDF 28KB, compressed PostScript 42KB, PostScript 91KB]. Richard Golding, Elizabeth Shriver, Tim Sullivan, and John Wilkes. A position paper for the Workshop on Modeling and Specification of I/O (San Antonio, TX, 26 Oct. 1995).

Storage network design

Appia and the HP SAN Designer: automatic storage area network fabric design [PDF 252KB]. Julie Ward, Michael O'Sullivan, Troy Shahoumian, John Wilkes, Ren Wu, and Dirk Beyer. HP Technical Conference 2003, April 2003 (Keystone, CO). An internal HP technical conference.

Appia: automatic storage area network design [PDF 505KB]. Julie Ward, Michael O'Sullivan, Troy Shahoumian, and John Wilkes. Conference on File and Storage Technology (FAST'02), pp. 203-217, 28-30 January 2002 (Monterey, CA). Published by USENIX, Berkeley, CA.

Appia: formalization of its topology assignment problem [PDF 49KB]. Li-Shiuan Peh. HP Laboratories SSP technical report HPL-SSP-98-12, September 1998.

The Appia topology solver: implementation [PDF 130KB]. Li-Shiuan Peh. HP Laboratories SSP technical report HPL-SSP-98-13, September 1998.

Storage systems

Storage, data, and information systems [Amazon.com] John Wilkes, Christopher Hoover, Beth Keer, Pankaj Mehra, and Alistair Veitch. HP Laboratories, Palo Alto, CA. ISBN-10: 1-4243-1731-2. 95 pages. 5th edition, January 2008. Available from Amazon.com.

Hibernator: helping disk arrays sleep through the winter [PDF 640K] Q. Zhu, Z. Chen, L. Tan, Y. Zhou, K. Keeton and J. Wilkes, Proc. of Symposium on Operating Systems Principles, pp. 177-190, October 2005.

On the erasure recoverability of MDS codes under concurrent updates [PDF 106K] Marcos K. Aguilera, Ramaprabhu Janakiraman, and Lihao Xu. Proc. of 2005 IEEE International Symposium on Information Theory, pp. 1358-1362, September 2005, Adelaide, Australia.

Using erasure codes efficiently for storage in a distributed system [PDF 165K] Marcos K. Aguilera, Ramaprabhu Janakiraman, and Lihao Xu. Proc. of International Conference on Dependable Systems and Networks, pp. 336-345, June 2005, Yokohama, Japan.

FAB: building distributed enterprise disk arrays from commodity components Yasushi Saito, Svend Frølund, Alistair Veitch, Arif Merchant, and Susan Spence. ASPLOS 2004 (October 2004, Boston, MA).

A decentralized algoritnhm for erasure-coded virtual disks Svend Frølund, Arif Merchant, Yasushi Saito, Susan Spence, and Alistair Veitch DSN 2004 (June 2004, Florence, Italy).

Buttress: a toolkit for flexible and high fidelity I/O benchmarking Eric Anderson, Mahesh Kallahalla, Mustafa Uysal, Ram Swaminathan FAST 2004 (March-April 2004, San Fransisco, CA). Published by USENIX, Berkeley, CA.

Seneca: remote mirroring done write [PDF 219KB] [Powerpoint slides 441KB] . Minwen Ji, Alistair Veitch, and John Wilkes. USENIX Technical Conference (USENIX'03) pp. 253-268 (June 2003, San Antonio, TX). Published by USENIX, Berkeley, CA.

FAB: enterprise storage systems on a shoestring Svend Frølund, Arif Merchant, Yasushi Saito, Susan Spence, and Alistair Veitch HOTOS 2003 (May 2003, Kauai, HI).

Building storage registers from crash-recovery processes Svend Frølund, Arif Merchant, Yasushi Saito, Susan Spence, and Alistair Veitch HP Labs technical report SSP-2003-14 (September 2003).

Using MEMS-based storage in disk arrays [PDF 164KB, PostScript 975KB]. Mustafa Uysal, Arif Merchant and Guillermo Alvarez. Conference on File and Storage Technology (FAST'03) pp. 89-102 (31 Mar - 2 Apr 2003, San Francisco, CA). Published by USENIX, Berkeley, CA. Best paper award.

Façade: virtual storage devices with performance guarantees [PDF 704KB, PostScript 2896KB]. Christopher Lumb, Arif Merchant and Guillermo Alvarez. Conference on File and Storage Technology (FAST'03) pp. 131-144 (31 Mar - 2 Apr 2003, San Francisco, CA). Published by USENIX, Berkeley, CA.

My cache or yours? Making storage more exclusive [PDF 251KB]. Theodore M. Wong and John Wilkes. USENIX Annual Technical Conference (USENIX 2002), pp 161-175 (Monterey, CA, June 2002).
Presentation: 2up slides [PDF 161KB], with notes [PDF 204KB].

A backup appliance composed of high-capacity disk drives [PDF 32KB]. Kimberly Keeton and Eric Anderson. HP Laboratories SSP technical report HPL-SSP-2001-3, April 2001.

A one page version also appears as a position summary in the proceedings of the 8th Workshop on Hot Topics in Operating Systems (HotOS-VIII), p. 171, 20-23 May 2001, Schloss Elmau, Germany, (C) Copyright 2001 IEEE.

Characterizing data-intensive workloads on modern disk arrays [PostScript 9644KB, compressed PostScript 1272KB, PDF 636KB]. Guillermo Alvarez, Kim Keeton, Erik Riedel, and Mustafa Uysal. Workshop on Computer Architecture Evaluation using Commercial Workloads (CAECW'01, held in conjunction with HPCA-7) (Monterrey, Mexico, January 2001).

Highly concurrent shared storage [PostScript 929KB, compressed PostScript 375KB, PDF 105KB]. Khalil Amiri, Garth A. Gibson, and Richard Golding. Proc. International Conference on Distributed Computing Systems (ICDCS 2000) (Taipei, Taiwan, April 2000).

Fault-tolerant replication management in large-scale distributed storage systems [PostScript 212KB, compressed PostScript 85KB, PDF 141KB]. Richard Golding and Elizabeth Borowsky. Proc. 18th IEEE Sumposium on Reliable Distributed Systems (SRDS'99) (Lausanne, Switzerland), 19-22 October 1999.

The Palladio access protocol [PDF 200KB]. Richard Golding. HP Laboratories technical report HPL-SSP-99-2 (November 1999).

The Palladio layout control protocol [PDF 179KB]. Richard Golding and Liz Borowsky. HP Laboratories technical report HPL-SSP-99-1rev2 (August 1999).

An analytic behavior model for disk drives with readahead caches and request reordering [PostScript 432KB, compressed PostScript 147KB, PDF 283KB]. Elizabeth Shriver, Arif Merchant and John Wilkes. Proc. SIGMETRICS'98 (Madison, WI, June 1998).

Storage system function list John Wilkes and Doug Voigt. Revision 5, 17 December 1997. A list of the functions that might need to be distributed across a storage system architecture.

On-line extraction of SCSI disk drive parameters [PostScript 1.46MB, compressed PostScript 569KB, PDF 318KB]. Bruce L. Worthington, Gregory R. Ganger, Yale N. Patt and John Wilkes. HP Laboratories technical report HPL-97-02 (January 1997), and University of Michigan Technical report CSE-TR-323-96 (December 1996).
An earlier version was published as Proc. SIGMETRICS'95 (Ottawa, Canada, May 1995), pages 146-156.

AFRAID -- A Frequently Redundant Array of Independent Disks [PDF 111KB]. Stefan Savage and John Wilkes. Proc. 1996 USENIX Technical Conference (San Diego, CA, 22-26 January 1996), pages 27-39. (This paper won one of the two USENIX awards for "best paper with a student author".)

Frequently-redundant array of independent disks [PDF 1.2MB]. John Wilkes and Stefan Savage. US patent 5720025, granted 17 Feb. 1998, filed 18 Jan. 1996. The AFRAID patent.

Techniques for file system simulation. Chandramohan A. Thekkath, John Wilkes and Edward D. Lazowska. No longer available: published in Software--Practice and Experience 24(11):981-999, November 1994. (C) Copyright 1994 John Wiley and Sons Ltd.

An introduction to disk drive modelling [PostScript 494KB, compressed PostScript 210KB, PDF 134KB]. Chris Ruemmler and John Wilkes. Published in IEEE Computer 27(3):17-28, March 1994. (C) Copyright 1994 IEEE. This paper supercedes "Modelling disks", HP Laboratories technical report HPL-93-68rev1, by the same authors.
David Kotz, Song Bac and Sriram Radhakrishnan at Dartmouth University developed (and make available) a simulation model of an HP97560 disk drive based on our description in this paper.

The TickerTAIP parallel RAID architecture [PDF 2.04MB]. Pei Cao, Swee Boon Lim, Shivakumar Venkataraman, John Wilkes. Published in ACM Transactions on Computer Systems 12(3):236-267, August 1994.

Mime: a high performance parallel storage device with strong recovery guarantees [PostScript 179KB, compressed PostScript 78KB, PDF 78KB]. Chia Chao, Robert English, David Jacobson, Alexander Stepanov and John Wilkes. HP Laboratories technical report HPL-CSP-92-9rev1 (March 1992, revised November 1992).

Loge: a self-organizing disk controller [PostScript 250KB, compressed PostScript 125KB, PDF 92KB]. Robert M. English and Alexander A. Stepanov. Published in Proceedings of USENIX Winter 1992 Technical Conference, January 20-24, 1992 (San Francisco, CA), pp 237-251.

UNIX disk access patterns [PostScript 1121KB, compressed PostScript 408KB, PDF 319KB]. Chris Ruemmler and John Wilkes. Published in Proceedings of the Winter'93 USENIX Conference pages 405-420 (January 1993), and also as HP Laboratories Technical Report HPL-92-152 (December 1992).

• Tabular versions of the graphs in this paper are available as a compressed tar file [659KB].
• Another analysis of the same data can be found in: A trace-driven analysis of disk working set sizes [PostScript 469KB, compressed PostScript 176KB, PDF 154KB]. Chris Ruemmler and John Wilkes. HP Laboratories Technical Report HPL-OSR-93-23 (April 1993).
• The I/O traces themselves are also available to qualified researchers.

Disk shuffling [PostScript 1.04MB, compressed PostScript 380KB, PDF 269KB]. Chris Ruemmler and John Wilkes. HP Laboratories Technical Report HPL-91-156 (October 1991).

Disk scheduling algorithms based on rotational position [PostScript 814KB, compressed PostScript 394KB, PDF 516KB]. David M. Jacobson and John Wilkes. HP Laboratories technical report HPL-CSP-91-7rev1 (February 1991, revised March 1991).

Improving the efficiency of UNIX file buffer caches [PostScript 835KB, compressed PostScript 328KB, PDF 342KB]. Andrew Braunstein, Mark Riley, and John Wilkes. This paper originally appeared in: Proceedings of the 12th ACM Symposium on Operating Systems Principles (SOSP), The Wigwam, Litchfield Park, Arizona, and was published as Operating Systems Review 23(5):71-82, December 1989.

Ivy: a study on replicating data for performance improvement [PDF 1.43MB]. Sai-Li Lo. HP Laboratories technical report HPL-CSP-90-48 (December 1990).

Specifying data availability in multi-device file systems [PostScript 146KB, compressed PostScript 60KB, PDF 34KB]. John Wilkes and Raymie Stata. This paper was a position paper for the 4th ACM-SIGOPS European Workshop, Bologna, Italy (3-5 September 1990), and was published as Operating Systems Review 25(1):56-59, January 1991.

HP AutoRAID

The HP AutoRAID hierarchical storage system [PDF 343KB]. John Wilkes, Richard Golding, Carl Staelin, and Tim Sullivan. Appears in ACM Transactions on Computer Systems 14 (1):108-136, February 1996. Copyright (C) 1996 Association for Computing Machinery.

• An earlier version [PDF 113KB] with the same title and authors can be found in the Proceedings of the 15th ACM Symposium on Operating System Principles, Copper Mountain Resort, Colorado, 3-6 December 1995. Published as Operating Systems Review 29(5):96-108. (C) Copyright 1995 Association for Computing Machinery.
• Some of the Pantheon simulation software used for this project is available under a non-disclosure agreement.

HP AutoRAID field performance [PDF 143kB]. Doug Voigt. HP World Presentation #3354, August 1988.

DataMesh

DataMesh Architecture 1.0 [PostScript 162KB, compressed PostScript 70KB, PDF 71KB]. Chia Chao, Robert English, David Jacobson, Bart Sears, Alexander Stepanov and John Wilkes. HP Laboratories technical report HPL-92-153 (December 1992).

DataMesh research project, phase 1. [PostScript 109KB, compressed PostScript 49KB, PDF 36KB]. John Wilkes, with Chia Chao, Robert English, David Jacobson, Sai-Lai Lo, Chris Ruemmler, Bart Sears, Alex Stepanov, and Rebecca Wright. Published in Proc. USENIX Workshop on File Systems (Ann Arbor, MI), pages 63-69, 21-22 May 1992.

The next two documents form a pair.

DataMeshTM --- scope and objectives [PDF 12KB]
John Wilkes. HP Laboratories technical report HPL-DSD-89-37rev1 (19 July 1989).
DataMesh --- scope and objectives: a commentary [PDF 84KB] John Wilkes. HP Laboratories technical report HPL-DSD-89-44 (19 July 1989).

Tuscany - economy-based, service oriented architectures

Utility functions, prices, and negotiation [PDF 0.2MB]. John Wilkes. HP Laboratories Technical report HPL-2008-81 (6 July 2008).

Keynote [PDF 2.3MB] and panel [PDF 3.2MB] presentations. John Wilkes. Invited talks at 3rd International Workshop on Self-Managing Database Systems (SMDB'08), 7 April 2008 (Cancun, Mexico).

BDIM keynote [PDF 2.4MB]. John Wilkes. Invited talk at 2nd IEEE/IFIP Intl. Workshop on Business-Driven IT Management (BDIM'07), 20 May 2007 (Munich, Germany).

What's IT management worth to the business? [PDF 5.0MB]. John Wilkes. Panel presentation at Integrated Network Management 2007 (IM 2007), May 2007 (Munich, Germany).

Utilification redux (invited keynote) [PDF 7.4MB] and panel presentation [PDF 220KB]. John Wilkes. Middleware 2006, November 2006 (Melbourne, Australia).

Storage, data, and information: a brief overview. John Wilkes, Beth Keer, Christopher Hoover, Alistair Veitch, and Pankaj Mehra. Invited talk at National ICT Australia, Melbourne University, December 2006.

Service contracts and aggregate utility functions [PDF 240KB]. Alvin AuYoung, Laura Grit, Janet Wiener, and John Wilkes. 15th IEEE International Symposium on High Performance Distributed Computing (HPDC-15), pp. 119-131, June 2006 (Paris, France).

Profitable services in an uncertain world [PDF 260KB]. Florentina I. Popovici and John Wilkes. Supercomputing'05 (SC|05) November 2005 (Seattle, WA).

Value-maximizing deadline scheduling and its application to animation rendering [PDF 209KB]. Eric Anderson, Dirk Beyer, Kamalika Chaudhuri, Terence Kelly, Norman Salazar, Cipriano Santos, Ram Swaminathan, Robert Endre Tarjan, Janet L. Wiener, Yunhong Zhou. Symposium on Parallel Architectures and Algorithms (SPAA'05) July 2005 (Las Vegas, NV).

An extended evaluation of two-phase scheduling methods for animation rendering [PDF 474KB]. Yunhong Zhou, Terence Kelly, Janet Wiener, and Eric Anderson. Job scheduling strategies for parallel processing (JSSPP'05) June, 2005 (Cambridge, MA).

Global data placement

Choosing replica placement heuristics for wide-area systems [PDF 204KB]. Magnus Karlsson and Christos Karamanolis. 24th International Conference on Distributed Computing Systems (ICDCS'04) (24-26 March 2004, Hachioji, Tokyo, Japan). This is the extended version of the ICDCS paper, featuring all proofs and algorithms.

Pangaea

Name space consistency in the Pangaea wide-area file system [PDF 330KB]. Yasushi Saito and Christos Karamanolis. HP Laboratories SSP technical report HPL-SSP-2002-12, December 2002.

Taming aggressive replication in the Pangaea wide-area file system [PDF 218KB]. Yasushi Saito, Christos Karamanolis, Magnus Karlsson, and Mallik Mahalingam. 5th Symposium on Operating System Design and Implementation (OSDI'02) (9-12 December 2002, Boston, MA, USA).

The Pangaea symbiotic wide-area file system [PDF 93KB]. Yasushi Saito and Christos Karamanolis. Proceedings of the 10th ACM-SIGOPS European Workshop, Saint-Emilion, France, September 2002.

File virtualization with DirectNFS [PDF 144KB]. Anupam Bhide, Anu Engineer, Anshuman Kanetkar, Aditya Kini, Christos Karamanolis, Dan Muntz, Zheng Zhang, Gary Thunquest. 10th NASA Goddard Conference on Mass Storage Systems and Technologies (15-18 April 2002, Adelphi, Maryland, USA).

Locating logical volumes in large-scale networks [PDF 67KB]. Mallik Mahalingam, Christos Karamanolis, Magnus Karlsson, Zhichen Xu. 10th NASA Goddard Conference on Mass Storage Systems and Technologies (15-18 April 2002, Adelphi, Maryland, USA).

Designing a robust namespace for distributed file services [PDF 105KB, compressed PostScript 218KB, PostScript 793KB]. Zheng Zhang, Christos Karamanolis. 20th IEEE Symposium on Reliable Distributed Systems (28-31 October 2001, New Orleans, LA, USA).

When local becomes global: an application study of data consistency in a networked world [PDF 71KB, PostScript 825KB, talk slides 1.2MB]. Erik Riedel, Susan Spence, Alistair Veitch. Presented at the 20th IEEE International Performance, Computing, and Communications Conference (IPCCC) (4-6 April 2001, Phoenix, AZ).

Towards global storage management and data placement [PDF 63KB, PostScript 540KB]. Alistair Veitch, Erik Riedel, Simon Towers, John Wilkes. HP Laboratories SSP technical report (March 2001).
A one page version also appears as a position summary in the proceedings of the 8th Workshop on Hot Topics in Operating Systems (HotOS-VIII, 20-23 May 2001, Schloss Elmau, Germany), (C) Copyright 2001 IEEE.

Security

DDoS-resilient scheduling to counter application layer attacks under imperfect detection [PDF 276KB]. Supranamaya Ranjan, Ram Swaminathan, Mustafa Uysal, Edward Knightly. Proc. IEEE Infocom 2006, Barcelona, Spain, April 2006.

Plutus: Scalable secure file sharing on untrusted storage [PDF 117KB, PostScript 658KB, talk slides 181KB]. Mahesh Kallahalla, Erik Riedel, Ram Swaminathan, Qian Wang, and Kevin Fu. Conference on File and Storage Technology (FAST'03) pp. 29-42 (31 Mar - 2 Apr 2003, San Francisco, CA). Published by USENIX, Berkeley, CA.

A framework for evaluating storage system security [PDF 256KB, PostScript 980KB, talk slides 590KB]. Erik Riedel, Mahesh Kallahalla, and Ram Swaminathan. Conference on File and Storage Technology (FAST'02) pp. 15-30 (28-30 January 2002, Monterey, CA). Published by USENIX, Berkeley, CA.

Paranoia vs. performance - a quantitative evaluation of storage system security [PDF 465KB, PostScript 2.1MB]. Erik Riedel, Mahesh Kallahalla, Ram Swaminathan. HP Laboratories SSP technical report HPL-SSP-2001-06, February 2001.

Software and traces

Pantheon is the the event/trace-driven simulation system that we developed for the HP AutoRAID project. It is available to qualified researchers, and described in the following technical reports:

The Pantheon storage-system simulator [PDF 107KB, PostScript 266KB, compressed PostScript 102KB]. John Wilkes. HP Laboratories technical report HPL-SSP-95-14 (revision 1, May 1996). Oculus: a visual user interface for the Pantheon storage system simulator [PDF 334KB]. Uwe Aicheler. HP Laboratories technical report HPL-SSP-96-1 (January 1996).

The HP-UX I/O traces that we gathered - and use in much of our work - are also available to qualified researchers.

Hamlyn: sender-directed communication

Method and apparatus for determining when all packets of a message have arrived [PDF 0.95MB]. David M. Jacobson. US patent number 5,587,997. Granted 24th Dec. 1996. This is the Hamlyn packet-counting patent.

An implementation of the Hamlyn sender-managed interface architecture [PDF 230KB, PostScript 638KB, compressed PostScript 263KB]. Greg Buzzard, David Jacobson, Milon Mackey, Scott Marovich and John Wilkes. Published in Proceedings of the 2nd Symposium on Operating Systems Design and Implementation (OSDI'96), Seattle, Washington, 28-31 October 1996, pages 245-259.
Plus: Slides for the Hamlyn presentation at OSDI [PostScript 578K, compressed PostScript 227K, PDF 5797K].

Inter-processor communication system in which messages are stored at locations specified by the sender [PDF 24KB] A. J. Wilkes. US patent number 5,448,698. Granted 5th Sept. 1995. This is the first Hamlyn patent.

Hamlyn: a high-performance network interface with sender-based memory management [PostScript 187KB, compressed PostScript 82KB, PDF 78KB]. Greg Buzzard, David Jacobson, Scott Marovich and John Wilkes. Presented at the Hot Interconnects III Symposium (Stanford University, Palo Alto, CA), 10-12 August 1995.

Hamlyn--an interface for sender-based communications [PDF 103KB, compressed PostScript 106KB, PostScript 247KB]. John Wilkes. HP Laboratories technical report HPL-OSR-92-13 (November 1992). This is the original Hamlyn paper.

Related links

• Myricom, the vendor of the switch used for the Hamlyn prototype;
• UC Berkeley's NOW (Network Of Workstations) project, where Thorsten von Eicken developed Active Messages (a new version is now being developed at UCB);
• Cornell's U-Net and CMU's CreditNet, ATM interfaces with some similarities to Hamlyn.

Idleness and its exploitation

Idleness is not sloth [PostScript 153KB, compressed PostScript 69KB, PDF 62KB]. Richard Golding, Peter Bosch, Carl Staelin, Tim Sullivan, and John Wilkes. Published in Proceedings of the Winter'95 USENIX Conference, New Orleans, LA, 16-19 January 1995, pp 201-22. (C) Copyright 1995 Usenix Association.
Plus: Slides for the presentation given at USENIX [PostScript 676KB, compressed PostScript 174KB, PDF 124KB].

Predictive power conservation [PostScript 27KB, compressed PostScript 59KB, PDF 13KB]. John Wilkes. HP Laboratories technical report HPL-CSP-92-5 (14th Febuary 1992). 1 page.

Digital preservation

A fresh look at the reliability of long-term digital storage. M. Baker, M. Shah, D. Rosenthal, M. Roussopoulos, P. Maniatis, TJ Giuli, P. Bungale EuroSys 2006, Leuven, Belgium, April 2006.

Why traditional storage systems don't help us save stuff forever. M. Baker, K. Keeton, and S. Martin. 1st IEEE Workshop on Hot Topics in System Dependability, Japan, June 2005.

Attrition defenses for a peer-to-peer digital preservation system. TJ Giuli, Petros Maniatis, Mary Baker, David S. H. Rosenthal, and Mema Roussopoulos. Proceedings of the Annual USENIX Technical Conference, April 2005.

The LOCKSS peer-to-peer digital preservation system. Petros Maniatis, Mema Roussopoulos, TJ Giuli, David S. H. Rosenthal, and Mary Baker. ACM Transactions on Computer Systems (TOCS), Volume 23, Issue 1, February 2005.

Impeding attrition attacks in P2P systems. Petros Maniatis, TJ Giuli, Mema Roussopoulos, David S. H. Rosenthal, and Mary Baker. Proceedings of the 11th ACM SIGOPS European Workshop, September 2004.

Using hard disks for digital preservation. David S. H. Rosenthal, Mema Roussopoulos, TJ Giuli, Petros Maniatis, and Mary Baker. Proceedings of the IS&T Archiving Conference, April 2004.

Miscellaneous papers

SoftUDC: A software-based data center for utility computing [PDF 371KB]. Mahesh Kallahalla, Mustafa Uysal, Ram Swaminathan, David E. Lowell, Mike Wray, Tom Christian, Nigel Edwards, Chris I. Dalton, Frederic Gittler. IEEE Computer, Volume 37, Number 11, pages 38-46, November 2004.

Utilification [PDF 193KB]. John Wilkes, Jeffrey Mogul, and Jaap Suermondt. Proceedings of the 11th ACM-SIGOPS European Workshop, Leuven, Belgium, 22-24 September 2004.

The semiotics of umbrellas [PDF 163KB]. John Wilkes.
HP Laboratories, Palo Alto, CA, February 2003.
Written for the collection Computer systems: papers for Roger Needham to mark 50 years in Cambridge and 5 years at Microsoft Research, edited by Andrew Herbert and Karen Spärck Jones, Cambridge, February 2003, Microsoft, pp. 257-260.

eOS - the dawn of the resource economy [PDF 86KB, PostScript 408KB]. John Wilkes, Patrick Goldsack, G. (John) Janakiraman, Lance Russell, Sharad Singhal, and Andrew Thomas. HP Laboratories technical report (May 2001).
A one page version [PDF 97KB, PostScript 96KB] also appears as a position summary in the proceedings of the 8th Workshop on Hot Topics in Operating Systems (HotOS-VIII, 20-23 May 2001, Schloss Elmau, Germany), (C) Copyright 2001 IEEE.

Life after grad school - heaven or hell? [PDF 1.2MB]. Erik Riedel. Presented at Carnegie Mellon University (November 2001). Slides from the SCS "emigration course" for graduating students. Compares industrial research against startup jobs for PhD grads.

Is work hell? Life in industrial research [PostScript 794KB, compressed PostScript 222KB, PDF 247KB]. John Wilkes. HP Laboratories technical report HPL-SSP-97-1 (January 1997). Slides and reader notes for the CMU "emigration course", designed to help graduating students look for a job.

A longitudinal survey of Internet host reliability [PostScript 217KB, compressed PostScript 84KB, PDF 57KB]. Darrell Long, Andrew Muir, and Richard Golding. HP Laboratories technical report HPL-CCD-95-4 and Information Sciences Board of the University of California at Santa Cruz technical report UCSC-CRL-95-16 (February 1995).

"Tcl cures 98.3% of all known simulation configuration problems" claims astonished researcher! [PostScript 84KB, compressed PostScript 40KB, PDF 26KB]. Richard Golding, Carl Staelin, Tim Sullivan, John Wilkes. Presented at the Tcl Workshop, New Orleans, May 1994. Describes the system we use in our Pantheon simulator to link Tcl and C++ code. This allows us to construct a wide variety of simulations from a kit of parts at runtime using an interpreted language, and yet still have the simulation execution run at full speed.
Plus: Source code [73KB] for the scheme described in this paper.

The refdbms distributed bibliographic database system [PostScript 122KB, compressed PostScript 56KB, PDF 128KB]. Richard Golding, Darrell Long and John Wilkes. Published in Proceedings of Winter'94 USENIX technical conference, pages 47-62 (January 1994). (C) Copyright 1994 Usenix Association. [We regret that this file does not print on A4 paper.]

Brevix design 1.01 [PDF 315KB]. Martin Fouts, Tim Connors, Steve Hoyle, Bart Sears, Tim Sullivan, and John Wilkes. HP Laboratories technical report HPL-OSR-93-22 (1 April 1993). 80 pages.

A comparison of Protection Lookaside Buffers and the PA-RISC protection architecture [PostScript 106KB, compressed PostScript 48KB, PDF 45KB]. John Wilkes and Bart Sears. HP Laboratories technical report HPL-92-55 (March 1992).

CSP project startup documents [PostScript 59KB, compressed PostScript 27KB, PDF 12KB]. John Wilkes. HP Laboratories technical report HPL-CSP-90-42 (October 1990). 1 page. A set of guidelines for helping to crystallize the purpose, goals and rationale of a new project or research activity.