Prefix Hash Tree
An Indexing Data Structure over Distributed Hash Tables
Sriram Ramabhadran
∗
University of California, San Diego
Sylvia Ratnasamy
Intel Research, Berkeley
Joseph M. Hellerstein
University of California, Berkeley
and
Intel Research, Berkeley
Scott Shenker
International Comp. Science Institute, Berkeley
and
University of California, Berkeley
ABSTRACT
Distributed Hash Tables are scalable, robust, and
self-organizing peer-to-peer systems that support
exact match lookups. This paper describes the de-
sign and implementation of a Prefix Hash Tree -
a distributed data structure that enables more so-
phisticated queries over a DHT. The Prefix Hash
Tree uses the lookup interface of a DHT to con-
struct a trie-based structure that is both efficient
(updates are doubly logarithmic in the size of the
domain being indexed), and resilient (the failure
of any given node in the Prefix Hash Tree does
not affect the availability of data stored at other
nodes).
Categories and Subject Descriptors
C.2.4 [Comp. Communication Networks]: Dis-
tributed Systems—distributed applications; E.1 [Data
Structures]: distributed data structures; H.3.1 [Info-
rmation Storage and Retrieval]: Content Anal-
ysis and Indexing—indexing methods
General Terms
Algorithms, Design, Performance
Keywords
distributed hash tables, data structures, range queries
1. INTRODUCTION
The explosive growth but primitive design of peer-
to-peer file-sharing applications such as Gnutella
[7] and KaZaa [29] inspired the research community
to invent Distributed Hash Tables (DHTs) [31, 24,
14, 26, 22, 23]. Using a structured overlay network,
DHTs map a given key to the node in the network
holding the object associated with that key; this
lookup operation lookup(key) can be used to sup-
port the canonical put(key,value) and get(key)
hash table operations. The broad applicability of
∗
email sriram@cs.ucsd.edu
this lookup interface has allowed a wide variety of
system to be built on top DHTs, including file sys-
tems [9, 27], indirection services [30], event notifi-
cation [6], content distribution networks [10] and
many others.
DHTs were designed in the Internet style: scala-
bility and ease of deployment triumph over strict
semantics. In particular, DHTs are self-organizing,
requiring no centralized authority or manual con-
figuration. They are robust against node failures
and easily accommodate new nodes. Most impor-
tantly, they are scalable in the sense that both la-
tency (in terms of the number of hops per lookup)
and the local state required typically grow loga-
rithmically in the number of nodes; this is crucial
since many of the envisioned scenarios for DHTs
involve extremely large systems (such as P2P mu-
sic file sharing). However, DHTs, like the Internet,
deliver ”best-effort” semantics; put’s and get’s are
likely to succeed, but the system provides no guar-
antees. As observed by others [36, 5], this conflict
between scalability and strict semantics appears
to be inevitable and, for many large-scale Inter-
net systems, the former is deemed more important
than the latter.
While DHTs have enjoyed some success as a build-
ing block for Internet-scale applications, they are
seriously deficient in one regard: they only directly
support exact match queries. Keyword queries can
be derived from these exact match queries in a
straightforward but inefficient manner; see [25, 20]
for applications of this to DHTs. Equality joins
can also be supported within a DHT framework;
see [15]. However, range queries, asking for all ob-
jects with values in a certain range, are particularly
difficult to implement in DHTs. This is because
DHTs use hashing to distribute keys uniformly and
so can’t rely on any structural properties of the key
space, such as an ordering among keys.
Range queries arise quite naturally in a number of