The Google File System
Sanjay Ghemawat, Howard Gobioff, and Shun-Tak Leung
Google
∗
ABSTRACT
We have designed and implemented the Google File Sys-
tem, a scalable distributed file system for large distributed
data-intensive applications. It provides fault tolerance while
running on inexpensive commodity hardware, and it delivers
high aggregate performance to a large number of clients.
While sharing many of the same goals as previous dis-
tributed file systems, our design has been driven by obser-
vations of our application workloads and technological envi-
ronment, both current and anticipated, that reflect a marked
departure from some earlier file system assumptions. This
has led us to reexamine traditional choices and explore rad-
ically different design points.
The file system has successfully met our storage needs.
It is widely deployed within Google as the storage platform
for the generation and processing of data used by our ser-
vice as well as research and development efforts that require
large data sets. The largest cluster to date provides hun-
dreds of terabytes of storage across thousands of disks on
over a thousand machines, and it is concurrently accessed
by hundreds of clients.
In this paper, we present file system interface extensions
designed to support distributed applications, discuss many
aspects of our design, and report measurements from both
micro-benchmarks and real world use.
Categories and Subject Descriptors
D[4]: 3—Distributed file systems
General Terms
Design, reliability, performance, measurement
Keywords
Fault tolerance, scalability, data storage, clustered storage
∗
The authors can be reached at the following addresses:
{sanjay,hgobioff,shuntak}@google.com.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
SOSP’03, October 19–22, 2003, Bolton Landing, New York, USA.
Copyright 2003 ACM 1-58113-757-5/03/0010 ...
$5.00.
1. INTRODUCTION
We have designed and implemented the Google File Sys-
tem (GFS) to meet the rapidly growing demands of Google’s
data processing needs. GFS shares many of the same goals
as previous distributed file systems such as performance,
scalability, reliability, and availability. However, its design
has been driven by key observations of our application work-
loads and technological environment, both current and an-
ticipated, that reflect a marked departure from some earlier
file system design assumptions. We have reexamined tradi-
tional choices and explored radically different points in the
design space.
First, component failures are the norm rather than the
exception. The file system consists of hundreds or even
thousands of storage machines built from inexpensive com-
modity parts and is accessed by a comparable number of
client machines. The quantity and quality of the compo-
nents virtually guarantee that some are not functional at
any given time and some will not recover from their cur-
rent failures. We have seen problems caused by application
bugs, operating system bugs, human errors, and the failures
of disks, memory, connectors, networking, and power sup-
plies. Therefore, constant monitoring, error detection, fault
tolerance, and automatic recovery must be integral to the
system.
Second, files are huge by traditional standards. Multi-GB
files are common. Each file typically contains many applica-
tion objects such as web documents. When we are regularly
working with fast growing data sets of many TBs comprising
billions of objects, it is unwieldy to manage billions of ap-
proximately KB-sized files even when the file system could
support it. As a result, design assumptions and parameters
such as I/O operation and block sizes have to be revisited.
Third, most files are mutated by appending new data
rather than overwriting existing data. Random writes within
a file are practically non-existent. Once written, the files
are only read, and often only sequentially. A variety of
data share these characteristics. Some may constitute large
repositories that data analysis programs scan through. Some
may be data streams continuously generated by running ap-
plications. Some may be archival data. Some may be in-
termediate results produced on one machine and processed
on another, whether simultaneously or later in time. Given
this access pattern on huge files, appending becomes the fo-
cus of performance optimization and atomicity guarantees,
while caching data blocks in the client loses its appeal.
Fourth, co-designing the applications and the file system
API benefits the overall system by increasing our flexibility.