Preface
Welcome to this first edition of Spark: The Definitive Guide! We are excited to bring you the
most complete resource on Apache Spark today, focusing especially on the new generation of
Spark APIs introduced in Spark 2.0.
Apache Spark is currently one of the most popular systems for large-scale data processing, with
APIs in multiple programming languages and a wealth of built-in and third-party libraries.
Although the project has existed for multiple years—first as a research project started at UC
Berkeley in 2009, then at the Apache Software Foundation since 2013—the open source
community is continuing to build more powerful APIs and high-level libraries over Spark, so
there is still a lot to write about the project. We decided to write this book for two reasons. First,
we wanted to present the most comprehensive book on Apache Spark, covering all of the
fundamental use cases with easy-to-run examples. Second, we especially wanted to explore the
higher-level “structured” APIs that were finalized in Apache Spark 2.0—namely DataFrames,
Datasets, Spark SQL, and Structured Streaming—which older books on Spark don’t always
include. We hope this book gives you a solid foundation to write modern Apache Spark
applications using all the available tools in the project.
In this preface, we’ll tell you a little bit about our background, and explain who this book is for
and how we have organized the material. We also want to thank the numerous people who
helped edit and review this book, without whom it would not have been possible.
About the Authors
Both of the book’s authors have been involved in Apache Spark for a long time, so we are very
excited to be able to bring you this book.
Bill Chambers started using Spark in 2014 on several research projects. Currently, Bill is a
Product Manager at Databricks where he focuses on enabling users to write various types of
Apache Spark applications. Bill also regularly blogs about Spark and presents at conferences and
meetups on the topic. Bill holds a Master’s in Information Management and Systems from the
UC Berkeley School of Information.
Matei Zaharia started the Spark project in 2009, during his time as a PhD student at UC
Berkeley. Matei worked with other Berkeley researchers and external collaborators to design the
core Spark APIs and grow the Spark community, and has continued to be involved in new
initiatives such as the structured APIs and Structured Streaming. In 2013, Matei and other
members of the Berkeley Spark team co-founded Databricks to further grow the open source
project and provide commercial offerings around it. Today, Matei continues to work as Chief
Technologist at Databricks, and also holds a position as an Assistant Professor of Computer
Science at Stanford University, where he does research on large-scale systems and AI. Matei
received his PhD in Computer Science from UC Berkeley in 2013.
- 1
- 2
前往页