Big Data Glossary

Big Data Glossary A Guide to the New Generation of Data Tools By Pete Warden Publisher: O'Reilly Media Released: September 2011 Pages: 60 Print ISBN: 978-1-4493-1459-0 ISBN 10: 1-4493-1459-7 Ebook ISBN: 978-1-4493-1458-3 ISBN 10: 1-4493-1458-9 To help you navigate the large number of new data tools available, this guide describes 60 of the most recent innovations, from NoSQL databases and MapReduce approaches to machine learning and visualization tools. Descriptions are based on first-hand experience with these tools in a production environment. This handy glossary also includes a chapter of key terms that help define many of these tool categories: NoSQL Databases—Document-oriented databases using a key/value interface rather than SQL MapReduce—Tools that support distributed computing on large datasets Storage—Technologies for storing data in a distributed way Servers—Ways to rent computing power on remote machines Processing—Tools for extracting valuable information from large datasets Natural Language Processing—Methods for extracting information from human-created text Machine Learning—Tools that automatically perform data analyses, based on results of a one-off analysis Visualization—Applications that present meaningful data graphically Acquisition—Techniques for cleaning up messy public data sources Serialization—Methods to convert data structure or object state into a storable format TABLE OF CONTENTS Chapter 1 Terms Document-Oriented Key/Value Stores Horizontal or Vertical Scaling MapReduce Sharding Chapter 2 NoSQL Databases MongoDB CouchDB Cassandra Redis BigTable HBase Hypertable Voldemort Riak ZooKeeper Chapter 3 MapReduce Hadoop Hive Pig Cascading Cascalog mrjob Caffeine S4 MapR Acunu Flume Kafka Azkaban Oozie Greenplum Chapter 4 Storage S3 Hadoop Distributed File System Chapter 5 Servers EC2 Google App Engine Elastic Beanstalk Heroku Chapter 6 Processing R Yahoo! Pipes Mechanical Turk Solr/Lucene ElasticSearch Datameer BigSheets Tinkerpop Chapter 7 NLP Natural Language Toolkit OpenNLP Boilerpipe OpenCalais Chapter 8 Machine Learning WEKA Mahout scikits.learn Chapter 9 Visualization Gephi GraphViz Processing Protovis Fusion Tables Tableau Chapter 10 Acquisition Google Refine Needlebase ScraperWiki Chapter 11 Serialization JSON BSON Thrift Avro Protocol Buffers