Learning PySpark by Tomasz Drabas

Learning

PySpark

Tomasz

需积分: 15 35 下载量 83 浏览量 2017-05-19 16:17:39 上传评论 1 收藏 11.5MB PDF 举报

温馨提示

试读

380页

Learning PySpark by Tomasz Drabas

资源推荐

资源详情

资源评论

ProjektPK4:Projekt na przedmiot PK4，napisany przez Mateusz Lipina i Tomasz Pelan

【标题】"ProjektPK4:Projekt na przedmiot PK4，napisany przez Mateusz Lipina i Tomasz Pelan" 涉及的是一个由Mateusz Lipina和Tomasz Pelan合作完成的PK4课程项目。从标题我们可以推测，这可能是一个计算机科学...

wierzbickitomasz.github.io

这个网站可能包含了作者Tomasz Wierzbicki的个人信息、作品展示或者某个特定项目的详细信息。由于标签是"CSS"，我们可以推测这个网站的构建和设计很大程度上依赖于层叠样式表（CSS）技术，这是网页设计中用于描述...

tomaszbednarz.github.io

tomaszbednarz.github.io

kgb档案压缩console版+源码

build-an-app-with-the-AWS-cloud-development-kit-notes:社区笔记，旨在通过AWS Cloud Development Kit课程补充Tomasz的“构建应用程序”

关于 :magnifying_glass_tilted_left: 此包含的Egghead课程的注释，该课程。这些笔记包含与抄录相同的结构，以及额外的重写，与资源的链接以及个人上课的内容。随时提交这些注释的补充，但请不要删除任何内容...

CppCon2017：CppCon 2017的幻灯片和其他材料

Mathieu Ropert的Boris Kolpackov [] Tomasz Kapela的由Fedor Pikus Chuck Allison撰写的《的 by Kenny Kerr Bryce Adelstein Lelbach的约翰·麦克法兰（John McFarlane）的[ ] 作者：Anthony Williams

River Simulations and Python:使用 Python 管理河流模拟-开源

该论文由 Tomasz Dysarz 编写。软件名称：HEC-RAS 和 Python 开发人员：Tomasz Dysarz 邮件地址：波兹南生命科学大学水利与卫生工程系，ul。 Piatkowska 94A, 60-649 Poznan, POLAND 电子邮件地址：tdysarz@gmail....

[WP] Windows Phone 8 应用开发核心技术 (英文版)

5星 · 资源好评率100%

☆ 资源说明：☆ [Packt Publishing] Windows Phone 8 应用开发...[作者信息] Tomasz Szostak [出版机构] Packt Publishing [出版日期] 2013年10月17日 [图书页数] 118页 [图书语言] 英语 [图书格式] PDF 格式

SUMS30 Mathematics for Finance -- An Introduction to Financial Engineering

SUMS30 Mathematics for Finance -- An Introduction to Financial Engineering, Marek Capiński, Tomasz Zastawniak (2003).pdf

Learning PySpark

Table of Contents
Learning PySpark
Credits
Foreword
About the Authors
About the Reviewer
www.PacktPub.com
Customer Feedback
Preface
What this book covers
What you need for this book
Who this book is for
Conventions
Reader feedback
Customer support
Downloading the example code
Downloading the color images of this book
Errata
Piracy
Questions
1. Understanding Spark
What is Apache Spark?
Spark Jobs and APIs
Execution process
Resilient Distributed Dataset
DataFrames
Datasets
Catalyst Optimizer
Project Tungsten
Spark 2.0 architecture
Unifying Datasets and DataFrames
Introducing SparkSession
Tungsten phase 2

Structured Streaming
Continuous applications
Summary
2. Resilient Distributed Datasets
Internal workings of an RDD
Creating RDDs
Schema
Reading from files
Lambda expressions
Global versus local scope
Transformations
The .map(...) transformation
The .filter(...) transformation
The .flatMap(...) transformation
The .distinct(...) transformation
The .sample(...) transformation
The .leftOuterJoin(...) transformation
The .repartition(...) transformation
Actions
The .take(...) method
The .collect(...) method
The .reduce(...) method
The .count(...) method
The .saveAsTextFile(...) method
The .foreach(...) method
Summary
3. DataFrames
Python to RDD communications
Catalyst Optimizer refresh
Speeding up PySpark with DataFrames
Creating DataFrames
Generating our own JSON data
Creating a DataFrame
Creating a temporary table
Simple DataFrame queries
DataFrame API query

SQL query
Interoperating with RDDs
Inferring the schema using reflection
Programmatically specifying the schema
Querying with the DataFrame API
Number of rows
Running filter statements
Querying with SQL
Number of rows
Running filter statements using the where Clauses
DataFrame scenario – on-time flight performance
Preparing the source datasets
Joining flight performance and airports
Visualizing our flight-performance data
Spark Dataset API
Summary
4. Prepare Data for Modeling
Checking for duplicates, missing observations, and outliers
Duplicates
Missing observations
Outliers
Getting familiar with your data
Descriptive statistics
Correlations
Visualization
Histograms
Interactions between features
Summary
5. Introducing MLlib
Overview of the package
Loading and transforming the data
Getting to know your data
Descriptive statistics
Correlations
Statistical testing
Creating the final dataset