## sparktraining
Examples for Spark Training in aura.cn
## 本地运行Spark方法
- 下载spark安装包
- 解压spark安装包
- 进入spark解压目录下,运行:
```bash
$ bin/spark-shell
```
- 在命令行提示符下拷贝以下代码并查看执行结果
```scala
import scala.math.random
val tasks = 10
val n = tasks * 100000
val count = sc.parallelize(1 until n, tasks).map { i =>
val x = random * 2 - 1
val y = random * 2 - 1
if (x*x + y*y <= 1) 1 else 0
}.reduce(_ + _)
println("Pi is roughly " + 4.0 * count / n )
```
## 分布式运行Spark方法
### 搭建hadoop集群
Hadoop YARN/HDFS配置文件参考:conf/hadoop目录
### 配置Spark客户端,并启动spark history server
- Spark客户端配置文件参考:conf/spark目录
- 启动spark history server: sbin/start-history-server.sh
### 将spark-shell运行在yarn client或cluster模式
- yarn client模式:bin/spark-shell --master yarn --deploy-mode client
- yarn cluster:bin/spark-shell --master yarn --deploy-mode cluster
没有合适的资源?快使用搜索试试~ 我知道了~
光环大数据培训spark体系学习文档
共242个文件
csv:99个
scala:47个
pdf:12个
1星 需积分: 10 57 下载量 13 浏览量
2019-01-27
21:44:31
上传
评论
收藏 62.41MB ZIP 举报
温馨提示
光环大数据培训spark体系课程文档,花钱上课的课件文档!
资源推荐
资源详情
资源评论
收起资源包目录
光环大数据培训spark体系学习文档 (242个子文件)
AuraSparkTraining 180B
basketball 142B
bin 179B
spark-defaults.conf 301B
spark-defaults.conf 141B
conf 141B
core 179B
core 179B
Emails.csv 24.43MB
movie_metadata.csv 1.43MB
EmailReceivers.csv 117KB
leagues_NBA_2015_per_game_per_game.csv 66KB
leagues_NBA_2014_per_game_per_game.csv 64KB
leagues_NBA_2012_per_game_per_game.csv 64KB
leagues_NBA_2013_per_game_per_game.csv 63KB
leagues_NBA_2005_per_game_per_game.csv 62KB
leagues_NBA_2016_per_game_per_game.csv 61KB
leagues_NBA_2007_per_game_per_game.csv 61KB
leagues_NBA_2006_per_game_per_game.csv 61KB
leagues_NBA_2011_per_game_per_game.csv 60KB
leagues_NBA_2008_per_game_per_game.csv 60KB
leagues_NBA_2009_per_game_per_game.csv 59KB
leagues_NBA_2010_per_game_per_game.csv 59KB
leagues_NBA_2004_per_game_per_game.csv 59KB
leagues_NBA_1997_per_game_per_game.csv 59KB
leagues_NBA_2001_per_game_per_game.csv 59KB
leagues_NBA_2002_per_game_per_game.csv 59KB
leagues_NBA_1998_per_game_per_game.csv 58KB
leagues_NBA_2000_per_game_per_game.csv 58KB
leagues_NBA_1999_per_game_per_game.csv 58KB
leagues_NBA_1996_per_game_per_game.csv 57KB
leagues_NBA_2003_per_game_per_game.csv 57KB
leagues_NBA_1995_per_game_per_game.csv 54KB
leagues_NBA_1994_per_game_per_game.csv 54KB
leagues_NBA_1993_per_game_per_game.csv 52KB
leagues_NBA_1991_per_game_per_game.csv 52KB
leagues_NBA_1992_per_game_per_game.csv 51KB
leagues_NBA_1990_per_game_per_game.csv 51KB
leagues_NBA_1989_per_game_per_game.csv 47KB
leagues_NBA_1987_per_game_per_game.csv 45KB
leagues_NBA_1988_per_game_per_game.csv 44KB
leagues_NBA_1986_per_game_per_game.csv 43KB
leagues_NBA_1985_per_game_per_game.csv 43KB
leagues_NBA_1983_per_game_per_game.csv 42KB
leagues_NBA_1982_per_game_per_game.csv 42KB
leagues_NBA_1984_per_game_per_game.csv 41KB
leagues_NBA_1981_per_game_per_game.csv 40KB
leagues_NBA_1980_per_game_per_game.csv 38KB
leagues_NBA_1977_per_game_per_game.csv 35KB
leagues_NBA_1978_per_game_per_game.csv 35KB
leagues_NBA_1979_per_game_per_game.csv 34KB
leagues_NBA_1976_per_game_per_game.csv 28KB
leagues_NBA_1975_per_game_per_game.csv 28KB
leagues_NBA_1974_per_game_per_game.csv 27KB
leagues_NBA_1971_per_game_per_game.csv 23KB
leagues_NBA_1972_per_game_per_game.csv 23KB
leagues_NBA_1973_per_game_per_game.csv 23KB
Aliases.csv 20KB
leagues_NBA_1970_per_game_per_game.csv 18KB
Persons.csv 10KB
leagues_NBA_1990_per_game_per_game.csv 141B
leagues_NBA_2008_per_game_per_game.csv 141B
leagues_NBA_1974_per_game_per_game.csv 141B
leagues_NBA_1980_per_game_per_game.csv 141B
leagues_NBA_1975_per_game_per_game.csv 141B
leagues_NBA_1977_per_game_per_game.csv 141B
leagues_NBA_1995_per_game_per_game.csv 141B
leagues_NBA_2009_per_game_per_game.csv 141B
leagues_NBA_2011_per_game_per_game.csv 141B
leagues_NBA_2003_per_game_per_game.csv 141B
leagues_NBA_1991_per_game_per_game.csv 141B
leagues_NBA_2014_per_game_per_game.csv 141B
leagues_NBA_2010_per_game_per_game.csv 141B
leagues_NBA_2004_per_game_per_game.csv 141B
leagues_NBA_2015_per_game_per_game.csv 141B
leagues_NBA_2016_per_game_per_game.csv 141B
leagues_NBA_1984_per_game_per_game.csv 141B
leagues_NBA_1972_per_game_per_game.csv 141B
leagues_NBA_1981_per_game_per_game.csv 141B
leagues_NBA_1973_per_game_per_game.csv 141B
leagues_NBA_2001_per_game_per_game.csv 141B
leagues_NBA_1979_per_game_per_game.csv 141B
leagues_NBA_1989_per_game_per_game.csv 141B
leagues_NBA_1986_per_game_per_game.csv 141B
leagues_NBA_1982_per_game_per_game.csv 141B
leagues_NBA_1988_per_game_per_game.csv 141B
leagues_NBA_1970_per_game_per_game.csv 141B
leagues_NBA_1987_per_game_per_game.csv 141B
leagues_NBA_1976_per_game_per_game.csv 141B
leagues_NBA_1983_per_game_per_game.csv 141B
leagues_NBA_1971_per_game_per_game.csv 141B
leagues_NBA_2012_per_game_per_game.csv 141B
leagues_NBA_1992_per_game_per_game.csv 141B
leagues_NBA_1985_per_game_per_game.csv 141B
leagues_NBA_1996_per_game_per_game.csv 141B
leagues_NBA_1998_per_game_per_game.csv 141B
leagues_NBA_2005_per_game_per_game.csv 141B
leagues_NBA_2002_per_game_per_game.csv 141B
leagues_NBA_2013_per_game_per_game.csv 141B
leagues_NBA_2007_per_game_per_game.csv 141B
共 242 条
- 1
- 2
- 3
资源评论
- weixin_416690732021-02-14下载下来是假的
tetty
- 粉丝: 2
- 资源: 9
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功