# Apache Spark
Spark is a fast and general cluster computing system for Big Data. It provides
high-level APIs in Scala, Java, Python, and R, and an optimized engine that
supports general computation graphs for data analysis. It also supports a
rich set of higher-level tools including Spark SQL for SQL and DataFrames,
MLlib for machine learning, GraphX for graph processing,
and Spark Streaming for stream processing.
<http://spark.apache.org/>
## Online Documentation
You can find the latest Spark documentation, including a programming
guide, on the [project web page](http://spark.apache.org/documentation.html)
## Python Packaging
This README file only contains basic information related to pip installed PySpark.
This packaging is currently experimental and may change in future versions (although we will do our best to keep compatibility).
Using PySpark requires the Spark JARs, and if you are building this from source please see the builder instructions at
["Building Spark"](http://spark.apache.org/docs/latest/building-spark.html).
The Python packaging for Spark is not intended to replace all of the other use cases. This Python packaged version of Spark is suitable for interacting with an existing cluster (be it Spark standalone, YARN, or Mesos) - but does not contain the tools required to set up your own standalone Spark cluster. You can download the full version of Spark from the [Apache Spark downloads page](http://spark.apache.org/downloads.html).
**NOTE:** If you are using this with a Spark standalone cluster you must ensure that the version (including minor version) matches or you may experience odd errors.
## Python Requirements
At its core PySpark depends on Py4J (currently version 0.10.7), but some additional sub-packages have their own extra requirements for some features (including numpy, pandas, and pyarrow).
没有合适的资源?快使用搜索试试~ 我知道了~
资源推荐
资源详情
资源评论
收起资源包目录
python基于spark开发插件库、用于离线安装,在线安装都行 (546个子文件)
beeline 1KB
setup.cfg 108B
find-spark-home.cmd 3KB
spark-class2.cmd 2KB
load-spark-env.cmd 2KB
spark-shell2.cmd 2KB
pyspark2.cmd 2KB
run-example.cmd 1KB
spark-class.cmd 1KB
spark-submit.cmd 1KB
spark-shell.cmd 1KB
spark-sql.cmd 1KB
pyspark.cmd 1KB
sparkR.cmd 1KB
spark-submit2.cmd 1KB
spark-sql2.cmd 1KB
sparkR2.cmd 1KB
beeline.cmd 1KB
lpsa.data 10KB
test.data 128B
find-spark-home 2KB
MANIFEST.in 1KB
scala-compiler-2.11.12.jar 14.89MB
breeze_2.11-0.13.2.jar 14.41MB
spark-core_2.11-2.4.3.jar 12.89MB
hive-exec-1.2.1.spark2.jar 10.97MB
spark-catalyst_2.11-2.4.3.jar 9.74MB
spire_2.11-0.13.0.jar 9.65MB
spark-sql_2.11-2.4.3.jar 9.43MB
kubernetes-model-4.1.2.jar 8.95MB
hadoop-hdfs-2.7.3.jar 7.93MB
spark-mllib_2.11-2.4.3.jar 7.65MB
mesos-1.4.0-shaded-protobuf.jar 7MB
scala-library-2.11.12.jar 5.48MB
hive-metastore-1.2.1.spark2.jar 5.25MB
scala-reflect-2.11.12.jar 4.41MB
netty-all-4.1.17.Final.jar 3.6MB
shapeless_2.11-2.3.2.jar 3.36MB
calcite-core-1.2.0-incubating.jar 3.36MB
hadoop-common-2.7.3.jar 3.32MB
derby-10.12.1.1.jar 3.08MB
parquet-hadoop-bundle-1.6.0.jar 2.67MB
spark-network-common_2.11-2.4.3.jar 2.28MB
zstd-jni-1.3.2-2.jar 2.23MB
guava-14.0.1.jar 2.09MB
spark-streaming_2.11-2.4.3.jar 2.07MB
hadoop-yarn-api-2.7.3.jar 1.94MB
commons-math3-3.4.1.jar 1.94MB
snappy-java-1.1.7.3.jar 1.93MB
datanucleus-core-3.2.10.jar 1.8MB
spark-hive-thriftserver_2.11-2.4.3.jar 1.73MB
datanucleus-rdbms-3.2.9.jar 1.73MB
hadoop-yarn-common-2.7.3.jar 1.6MB
hppc-0.7.2.jar 1.59MB
orc-core-1.5.5-nohive.jar 1.49MB
avro-1.8.2.jar 1.48MB
hadoop-mapreduce-client-core-2.7.3.jar 1.48MB
htrace-core-3.1.0-incubating.jar 1.41MB
spark-hive_2.11-2.4.3.jar 1.28MB
netty-3.9.9.Final.jar 1.27MB
arrow-vector-0.10.0.jar 1.26MB
ivy-2.4.0.jar 1.22MB
xercesImpl-2.9.1.jar 1.17MB
arpack_combined_all-0.1.jar 1.14MB
jackson-databind-2.6.7.1.jar 1.11MB
parquet-column-1.10.1.jar 1.05MB
parquet-jackson-1.10.1.jar 1024KB
leveldbjni-all-1.8.jar 1021KB
jersey-guava-2.22.2.jar 949KB
jersey-server-2.22.2.jar 929KB
parquet-encoding-1.10.1.jar 829KB
orc-mapreduce-1.5.5-nohive.jar 793KB
janino-3.0.9.jar 783KB
zookeeper-3.4.6.jar 774KB
jackson-mapper-asl-1.9.13.jar 762KB
hadoop-mapreduce-client-common-2.7.3.jar 758KB
httpclient-4.5.6.jar 749KB
jtransforms-2.4.0.jar 747KB
parquet-format-2.4.0.jar 706KB
javassist-3.18.1-GA.jar 697KB
guice-3.0.jar 694KB
spark-graphx_2.11-2.4.3.jar 692KB
jersey-common-2.22.2.jar 682KB
apacheds-kerberos-codec-2.0.0-M15.jar 675KB
json4s-core_2.11-3.5.3.jar 674KB
spark-mesos_2.11-2.4.3.jar 668KB
spark-yarn_2.11-2.4.3.jar 657KB
scala-xml_2.11-1.0.5.jar 655KB
joda-time-2.9.3.jar 613KB
json4s-scalap_2.11-3.5.3.jar 600KB
commons-collections-3.2.2.jar 575KB
kubernetes-client-4.1.2.jar 531KB
hadoop-mapreduce-client-app-2.7.3.jar 530KB
jetty-6.1.26.jar 527KB
spark-kubernetes_2.11-2.4.3.jar 525KB
protobuf-java-2.5.0.jar 521KB
jackson-module-scala_2.11-2.6.7.1.jar 504KB
log4j-1.2.17.jar 478KB
commons-lang3-3.5.jar 469KB
scala-parser-combinators_2.11-1.1.0.jar 461KB
共 546 条
- 1
- 2
- 3
- 4
- 5
- 6
资源评论
Shaw_Bigdata
- 粉丝: 29
- 资源: 25
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功