没有合适的资源?快使用搜索试试~ 我知道了~
藏经阁-Improved Reliable Streaming Processing_ Apache Storm as exam
需积分: 5 0 下载量 133 浏览量
2023-08-29
21:00:36
上传
评论
收藏 1.11MB PDF 举报
温馨提示
试读
34页
藏经阁-Improved Reliable Streaming Processing_ Apache Storm as exam
资源推荐
资源详情
资源评论
1© Copyright 2016 EMC Corporation. All rights reserved.
Improved Reliable Streaming Processing:
Apache Storm as example
Frank Zhao, EMC CTO Office,
Fenghao Zhang*, Microsoft Bing,
Yusong Lv*, Peking University
Special thanks to EMC Ken Taylor, John Cardente and Lincourt Robert
*Zhang and Lv contributed to the research when they worked at EMC China COE
2© Copyright 2016 EMC Corporation. All rights reserved.
The technology concepts being discussed and demonstrated are
the result of research conducted by the Advanced Research &
Development (ARD) team from the EMC Office of the CTO. Any
demonstrated capability is only for research purpose and at a
prototype phase, therefore : THERE ARE NO IMMEDIATE PLANS
NOR INDICATION OF SUCH PLANS FOR PRODUCTIZATION OF
THESE CAPABILITIES AT THE TIME OF PRESENTATION. THINGS
MAY OR MAY NOT CHANGE IN THE FUTURE.
DISCLAIMER
3© Copyright 2016 EMC Corporation. All rights reserved.
• Distributed Streaming System
• Reliable Processing
• Apache Storm’s Solution, the Challenge
• New Proposed Approaches
– Fingerprint, and share-split
• Prototyping with Apache Storm and Benchmark
• Summary and Outlook
Agenda
4© Copyright 2016 EMC Corporation. All rights reserved.
• As service, continuously process data (a.k.a message or tuple)
in scalable, reliable and high-performance way (msec)
– Open-source: Storm, Flink, Spark-Streaming, Samza
Streaming processing
5© Copyright 2016 EMC Corporation. All rights reserved.
Streaming Processing
(Storm, Spark Streaming)
Batch processing
(Hadoop MR)
Type
Continuous(never-stop),
real-time (ms level)
Batch/Period
Model
DAG/graph
MapReduce like Jobs
Workload
CPU/Memory intensive
CPU/mem and IO internsive
State
Stateless, may period ckpt
Stateful
Cluster
Master-Slave w/ Zookeeper (Storm)
Master-Slave or Job-task
Fault-
tolerance
Fault-tolerance/HA
Fault-tolerance/HA
Streaming vs. batch processing
剩余33页未读,继续阅读
资源评论
weixin_40191861_zj
- 粉丝: 64
- 资源: 1万+
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功