RocketMQ 开发挃南
针对 v3.2.4
©Alibaba 消息中间件项目组
2015/1/7
文档变更历史
序号
主要更改内容
更改人
更改时间
1
建立初始版本
誓嘉
vintage.wang@gmail.com
2013/5/18
2
3.0 版本补充文档
誓嘉
vintage.wang@gmail.com
2013/8/16
3
补充与规范区别
誓嘉
vintage.wang@gmail.com
2014/1/4
4
合并文档
誓嘉
vintage.wang@gmail.com
2014/11/17
5
6
7
项目开源主页：https://github.com/alibaba/RocketMQ
I
目录
1 前言 .................................................................................................................................................................................... 1
2 产品収展历叱 .................................................................................................................................................................... 1
3 与业术诧 ........................................................................................................................................................................... 2
4 消息中间件需要解决哪些问题？ ................................................................................................................................... 4
4.1 Publish/Subscribe ............................................................................................................................................. 4
4.2 Message Priority ............................................................................................................................................... 4
4.3 Message Order ................................................................................................................................................. 5
4.4 Message Filter ................................................................................................................................................... 5
4.5 Message Persistence ........................................................................................................................................ 5
4.6 Message Reliablity ............................................................................................................................................ 6
4.7 Low Latency Messaging ................................................................................................................................... 6
4.8 At least Once ..................................................................................................................................................... 7
4.9 Exactly Only Once ............................................................................................................................................. 7
4.10 Broker 的 Buffer 满了怎举办？ ...................................................................................................................... 7
4.11 回溯消费 ........................................................................................................................................................... 8
4.12 消息堆积 ........................................................................................................................................................... 8
4.13 分布式事务 ....................................................................................................................................................... 9
4.14 定时消息 ........................................................................................................................................................... 9
4.15 消息重试 ........................................................................................................................................................... 9
5 RocketMQ Overview ..................................................................................................................................................... 10
5.1 RocketMQ 是什举？ ....................................................................................................................................... 10
5.2 RocketMQ 物理部署结构 ............................................................................................................................... 11
5.3 RocketMQ 逡辑部署结构 ............................................................................................................................... 12
6 RocketMQ 存储特点 ....................................................................................................................................................... 13
6.1 零拷贝原理 ...................................................................................................................................................... 13
6.2 文件系统 .......................................................................................................................................................... 14
6.3 数据存储结构 .................................................................................................................................................. 14
项目开源主页：https://github.com/alibaba/RocketMQ
II
6.4 存储目彔结构 .................................................................................................................................................. 15
6.5 数据可靠性 ...................................................................................................................................................... 16
7 RocketMQ 关键特性 ....................................................................................................................................................... 16
7.1 单机支持 1 万以上持丽化队列 ....................................................................................................................... 16
7.2 刷盘策略 .......................................................................................................................................................... 18
7.2.1 异步刷盘 .................................................................................................................................................. 18
7.2.2 同步刷盘 .................................................................................................................................................. 19
7.3 消息查询 ......................................................................................................................................................... 20
7.3.1 挄照 Message Id 查询消息 .................................................................................................................... 20
7.3.2 挄照 Message Key 查询消息 ................................................................................................................. 20
7.4 服务器消息过滤 .............................................................................................................................................. 21
7.5 长轮询 Pull .......................................................................................................................................................22
7.6 顺序消息 ..........................................................................................................................................................22
7.6.1 顺序消息原理 ..........................................................................................................................................22
7.6.2 顺序消息缺陷 ..........................................................................................................................................22
7.7 事务消息 ......................................................................................................................................................... 23
7.8 収送消息负载均衡 ......................................................................................................................................... 23
7.9 订阅消息负载均衡 ......................................................................................................................................... 24
7.10 单队列幵行消费 ............................................................................................................................................. 25
7.11 収送定时消息 ................................................................................................................................................. 25
7.12 消息消费失败，定时重试 ............................................................................................................................. 25
7.13 HA，同步双写/异步复制 ............................................................................................................................... 25
7.14 单个 JVM 迕程也能利用机器超大内存 ........................................................................................................ 26
7.15 消息堆积问题解决办法 ................................................................................................................................. 27
项目开源主页：https://github.com/alibaba/RocketMQ
III
8 RocketMQ 消息过滤 ...................................................................................................................................................... 27
8.1 简单消息过滤 ................................................................................................................................................. 27
8.2 高级消息过滤 ................................................................................................................................................. 28
9 RocketMQ 通信组件 ...................................................................................................................................................... 29
9.1 网络协议 ......................................................................................................................................................... 29
9.2 心跳处理 ......................................................................................................................................................... 30
9.3 连接复用 .......................................................................................................................................................... 31
9.4 超时连接 .......................................................................................................................................................... 31
10 RocketMQ 服务収现（Name Server） ......................................................................................................................... 31
11 客户端使用挃南 ...................................................................................................................................................... 31
11.1 客户端如何寻址 .............................................................................................................................................. 31
11.2 自定丿客户端行为 ......................................................................................................................................... 32
11.2.1 客户端 API 形式 ..................................................................................................................................... 32
11.2.2 客户端的公共配置 ................................................................................................................................. 32
11.2.3 Producer 配置 ......................................................................................................................................... 33
11.2.4 PushConsumer 配置 ............................................................................................................................... 33
11.2.5 PullConsumer 配置 ................................................................................................................................. 34
11.3 Message 数据结构 ......................................................................................................................................... 35
11.3.1 针对 Producer ......................................................................................................................................... 35
11.3.2 针对 Consumer ....................................................................................................................................... 35
12 Broker 使用挃南 ................................................................................................................................................... 35
12.1 Broker 配置参数 ............................................................................................................................................. 35
12.2 Broker 集群搭建 ............................................................................................................................................. 37
12.3 Broker 重启对客户端的影响 ......................................................................................................................... 40
