![karafka logo](https://raw.githubusercontent.com/karafka/misc/master/logo/karafka_logotype_transparent2.png)
[![Build Status](https://github.com/karafka/karafka/actions/workflows/ci.yml/badge.svg)](https://github.com/karafka/karafka/actions/workflows/ci.yml)
[![Gem Version](https://badge.fury.io/rb/karafka.svg)](http://badge.fury.io/rb/karafka)
[![Join the chat at https://slack.karafka.io](https://raw.githubusercontent.com/karafka/misc/master/slack.svg)](https://slack.karafka.io)
**Note**: Upgrade instructions for migration from Karafka `1.4` to Karafka `2.0` can be found [here](https://karafka.io/docs/Upgrades-2.0/).
## About Karafka
Karafka is a Ruby and Rails multi-threaded efficient Kafka processing framework that:
- Has a built-in [Web UI](https://karafka.io/docs/Web-UI-Features/) providing a convenient way to monitor and manage Karafka-based applications.
- Supports parallel processing in [multiple threads](https://karafka.io/docs/Concurrency-and-multithreading) (also for a [single topic partition](https://karafka.io/docs/Pro-Virtual-Partitions) work)
- [Automatically integrates](https://karafka.io/docs/Integrating-with-Ruby-on-Rails-and-other-frameworks#integrating-with-ruby-on-rails) with Ruby on Rails
- Has [ActiveJob backend](https://karafka.io/docs/Active-Job) support (including [ordered jobs](https://karafka.io/docs/Pro-Enhanced-Active-Job#ordered-jobs))
- Has a seamless [Dead Letter Queue](https://karafka.io/docs/Dead-Letter-Queue/) functionality built-in
- Supports in-development [code reloading](https://karafka.io/docs/Auto-reload-of-code-changes-in-development)
- Is powered by [librdkafka](https://github.com/edenhill/librdkafka) (the Apache Kafka C/C++ client library)
- Has an out-of the box [StatsD/DataDog monitoring](https://karafka.io/docs/Monitoring-and-logging) with a dashboard template.
```ruby
# Define what topics you want to consume with which consumers in karafka.rb
Karafka::App.routes.draw do
topic 'system_events' do
consumer EventsConsumer
end
end
# And create your consumers, within which your messages will be processed
class EventsConsumer < ApplicationConsumer
# Example that utilizes ActiveRecord#insert_all and Karafka batch processing
def consume
# Store all of the incoming Kafka events locally in an efficient way
Event.insert_all messages.payloads
end
end
```
Karafka **uses** threads to handle many messages simultaneously in the same process. It does not require Rails but will integrate tightly with any Ruby on Rails applications to make event processing dead simple.
## Getting started
![karafka web ui](https://raw.githubusercontent.com/karafka/misc/master/printscreens/web-ui.png)
If you're entirely new to the subject, you can start with our "Kafka on Rails" articles series, which will get you up and running with the terminology and basic ideas behind using Kafka:
- [Kafka on Rails: Using Kafka with Ruby on Rails – Part 1 – Kafka basics and its advantages](https://mensfeld.pl/2017/11/kafka-on-rails-using-kafka-with-ruby-on-rails-part-1-kafka-basics-and-its-advantages/)
- [Kafka on Rails: Using Kafka with Ruby on Rails – Part 2 – Getting started with Rails and Kafka](https://mensfeld.pl/2018/01/kafka-on-rails-using-kafka-with-ruby-on-rails-part-2-getting-started-with-ruby-and-kafka/)
If you want to get started with Kafka and Karafka as fast as possible, then the best idea is to visit our [Getting started](https://karafka.io/docs/Getting-Started) guides and the [example apps repository](https://github.com/karafka/example-apps).
We also maintain many [integration specs](https://github.com/karafka/karafka/tree/master/spec/integrations) illustrating various use-cases and features of the framework.
### TL;DR (1 minute from setup to publishing and consuming messages)
**Prerequisites**: Kafka running. You can start it by following instructions from [here](https://karafka.io/docs/Setting-up-Kafka).
1. Add and install Karafka:
```bash
# Make sure to install Karafka 2.0
bundle add karafka --version ">= 2.0.28"
bundle exec karafka install
```
2. Dispatch a message to the example topic using the Rails or Ruby console:
```ruby
Karafka.producer.produce_sync(topic: 'example', payload: { 'ping' => 'pong' }.to_json)
```
3. Run Karafka server and see the consumption magic happen:
```bash
bundle exec karafka server
[86d47f0b92f7] Polled 1 message in 1000ms
[3732873c8a74] Consume job for ExampleConsumer on example started
{"ping"=>"pong"}
[3732873c8a74] Consume job for ExampleConsumer on example finished in 0ms
```
## Want to Upgrade? LGPL is not for you? Want to help?
I also sell Karafka Pro subscriptions. It includes a commercial-friendly license, priority support, architecture consultations, enhanced Web UI and high throughput data processing-related features (virtual partitions, long-running jobs, and more).
**10%** of the income will be distributed back to other OSS projects that Karafka uses under the hood.
Help me provide high-quality open-source software. Please see the Karafka [homepage](https://karafka.io/#become-pro) for more details.
## Support
Karafka has [Wiki pages](https://karafka.io/docs) for almost everything and a pretty decent [FAQ](https://karafka.io/docs/FAQ). It covers the installation, setup, and deployment, along with other useful details on how to run Karafka.
If you have questions about using Karafka, feel free to join our [Slack](https://slack.karafka.io) channel.
Karafka has [priority support](https://karafka.io/docs/Pro-Support) for technical and architectural questions that is part of the Karafka Pro subscription.
没有合适的资源?快使用搜索试试~ 我知道了~
Ruby和Rails高效的Kafka处理框架_Ruby_下载.zip
共1132个文件
rb:1068个
gemfile:20个
yml:7个
1.该资源内容由用户上传,如若侵权请联系客服进行举报
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
2.虚拟产品一经售出概不退款(资源遇到问题,请及时私信上传者)
版权申诉
0 下载量 8 浏览量
2023-04-26
11:02:40
上传
评论
收藏 966KB ZIP 举报
温馨提示
Ruby和Rails高效的Kafka处理框架_Ruby_下载.zip
资源推荐
资源详情
资源评论
收起资源包目录
Ruby和Rails高效的Kafka处理框架_Ruby_下载.zip (1132个子文件)
benchmarks 2KB
.console_irbrc 336B
create_token 586B
karafka.rb.erb 2KB
example_consumer.rb.erb 369B
application_consumer.rb.erb 286B
Gemfile 431B
Gemfile 431B
Gemfile 376B
Gemfile 232B
Gemfile 232B
Gemfile 214B
Gemfile 214B
Gemfile 200B
Gemfile 183B
Gemfile 165B
Gemfile 165B
Gemfile 165B
Gemfile 165B
Gemfile 165B
Gemfile 165B
Gemfile 150B
Gemfile 150B
Gemfile 139B
Gemfile 127B
Gemfile 127B
karafka.gemspec 2KB
.gitignore 1023B
.gitkeep 0B
integrations 9KB
dashboard.json 16KB
renovate.json 107B
karafka 252B
LICENSE 677B
LICENSE-COMM 16KB
LICENSE-LGPL 7KB
Gemfile.lock 2KB
CHANGELOG.md 34KB
README.md 5KB
CODE_OF_CONDUCT.md 3KB
CONTRIBUTING.md 2KB
bug_report.md 1KB
feature_request.md 593B
private_key_2.pem 3KB
private_key_1.pem 3KB
cert_chain.pem 2KB
public_key_2.pem 800B
public_key_1.pem 800B
karafka-pro.pem 601B
client.rb 17KB
base_consumer_spec.rb 14KB
listener.rb 14KB
base_consumer_spec.rb 13KB
strategy_selector_spec.rb 13KB
logger_listener_spec.rb 13KB
config.rb 11KB
integrations_helper.rb 11KB
logger_listener.rb 11KB
attributes_map.rb 10KB
metrics_listener.rb 10KB
iterator.rb 10KB
admin.rb 9KB
config_spec.rb 9KB
builder_spec.rb 9KB
pause_spec.rb 9KB
status_spec.rb 8KB
base_consumer.rb 8KB
server_spec.rb 8KB
admin_spec.rb 7KB
jobs_queue_spec.rb 6KB
coordinator.rb 6KB
filters_applier_spec.rb 6KB
raw_messages_buffer_spec.rb 6KB
server.rb 6KB
executor.rb 6KB
jobs_queue.rb 6KB
logger_listener.rb 5KB
default.rb 5KB
dispatcher_spec.rb 5KB
railtie.rb 5KB
default.rb 5KB
topics.rb 5KB
karafka.rb 5KB
topic_spec.rb 4KB
bindings.rb 4KB
consumer_spec.rb 4KB
executor_spec.rb 4KB
coordinator_spec.rb 4KB
coordinator.rb 4KB
topic.rb 4KB
consumer_group.rb 4KB
collapse_and_continuity_spec.rb 4KB
filters_applier.rb 4KB
pause.rb 4KB
benchmarks_helper.rb 4KB
dispatcher_spec.rb 4KB
throttled_collapse_and_continuity_spec.rb 4KB
coordinator_spec.rb 4KB
with_static_membership_reconnect_spec.rb 4KB
with_static_membership_reconnect_spec.rb 4KB
共 1132 条
- 1
- 2
- 3
- 4
- 5
- 6
- 12
资源评论
快撑死的鱼
- 粉丝: 1w+
- 资源: 9156
上传资源 快速赚钱
- 我的内容管理 展开
- 我的资源 快来上传第一个资源
- 我的收益 登录查看自己的收益
- 我的积分 登录查看自己的积分
- 我的C币 登录后查看C币余额
- 我的收藏
- 我的下载
- 下载帮助
最新资源
- mybatis动态sql及其JAVA示例
- 微软常用运行库 游戏运行库 VC++各个版本
- 微信小程序开发教程.pptx
- MyBatis动态SQL是一种强大的特性,它允许我们在SQL语句中根据条件动态地添加或删除某些部分,从而实现更加灵活和高效的数据
- 锐捷网络认证中心网络管理.pdf
- MyBatis动态SQL是一种强大的特性,它允许我们在SQL语句中根据条件动态地添加或删除某些部分,从而实现更加灵活和高效的数据
- SD8233LF是一款用于单按键触摸及接近感应开关,其用途是替代传统的机械型开关芯片IC
- 基于YOLOv5的烟雾火焰检测算法研究
- 基于STM32的联合调试侦听设备解决方案原理图PCB源文件调试工具视频(大赛作品)
- MyBatis动态SQL是一种强大的特性,它允许我们在SQL语句中根据条件动态地添加或删除某些部分,从而实现更加灵活和高效的数据
资源上传下载、课程学习等过程中有任何疑问或建议,欢迎提出宝贵意见哦~我们会及时处理!
点击此处反馈
安全验证
文档复制为VIP权益,开通VIP直接复制
信息提交成功