2022年度最新Apache Pulsar versus Apache Kafka

Table of Contents

Executive Summary 3

○ Figure 1: Pulsar vs. Kafka Monthly Active Contributors 3

I. Key Benchmark Findings 4

Benchmark Tests 5

I. What We Tested 5

1. Test #1 / Case #1: Maximum Throughput with 1 Partition 8

○ Figure 2: Single Partition Max Write Throughput 9

2. Test #1 / Case #2: Maximum Throughput with 100 Partitions 10

○ Figure 3: 100 Partitions Max Write Throughput 11

B. Test #2: Publish Latency 12

○ Figure 5c: Impact Publish Latency during Catchup Read 19

Conclusion 20

About StreamNative 21

References 22

Executive Summary

has happened since then. In 2021, Pulsar ranked as a Top 5 Apache Software Foundation

project and surpassed Apache Kafka in monthly active contributors as shown in the chart

below. Pulsar also averaged more monthly active contributors than Kafka for most of the

past 18 months.

These contributions led to major performance improvements for Pulsar. To measure the

impact of the improvements, the engineering team at StreamNative, led by Matteo Merli,

one of the original creators of Apache Pulsar, Apache Pulsar PMC Chairperson, performed

a benchmark study using the Linux Foundation Open Messaging benchmark.

Below we provide details on how the benchmark was performed and its results.

Benchmark Tests

Using the Linux Foundation Open Messaging benchmark [1], we ran the latest versions of

Pulsar’s default configuration includes Journaling, which offers a higher durability

guarantee than Kafka’s default configuration. Pulsar without Journaling provides the same

This test measures the maximum data throughput the system can deliver when consumers

are keeping up with the incoming trac.

We ran this test in two scenarios to test the upper boundary performance and to test the

cost profile for each system:

1. Topic with a single partition. This scenario tests the upper boundary performance

for a total-order use case or, in the worst case, where partition keys’ data is skewed.

At some scale, the design of a system that relies upon single ordering or handling

large amounts of skewed data will need to be reconsidered. Pulsar has the ability to

handle situations where total ordering is required at higher scale or large amounts

of skew arise.