Convert Figma logo to code with AI

apache logopulsar

Apache Pulsar - distributed pub-sub messaging system

15,353
3,777
15,353
479
View on GitHub

Top Related Projects

apache's avatar

kafka

34,115

Apache Kafka - A distributed event streaming platform

rabbitmq's avatar

rabbitmq-server

13,940

Open source RabbitMQ: core server and tier 1 (built-in) plugins

apache's avatar

rocketmq

22,568

Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.

apache's avatar

activemq

2,443

Apache ActiveMQ

nats-io's avatar

nats-server

20,630

High-Performance server for NATS.io, the cloud and edge native messaging system.

Quick Overview

Apache Pulsar is a cloud-native, distributed messaging and streaming platform. It provides a unified messaging model and API for both streaming and queuing use cases, with built-in features for geo-replication, multi-tenancy, and durability. Pulsar is designed to handle high-throughput, low-latency messaging at scale.

Pros

  • Unified messaging model for both streaming and queuing
  • Built-in geo-replication and multi-tenancy support
  • Scalable and high-performance architecture
  • Strong durability guarantees with tiered storage

Cons

  • Steeper learning curve compared to some simpler messaging systems
  • Requires more resources to run and manage compared to lightweight alternatives
  • Less mature ecosystem compared to Apache Kafka
  • Configuration and setup can be complex for advanced features

Code Examples

  1. Producing messages:
PulsarClient client = PulsarClient.builder()
    .serviceUrl("pulsar://localhost:6650")
    .build();

Producer<String> producer = client.newProducer(Schema.STRING)
    .topic("my-topic")
    .create();

producer.send("Hello, Pulsar!");
  1. Consuming messages:
Consumer<String> consumer = client.newConsumer(Schema.STRING)
    .topic("my-topic")
    .subscriptionName("my-subscription")
    .subscribe();

while (true) {
    Message<String> msg = consumer.receive();
    System.out.println("Received message: " + msg.getValue());
    consumer.acknowledge(msg);
}
  1. Using Pulsar Functions:
public class WordCountFunction implements Function<String, Void> {
    @Override
    public Void process(String input, Context context) throws Exception {
        String[] words = input.split(" ");
        for (String word : words) {
            String counterKey = word.toLowerCase();
            context.incrCounter(counterKey, 1);
        }
        return null;
    }
}

Getting Started

  1. Download and start Pulsar:
wget https://archive.apache.org/dist/pulsar/pulsar-2.8.1/apache-pulsar-2.8.1-bin.tar.gz
tar xvfz apache-pulsar-2.8.1-bin.tar.gz
cd apache-pulsar-2.8.1
bin/pulsar standalone
  1. Add Pulsar client dependency to your project:
<dependency>
    <groupId>org.apache.pulsar</groupId>
    <artifactId>pulsar-client</artifactId>
    <version>2.8.1</version>
</dependency>
  1. Create a client and start producing/consuming messages using the code examples provided above.

Competitor Comparisons

apache logo

kafka

34,115

Apache Kafka - A distributed event streaming platform

Pros of Kafka

  • Higher throughput and lower latency for simple use cases
  • More mature ecosystem with extensive tooling and community support
  • Simpler architecture, easier to set up and manage for small-scale deployments

Cons of Kafka

  • Limited multi-tenancy support
  • Lacks built-in geo-replication and disaster recovery features
  • Storage and compute are tightly coupled, making scaling less flexible

Code Comparison

Kafka producer example:

Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
Producer<String, String> producer = new KafkaProducer<>(props);

Pulsar producer example:

PulsarClient client = PulsarClient.builder()
    .serviceUrl("pulsar://localhost:6650")
    .build();
Producer<byte[]> producer = client.newProducer()
    .topic("my-topic")
    .create();

Both Kafka and Pulsar are powerful distributed messaging systems, but they have different strengths. Kafka excels in high-throughput scenarios and has a more established ecosystem, while Pulsar offers better multi-tenancy, geo-replication, and flexible scaling options. The choice between them depends on specific use cases and requirements.

rabbitmq logo

rabbitmq-server

13,940

Open source RabbitMQ: core server and tier 1 (built-in) plugins

Pros of RabbitMQ

  • Simpler setup and configuration, making it easier for beginners
  • Mature ecosystem with extensive plugins and integrations
  • Lower resource consumption, suitable for smaller-scale deployments

Cons of RabbitMQ

  • Limited scalability compared to Pulsar's multi-layer architecture
  • Lacks built-in multi-tenancy support, which Pulsar provides out-of-the-box
  • Less efficient for high-throughput scenarios with large message volumes

Code Comparison

RabbitMQ (Erlang):

basic_publish(Channel, <<"my_exchange">>, <<"routing_key">>, <<"Hello, World!">>),
{#'basic.get_ok'{}, Content} = basic_get(Channel, <<"my_queue">>, no_ack),
io:format("Received: ~p~n", [Content]).

Pulsar (Java):

producer.send("Hello, World!".getBytes());
Message<byte[]> msg = consumer.receive();
System.out.println("Received: " + new String(msg.getData()));
consumer.acknowledge(msg);

Both examples demonstrate basic message publishing and consuming. RabbitMQ uses exchanges and routing keys, while Pulsar uses a simpler topic-based approach. Pulsar requires explicit message acknowledgment, which is optional in RabbitMQ.

apache logo

rocketmq

22,568

Apache RocketMQ is a cloud native messaging and streaming platform, making it simple to build event-driven applications.

Pros of RocketMQ

  • Simpler architecture and easier to deploy
  • Lower latency for small messages
  • Better support for traditional messaging patterns

Cons of RocketMQ

  • Limited support for streaming use cases
  • Less scalable for very large deployments
  • Fewer advanced features like tiered storage

Code Comparison

RocketMQ producer example:

DefaultMQProducer producer = new DefaultMQProducer("ProducerGroupName");
producer.setNamesrvAddr("127.0.0.1:9876");
producer.start();

Message msg = new Message("TopicTest", "TagA", "Hello RocketMQ".getBytes(RemotingHelper.DEFAULT_CHARSET));
SendResult sendResult = producer.send(msg);

Pulsar producer example:

PulsarClient client = PulsarClient.builder()
    .serviceUrl("pulsar://localhost:6650")
    .build();
Producer<byte[]> producer = client.newProducer()
    .topic("my-topic")
    .create();
producer.send("Hello Pulsar".getBytes());

Both RocketMQ and Pulsar are powerful distributed messaging systems, but they have different strengths. RocketMQ is often favored for its simplicity and performance with small messages, while Pulsar offers more advanced features and better scalability for large-scale deployments. The choice between them depends on specific use cases and requirements.

apache logo

activemq

2,443

Apache ActiveMQ

Pros of ActiveMQ

  • Mature and well-established project with a large user base
  • Supports multiple protocols (AMQP, MQTT, STOMP) out of the box
  • Easier to set up and configure for simple use cases

Cons of ActiveMQ

  • Limited scalability compared to Pulsar's multi-layer architecture
  • Lacks built-in geo-replication capabilities
  • Less efficient in handling high-throughput scenarios

Code Comparison

ActiveMQ (Java):

ConnectionFactory factory = new ActiveMQConnectionFactory("tcp://localhost:61616");
Connection connection = factory.createConnection();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
Destination destination = session.createQueue("myQueue");
MessageProducer producer = session.createProducer(destination);

Pulsar (Java):

PulsarClient client = PulsarClient.builder()
    .serviceUrl("pulsar://localhost:6650")
    .build();
Producer<byte[]> producer = client.newProducer()
    .topic("my-topic")
    .create();

Both ActiveMQ and Pulsar are robust messaging systems, but they cater to different use cases. ActiveMQ is more suitable for traditional messaging scenarios, while Pulsar excels in high-throughput, scalable environments with features like geo-replication and multi-tenancy. The code comparison shows that Pulsar's API is slightly more concise, reflecting its modern design approach.

nats-io logo

nats-server

20,630

High-Performance server for NATS.io, the cloud and edge native messaging system.

Pros of NATS Server

  • Lightweight and simple architecture, resulting in lower resource usage and easier deployment
  • Extremely high performance, capable of handling millions of messages per second
  • Supports multiple messaging patterns (pub/sub, request/reply, queueing) out of the box

Cons of NATS Server

  • Limited built-in persistence options compared to Pulsar's tiered storage
  • Less robust multi-tenancy features and resource isolation
  • Fewer advanced streaming capabilities, such as delayed message delivery or message replay

Code Comparison

NATS Server (Go):

nc, err := nats.Connect(nats.DefaultURL)
if err != nil {
    log.Fatal(err)
}
nc.Publish("foo", []byte("Hello World"))

Pulsar (Java):

PulsarClient client = PulsarClient.builder()
    .serviceUrl("pulsar://localhost:6650")
    .build();
Producer<byte[]> producer = client.newProducer()
    .topic("my-topic")
    .create();
producer.send("Hello World".getBytes());

Both examples demonstrate basic message publishing, but NATS Server's API is more concise and straightforward, while Pulsar's API offers more configuration options and type safety.

Convert Figma logo designs to code with AI

Visual Copilot

Introducing Visual Copilot: A new AI model to turn Figma designs to high quality code using your components.

Try Visual Copilot

README

logo

docker pull contributors last commit release downloads Ask DeepWiki

Pulsar is a distributed pub-sub messaging platform with a very flexible messaging model and an intuitive client API.

Learn more about Pulsar at https://pulsar.apache.org

Main features

  • Horizontally scalable (Millions of independent topics and millions of messages published per second)
  • Strong ordering and consistency guarantees
  • Low latency durable storage
  • Topic and queue semantics
  • Load balancer
  • Designed for being deployed as a hosted service:
    • Multi-tenant
    • Authentication
    • Authorization
    • Quotas
    • Support mixing very different workloads
    • Optional hardware isolation
  • Keeps track of consumer cursor position
  • REST API for provisioning, admin and stats
  • Geo replication
  • Transparent handling of partitioned topics
  • Transparent batching of messages

Repositories

This repository is the main repository of Apache Pulsar. Pulsar PMC also maintains other repositories for components in the Pulsar ecosystem, including connectors, adapters, and other language clients.

Helm Chart

Ecosystem

Clients

Dashboard & Management Tools

Website

CI/CD

Archived/Halted

Pulsar Runtime Java Version Recommendation

Note:

When using Java versions, it is recommended to use a recent version of a particular Java release (17 or 21) with the most recent bug fixes and security patches. For example, the JVM bug JDK-8351933 can cause stability issues in Pulsar. JDK-8351933 was fixed in Java 17.0.17+ and Java 21.0.8+. Pulsar Docker images come with the most recent Java version at the time of release.

When using the Pulsar Java client, it is recommended to set specific system properties and JVM options to allow optimal performance.

pulsar ver >= 4.1 and master branch

ComponentJava Version
Broker21
Functions / IO21
CLI17 or 21
Java Client17 or 21

Docker image Java runtime: 21

3.3 <= pulsar ver <= 4.0

ComponentJava Version
Broker21
Functions / IO21
CLI17 or 21
Java Client8, 11, 17 or 21

Docker image Java runtime: 21

2.10 <= pulsar ver <= 3.0

ComponentJava Version
Broker17
Functions / IO17
CLI17
Java Client8 or 11 or 17

Docker image Java runtime: 17

2.8 <= pulsar ver <= 2.10

ComponentJava Version
Broker11
Functions / IO11
CLI8 or 11
Java Client8 or 11

pulsar ver < 2.8

ComponentJava Version
All8 or 11

Build Pulsar

The quick start below covers the essentials; see CONTRIBUTING.md → Building for the full build and lint commands, and ARCHITECTURE.md → Build infrastructure for the Gradle build infrastructure and how to change build files.

Requirements

There is also a guide for setting up the tooling for building Pulsar.

Note:

This project includes a Gradle Wrapper so no separate Gradle installation is needed. Use ./gradlew on Linux/macOS and gradlew.bat on Windows.

For a better developer experience, install Gradle command-line completion (gradle-completion installation) for bash and zsh shells.

Build

./gradlew assemble                                                       # compile and assemble
./gradlew :pulsar-client-original:test --tests "ConsumerBuilderImplTest" # run a single test
bin/pulsar standalone                                                    # run a standalone service
./gradlew quickCheck                                                     # license headers + checkstyle, no compile
./gradlew sanityCheck                                                    # quickCheck + compile main/test (pre-PR)

For the full build, lint, test, and PR workflow — test groups, integration tests, Personal CI, and PR conventions — see CONTRIBUTING.md. For the module map and the Gradle build infrastructure see ARCHITECTURE.md, and for coding conventions see CODING.md. AI coding agents should start from AGENTS.md.

Check https://pulsar.apache.org for documentation and examples.

Build custom docker images

The commands used in the Apache Pulsar release process can be found in the release process documentation.

Here are some general instructions for building custom docker images:

  • Java 21 is the recommended JDK version since branch-4.0.

The following command builds the docker image apachepulsar/pulsar:latest:

./gradlew docker

After the image is built, it can be tagged and pushed to your custom repository. Here's an example of a bash script that tags the docker image with the current version and git revision and pushes it to localhost:32000/apachepulsar.

image_repo_and_project=localhost:32000/apachepulsar
pulsar_version=$(src/get-pulsar-version.sh)
gitrev=$(git rev-parse HEAD | colrm 10)
tag="${pulsar_version}-${gitrev}"
echo "Using tag $tag"
docker tag apachepulsar/pulsar:latest ${image_repo_and_project}/pulsar:$tag
docker push ${image_repo_and_project}/pulsar:$tag

Setting up your IDE

Read https://pulsar.apache.org/contribute/setup-ide for setting up IntelliJ IDEA or Eclipse for developing Pulsar.

Documentation

Note:

For how to make contributions to Pulsar documentation, see Pulsar Documentation Contribution Guide.

Contact

Mailing lists
  • The mailing lists are the primary contact for the Apache Pulsar project.
  • Your email to the mailing list might be placed in a moderation queue if you haven't joined the mailing list by subscribing before posting.

Note

Please note that security-related issues or concerns should not be reported in public channels. Follow the instructions in the Security Policy to contact the ASF Security Team and the Apache Pulsar PMC directly.

NameScopeSubscribeUnsubscribeArchives
users@pulsar.apache.orgUser-related discussionsSubscribeUnsubscribeArchives
dev@pulsar.apache.orgDevelopment-related discussionsSubscribeUnsubscribeArchives
Slack

Pulsar slack channel at https://apache-pulsar.slack.com/

You can self-register at https://communityinviter.com/apps/apache-pulsar/apache-pulsar

Security Policy

See the Pulsar security policy for the full policy. In short: never disclose a suspected vulnerability publicly (issue, PR, or discussion) before a fix is released, and a human must verify and take responsibility for a report — autonomous agents must not file security reports on their own.

Reporting a security vulnerability

Report privately by email to the Apache Security Team at security@apache.org; you may also copy the Apache Pulsar PMC's private list, private@pulsar.apache.org. See the Pulsar security policy for more detail.

Checking exposure to an already-public CVE

An already-public CVE is not a private disclosure. Search PRs and issues by the CVE id at https://github.com/apache/pulsar/pulls (including closed ones), then ask via a GitHub issue or the dev@pulsar.apache.org mailing list if needed. Pulsar's supported versions are listed at https://pulsar.apache.org/contribute/release-policy/; upgrade to a supported version to receive security updates. See the Pulsar security policy.

License

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0

Crypto Notice

This distribution includes cryptographic software. The country in which you currently reside may have restrictions on the import, possession, use, and/or re-export to another country, of encryption software. BEFORE using any encryption software, please check your country's laws, regulations and policies concerning the import, possession, or use, and re-export of encryption software, to see if this is permitted. See The Wassenaar Arrangement for more information.

The U.S. Government Department of Commerce, Bureau of Industry and Security (BIS), has classified this software as Export Commodity Control Number (ECCN) 5D002.C.1, which includes information security software using or performing cryptographic functions with asymmetric algorithms. The form and manner of this Apache Software Foundation distribution makes it eligible for export under the License Exception ENC Technology Software Unrestricted (TSU) exception (see the BIS Export Administration Regulations, Section 740.13) for both object code and source code.

The following provides more details on the included cryptographic software: Pulsar uses the SSL library from Bouncy Castle written by http://www.bouncycastle.org.