mshauneu/php-rdkafka-bundle

Integrates php-rdkafka with Symfony2|3

KafkaBundle

About

This Symfony bundle provides connectivity to the Kafka publish-subscribe messaging system based on rdkafka binding to librdkafka, _(*1)

Installation

Add the dependency in your composer.json, _(*2)

{
    "require": {
        "mshauneu/php-rdkafka-bundle"
    }
}

Enable the bundle in your application kernel, _(*3)

// app/AppKernel.php
public function registerBundles() {
    $bundles = array(
        // ...
        new Mshauneu\RdKafkaBundle\MshauneuRdKafkaBundle(),
    );
}

Configuration

Simple configuration could look like:, _(*4)

mshauneu_rd_kafka:
  producers: 
    test_producer: 
      brokers: 127.0.0.1:9092
      topic: test_topic   
  consumers:
    test_consumer:
      brokers: 127.0.0.1:9092
      topic: test_topic   
      properties: 
        group_id: "test_group_id"
      topic_properties: 
        offset_store_method: broker           
        auto_offset_reset: smallest
        auto_commit_interval_ms: 100

Configuration properties are documented: - for producer or consumer in CommunicatorConfiguration.php - for topic to produce in TopicProducerConfiguration.php - for topic to consume in TopicConsumerConfiguration.php, _(*5)

Usage

Publishing messages to a Kafka topic

From a Symfony controller:, _(*6)

$payload = 'test_message';
$topicProducer = $container->get('mshauneu_rd_kafka')->getProducer("test_producer");
$topicProducer->produceStart();
$topicProducer->produce("message");
$topicProducer->produceStop();

By CLI:, _(*7)

./app/console kafka:producer --producer test_producer test_message 

Consume messages out of a Kafka topic:

Implement ConsumerInterface, _(*8)

class MessageHandler implements ConsumerInterface {
    public function consume($topic, $partition, $offset, $key, $payload) {
        echo "Received payload: " . $payload . PHP_EOL;
    }
}

Register it:, _(*9)

test_message_handler:
    class: MessageHandler

From a Symfony controller:, _(*10)

$topicConsumer = $container->get('mshauneu_rd_kafka')->getConsumer("test_producer");
$topicConsumer->consumeStart(TopicCommunicator::OFFSET_STORED);
$topicConsumer->consume($consumerImpl);
$topicConsumer->consumeStop();

By CLI:, _(*11)

./app/console kafka:consumer --consumer test_consumer --handler test_message_handler 

License

This project is under the MIT License. See the LICENSE file for the full license text., _(*12)