rafka-rb

rafka-rb is a Ruby client library for Rafka, providing Consumer and Producer implementations with simple semantics.

Refer to the API documentation for more information.

Features

• Consumer
  • consumer groups
  • support for consuming in batches
  • offsets may be managed automatically or manually
• Producer
  • support for partition hashing key

Getting started

Install rafka-rb:

$ gem install rafka

If you're using Bundler, add it to your Gemfile:

gem "rafka"

and run bundle install.

Usage

Producer

producer = Rafka::Producer.new(host: "localhost", port: 6380)
producer.produce("greetings", "Hello there!")

Refer to the Producer API documentation for more information.

Consumer

consumer = Rafka::Consumer.new(topic: "greetings", group: "myapp")
msg = consumer.consume
msg.value # => "Hello there!"

# with a block
consumer.consume { |msg| puts "Received: #{msg.value}" } # => "Hello there!"

Offsets are managed automatically by default. If you need more control you can turn off the feature and manually commit offsets:

consumer = Rafka::Consumer.new(topic: "greetings", group: "myapp", auto_commit: false)

# commit a single offset
msg = consumer.consume
consumer.commit(msg) # => true

# or commit a bunch of offsets
msg1 = consumer.consume
msg2 = consumer.consume
consumer.commit(msg1, msg2) # => true

Consumers may also set their own custom librdkafka configuration:

consumer = Rafka::Consumer.new(
  topic: "greetings", group: "myapp", librdkafka: { "auto.offset.reset" => "earliest" }
)

Refer to the Consumer API documentation for more information.

Development

Running Rubocop:

$ bundle exec rake rubocop

Unit tests run as follows:

$ bundle exec rake test

rafka-rb is indirectly tested by Rafka's end-to-end tests.

License

rafka-rb is released under the GNU General Public License version 3. See COPYING.