« Changelog History


Karafka v1.0.0 Release Notes

  • Closed issues:

    • #103 - Env for logger is loaded 2 early (on gem load not on app init)
    • 👍 #142 - Possibility to better control Kafka consumers (consumer groups management)
    • #150 - Add support for start_from_beginning on a per topic basis
    • #154 - Support for min_bytes and max_wait_time on messages consuming
    • 💎 #160 - Reorganize settings to better resemble ruby-kafka requirements
    • 🔧 #164 - If we decide to have configuration per topic, topic uniqueness should be removed
    • #165 - Router validator
    • #166 - Params and route reorganization (new API)
    • 🚚 #167 - Remove Sidekiq UI from Karafka
    • #168 - Introduce unique IDs of routes
    • 📇 #171 - Add kafka message metadata to params
    • #176 - Transform Karafka::Connection::Consumer into a module
    • #177 - Monitor not reacting when kafka killed with -9
    • #175 - Allow single consumer to subscribe to multiple topics
    • 🚚 #178 - Remove parsing failover when cannot unparse data
    • #174 - Extended config validation
    • 📜 #180 - Switch from JSON parser to yajl-ruby
    • 👻 #181 - When responder is defined and not used due to respond_with not being triggered in the perform, it won't raise an exception.
    • #188 - Rename name in config to client id
    • #186 - Support ruby-kafka ssl_ca_cert_file_path config
    • #189 - karafka console does not preserve history on exit
    • 💎 #191 - Karafka 0.6.0rc1 does not work with jruby / now it does :-)
    • 📜 Switch to multi json so everyone can use their favourite JSON parser
    • ➕ Added jruby support in general and in Travis
    • #196 - Topic mapper does not map topics when subscribing thanks to @webandtech
    • #96 - Karafka server - possibility to run it only for a certain topics
    • ⏪ karafka worker cli option is removed (please use sidekiq directly) - restored, bad idea
    • (optional) pausing upon processing failures pause_timeout
    • Karafka console main process no longer intercepts irb errors
    • ⚡️ Wiki updates
    • #204 - Long running controllers
    • 👍 Better internal API to handle multiple usage cases using Karafka::Controllers::Includer
    • #207 - Rename before_enqueued to after_received
    • #147 - De-attach Karafka from Sidekiq by extracting Sidekiq backend

    🆕 New features and improvements

    • batch processing thanks to #batch_consuming flag and #params_batch on controllers
    • #topic method on an controller instance to make a clear distinction in between params and route details
    • 🔄 Changed routing model (still compatible with 0.5) to allow better resources management
    • Lower memory requirements due to object creation limitation (2-3 times less objects on each new message)
    • Introduced the #batch_consuming config flag (config for #126) that can be set per each consumer_group
    • ➕ Added support for partition, offset and partition key in the params hash
    • name option in config renamed to client_id
    • 0️⃣ Long running controllers with persistent flag on a topic config level, to make controller instances persistent between messages batches (single controller instance per topic per partition no per messages batch) - turned on by default

    Incompatibilities

    • 0️⃣ Default boot file is renamed from app.rb to karafka.rb
    • ✂ Removed worker glass as dependency (now and independent gem)
    • 👀 kafka.hosts option renamed to kafka.seed_brokers - you don't need to provide all the hosts to work with Kafka
    • start_from_beginning moved into kafka scope (kafka.start_from_beginning)
    • Router no longer checks for route uniqueness - now you can define same routes for multiple kafkas and do a lot of crazy stuff, so it's your responsibility to check uniqueness
    • ⬆️ Change in the way we identify topics in between Karafka and Sidekiq workers. If you upgrade, please make sure, all the jobs scheduled in Sidekiq are finished before the upgrade.
    • batch_mode renamed to batch_fetching
    • 💎 Renamed content to value to better resemble ruby-kafka internal messages naming convention
    • 👻 When having a responder with required topics and not using #respond_with at all, it will raise an exception
    • Renamed inline_mode to inline_processing to resemble other settings conventions
    • 📇 Renamed inline_processing to backend to reach 1.0 future compatibility
    • Single controller needs to be used for a single topic consumption
    • Renamed before_enqueue to after_received to better resemble internal logic, since for inline backend, there is no enqueue.
    • 👷 Due to the level on which topic and controller are related (class level), the dynamic worker selection is no longer available.
    • 📇 Renamed params #retrieve to params #retrieve! to better reflect what it does

    Other changes

    • 💅 PolishGeeksDevTools removed (in favour of Coditsu)
    • 👍 Waaaaaay better code quality thanks to switching from dev tools to Coditsu
    • Gem bump
    • Cleaner internal API
    • SRP
    • 👍 Better settings proxying and management between ruby-kafka and karafka
    • All internal validations are now powered by dry-validation
    • 👍 Better naming conventions to reflect Kafka reality
    • ✂ Removed Karafka::Connection::Message in favour of direct message details extraction from Kafka::FetchedMessage