All Versions
112
Latest Version
Avg Release Cycle
5 days
Latest Release
655 days ago

Changelog History
Page 9

  • v1.0.0 Changes

    Closed issues:

    • #103 - Env for logger is loaded 2 early (on gem load not on app init)
    • 👍 #142 - Possibility to better control Kafka consumers (consumer groups management)
    • #150 - Add support for start_from_beginning on a per topic basis
    • #154 - Support for min_bytes and max_wait_time on messages consuming
    • 💎 #160 - Reorganize settings to better resemble ruby-kafka requirements
    • 🔧 #164 - If we decide to have configuration per topic, topic uniqueness should be removed
    • #165 - Router validator
    • #166 - Params and route reorganization (new API)
    • 🚚 #167 - Remove Sidekiq UI from Karafka
    • #168 - Introduce unique IDs of routes
    • 📇 #171 - Add kafka message metadata to params
    • #176 - Transform Karafka::Connection::Consumer into a module
    • #177 - Monitor not reacting when kafka killed with -9
    • #175 - Allow single consumer to subscribe to multiple topics
    • 🚚 #178 - Remove parsing failover when cannot unparse data
    • #174 - Extended config validation
    • 📜 #180 - Switch from JSON parser to yajl-ruby
    • 👻 #181 - When responder is defined and not used due to respond_with not being triggered in the perform, it won't raise an exception.
    • #188 - Rename name in config to client id
    • #186 - Support ruby-kafka ssl_ca_cert_file_path config
    • #189 - karafka console does not preserve history on exit
    • 💎 #191 - Karafka 0.6.0rc1 does not work with jruby / now it does :-)
    • 📜 Switch to multi json so everyone can use their favourite JSON parser
    • ➕ Added jruby support in general and in Travis
    • #196 - Topic mapper does not map topics when subscribing thanks to @webandtech
    • #96 - Karafka server - possibility to run it only for a certain topics
    • ⏪ karafka worker cli option is removed (please use sidekiq directly) - restored, bad idea
    • (optional) pausing upon processing failures pause_timeout
    • Karafka console main process no longer intercepts irb errors
    • ⚡️ Wiki updates
    • #204 - Long running controllers
    • 👍 Better internal API to handle multiple usage cases using Karafka::Controllers::Includer
    • #207 - Rename before_enqueued to after_received
    • #147 - De-attach Karafka from Sidekiq by extracting Sidekiq backend

    🆕 New features and improvements

    • batch processing thanks to #batch_consuming flag and #params_batch on controllers
    • #topic method on an controller instance to make a clear distinction in between params and route details
    • 🔄 Changed routing model (still compatible with 0.5) to allow better resources management
    • Lower memory requirements due to object creation limitation (2-3 times less objects on each new message)
    • Introduced the #batch_consuming config flag (config for #126) that can be set per each consumer_group
    • ➕ Added support for partition, offset and partition key in the params hash
    • name option in config renamed to client_id
    • 0️⃣ Long running controllers with persistent flag on a topic config level, to make controller instances persistent between messages batches (single controller instance per topic per partition no per messages batch) - turned on by default

    Incompatibilities

    • 0️⃣ Default boot file is renamed from app.rb to karafka.rb
    • ✂ Removed worker glass as dependency (now and independent gem)
    • 👀 kafka.hosts option renamed to kafka.seed_brokers - you don't need to provide all the hosts to work with Kafka
    • start_from_beginning moved into kafka scope (kafka.start_from_beginning)
    • Router no longer checks for route uniqueness - now you can define same routes for multiple kafkas and do a lot of crazy stuff, so it's your responsibility to check uniqueness
    • ⬆️ Change in the way we identify topics in between Karafka and Sidekiq workers. If you upgrade, please make sure, all the jobs scheduled in Sidekiq are finished before the upgrade.
    • batch_mode renamed to batch_fetching
    • 💎 Renamed content to value to better resemble ruby-kafka internal messages naming convention
    • 👻 When having a responder with required topics and not using #respond_with at all, it will raise an exception
    • Renamed inline_mode to inline_processing to resemble other settings conventions
    • 📇 Renamed inline_processing to backend to reach 1.0 future compatibility
    • Single controller needs to be used for a single topic consumption
    • Renamed before_enqueue to after_received to better resemble internal logic, since for inline backend, there is no enqueue.
    • 👷 Due to the level on which topic and controller are related (class level), the dynamic worker selection is no longer available.
    • 📇 Renamed params #retrieve to params #retrieve! to better reflect what it does

    Other changes

    • 💅 PolishGeeksDevTools removed (in favour of Coditsu)
    • 👍 Waaaaaay better code quality thanks to switching from dev tools to Coditsu
    • Gem bump
    • Cleaner internal API
    • SRP
    • 👍 Better settings proxying and management between ruby-kafka and karafka
    • All internal validations are now powered by dry-validation
    • 👍 Better naming conventions to reflect Kafka reality
    • ✂ Removed Karafka::Connection::Message in favour of direct message details extraction from Kafka::FetchedMessage

  • v0.5.0 Changes

    • ✂ Removed Zookeeper totally as dependency
    • 👍 Better group and partition rebalancing
    • Automatic thread management (no need for tunning) - each topic is a separate actor/thread
    • 🚚 Moved from Poseidon into Ruby-Kafka
    • No more max_concurrency setting
    • After you define your App class and routes (and everything else) you need to add execute App.boot!
    • Manual consuming is no longer available (no more karafka consume command)
    • Karafka topics CLI is no longer available. No Zookeeper - no global topic discovery
    • ⬇️ Dropped ZK as dependency
    • 🖨 karafka info command no longer prints details about Zookeeper
    • 👍 Better shutdown
    • No more autodiscovery via Zookeeper - instead, the whole cluster will be discovered directly from Kafka
    • 👍 No more support for Kafka 0.8
    • 👌 Support for Kafka 0.9
    • No more need for ActorCluster, since now we have a single thread (and Kafka connection) per topic
    • 💎 Ruby 2.2.* support dropped
    • Using App name as a Kafka client_id
    • Automatic Capistrano integration
    • 👀 Responders support for handling better responses pipelining and better responses flow description and design (see README for more details)
    • Gem bump
    • ⚡️ Readme updates
    • 🖨 karafka flow CLI command for printing the application flow
    • 🔨 Some internal refactorings

  • v0.5.0.3 Changes

    August 02, 2017
    • #132 - When Kafka is gone, should reconnect after a time period
    • 💎 #136 - new ruby-kafka version + other gem bumps
    • ⚡️ ruby-kafka update
    • 👍 #135 - NonMatchingRouteError - better error description in the code
    • 🚚 #140 - Move Capistrano Karafka to a different specific gem
    • 🏗 #110 - Add call method on a responder class to alias instance build and call
    • #76 - Configs validator
    • 👷 #138 - Possibility to have no worker class defined if inline_mode is being used
    • #145 - Topic Mapper
    • ⚡️ Ruby update to 2.4.1
    • Gem bump x2
    • ⚡️ #158 - Update docs section on heroku usage
    • #150 - Add support for start_from_beginning on a per topic basis
    • #148 - Lower Karafka Sidekiq dependency
    • 👍 Allow karafka root to be specified from ENV
    • 🚀 Handle SIGTERM as a shutdown command for kafka server to support Heroku deployment

  • v0.5.0.2 Changes

    • ⚡️ Gems update x3
    • 0️⃣ Default Ruby set to 2.3.3
    • 0️⃣ Default Ruby set to 2.4.0
    • ⚡️ Readme updates to match bug fixes and resolved issues
    • #95 - Allow options into responder
    • 📜 #98 - Use parser when responding on a topic
    • 🔧 #114 - Option to configure waterdrop connection pool timeout and concurrency
    • #118 - Added dot in topic validation format
    • 👍 #119 - add support for authentication using SSL
    • 0️⃣ #121 - JSON as a default for standalone responders usage
    • #122 - Allow on capistrano role customization
    • 👍 #125 - Add support to batch incoming messages
    • #130 - start_from_beginning flag on routes and default
    • #128 - Monitor caller_label not working with super on inheritance
    • Renamed inline to inline_mode to stay consistent with flags that change the way karafka works (#125)
    • 🔧 Dry-configurable bump to 0.5 with fixed proc value evaluation on retrieve patch (internal change)

  • v0.5.0.1 Changes

    • 🛠 Fixed inconsistency in responders non-required topic definition. Now only required: false available
    • #101 - Responders fail when multiple_usage true and required false
    • 🛠 fix error on startup from waterdrop #102
    • Waterdrop 0.3.2.1 with kafka.hosts instead of kafka_hosts
    • #105 - Karafka::Monitor#caller_label not working with inherited monitors
    • #99 - Standalone mode (without Sidekiq)
    • #97 - Buffer responders single topics before send (pre-validation)
    • 👍 Better control over consumer thanks to additional config options
    • 👷 #111 - Dynamic worker assignment based on the income params
    • Long shutdown time fix

  • v0.4.2 Changes

    • 🚅 #87 - Re-consume mode with crone for better Rails/Rack integration
    • 🚚 Moved Karafka server related stuff into separate Karafka::Server class
    • 📇 Renamed Karafka::Runner into Karafka::Fetcher
    • Gem bump
    • ➕ Added chroot option to Zookeeper options
    • 🚚 Moved BROKERS_PATH into config from constant
    • ➕ Added Karafka consume CLI action for a short running single consumption round
    • 🛠 Small fixes to close broken connections
    • ⚡️ Readme updates

  • v0.4.1 Changes

    • 🚅 Explicit throw(:abort) required to halt before_enqueue (like in Rails 5)
    • #61 - autodiscovery of Kafka brokers based on Zookeeper data
    • #63 - Graceful shutdown with current offset state during data processing
    • #65 - Example of NewRelic monitor is outdated
    • #71 - Setup should be executed after user code is loaded
    • Gem bump x3
    • Rubocop remarks
    • 🚚 worker_timeout config option has been removed. It now needs to be defined manually by the framework user because WorkerGlass::Timeout can be disabled and we cannot use Karafka settings on a class level to initialize user code stuff
    • 🚚 Moved setup logic under setup/Setup namespace
    • 👍 Better defaults handling
    • #75 - Kafka and Zookeeper options as a hash
    • #82 - Karafka autodiscovery fails upon caching of configs
    • 🔧 #81 - Switch config management to dry configurable
    • 🔖 Version fix
    • ⬇️ Dropped support for Ruby 2.1.*
    • 💎 Ruby bump to 2.3.1

  • v0.4.0 Changes

    • ➕ Added WaterDrop gem with default configuration
    • 🔨 Refactoring of config logic to simplify adding new dependencies that need to be configured based on #setup data
    • Gem bump
    • ⚡️ Readme updates
    • 📇 Renamed cluster to actor_cluster for method names
    • 👷 Replaced SidekiqGlass with generic WorkerGlass lib
    • Application bootstrap in app.rb no longer required
    • ⚡️ Karafka.boot needs to be executed after all the application files are loaded (template updated)
    • 🔨 Small loader refactor (no API changes)
    • 0️⃣ Ruby 2.3.0 support (default)
    • No more rake tasks
    • Karafka CLI instead of rake tasks
    • 👷 Worker cli command allows passing additional options directly to Sidekiq
    • 📇 Renamed concurrency to max_concurrency - it describes better what happens - Karafka will use this number of threads only when required
    • ➕ Added wait_timeout that allows us to tune how long should we wait on a single socket connection (single topic) for new messages before going to next one (this applies to each thread separately)
    • Rubocop remarks
    • ✂ Removed Sinatra and Puma dependencies
    • Karafka Cli internal reorganization
    • Karafka Cli routes task
    • 📜 #37 - warn log for failed parsing of a message
    • #43 - wrong constant name
    • #44 - Method name conflict
    • #48 - Cannot load such file -- celluloid/current
    • #46 - Loading application
    • #45 - Set up monitor in config
    • #47 - rake karafka:run uses app.rb only
    • ⚡️ #53 - README update with Sinatra/Rails integration description
    • #41 - New Routing engine
    • 🚚 #54 - Move Karafka::Workers::BaseWorker to Karafka::BaseWorker
    • 👷 #55 - ApplicationController and ApplicationWorker

  • v0.3.2 Changes

    • 🔀 Karafka::Params::Params lazy load merge keys with string/symbol names priorities fix

  • v0.3.1 Changes

    • 📇 Renamed Karafka::Monitor to Karafka::Process to represent a Karafka process wrapper
    • ➕ Added Karafka::Monitoring that allows to add custom logging and monitoring with external libraries and systems
    • 🚚 Moved logging functionality into Karafka::Monitoring default monitoring
    • ➕ Added possibility to provide own monitoring as long as in responds to #notice and #notice_error
    • 🔊 Standardized logging format for all logs