All Versions
112
Latest Version
Avg Release Cycle
5 days
Latest Release
848 days ago

Changelog History
Page 4

  • v2.0.0-beta3 Changes

    June 14, 2022
    • 🏗 Jobs building responsibility extracted out of the listener code base.
    • 🛠 Fix a case where specs supervisor would try to kill no longer running process (#868)
    • 🛠 Fix an instable integration spec that could misbehave under load
    • ✅ Commit offsets prior to pausing partitions to ensure that the latest offset is always committed
    • 🛠 Fix a case where consecutive CTRL+C (non-stop) would case an exception during forced shutdown
    • ➕ Add missing consumer.prepared.error into LoggerListener
    • Delegate partition resuming from the consumers to listeners threads.
    • ➕ Add support for Long Running Jobs (LRJ) for ActiveJob [PRO]
    • ➕ Add support for Long Running Jobs for consumers [PRO]
    • 👷 Allow active_job_topic to accept a block for extra topic related settings
    • ✂ Remove no longer needed logger threads
    • Auto-adapt number of processes for integration specs based on the number of CPUs
    • 🖨 Introduce an integration spec runner that prints everything to stdout (better for development)
    • 👷 Introduce extra integration specs for various ActiveJob usage scenarios
    • 📇 Rename consumer method #prepared to #prepare to reflect better its use-case
    • ✅ For test and dev raise an error when expired license key is used (never for non dev)
    • ➕ Add worker related monitor events (worker.process and worker.processed)
    • ⚡️ Update LoggerListener to include more useful information about processing and polling messages

  • v2.0.0-beta2 Changes

    June 07, 2022
    • Abstract away notion of topics groups (until now it was just an array)
    • ⚡️ Optimize how jobs queue is closed. Since we enqueue jobs only from the listeners, we can safely close jobs queue once listeners are done. By extracting this responsibility from listeners, we remove corner cases and race conditions. Note here: for non-blocking jobs we do wait for them to finish while running the poll. This ensures, that for async jobs that are long-living, we do not reach max.poll.interval.
    • 👷 Shutdown jobs are executed in workers to align all the jobs behaviours.
    • 👷 Shutdown jobs are always blocking.
    • 👷 Notion of ListenersBatch was introduced similar to WorkersBatch to abstract this concept.
    • ⏱ Change default shutdown_timeout to be more than max_wait_time not to cause forced shutdown when no messages are being received from Kafka.
    • ⏱ Abstract away scheduling of revocation and shutdown jobs for both default and pro schedulers
    • Introduce a second (internal) messages buffer to distinguish between raw messages buffer and karafka messages buffer
    • 📇 Move messages and their metadata remap process to the listener thread to allow for their inline usage
    • 🔄 Change how we wait in the shutdown phase, so shutdown jobs can still use Kafka connection even if they run for a longer period of time. This will prevent us from being kicked out from the group early.
    • ⏱ Introduce validation that ensures, that shutdown_timeout is more than max_wait_time. This will prevent users from ending up with a config that could lead to frequent forceful shutdowns.

  • v2.0.0-beta1 Changes

    May 22, 2022
    • ⚡️ Update the jobs queue blocking engine and allow for non-blocking jobs execution
    • Provide #prepared hook that always runs before the fetching loop is unblocked
    • 🐎 [Pro] Introduce performance tracker for scheduling optimizer
    • Provide ability to pause (#pause) and resume (#resume) given partitions from the consumers
    • 🔨 Small integration specs refactoring + specs for pausing scenarios

  • v2.0.0-alpha6 Changes

    April 17, 2022
    • 🛠 Fix a bug, where upon missing boot file and Rails, railtie would fail with a generic exception (#818)
    • 🛠 Fix an issue with parallel pristine specs colliding with each other during bundle install (#820)
    • Replace consumer.consume with consumer.consumed event to match the behaviour
    • 👉 Make sure, that offset committing happens before the consumer.consumed event is propagated
    • 🛠 Fix for failing when not installed (just a dependency) (#817)
    • Evict messages from partitions that were lost upon rebalancing (#825)
    • Do not run #revoked on partitions that were lost and assigned back upon rebalancing (#825)
    • ✂ Remove potential duplicated that could occur upon rebalance with re-assigned partitions (#825)
    • ⚡️ Optimize integration test suite additional consumers shutdown process (#828)
    • ⚡️ Optimize messages eviction and duplicates removal on poll stopped due to lack of messages
    • ➕ Add static group membership integration spec

  • v2.0.0-alpha5 Changes

    April 03, 2022
    • 📇 Rename StdoutListener to LoggerListener (#811)

  • v2.0.0-alpha4 Changes

    March 20, 2022
    • 👷 Rails support without ActiveJob queue adapter usage (#805)

  • v2.0.0-alpha3 Changes

    March 16, 2022
    • ⏪ Restore 'app.initialized' state and add notification on it
    • 🛠 Fix the installation flow for Rails and add integration tests for this scenario
    • ➕ Add more integration tests covering some edge cases

  • v2.0.0-alpha2 Changes

    February 19, 2022
    • Require kafka keys to be symbols
    • 👷 [Pro] Added ActiveJob Pro adapter
    • ⚡️ Small updates to the license and docs

  • v2.0.0-alpha1 Changes

    January 30, 2022
    • 🔄 Change license to LGPL-3.0
    • [Pro] Introduce a Pro subscription
    • 💎 Switch from ruby-kafka to librdkafka as an underlying driver
    • ✅ Introduce fully automatic integration tests that go through the whole server lifecycle
    • ↔ Integrate WaterDrop tightly with autoconfiguration inheritance and an option to redefine it
    • 👷 Multi-threaded support for concurrent jobs consumption (when in separate topics and/or partitions)
    • 👍 Introduce subscriptions groups concept for better resources management
    • ✂ Remove completely all the callbacks in favour of finalizer method #on_shutdown
    • Provide on_revoked method for taking actions upon topic revoke
    • ✂ Remove single message consumption mode in favour of documentation on how to do it easily by yourself
    • 🔀 Provide sync and async offset management with async preferred
    • 💎 Introduce seamless Ruby on Rails integration via Rails::Railte
    • ⚡️ Update cli info to reflect the 2.0 details
    • ✂ Remove responders in favour of WaterDrop 2.0 producer
    • ✂ Remove pidfiles support
    • ✂ Remove daemonization support
    • 🔧 Stop validating kafka configuration beyond minimum as it is handled by librdkafka
    • ✂ Remove topics mappers concept
    • Reorganize monitoring to match new concepts
    • 👷 Notify on fatal worker processing errors
    • 📇 Rename Karafka::Params::BatchMetadata to Karafka::Messages::BatchMetadata
    • 📇 Rename Karafka::Params::Params to Karafka::Messages::Message
    • 📇 Rename #params_batch in consumers to #messages
    • 📇 Rename Karafka::Params::Metadata to Karafka::Messages::Metadata
    • 👍 Allow for processing work of multiple consumer groups by the same worker poll
    • 📇 Rename Karafka::Fetcher to Karafka::Runner and align notifications key names
    • ⚡️ Update install templates
    • 👍 sidekiq-backend is no longer supported
    • ⚡️ testing gem for RSpec has been updated
    • 👍 WaterDrop 2.1+ support
    • 💅 Simple routing style (0.5) now builds a single consumer group instead of one per topic
    • ⚡️ Example apps were updated
    • Hook for underlying statistics emitted from librdkafka have been added
    • Hook for underlying async errors emitted from librdkafka have been added
    • 👷 ActiveJob Rails adapter
    • ➕ Added benchmarks that can be used to profile Karafka
    • Standardize error hook for all error reporting

  • v1.4.13 Changes

    February 19, 2022
    • ⬇️ Drop support for ruby 2.6
    • ➕ Add mfa requirement