All Versions
112
Latest Version
Avg Release Cycle
5 days
Latest Release
118 days ago

Changelog History
Page 1

  • v20.0.21 Changes

    November 25, 2022
    • 👷 [Improvement] Make revocation jobs for LRJ topics non-blocking to prevent blocking polling when someone uses non-revocation aware LRJ jobs and revocation happens.

  • v2.0.20 Changes

    November 24, 2022
    • 👍 [Improvement] Support group.instance.id assignment (static group membership) for a case where a single consumer group has multiple subscription groups (#1173).

  • v2.0.19 Changes

    November 20, 2022
    • [Feature] Provide ability to skip failing messages without dispatching them to an alternative topic (DLQ).
    • 💎 [Improvement] Improve the integration with Ruby on Rails by preventing double-require of components.
    • 🚑 [Improvement] Improve stability of the shutdown process upon critical errors.
    • [Improvement] Improve stability of the integrations spec suite.
    • [Fix] Fix an issue where upon fast startup of multiple subscription groups from the same consumer group, a ghost queue would be created due to problems in Concurrent::Hash.

  • v2.0.18 Changes

    November 18, 2022
    • [Feature] Support quiet mode via TSTP signal. When used, Karafka will finish processing current messages, run shutdown jobs, and switch to a quiet mode where no new work is being accepted. At the same time, it will keep the consumer group quiet, and thus no rebalance will be triggered. This can be particularly useful during deployments.
    • 👷 [Improvement] Trigger #revoked for jobs in case revocation would happen during shutdown when jobs are still running. This should ensure, we get a notion of revocation for Pro LRJ jobs even when revocation happening upon shutdown (#1150).
    • [Improvement] Stabilize the shutdown procedure for consumer groups with many subscription groups that have non-aligned processing cost per batch.
    • 🚚 [Improvement] Remove double loading of Karafka via Rails railtie.
    • 📄 [Fix] Fix invalid class references in YARD docs.
    • [Fix] prevent parallel closing of many clients.
    • [Fix] fix a case where information about revocation for a combination of LRJ + VP would not be dispatched until all VP work is done.

  • v2.0.17 Changes

    November 10, 2022
    • 📇 [Fix] Few typos around DLQ and Pro DLQ Dispatch original metadata naming.
    • [Fix] Narrow the components lookup to the appropriate scope (#1114)

    ⬆️ Upgrade notes

    📇 1. Replace original-* references from DLQ dispatched metadata with original_*

  • v2.0.16 Changes

    November 09, 2022
    • [Breaking] Disable the root manual_offset_management setting and require it to be configured per topic. This is part of "topic features" configuration extraction for better code organization.
    • [Feature] Introduce Dead Letter Queue feature and Pro Enhanced Dead Letter Queue feature
    • [Improvement] Align attributes available in the instrumentation bus for listener related events.
    • [Improvement] Include consumer group id in consumption related events (#1093)
    • [Improvement] Delegate pro components loading to Zeitwerk
    • [Improvement] Include Datadog::LoggerListener for tracking logger data with DataDog (@bruno-b-martins)
    • 👀 [Improvement] Include seek_offset in the consumer.consume.error event payload (#1113)
    • 🔨 [Refactor] Remove unused logger listener event handler.
    • 🔨 [Refactor] Internal refactoring of routing validations flow.
    • 🔨 [Refactor] Reorganize how routing related features are represented internally to simplify features management.
    • 🔨 [Refactor] Extract supported features combinations processing flow into separate strategies.
    • 🔨 [Refactor] Auto-create topics in the integration specs based on the defined routing
    • 🔨 [Refactor] Auto-inject Pro components via composition instead of requiring to use Karafka::Pro::BaseConsumer (#1116)
    • [Fix] Fix a case where routing tags would not be injected when given routing definition would not be used with a block
    • 👷 [Fix] Fix a case where using #active_job_topic without extra block options would cause manual_offset_management to stay false.
    • 👷 [Fix] Fix a case when upon Pro ActiveJob usage with Virtual Partitions, correct offset would not be stored
    • [Fix] Fix a case where upon Virtual Partitions usage, same underlying real partition would be resumed several times.
    • [Fix] Fix LRJ enqueuing pause increases the coordinator counter (#115)
    • 🚀 [Fix] Release ActiveRecord connection to the pool after the work in non-dev envs (#1130)
    • [Fix] Fix a case where post-initialization shutdown would not initiate shutdown procedures.
    • [Fix] Prevent Karafka from committing offsets twice upon shutdown.
    • 🚦 [Fix] Fix for a case where fast consecutive stop signaling could hang the stopping listeners.
    • [Specs] Split specs into regular and pro to simplify how resources are loaded
    • [Specs] Add specs to ensure, that all the Pro components have a proper per-file license (#1099)

    ⬆️ Upgrade notes

    1. Remove the manual_offset_management setting from the main config if you use it:
    class KarafkaApp < Karafka::App
  setup do |config|
    # ...

    # This line needs to be removed:
    config.manual_offset_management = true
  end
end
    1. Set the manual_offset_management feature flag per each topic where you want to use it in the routing. Don't set it for topics where you want the default offset management strategy to be used.
    class KarafkaApp < Karafka::App
  routes.draw do
    consumer_group :group_name do
      topic :example do
        consumer ExampleConsumer
        manual_offset_management true
      end

      topic :example2 do
        consumer ExampleConsumer2
        manual_offset_management true
      end
    end
  end
end
    1. If you were using code to restart dead connections similar to this:
    class ActiveRecordConnectionsCleaner
  def on_error_occurred(event)
    return unless event[:error].is_a?(ActiveRecord::StatementInvalid)

    ::ActiveRecord::Base.clear_active_connections!
  end
end

Karafka.monitor.subscribe(ActiveRecordConnectionsCleaner.new)

    It should be removed. This code is no longer needed.

  • v2.0.15 Changes

    October 20, 2022
    • Sanitize admin config prior to any admin action.
    • 👉 Make messages partitioner outcome for virtual partitions consistently distributed in regards to concurrency.
    • 👌 Improve DataDog/StatsD metrics reporting by reporting per topic partition lags and trends.
    • 🔀 Replace synchronous offset commit with async on resuming paused partition (#1087).

  • v2.0.14 Changes

    October 16, 2022
    • 🚦 Prevent consecutive stop signals from starting multiple supervision shutdowns.
    • Provide Karafka::Embedded to simplify the start/stop process when running Karafka from within other process (Puma, Sidekiq, etc).
    • 🛠 Fix a race condition when un-pausing a long-running-job exactly upon listener resuming would crash the listener loop (#1072).

  • v2.0.13 Changes

    October 14, 2022
    • Early exit upon attempts to commit current or earlier offset twice.
    • ➕ Add more integration specs covering edge cases.
    • 0️⃣ Strip non producer related config when default producer is initialized (#776)

  • v2.0.12 Changes

    October 06, 2022
    • Commit stored offsets upon rebalance revocation event to reduce number of messages that are re-processed.
    • 👌 Support cooperative-sticky rebalance strategy.
    • Replace offset commit after each batch with a per-rebalance commit.
    • 👉 User instrumentation to publish internal rebalance errors.