All Versions
112
Latest Version
Avg Release Cycle
5 days
Latest Release
480 days ago

Changelog History
Page 1

  • v20.0.21 Changes

    November 25, 2022
    • ๐Ÿ‘ท [Improvement] Make revocation jobs for LRJ topics non-blocking to prevent blocking polling when someone uses non-revocation aware LRJ jobs and revocation happens.
  • v2.0.20 Changes

    November 24, 2022
    • ๐Ÿ‘ [Improvement] Support group.instance.id assignment (static group membership) for a case where a single consumer group has multiple subscription groups (#1173).
  • v2.0.19 Changes

    November 20, 2022
    • [Feature] Provide ability to skip failing messages without dispatching them to an alternative topic (DLQ).
    • ๐Ÿ’Ž [Improvement] Improve the integration with Ruby on Rails by preventing double-require of components.
    • ๐Ÿš‘ [Improvement] Improve stability of the shutdown process upon critical errors.
    • [Improvement] Improve stability of the integrations spec suite.
    • [Fix] Fix an issue where upon fast startup of multiple subscription groups from the same consumer group, a ghost queue would be created due to problems in Concurrent::Hash.
  • v2.0.18 Changes

    November 18, 2022
    • [Feature] Support quiet mode via TSTP signal. When used, Karafka will finish processing current messages, run shutdown jobs, and switch to a quiet mode where no new work is being accepted. At the same time, it will keep the consumer group quiet, and thus no rebalance will be triggered. This can be particularly useful during deployments.
    • ๐Ÿ‘ท [Improvement] Trigger #revoked for jobs in case revocation would happen during shutdown when jobs are still running. This should ensure, we get a notion of revocation for Pro LRJ jobs even when revocation happening upon shutdown (#1150).
    • [Improvement] Stabilize the shutdown procedure for consumer groups with many subscription groups that have non-aligned processing cost per batch.
    • ๐Ÿšš [Improvement] Remove double loading of Karafka via Rails railtie.
    • ๐Ÿ“„ [Fix] Fix invalid class references in YARD docs.
    • [Fix] prevent parallel closing of many clients.
    • [Fix] fix a case where information about revocation for a combination of LRJ + VP would not be dispatched until all VP work is done.
  • v2.0.17 Changes

    November 10, 2022
    • ๐Ÿ“‡ [Fix] Few typos around DLQ and Pro DLQ Dispatch original metadata naming.
    • [Fix] Narrow the components lookup to the appropriate scope (#1114)

    โฌ†๏ธ Upgrade notes

    ๐Ÿ“‡ 1. Replace original-* references from DLQ dispatched metadata with original_*

  • v2.0.16 Changes

    November 09, 2022
    • [Breaking] Disable the root manual_offset_management setting and require it to be configured per topic. This is part of "topic features" configuration extraction for better code organization.
    • [Feature] Introduce Dead Letter Queue feature and Pro Enhanced Dead Letter Queue feature
    • [Improvement] Align attributes available in the instrumentation bus for listener related events.
    • [Improvement] Include consumer group id in consumption related events (#1093)
    • [Improvement] Delegate pro components loading to Zeitwerk
    • [Improvement] Include Datadog::LoggerListener for tracking logger data with DataDog (@bruno-b-martins)
    • ๐Ÿ‘€ [Improvement] Include seek_offset in the consumer.consume.error event payload (#1113)
    • ๐Ÿ”จ [Refactor] Remove unused logger listener event handler.
    • ๐Ÿ”จ [Refactor] Internal refactoring of routing validations flow.
    • ๐Ÿ”จ [Refactor] Reorganize how routing related features are represented internally to simplify features management.
    • ๐Ÿ”จ [Refactor] Extract supported features combinations processing flow into separate strategies.
    • ๐Ÿ”จ [Refactor] Auto-create topics in the integration specs based on the defined routing
    • ๐Ÿ”จ [Refactor] Auto-inject Pro components via composition instead of requiring to use Karafka::Pro::BaseConsumer (#1116)
    • [Fix] Fix a case where routing tags would not be injected when given routing definition would not be used with a block
    • ๐Ÿ‘ท [Fix] Fix a case where using #active_job_topic without extra block options would cause manual_offset_management to stay false.
    • ๐Ÿ‘ท [Fix] Fix a case when upon Pro ActiveJob usage with Virtual Partitions, correct offset would not be stored
    • [Fix] Fix a case where upon Virtual Partitions usage, same underlying real partition would be resumed several times.
    • [Fix] Fix LRJ enqueuing pause increases the coordinator counter (#115)
    • ๐Ÿš€ [Fix] Release ActiveRecord connection to the pool after the work in non-dev envs (#1130)
    • [Fix] Fix a case where post-initialization shutdown would not initiate shutdown procedures.
    • [Fix] Prevent Karafka from committing offsets twice upon shutdown.
    • ๐Ÿšฆ [Fix] Fix for a case where fast consecutive stop signaling could hang the stopping listeners.
    • [Specs] Split specs into regular and pro to simplify how resources are loaded
    • [Specs] Add specs to ensure, that all the Pro components have a proper per-file license (#1099)

    โฌ†๏ธ Upgrade notes

    1. Remove the manual_offset_management setting from the main config if you use it:
    class KarafkaApp < Karafka::App
      setup do |config|
        # ...
    
        # This line needs to be removed:
        config.manual_offset_management = true
      end
    end
    
    1. Set the manual_offset_management feature flag per each topic where you want to use it in the routing. Don't set it for topics where you want the default offset management strategy to be used.
    class KarafkaApp < Karafka::App
      routes.draw do
        consumer_group :group_name do
          topic :example do
            consumer ExampleConsumer
            manual_offset_management true
          end
    
          topic :example2 do
            consumer ExampleConsumer2
            manual_offset_management true
          end
        end
      end
    end
    
    1. If you were using code to restart dead connections similar to this:
    class ActiveRecordConnectionsCleaner
      def on_error_occurred(event)
        return unless event[:error].is_a?(ActiveRecord::StatementInvalid)
    
        ::ActiveRecord::Base.clear_active_connections!
      end
    end
    
    Karafka.monitor.subscribe(ActiveRecordConnectionsCleaner.new)
    

    It should be removed. This code is no longer needed.

  • v2.0.15 Changes

    October 20, 2022
    • Sanitize admin config prior to any admin action.
    • ๐Ÿ‘‰ Make messages partitioner outcome for virtual partitions consistently distributed in regards to concurrency.
    • ๐Ÿ‘Œ Improve DataDog/StatsD metrics reporting by reporting per topic partition lags and trends.
    • ๐Ÿ”€ Replace synchronous offset commit with async on resuming paused partition (#1087).
  • v2.0.14 Changes

    October 16, 2022
    • ๐Ÿšฆ Prevent consecutive stop signals from starting multiple supervision shutdowns.
    • Provide Karafka::Embedded to simplify the start/stop process when running Karafka from within other process (Puma, Sidekiq, etc).
    • ๐Ÿ›  Fix a race condition when un-pausing a long-running-job exactly upon listener resuming would crash the listener loop (#1072).
  • v2.0.13 Changes

    October 14, 2022
    • Early exit upon attempts to commit current or earlier offset twice.
    • โž• Add more integration specs covering edge cases.
    • 0๏ธโƒฃ Strip non producer related config when default producer is initialized (#776)
  • v2.0.12 Changes

    October 06, 2022
    • Commit stored offsets upon rebalance revocation event to reduce number of messages that are re-processed.
    • ๐Ÿ‘Œ Support cooperative-sticky rebalance strategy.
    • Replace offset commit after each batch with a per-rebalance commit.
    • ๐Ÿ‘‰ User instrumentation to publish internal rebalance errors.