All Versions
112
Latest Version
Avg Release Cycle
5 days
Latest Release
118 days ago
Changelog History
Page 1
Changelog History
Page 1
-
v20.0.21 Changes
November 25, 2022- ๐ท [Improvement] Make revocation jobs for LRJ topics non-blocking to prevent blocking polling when someone uses non-revocation aware LRJ jobs and revocation happens.
-
v2.0.20 Changes
November 24, 2022- ๐ [Improvement] Support
group.instance.id
assignment (static group membership) for a case where a single consumer group has multiple subscription groups (#1173).
- ๐ [Improvement] Support
-
v2.0.19 Changes
November 20, 2022- [Feature] Provide ability to skip failing messages without dispatching them to an alternative topic (DLQ).
- ๐ [Improvement] Improve the integration with Ruby on Rails by preventing double-require of components.
- ๐ [Improvement] Improve stability of the shutdown process upon critical errors.
- [Improvement] Improve stability of the integrations spec suite.
- [Fix] Fix an issue where upon fast startup of multiple subscription groups from the same consumer group, a ghost queue would be created due to problems in
Concurrent::Hash
.
-
v2.0.18 Changes
November 18, 2022- [Feature] Support quiet mode via
TSTP
signal. When used, Karafka will finish processing current messages, runshutdown
jobs, and switch to a quiet mode where no new work is being accepted. At the same time, it will keep the consumer group quiet, and thus no rebalance will be triggered. This can be particularly useful during deployments. - ๐ท [Improvement] Trigger
#revoked
for jobs in case revocation would happen during shutdown when jobs are still running. This should ensure, we get a notion of revocation for Pro LRJ jobs even when revocation happening upon shutdown (#1150). - [Improvement] Stabilize the shutdown procedure for consumer groups with many subscription groups that have non-aligned processing cost per batch.
- ๐ [Improvement] Remove double loading of Karafka via Rails railtie.
- ๐ [Fix] Fix invalid class references in YARD docs.
- [Fix] prevent parallel closing of many clients.
- [Fix] fix a case where information about revocation for a combination of LRJ + VP would not be dispatched until all VP work is done.
- [Feature] Support quiet mode via
-
v2.0.17 Changes
November 10, 2022- ๐ [Fix] Few typos around DLQ and Pro DLQ Dispatch original metadata naming.
- [Fix] Narrow the components lookup to the appropriate scope (#1114)
โฌ๏ธ Upgrade notes
๐ 1. Replace
original-*
references from DLQ dispatched metadata withoriginal_*
-
v2.0.16 Changes
November 09, 2022- [Breaking] Disable the root
manual_offset_management
setting and require it to be configured per topic. This is part of "topic features" configuration extraction for better code organization. - [Feature] Introduce Dead Letter Queue feature and Pro Enhanced Dead Letter Queue feature
- [Improvement] Align attributes available in the instrumentation bus for listener related events.
- [Improvement] Include consumer group id in consumption related events (#1093)
- [Improvement] Delegate pro components loading to Zeitwerk
- [Improvement] Include
Datadog::LoggerListener
for tracking logger data with DataDog (@bruno-b-martins) - ๐ [Improvement] Include
seek_offset
in theconsumer.consume.error
event payload (#1113) - ๐จ [Refactor] Remove unused logger listener event handler.
- ๐จ [Refactor] Internal refactoring of routing validations flow.
- ๐จ [Refactor] Reorganize how routing related features are represented internally to simplify features management.
- ๐จ [Refactor] Extract supported features combinations processing flow into separate strategies.
- ๐จ [Refactor] Auto-create topics in the integration specs based on the defined routing
- ๐จ [Refactor] Auto-inject Pro components via composition instead of requiring to use
Karafka::Pro::BaseConsumer
(#1116) - [Fix] Fix a case where routing tags would not be injected when given routing definition would not be used with a block
- ๐ท [Fix] Fix a case where using
#active_job_topic
without extra block options would causemanual_offset_management
to stay false. - ๐ท [Fix] Fix a case when upon Pro ActiveJob usage with Virtual Partitions, correct offset would not be stored
- [Fix] Fix a case where upon Virtual Partitions usage, same underlying real partition would be resumed several times.
- [Fix] Fix LRJ enqueuing pause increases the coordinator counter (#115)
- ๐ [Fix] Release
ActiveRecord
connection to the pool after the work in non-dev envs (#1130) - [Fix] Fix a case where post-initialization shutdown would not initiate shutdown procedures.
- [Fix] Prevent Karafka from committing offsets twice upon shutdown.
- ๐ฆ [Fix] Fix for a case where fast consecutive stop signaling could hang the stopping listeners.
- [Specs] Split specs into regular and pro to simplify how resources are loaded
- [Specs] Add specs to ensure, that all the Pro components have a proper per-file license (#1099)
โฌ๏ธ Upgrade notes
- Remove the
manual_offset_management
setting from the main config if you use it:
class KarafkaApp < Karafka::App setup do |config| # ... # This line needs to be removed: config.manual_offset_management = true end end
- Set the
manual_offset_management
feature flag per each topic where you want to use it in the routing. Don't set it for topics where you want the default offset management strategy to be used.
class KarafkaApp < Karafka::App routes.draw do consumer_group :group_name do topic :example do consumer ExampleConsumer manual_offset_management true end topic :example2 do consumer ExampleConsumer2 manual_offset_management true end end end end
- If you were using code to restart dead connections similar to this:
class ActiveRecordConnectionsCleaner def on_error_occurred(event) return unless event[:error].is_a?(ActiveRecord::StatementInvalid) ::ActiveRecord::Base.clear_active_connections! end end Karafka.monitor.subscribe(ActiveRecordConnectionsCleaner.new)
It should be removed. This code is no longer needed.
- [Breaking] Disable the root
-
v2.0.15 Changes
October 20, 2022- Sanitize admin config prior to any admin action.
- ๐ Make messages partitioner outcome for virtual partitions consistently distributed in regards to concurrency.
- ๐ Improve DataDog/StatsD metrics reporting by reporting per topic partition lags and trends.
- ๐ Replace synchronous offset commit with async on resuming paused partition (#1087).
-
v2.0.14 Changes
October 16, 2022- ๐ฆ Prevent consecutive stop signals from starting multiple supervision shutdowns.
- Provide
Karafka::Embedded
to simplify the start/stop process when running Karafka from within other process (Puma, Sidekiq, etc). - ๐ Fix a race condition when un-pausing a long-running-job exactly upon listener resuming would crash the listener loop (#1072).
-
v2.0.13 Changes
October 14, 2022- Early exit upon attempts to commit current or earlier offset twice.
- โ Add more integration specs covering edge cases.
- 0๏ธโฃ Strip non producer related config when default producer is initialized (#776)
-
v2.0.12 Changes
October 06, 2022- Commit stored offsets upon rebalance revocation event to reduce number of messages that are re-processed.
- ๐ Support cooperative-sticky rebalance strategy.
- Replace offset commit after each batch with a per-rebalance commit.
- ๐ User instrumentation to publish internal rebalance errors.