Consumer removal can race with ACK completion, redelivery, or mark-delete cleanup, causing subscription and broker unacked counters to become inconsistent. Broker throttling also has lock-order and registration races that can block consumer removal or leave a dispatcher blocked after its unacked count has fallen below the low watermark.
These races can disrupt flow control and prevent subscriptions from resuming delivery.
Modifications
Serialize each consumer's unacked accounting with removal, settle its remaining balance once, and reject subsequent accounting updates after removal.
Apply the settlement lifecycle to both modern and classic shared dispatchers.
Resolve subscriptions before acquiring the broker unacked lock and schedule resumed reads after releasing it.
Serialize broker blocking transitions with dispatcher registration, and recheck the low watermark after registration to cancel a late block.
Coalesce eligible whole-entry ACK completions by consecutive consumer owner, using the actual removed message count. Preserve the original path for single-entry, batch-index, and transaction-enabled processing, and flush previously returned deltas if a later removal fails.
Extend the existing accounting test class with deterministic removal, blocking, batched-entry ACK, and Key_Shared ownership-transfer coverage.
Add ConsumerUnackedMessagesBenchmark under microbench/ to measure accounting contention and grouped ACK completion.
Verifying this change
Make sure that the change passes the CI checks.
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
Added integration tests for end-to-end deployment with large payloads (10MB)
Extended integration test for recovery after broker failure
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Consumer removal can race with ACK completion, redelivery, or mark-delete cleanup, causing subscription and broker unacked counters to become inconsistent. Broker throttling also has lock-order and registration races that can block consumer removal or leave a dispatcher blocked after its unacked count has fallen below the low watermark.
These races can disrupt flow control and prevent subscriptions from resuming delivery.
Modifications
Verifying this change
(Please pick either of the following options)
This change is a trivial rework / code cleanup without any test coverage.
(or)
This change is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes