Skip to content

Fix producer flush hangs when undrained batches expire - #3165

Open
zhechen-cs wants to merge 1 commit into
dpkp:masterfrom
zhechen-cs:fix-producer-expiry-unmute
Open

zhechen-cs wants to merge 1 commit into
dpkp:masterfrom
zhechen-cs:fix-producer-expiry-unmute

Conversation

@zhechen-cs

Copy link
Copy Markdown

Completing an undrained expired batch with message ordering enabled removes a partition that was never muted. The resulting KeyError interrupts expiry processing, leaving other batches' futures unresolved and causing flush() to wait indefinitely.

Use discard() when unmuting partitions in both completion paths. Regression tests cover multiple undrained expired batches, late success/error responses after expiry, and normal successful completion with message ordering enabled or disabled.

Fixes #3159.

Validation:

  • The three regression cases fail with KeyError on unmodified master and pass with the fix.
  • pytest -q test --ignore=test/integration: 2743 passed, 1 skipped (Python 3.12.13).
  • make lint and git diff --check passed.

Kafka broker integration tests were not run locally.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Sender: KeyError from muted.remove() while expiring batches orphans the rest; flush() hangs forever

1 participant