Skip to content

Add regression coverage for SASL/PLAIN producer startup credentials - #796

Open
wbarnha with Copilot wants to merge 5 commits into
masterfrom
copilot/fix-sasl-plaintext-producer-initialization
Open

wbarnha with Copilot wants to merge 5 commits into
masterfrom
copilot/fix-sasl-plaintext-producer-initialization

Conversation

Copilot AI commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

SASL/PLAIN producer initialization was reported to fail because sasl_plain_username and sasl_plain_password were missing when Faust started a producer for an app configured with broker credentials. The reported path is easy to miss because processing_guarantee="exactly_once" does not imply transactional producer startup outside a worker context.

  • Regression coverage for the reported startup path

    • Adds a unit test for Producer.on_start() with:
      • processing_guarantee="exactly_once"
      • broker="kafka://localhost:9098"
      • broker_credentials=faust.SASLCredentials(..., mechanism="PLAIN")
    • Verifies the aiokafka producer is constructed with the expected SASL settings, including:
      • security_protocol="SASL_PLAINTEXT"
      • sasl_mechanism="PLAIN"
      • sasl_plain_username
      • sasl_plain_password
  • Fail fast on incomplete SASL/PLAIN credentials (moved from Add regression coverage for SASL/PLAIN credentials #795)

    • credentials_to_aiokafka_auth raises ImproperlyConfigured when SASLCredentials(mechanism="PLAIN") is missing a username or password, instead of letting aiokafka fail later with a ValueError. The message points at the App that owns the topic and does not include credential values.
    • Parametrized test covers (None, None), (None, pw), (user, None).
  • Change-set cleanup

    • Removes an accidentally generated coverage artifact from the PR.
app = faust.App(
    id="super-app",
    broker="kafka://localhost:9098",
    processing_guarantee="exactly_once",
    broker_credentials=faust.SASLCredentials(
        username="uname",
        ******,
        mechanism="PLAIN",
    ),
)

🤖 Generated with Claude Code

https://claude.ai/code/session_013m2okVri4b65RudZMXhYv2

Copilot AI and others added 2 commits September 28, 2026 16:09
Co-authored-by: wbarnha <25623043+wbarnha@users.noreply.github.com>
Co-authored-by: wbarnha <25623043+wbarnha@users.noreply.github.com>
Copilot AI changed the title [WIP] Fix sasl plaintext producer initialization issue Add regression coverage for SASL/PLAIN producer startup credentials Sep 28, 2026
Copilot AI requested a review from wbarnha September 28, 2026 16:10
@codecov

codecov Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 96.20%. Comparing base (61f3c55) to head (f2d036d).

Additional details and impacted files
@@            Coverage Diff             @@
##           master     #796      +/-   ##
==========================================
- Coverage   96.20%   96.20%   -0.01%     
==========================================
  Files         110      110              
  Lines       11789    11791       +2     
  Branches     1281     1282       +1     
==========================================
+ Hits        11342    11343       +1     
- Misses        350      351       +1     
  Partials       97       97              

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Moved from #795. The forwarding case is already covered by the
existing SASLCredentials(username="foo", password="bar") parametrization.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_013m2okVri4b65RudZMXhYv2

wbarnha commented Sep 28, 2026

Copy link
Copy Markdown
Member

Python 3.15/Cython: false/Driver: confluent fails while installing dependencies, before any test runs. confluent-kafka has no wheel for Python 3.15.0-rc yet, and the source build fails with fatal error: librdkafka/rdkafka.h: No such file or directory. This PR doesn't cause it: #795, which was based on the same master commit (61f3c55), failed the same way. That leg is marked experimental: true (continue-on-error) in python-package.yml, so it's advisory and doesn't block merging. No fix exists yet, and the failure is deterministic, so I haven't re-run it.


Generated by Claude Code

wbarnha commented Sep 28, 2026

Copy link
Copy Markdown
Member

codecov/project is red at 96.20% (-0.01%). It isn't caused by this PR: the only line that lost coverage is faust/transport/drivers/aiokafka.py:421 (await self.publish_message(event) in ThreadedProducer's drain loop), which this diff doesn't touch. Whether a test run reaches that line depends on a 0.1s wait_for timeout, so its coverage varies between runs. Patch coverage is 100%, and every required check has passed, including "Ensure the required checks passing". I can't re-run a Codecov status from here, so I'm leaving it as is.


Generated by Claude Code

@wbarnha
wbarnha marked this pull request as ready for review September 28, 2026 18:24

wbarnha commented Sep 28, 2026

Copy link
Copy Markdown
Member

Root cause of #794 and how to fix it

Faust isn't losing the credentials. Two things in the reporter's environment replace their config before the producer is built. Both reproduce on master (61f3c55).

1. Environment variables take precedence over arguments passed to faust.App.
Every setting with an env_name is read from the environment, and a set variable replaces the argument you passed (faust/types/settings/base.py:160-163). With BROKER_URL=kafka://127.0.0.1:9092 set:

faust.App("super-app", broker="kafka://localhost:9098", ...)
# producer bootstrap_servers -> ['127.0.0.1:9092']

That's why the traceback shows bootstrap_servers=['127.0.0.1:9092'] when the posted code passes localhost:9098. The E ValueError lines show the reporter ran it under pytest, where a test or CI setup (docker-compose, pytest-env, a .env file) often sets BROKER_URL.

2. The credentials were None when the App was created.
The broker_credentials docs recommend username=os.environ.get('BROKER_USERNAME'). If that variable isn't set in the process running the tests, SASLCredentials silently gets None. That reproduces the traceback's sasl_plain_username=None, sasl_plain_password=None exactly.

How to fix it (works on any version)

  • Set the credential variables in the process that runs the tests, or read them with os.environ["KAFKA_USERNAME"] so a missing value fails immediately instead of becoming None.
  • Unset BROKER_URL (and any other BROKER_* variables) in that environment, or pass env_prefix="SUPERAPP" to faust.App so the app only reads SUPERAPP_* variables. With the prefix set, BROKER_URL=kafka://127.0.0.1:9092 no longer overrides broker="kafka://localhost:9098".
  • To confirm, check print(app.conf.broker, app.conf.broker_credentials.username) right before sending.

What this PR changes

This PR doesn't fix the reporter's setup. What it changes: when SASLCredentials(mechanism="PLAIN") is missing a username or password, the failure is faust's ImproperlyConfigured, which points at broker_credentials, instead of aiokafka's ValueError. That's why the PR says "Refs #794" rather than "Fixes".

Possible follow-up (not in this PR)

Environment variables silently overriding explicit faust.App(...) arguments isn't mentioned in the settings docs. That could be documented, or faust could log a warning when an environment variable replaces an explicitly passed value. Reversing the precedence would break setups that rely on it today.


Generated by Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] SASL PLAINTEXT producer initialization fails : missing sasl_plain_username/password

3 participants