Skip to content

feat(tools/mail-source): add Mailman 3 / Hyperkitty archive backend - #1474

Open
oscerd wants to merge 1 commit into
apache:mainfrom
oscerd:feature/306-mailman3-mail-source
Open

oscerd wants to merge 1 commit into
apache:mainfrom
oscerd:feature/306-mailman3-mail-source

Conversation

@oscerd

@oscerd oscerd commented Sep 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds tools/mail-source/mailman3/, a read-only mail-source backend for Mailman 3 archives served by Hyperkitty.
    Hyperkitty exposes a JSON API, so the adapter is a README of curl recipes with nothing to install: list_recent_threads, read_thread and thread_url, keyed by the root Message-ID like imap and mbox.
    Same capability shape as PonyMail: no drafts, no sent-mail view.
  • A private archive (the usual case for <security-list>) only serves signed-in subscribers.
    The adapter declines it (HTTP 403 means backend unavailable), so the resolution rule falls through to gmail or imap.
    Authenticated reads are left for a follow-up.
  • Refreshes the hand-maintained cross-references per tools/AGENTS.md: the contract's capability matrix, tools/mail-source/README.md, docs/adapters/registry.md and docs/vendor-neutrality.md (Mailman 3 moves from extension point to shipping), a mailman3_archive_url key in the setup template, the quick-start prerequisites, specs/adapters.md, and the CONTRIBUTING.md good-first-issue list.

Type of change

  • Skill change (.claude/skills/<name>/) — eval fixtures updated below
  • Tool / bridge contract (tools/<system>/*.md)
  • Python package (tools/*/ with pyproject.toml)
  • Groovy reference impl
  • Cross-cutting (RFC, AGENTS.md, sandbox, privacy-LLM)
  • Documentation (docs/, README.md, CONTRIBUTING.md)
  • Project template (projects/_template/)
  • CI / dev loop (prek, workflows, validators)
  • Other:

Test plan

  • prek run --all-files passes: every hook passes on this branch except two that fail in my environment and that this diff does not reach:
    • skill-token-count: the tokenizer cache was never prepared on this machine (--prepare-tokenizer); no SKILL.md changed.
    • pytest (workspace): tools/container-gateway test_run_unlinks_pid_file_before_closing_the_lock_fd fails 3/3 here; that tool is untouched, so the result is the same on main.
    • The per-file run over the changed files (doctoc, markdownlint, typos, lychee, check-placeholders, vendor-neutrality score, validator, spec-validate, …) is all green.
  • For Python packages touched: uv run pytest / ruff check / mypy passes
  • For Groovy bridges touched: command-line invocation tested end-to-end
  • For skill changes: eval suite passes for the affected skill
  • For skill behaviour changes: a new or updated eval fixture is included in this PR
  • Other:
    • Endpoints, paging, ordering, thread keys and the private-list checks were checked against the Hyperkitty source (urls.py, api/thread.py, api/email.py, lib/view_helpers.py, models/email.py).
      The mailman-web settings define no REST_FRAMEWORK, so the Django REST Framework defaults apply: no page size (hence "always pass limit") and session auth first (hence 403 for anonymous reads).
    • The Message-ID hash recipe gives JJIGKPKB6CVDX6B2CUG4IHAJRIQIOUTP for <87myycy5eh.fsf@uwakimon.sk.tsukuba.ac.jp>, with or without the angle brackets, and sha1sum | xxd -r -p | base32 agrees.
    • No eval re-run needed: the only eval step that extracts from a changed file (security-issue-sync/step-security-cc, the ## Abstract operations section of contract.md) extracts a byte-identical section, and no SKILL.md changed.
    • Not run against a live Hyperkitty server: the recipes are checked against the source only.

RFC-AI-0004 compliance

  • HITL — any new mutation is gated on explicit user confirmation
  • Sandbox — no new unrestricted host access; network reach declared in the adapter
  • Vendor neutrality — placeholders (<PROJECT>, <tracker>, <upstream>, <security-list>) used in all skill / tool prose (the check-placeholders prek hook is the mechanical gate)
  • Conversational + correctable — agentic-override path documented if behaviour is adopter-tunable
  • Write-access discipline — no autonomous outbound messages; drafts only, sent on confirmation
  • Privacy LLM — private content does not reach a non-approved LLM; redactor invoked where needed

Linked issues

Closes #306

Notes for reviewers (optional)

  • The Hyperkitty API docs page the issue links (docs.mailman3.org/.../api.html) now returns 404, and the current docs have no API page, so the README links the Hyperkitty project instead.
  • The new README carries the **Capability:**, **Kind:** and **Vendor:** lines that specs/adapters.md expects of every adapter README.
    The validator, the labeler and the vendor-neutrality score only scan top-level tools/*/README.md, so they do not count this nested adapter, the same as imap/ and mbox/.
  • mbox (#304) and IMAP (#303) stay listed as open in docs/vendor-neutrality.md and as shipping in the registry, as before this change.

Generated-by: Claude Code (Opus 5.5)

🤖 Generated with Claude Code

Projects on Mailman 3 (Python, Fedora, GNU and many others) had no
mail-source backend besides Gmail. Hyperkitty, the Mailman 3 archiver,
serves its archive as a JSON API, so the adapter is a README of curl
recipes rather than code: list_recent_threads, read_thread and
thread_url, keyed by the root Message-ID like the IMAP and mbox
adapters. Like PonyMail it only reads. A private archive needs a
subscribed session the adapter does not wire, so it declines those and
the resolution rule falls through to a subscriber-side backend.

The endpoints, paging, thread keys and permission checks follow the
Hyperkitty and mailman-web sources, and the Message-ID hash recipe is
the computation of Hyperkitty's own get_message_id_hash.

The contract's capability matrix and the other lists of mail-source
backends now include it, and CONTRIBUTING no longer offers it as open
work.

Closes apache#306

Signed-off-by: Andrea Cosentino <ancosen@gmail.com>
Generated-by: Claude Code (Opus 5.5)
@github-actions github-actions Bot added the contract:mail-source Tool capability: inbound-mail ingestion (mbox / IMAP / ...) label Sep 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contract:mail-source Tool capability: inbound-mail ingestion (mbox / IMAP / ...)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(tools/mail-source/mailman3): add Mailman 3 / Hyperkitty archive backend

1 participant