Skip to content

Add documentation for routing, server settings, domains, users, workers and the postal command - #90

Open
benrfairless wants to merge 5 commits into
postalserver:mainfrom
benrfairless:docs/new-feature-pages
Open

benrfairless wants to merge 5 commits into
postalserver:mainfrom
benrfairless:docs/new-feature-pages

Conversation

@benrfairless

Copy link
Copy Markdown

This is the first of four PRs that update the documentation against the Postal 3.3.7 source (postalserver/postal at tag 3.3.7; unreleased changes on main were deliberately excluded). It adds pages for areas that currently have no documentation at all.

New pages

  • Features → Routing Incoming E-Mail – MX vs. forwarding address, route name rules, the five route modes (Endpoint/Accept/Hold/Bounce/Reject) and what happens at SMTP time vs. processing time, additional endpoints, processing order, HTTP/SMTP/address endpoint options, retries and bounces.
  • Features → Mail Server Settings – Live/Development mode, retention (defaults and how raw-message day tables are pruned), send limits (rolling hour, 90% warning, hourly notifications), every reason a message can be held and whether a manual release bypasses it, suppression list rules, admin-only advanced settings, suspension, deletion.
  • Features → Sending Domains – organization vs. server domains, DNS and e-mail verification, the four DNS checks with the exact records and statuses Postal expects, DKIM signing and fallback to the installation key, custom return path (psrp.) for DMARC alignment.
  • Features → Users & Permissions – global administrators vs. organization users, postal make-user, passwords and sessions.
  • Other → Workers & Background Tasks – worker threads, queue locking, batching, IP-pool affinity, the tasks role election, stale locks, and a table of all nine scheduled tasks with their schedules.
  • Getting Started → The postal command – reference for the postalserver/install helper script (commands, flags, override file, hooks).

The installation and DNS configuration pages gain short links to these pages.

Notes for reviewers

  • One upstream bug is documented as a known issue on the routing page: SMTP endpoints with SSL mode STARTTLS behave as None, because SMTPClient::SSLModes::STARTTLS is "STARTLS" while the model stores "STARTTLS".
  • Two console-only features (use_for_any on domains, organization suspension) are documented with postal console snippets since there is no UI for them. Happy to remove those if you'd rather not advertise them.
  • pnpm generate passes on this branch (177 routes prerendered, no 404s from the link crawler).

Follow-up PRs (each stacked on this one): configuration reference; corrections to the developer & feature pages; corrections to the installation & other-notes pages.

New feature pages written from the 3.3.7 source:
- Routing incoming e-mail: MX vs forwarding address, route name rules,
  the five route modes and their SMTP-time behaviour, additional
  endpoints, processing order, HTTP/SMTP/address endpoint options
  (including the STARTTLS/STARTLS mismatch), retries and bounces
- Mail server settings: Live/Development, retention behaviour and
  defaults, send limits (rolling hour, 90% warning, hourly notices),
  every reason a message can be held and whether release bypasses it,
  suppression list rules, advanced/admin-only settings, suspension,
  deletion
- Sending domains: org vs server domains, DNS and e-mail verification,
  the four DNS checks with exact expected records and statuses, DKIM
  signing and fallback, custom return path for DMARC alignment,
  use_for_any
Documents the install-repo helper script: run/setup/tool commands,
flags, override files and hooks.

@willpower232 willpower232 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

always nice to see some documentation updates, just don't want to be managing a minutely detailed encyclopedia of Postals behaviour 😅

1. Create an organization and a mail server.
2. Add the domain you want to send from and follow the [Sending domains](/features/sending-domains) page to verify it and publish its SPF, DKIM and return path records.
3. Create an **SMTP** or **API** [credential](/features/smtp-authentication) and send a test message.
4. Point the `smtp` section of `postal.yml` at your new mail server so that Postal can send its own notification e-mails, then `postal restart`.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wouldn't include this line here, I would say something in a separate section to separate "just sending emails" from "Postal sending its own emails for password resets"

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved into its own "Postal's own e-mails" section below the list, explaining that password resets / send limit warnings / suspension notices go via the smtp section of postal.yml rather than through a mail server.

<td>postal._domainkey.rp.postal.example.com</td>
<td>TXT</td>
<td>Value from <code>postal default-dkim-record</code></td>
<td>Value from <code>postal default-dkim-record</code> (the <code>postal</code> selector is the value of <code>dns.dkim_identifier</code>)</td>

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure this fully makes sense, maybe something like "if you wish you control the prefix then you can update the config files etc etc"

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reverted the table cell to the original wording and added a sentence under the table instead: "The postal part of the DKIM record name is the selector. If you wish to use a different selector, set dns.dkim_identifier in postal.yml before publishing the record."

## Route domain

If you wish to receive incoming e-mail by forwarding messages directly to routes in Postal, you'll need to configure a domain for this just to point to your server using an MX record.
If you wish to receive incoming e-mail by forwarding messages directly to routes in Postal (each route has a unique `{token}@routes.postal.example.com` address - see [Routing incoming e-mail](/features/routing-incoming-email)), you'll need to configure a domain for this just to point to your server using an MX record.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this should be two separate paragraphs

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Split - the first paragraph is now back to the original text, and the forwarding address / link to the routing page is a second paragraph.


Docker Compose automatically merges `docker-compose.override.yml` from the same directory. Use this for changes such as [log drivers](/features/logging#redirecting-logs-to-the-host-syslog), extra environment variables or resource limits, so that they survive upgrades (the generated `docker-compose.yml` is replaced on every upgrade).

### Hooks

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I appreciate these are in the running script but I don't think there is a common use case for them so it isn't really getting started material I would say

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the hooks section.


If a `hooks` directory exists in `/opt/postal/install`, the Bash script with the matching name in it is run at various points. Available hook names are `pre-start`, `post-start`, `pre-stop`, `post-stop`, `pre-restart`, `post-restart`, `pre-initialize-pull`, `pre-initialize`, `post-initialize`, `pre-upgrade-pull`, `post-upgrade-pull`, `pre-upgrade-db`, `post-upgrade-db`, `post-upgrade`, `pre-bootstrap`, `post-bootstrap` and `set-postal-version`.

## The generated compose file

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this section also feels a little too detailed, I don't know how this benefits the regular user and those experienced with docker compose would look in the file anyway

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the generated compose file section.

| **SSL mode** | `None` - never use TLS. `Auto` - use STARTTLS if the server offers it, without verifying the certificate, falling back to plain text if the TLS handshake fails. `TLS` - connect with implicit TLS (e.g. port 465) and verify the certificate. `STARTTLS` - see the note below. |

::callout{icon="i-heroicons-exclamation-triangle" color="amber"}
In Postal 3.3.7 the <code>STARTTLS</code> option on SMTP endpoints does not work as intended due to a mismatch in the source code (the endpoint stores <code>STARTTLS</code> but the SMTP client checks for <code>STARTLS</code>), so it currently behaves the same as <code>None</code>. Use <code>Auto</code> or <code>TLS</code> until this is fixed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is there a PR for this?

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not that I can find. The mismatch is still on main: app/lib/smtp_client/ssl_modes.rb has STARTTLS = "STARTLS" while SMTPEndpoint::SSL_MODES stores "STARTTLS", so the case @server.ssl_mode in SMTPClient::Endpoint#start_smtp_session falls through to the else branch and disables TLS. (The spec in spec/lib/smtp_client/endpoint_spec.rb passes because it uses the constant on both sides.) It's a one-character fix - happy to open a PR against postalserver/postal if that's useful. I've trimmed the callout here to not mention the internal constant names.

Comment thread content/3.features/sending-domains.md Outdated

**E-Mail** - Postal sends a 6 digit code to one of `webmaster@`, `postmaster@`, `admin@`, `administrator@` or `hostmaster@` at the domain (or any of its parent domains). Enter the code to complete verification. This requires the `smtp` section of Postal's configuration to be working.

Unverified domains cannot be used for sending, in routes, or for tracking domains.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this right? I'm not sure it is

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I re-checked against 3.3.7 and it holds, though the original sentence was too terse. Server#authenticated_domain_for_address only queries Domain.verified, so submissions from an unverified domain are rejected; Route#validate_domain_belongs_to_server adds "has not been verified yet"; and the domain dropdown used by both the route and tracking domain forms (domain_options_for_select) only lists verified domains. Reworded to: "Until a domain is verified it cannot be used: messages from addresses on it are rejected at submission, and it is not offered when creating routes or tracking domains."

Comment thread content/3.features/sending-domains.md Outdated

## Sending from any domain

A server-level domain can be flagged so that the server may send from **any** `From` address once the normal checks have failed. This is intended for trusted internal systems and is shown with an **Any** label in the domain list. There is no interface for setting this flag; an administrator can set it from `postal console`:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm learning things today haha, not sure how much of this is intended though

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed the section. It's a real code path (use_for_any fallback in Server#authenticated_domain_for_address) but with no UI for it and unclear intent, it doesn't belong in user docs. I also dropped the postal console snippet for organization suspension on the mail server settings page for the same reason.

* Users log in with their e-mail address and password. Passwords must be at least 8 characters long.
* Password resets are available from the login page and are sent using the `smtp` section of the Postal configuration, so make sure that is set up (you can test it with `postal test-app-smtp`).
* From **My Settings** a user can change their name, e-mail address, time zone and (after confirming their current password) their password. Times throughout the interface are shown in the user's time zone, which defaults to UTC.
* After logging in you are asked whether you would like to stay logged in. Choosing **Remember me** keeps the session alive across browser restarts; otherwise it ends when the browser is closed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may be so but it feels a little too obvious to be documented

Suggested change
* After logging in you are asked whether you would like to stay logged in. Choosing **Remember me** keeps the session alive across browser restarts; otherwise it ends when the browser is closed.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a lot of detail in this page which is probably not necessary for most people to care about and also is probably subject to change and those that change it might not come back here

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rewritten as a short overview (64 → ~30 lines). It now covers what workers do, that housekeeping runs on an elected worker, and how to scale / restart safely - without the per-task table, lock column names, job class names or election timings.

- Move the note about Postal's own e-mails out of the "Next steps"
  list on the installation page into its own section.
- Reword the DKIM selector note on the DNS page and split the route
  domain paragraph in two.
- Drop the hooks and generated compose file sections from the postal
  command page.
- Remove quoted internal hold/notice strings from the mail server
  settings page and the console snippets for use_for_any and
  organization suspension.
- Clarify what "unverified domains cannot be used" means.
- Remove the "Remember me" bullet from the users page.
- Rewrite the workers page as a short overview without per-task and
  per-lock implementation detail.

Assisted-by: Claude Code:anthropic.claude-fable-5-1
@benrfairless

Copy link
Copy Markdown
Author

Thanks for the review. All twelve comments addressed in 580ec3a - I've replied on each thread. Net change is -53 lines: the workers page is now a short overview, the quoted internal strings and postal console snippets are gone, and the hooks / compose-template detail is out of the postal command page. pnpm generate still passes.

Taking the "encyclopedia" point on board for the three follow-up PRs too.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants