build(deploy): bump the pinned cloudflared to 2026.9.3

CHANGELOG.md

@@ -29,6 +29,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - A failing `GET /api/v1/bash-commands` no longer renders as the "no command detail in this data" explainer. The Bash section branched on row count alone, so a request that errored looked identical to one that legitimately returned nothing, blaming Claude Code's telemetry for what was actually a server fault. Fetch failures now show the error
 
 ### Changed
+- The bundled `cloudflared` is **2026.9.3** in place of 2024.11.1, still an explicit pin rather than `latest` so rebuilding an old commit produces the binary it originally shipped. The two years in between hold one change that touches cotel: 2026.4.0 flipped the `--edge-ip-version` default from `4` to `auto`, which connects over whichever address family the system resolver answers with first and falls back to the other only after a connection has already failed - so a container whose resolver answers `AAAA` first but has no working IPv6 egress spends its first attempts failing, and then sticks with the fallback for ten minutes at a time. Token mode therefore sets `TUNNEL_EDGE_IP_VERSION=4` unless the operator sets it, holding the behaviour the old version had; local-config mode is deliberately left alone, because that env var silently outranks an `edge-ip-version` in the operator's own `config.yml`. The other new arrival is visible rather than behavioural: cloudflared now probes DNS, QUIC, HTTP/2 and the Cloudflare API at startup and logs a `CONNECTIVITY PRE-CHECKS` table, which is kept on - it runs concurrently with startup and does not gate it, a `FAIL` row does not stop the tunnel, and it is the fastest way to separate a blocked UDP path from a bad token, which is exactly the diagnosis the deploy health gate cannot make because it probes cotel's `/healthz` and not the tunnel. The bump does **not** address the token-in-logs leak and was kept out of that fix for the same reason the fix was kept out of this bump: measured on 2026.9.3, a token left in the environment is still printed in full by the startup env dump. `--token-file`, which the new version adds, is not adopted either - the token stays readable through `docker inspect` and `/proc/1/environ` regardless, so it would close no access path in exchange for a tmpfs mount and a secret on disk ([docs/operations/cloudflare-tunnel-remote.md](docs/operations/cloudflare-tunnel-remote.md))
 - The DuckDB driver is `github.com/duckdb/duckdb-go/v2`, which carries engine **1.5.6** in place of 1.1.3. The reason is not the version: `github.com/marcboeker/go-duckdb` is deprecated by its own author — the first line of its `go.mod` reads *"This module has moved to github.com/duckdb/duckdb-go"* and its last tag is a year old — so cotel was pinned to an address that is no longer the dependency and has no upstream to receive a bug report. This is **not** a fix for the ART-metadata corruption that took production down: upstream [duckdb#25360](https://github.com/duckdb/duckdb/issues/25360) reports the same class of fault still reproducing on 1.5.5, and what the newer engine changes is that it raises a Go error instead of `abort()`ing inside cgo. The on-disk format was the thing that needed measuring, since a newer engine writing a file the previous image cannot read would make the upgrade unrollbackable with production's only copy of the data inside it — and it does not: an existing file is never converted by ordinary open-and-write, not even when `storage_compatibility_version=v1.5.6` is asked for explicitly, and 1.1.3 reads a file 1.5.6 has written to, replays a WAL 1.5.6 left, and was confirmed doing so as the actual `ghcr.io/flopsstuff/cotel:latest` image. The door closes only for a *newly created* file at an explicit opt-in, so `storage.Open` now pins `storage_compatibility_version=v0.10.2` in the DSN and a test reads the format number straight out of the file header to hold it at 64 — the pin is a no-op against today's engine, which is the point: a default is behaviour, not a contract, and a future minor could raise it and leave no diff of ours to point at. Two further consequences: Go **1.24.0** is now the build floor (`go.mod`, the builder image and CI move together), and the runtime image grows 213 MB → 294 MB for the bigger static engine library — against which ICU is now linked in rather than fetched over the network on every cold start. A DuckDB `JSON` column no longer scans into a Go `*string` (v2 hands back a decoded `map[string]interface{}`), so `ExportSpans` and the session-detail query cast in SQL; any new query reading `attributes` or `resource_attrs` into a string needs the same `::VARCHAR` ([ADR-0018](docs/decisions/0018-duckdb-go-v2-driver.md))
 - The chart palette is re-derived — eight of its ten values change, `--color-chart-1` is unchanged in both schemes because it is `--color-accent`. The old palette broke two things at once. Its dark values were the light values lightened, which reverses loudness when the surface flips: the quietest series in light was the loudest in dark, so the Overview's cost line shouted over the spans area it annotates for any reader on a dark theme, and did not for anyone on a light one. And it was not the colour-independent palette it was documented to be — `chart-1` and `chart-2` measured ΔE 0.4 light / 0.3 dark under deuteranopia, the same colour to roughly one man in twelve, and the palette is handed out in order, so that was the pair every two-series chart in the product drew. The new values hold both rules with margin: identical salience rank in both schemes, and every one of the ten pairs ≥ 8.6 ΔE under protanopia, deuteranopia and tritanopia in both schemes, measured with Machado 2009 at severity 1.0 in OKLab. Each token keeps its hue to within 3° — the palette still reads blue, violet, green, amber, pink — and only lightness and chroma moved; the contrast floor against the surface rises from 3.19:1 to 4.08:1, above the 3:1 WCAG 1.4.11 asks of a 2 px stroke. `--color-chart-4` also stops being byte-identical to `--color-warning`, which it was in both schemes: the cost line was literally the warning colour. The rules, the ruler and the full pairwise measurement are in [tokens.md](docs/design/tokens.md)
 - The Overview's `Activity & Cost` block is `Spans & cost`, and its right-hand legend entry is `Cost (right axis)`. The old title collided with the `Span activity` grid two blocks above it — two headings saying *activity*, both charting spans, so a reader looking for span volume over time had no scent to follow and landed on the heat grid first; the new one names the two series instead, in the order the chart ranks them, and matches the sentence case of every other block title. The legend dropped `, USD` because the right axis is already tick-formatted in dollars. The axis callouts themselves stay: they are what keeps the two series identifiable without colour, which matters more than a tidy label. Below roughly 400 px the legend no longer floats over the plot — it is a row of the block that wraps above the chart, where `Cost (right axis, USD)` used to land on top of the right axis's `$12.00` and leave both unreadable — and the legend swatches now wear the same two marks as the series, a filled area and a bare line. Desktop is unchanged to the pixel: same position, same block height. Section headers generally wrap their links onto a second line rather than squeezing the title, which was breaking `Spans & cost` across two lines on a phone

Dockerfile

@@ -1,7 +1,7 @@
 # syntax=docker/dockerfile:1
 # Multi-stage build — Node frontend first, then CGo Go build.
 
-ARG CLOUDFLARED_VERSION=2024.11.1
+ARG CLOUDFLARED_VERSION=2026.9.3
 
 ##############################################################################
 # cloudflared binary (multi-arch: TARGETARCH = amd64 | arm64)

README.md

@@ -497,6 +497,7 @@ from the network at startup instead of bundling it).
 | `COTEL_RETENTION_INTERVAL` | `6h` | Retention worker tick interval (Go duration) |
 | `COTEL_WAL_AUTOCHECKPOINT` | `4MB` | DuckDB `checkpoint_threshold`: the write-ahead log is folded into the main file once it grows past this size. Lower values bound how much WAL an ungraceful kill leaves to replay on the next open; higher values checkpoint less often during ingest. DuckDB's own default is `16MB`. |
 | `CLOUDFLARE_TUNNEL_TOKEN` | _(unset)_ | When set, starts `cloudflared tunnel run` before cotel; enables public HTTPS access via Cloudflare Tunnel |
+| `TUNNEL_EDGE_IP_VERSION` | `4` in token mode | Read by `cloudflared`, not by cotel: the address family used to reach the Cloudflare edge (`4`, `6` or `auto`). cloudflared's own default became `auto` in 2026.4.0, which tries whichever family the resolver answers with first and falls back only after a connection has failed; token mode pins `4` unless you set this. Not set in local-config mode, where `config.yml` owns the setting. See [token mode](docs/operations/cloudflare-tunnel-remote.md#the-bundled-cloudflared) |
 | `COTEL_PUBLIC_INGEST_URL` | _(unset)_ | Absolute `http`/`https` URL of the public OTLP ingest endpoint (e.g. `https://cotel-ingest.yourdomain.com`). When set, the Setup page substitutes this URL into the copy-paste Claude Code snippets. |
 | `COTEL_DATA_VOLUME` | `cotel-data-repaired-20261004` | Read by `docker-compose.yml`, not by the binary: the Docker volume mounted at `/data`. Point it at a restored copy to bring an instance up *without* writing to the volume being restored from — Docker has no `volume rename`, so the only other way to serve repaired data under the expected name is to overwrite the damaged original, which is also the forensic evidence. The default is a restored copy, not the `cotel_cotel-data` name compose derives by itself: that volume holds a database the binary can no longer open and is kept untouched as evidence. A fresh install that has neither volume gets the default created empty, which is correct. |
 

docs/decisions/0006-cloudflare-tunnel-and-token-auth.md

@@ -29,7 +29,7 @@ Rolling custom TLS + auth in-app would inflate the deployment surface (a second
 
 Implementation details:
 
-- The Dockerfile downloads the `cloudflared` binary in a separate build stage, pinned to a specific version (`v2024.11.1`), multi-arch (amd64/arm64).
+- The Dockerfile downloads the `cloudflared` binary in a separate build stage, pinned to an explicit version via the `CLOUDFLARED_VERSION` build arg, multi-arch (amd64/arm64).
 - `scripts/entrypoint.sh` supports two modes (token takes precedence):
   - **Token mode:** `CLOUDFLARE_TUNNEL_TOKEN` env var → `cloudflared tunnel run --token …`. Ingress rules live in the Cloudflare dashboard.
   - **Local-config mode:** `/etc/cloudflared/config.yml` mounted into the container → `cloudflared tunnel --config … run`. Ingress rules live in a YAML file, reviewable in git.

docs/operations/cloudflare-tunnel-local.md

@@ -148,6 +148,7 @@ This gates the dashboard UI without any changes to cotel itself. The ingest endp
 - **HA / multi-replica:** the `<UUID>.json` credentials file must be present on every host that runs the container. Synchronizing credentials across hosts is out of scope — consider token mode for multi-host setups, as the token is just an env var.
 - **`cert.pem` is only needed for bootstrap.** You can delete or archive it after step 3; the container only needs `<UUID>.json` and `config.yml` at runtime.
 - **Migrating from token mode:** create a new tunnel (`tunnel create`), add DNS routes, write `config.yml`, remove `CLOUDFLARE_TUNNEL_TOKEN`, mount `~/.cloudflared`, restart. The old token-mode tunnel can be deleted in the Cloudflare dashboard after confirming the new one is healthy.
+- **`edge-ip-version` now defaults to `auto`.** cloudflared 2026.4.0 changed it from `4`; it connects over whichever address family the resolver answers with first and falls back only after a failed connection. In local-config mode cotel does not override this - your `config.yml` owns it. Add `edge-ip-version: 4` there if the container has no working IPv6 egress. (Token mode pins `4` instead, because it has no config file to put it in.)
 
 ## References
 

docs/operations/cloudflare-tunnel-remote.md

@@ -91,6 +91,19 @@ Do not lower cloudflared's log level to `debug` on a public deployment: `--logle
 
 If you need the token out of the container entirely, use [local-config mode](./cloudflare-tunnel-local.md) - the credentials then live in a file mounted from the host and never enter the container's environment or command line.
 
+## The bundled cloudflared
+
+The image carries a `cloudflared` pinned by `CLOUDFLARED_VERSION` in the `Dockerfile` (currently **2026.9.3**) rather than tracking `latest`, so a rebuild of an old commit produces the same binary it did originally. Override it at build time with `--build-arg CLOUDFLARED_VERSION=…`.
+
+Two things about recent versions are worth knowing when reading the startup log:
+
+- **Edge IP version.** 2026.4.0 changed the `--edge-ip-version` default from `4` to `auto`, which connects over whichever address family the system resolver answers with first and falls back to the other one only after a connection has already failed. In token mode the entrypoint sets `TUNNEL_EDGE_IP_VERSION=4` unless you set it yourself, because a container with a resolver that answers `AAAA` first but no working IPv6 egress would otherwise spend its first connection attempts failing. Set `TUNNEL_EDGE_IP_VERSION=auto` (or `6`) in your deployment to opt back in. This is token mode only - in local-config mode the setting belongs in your `config.yml`, and an env var would silently outrank it.
+- **Connectivity pre-checks.** Since 2026 cloudflared probes DNS, QUIC, HTTP/2 and the Cloudflare API at startup and logs a `CONNECTIVITY PRE-CHECKS` table (about twenty lines) before the tunnel registers. They are diagnostic only - they run concurrently with startup, do not gate it, and a `FAIL` row does not stop the tunnel. The table is the fastest way to tell a blocked UDP path from a bad token. `TUNNEL_NO_PRECHECKS=true` silences it.
+
+### Why not `--token-file`
+
+Newer cloudflared can read the token from a file (`--token-file`) instead of an argument, which would flip the last row of the table above to "No". cotel does not use it, because that row is the only one it changes: whoever can run `ps` inside the container can read `/proc/1/environ` too, so the same person reaches the same value by a path `--token-file` does not touch. It would need a tmpfs mount and a secret on disk to close nothing. Getting the token out of the container altogether means [local-config mode](./cloudflare-tunnel-local.md), where the credentials are a host-mounted file.
+
 ## Verifying the tunnel
 
 ```sh

scripts/entrypoint.sh

@@ -48,6 +48,12 @@ cfd_token="${CLOUDFLARE_TUNNEL_TOKEN:-}"
 unset CLOUDFLARE_TUNNEL_TOKEN
 
 if [ -n "${cfd_token}" ]; then
+    # cloudflared 2026.4.0 changed the --edge-ip-version default from 4 to auto,
+    # which follows whichever family the resolver answers with first and only
+    # falls back to the other one after a connection has already failed. Keep
+    # the old default here, where cotel owns the whole invocation; local-config
+    # mode is left alone because this env var outranks a config.yml setting.
+    export TUNNEL_EDGE_IP_VERSION="${TUNNEL_EDGE_IP_VERSION:-4}"
     cloudflared tunnel run --token "${cfd_token}" &
     CLOUDFLARED_PID=$!
     echo "entrypoint: cloudflared started in token mode (PID ${CLOUDFLARED_PID})"