Skip to content

Remove per-row codec processes from parallel report spooling #1371

Closed

Description

@Chemaclass

A passing parallel test still launches several codec processes just to send its filename and name to the parent report writer. Reduce that reporting overhead while keeping every public report format unchanged.

The spool writer encodes four arbitrary-text fields; the reader decodes them. Empty messages/output use a sentinel, but every ordinary passing row still encodes and decodes its nonempty file and test name. A census over 100 passing tests with --parallel --simple --report-json <report> recorded 400 base64 processes attributable to the spool, plus 200 tr processes on macOS, before accounting for JSON escaping. There was one additional startup base64 probe.

Measured on macOS arm64, Bash 3.2.57, bashunit 0.51.0 at 860c797a48e34be20b019a70f58c0e06d15c8d33. Wall times are medians of five runs after one warm-up, with the same fixtures, working directory, output mode and bootstrap settings. External command counts come from PATH shims, not re-echoed shell traces.

#1289 and #1290 already removed encoding of numeric/status fields. This issue targets the four text fields left after that fix. Do not repeat the old fourteen-process claim. No post-change wall-time improvement is claimed yet; measure this transport independently of JSON escaping and clock reads, including CPU/process count as well as elapsed time.

Use a lossless internal record representation which avoids repeated codec launches. Separate worker records or another proven atomic publication method must preserve records under concurrency; appending several raw fields to one shared file risks interleaving. NUL-delimited fields are a candidate, not a required design. Keep hook-failure rows, replay order and exactly-once collection intact.

  • A normal passing row requires no external base64 encode/decode process for its text fields.
  • Concurrent workers preserve complete rows, including paths/names/messages/output with quotes, backslashes, tabs, embedded newlines, Unicode, unit separators and empty fields.
  • Large captured output has no material performance regression or truncation.
  • Parent/file-level hook failures, provider failures, annotation aborts, skips, retries and ordinary worker rows are reported exactly once.
  • Sequential reports are unchanged; every parallel report keeps its schema and counts.
  • Scratch records do not collide across files with duplicate basenames or provider rows, and are removed on cleanup.
  • A transport benchmark establishes a real benefit before the representation is changed; fork regressions and full compatibility/local gates pass.

Keep Bash 3.0 as the minimum. Verify the source runner and standalone artifact, actual Bash 3.0 in sequential, parallel, simple, simple-parallel and strict modes, macOS Bash 3.2, make test, make test/parallel, make test/strict, make sa and make lint. Measure with the same benchmark on current main and the final branch; keep elapsed-time checks out of normal tests.

Scope is report collection/spooling. JSON/XML escaping, clock behavior and provider argument parsing remain separate deliverables.

Activity

  1. self-assigned this
    on Oct 2, 2026
  2. Chemaclass commented on Oct 2, 2026

    @Chemaclass
    MemberAuthor

    [Codex] Done in #1378 (merged).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

  • Status
    Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions