Skip to content

Add detail to traced values where str() is ambiguous - #729

Open
RonnyPfannschmidt wants to merge 3 commits into
pytest-dev:mainfrom
RonnyPfannschmidt:trace-value-heuristic
Open

RonnyPfannschmidt wants to merge 3 commits into
pytest-dev:mainfrom
RonnyPfannschmidt:trace-value-heuristic

Conversation

@RonnyPfannschmidt

@RonnyPfannschmidt RonnyPfannschmidt commented Sep 12, 2026 •

Copy link
Copy Markdown
Member

AI-authored. I asked Claude Code (Opus 5) to work out a rendering for traced
values that keeps pytest's trace readable while surfacing the details str()
drops. The code, the tests, the measurements and the text below are the agent's
work. I read it and I am posting it, and I will follow up on review comments
myself.

Why

#728 fixes the two tracing crashes without touching the output, and says explicitly that
the type-visibility idea from #627/#681 should be argued on its own. This is that
argument.

Rendering every value with repr(), as #681 proposed, changes 216 of 1329 lines in a
real pytest --debug run, and the overwhelming majority of that is quotes added around
strings that were already readable. But str() does drop things a reader needs:

in the trace today the problem
left: / right: an empty string is indistinguishable from no value at all — in a comparison trace
val: with space no visible boundaries; leading and trailing space are invisible
exitstatus: 1 an IntEnum prints as a bare number, the member name is lost
collection_path: /x
path: /x
a PosixPath and a py.path.local pointing at the same place look identical
a multi-line value runs out to column 0 and reads as a trace line of its own

The rule

  • a string that reads unambiguously as itself — non-empty, printable, no spaces — stays bare
  • a string that is empty, or carries whitespace, gets repr()
  • an enum.Enum or an os.PathLike gets repr()
  • a multi-line value is drawn as a block
  • everything else keeps str(), unchanged

What it costs

Measured on a real pytest --debug run: 45 of 1329 lines change (3.4%), against 216
for blanket repr(). By category: 21 paths, 11 whitespace strings, 3 empty strings, 2
enums, 2 blocks.

  exitstatus: 1                      →  exitstatus: <ExitCode.TESTS_FAILED: 1>
  left:                              →  left: ''
  right:                             →  right: ''
  collection_path: DIR/.benchmarks   →  collection_path: PosixPath('DIR/.benchmarks')
  path: DIR/.benchmarks              →  path: local('DIR/.benchmarks')
  val: with space                    →  val: 'with space'
  plugin_name: lfplugin              →  plugin_name: lfplugin        (unchanged)
  config: <_pytest.config.Config…>   →  config: <_pytest.config.Config…>   (unchanged)

Multi-line values

Today a multi-line value breaks the layout — the continuation escapes to column 0 and
reads as a top-level trace line:

            orig: isinstance(f, pathlib.Path)
            expl: True
 +  where True = isinstance(PosixPath('/tmp/pytest-N/test_path0'), <class 'pathlib.Path'>)
 +    where <class 'pathlib.Path'> = pathlib.Path
        finish pytest_assertion_pass --> [] [hook]

With this change the value is boxed, so its extent is visible at a glance:

            orig: 'isinstance(f, pathlib.Path)'
            expl:
              | True
              |  +  where True = isinstance(PosixPath('/tmp/pytest-N/test_path0'), <class 'pathlib.Path'>)
              \  +    where <class 'pathlib.Path'> = pathlib.Path
        finish pytest_assertion_pass --> [] [hook]

Rough edges, for review

  • nodeid: test_mix.py::test_p[with space] gains quotes, because the parameter id
    contains a space. The rule fires correctly, but on something that is not really "a
    string with whitespace" in spirit.
  • orig: n or n == "" becomes orig: 'n or n == ""' — nested quotes on source text.
  • Lone surrogates now render '\ud800' rather than bare, since they are not
    isprintable(). Arguably clearer; it does change two tests added in Make hook tracing unable to fail a hook call #728.
  • Rendering a traced value costs ~0.8 µs rather than ~0.2 µs, only when tracing is
    enabled — around 0.5 ms across a whole pytest run.

Testing

  • uv run pytest — 203 passed.
  • uv run pre-commit run -a — all hooks pass.
  • _tracing.py at 100% statement and branch coverage.
  • 8 new tests: 6 covering each branch of the rule, and 2 for the repr guards this change
    newly reaches — a path-like with a broken repr, and KeyboardInterrupt raised from a
    repr.
  • pytest --debug output diffed against main: 45 changed lines, each one listed above.

🤖 Generated with Claude Code

RonnyPfannschmidt and others added 3 commits September 24, 2026 18:15
Trace output stays str() based, because that is what makes it readable,
but str() hides things a reader needs often enough to be worth fixing
case by case:

- an empty string is indistinguishable from no value at all, which is
  exactly wrong for a comparison trace showing left and right
- a string carrying whitespace has no visible boundaries
- an IntEnum prints as a bare number, losing the member name
- a path prints as text, so a PosixPath and a py.path.local argument
  pointing at the same place look identical
- a multi line value runs into column 0 and reads as a trace line of its
  own rather than as the value of its key

So values that read unambiguously as themselves -- a non empty printable
string without spaces, and every type whose repr adds nothing -- stay
bare, and the rest gain quotes, their type, or a block.

Multi line values are drawn as a box, each line prefixed with | and the
last with \\, so the extent of the value is visible at a glance.

Measured on a real pytest --debug run, this changes 45 of 1329 trace
lines, against 216 for rendering every value with repr(). Every changed
line carries information the previous rendering dropped.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
_safe_repr is only reached for enums, paths and quoted strings, which
all have working reprs in the existing tests, so its guards were dead in
coverage. A path-like with a broken repr is the pytest-dev#424 scenario applied to
a value the heuristic sends through repr.

_tracing.py is back at 100% statement and branch coverage.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Code <noreply@anthropic.com>
Comment thread changelog/729.feature.rst
Comment on lines +1 to +5
Traced values now gain detail where ``str()`` is ambiguous: an empty string or a string
carrying whitespace is quoted, an enum member shows its name, and a path shows its type,
so that two arguments pointing at the same place are distinguishable. Values that read
unambiguously as themselves are unchanged, and a value spanning several lines is drawn
as a block attached to its key instead of running into the surrounding trace.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Too verbose IMO. My suggestion to write it in human language:

Suggested change
Traced values now gain detail where ``str()`` is ambiguous: an empty string or a string
carrying whitespace is quoted, an enum member shows its name, and a path shows its type,
so that two arguments pointing at the same place are distinguishable. Values that read
unambiguously as themselves are unchanged, and a value spanning several lines is drawn
as a block attached to its key instead of running into the surrounding trace.
Tracing now displays details with more detail or quoting when ``str()`` is ambiguous (like an empty string
or string containing whitespace, enums and paths).
A value spanning several lines is now displayed as a block instead of running into the surrounding trace.

Comment thread docs/index.rst
Comment on lines +1046 to +1050
Values are rendered with :func:`str` wherever that reads unambiguously, and with
:func:`repr` where it does not: an empty string, a string carrying whitespace,
an enum member, or a path, whose type is otherwise easy to lose. A value
spanning several lines is drawn as a block so that it stays attached to its key
instead of running into the surrounding trace.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example seems useful but I think this paragraph is unnecessary, would just remove it.

Comment thread src/pluggy/_tracing.py
def _format_block(indent: str, text: str) -> list[str]:
"""Draw a multi line value as a box, so it reads as one value.

The left edge marks every line as continuation, and the final ``\\``

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the \ for the final line conventional? To me it seems confusing (I don't know what it means). I think the indentation is sufficient to indicate when the block ends?

Comment thread src/pluggy/_tracing.py
def _format_block(indent: str, text: str) -> list[str]:
"""Draw a multi line value as a box, so it reads as one value.

The left edge marks every line as continuation, and the final ``\\``

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In plaintext the \\ is a bit confusing, would make the docstring raw and then can write a single \

Comment thread src/pluggy/_tracing.py
around it.
"""
body = text.split("\n")
edges = ["|"] * (len(body) - 1) + ["\\"]

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems a bit inefficient to create the entire list just for this, maybe replace with an if? But it's OK if you prefer it

Comment thread src/pluggy/_tracing.py
Comment on lines +71 to +74
Most values keep their plain ``str`` rendering, which is what makes a trace
readable. ``repr`` is used only where ``str`` hides something the reader
needs: the type of a path, the name of an enum member, or the boundaries of
a string that is empty or carries whitespace.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would remove this paragraph as too verbose. The code itself is clear enough I'd say.

@feiiiiii5

Copy link
Copy Markdown
Contributor

Verified the three rendering rules on the branch (203 passed) and found two shapes that reach the code but not the rules. Neither is a regression — the base commit behaves the same way for both — but both are cases where the documented behaviour does not hold.

enum.StrEnum never reaches the enum branch. _render_value tests isinstance(obj, str) first, and StrEnum.__str__ is str.__str__, so a member whose value is a plain token satisfies _is_plain_token and comes back bare:

IntEnum(1)              -> '<IntEnumCase.ONE: 1>'      name shown
StrEnum('value')       -> 'value'                     name lost
StrEnum('has space')   -> "<StrEnumCase.SPACED: 'has space'>"   name shown
(str, Enum) A='a'      -> 'StrEnumSubclassOfStr.A'     name shown

The trigger is specifically StrEnum and a value that happens to be a plain token; a StrEnum with a space in its value is quoted and therefore does get repr(). And a class X(str, Enum) is fine either way, because there Enum.__str__ is what runs. The added enum test uses IntEnum, which is not a str, so it passes.

For contrast, on pr-729~3 the IntEnum case rendered as '1' — so the enum branch is a real improvement, StrEnum is just the one member of the family that cannot reach it. That contradicts "an enum member shows its name" for the StrEnum case.

The fix is to test the enum/PathLike case first. I tried it: over str plain/spaced/empty/multi-line, IntEnum, StrEnum plain and spaced, Path, int, None, list, bytes — 11 of 12 renderings are byte-identical, the only change being StrEnum('value') -> <S.PLAIN: 'value'>, and 203 passed still.

A bare CR is still emitted inline. _render_value treats "\r" in obj as a reason to return the value unquoted, but _format_message only draws a block if "\n" in rendered, and _format_block splits on "\n":

'a\nb'     -> boxed, no stray CR        (the case the tests cover)
'a\r\nb'   -> boxed, but one \r left at the end of the first block line
'a\rb'     -> not boxed, \r inline
'x\ry\rz'  -> not boxed, two \r inline

The bare-CR case is the one that matters for the stated goal: a terminal acts on the \r by returning the cursor to column 0, so k: a\rb overwrites itself and the reader sees only b — exactly the "runs into the surrounding trace" failure the block drawing exists to prevent. The base commit did the same thing, so nothing regressed; but the renderer now reasons about \r while the boxing decision does not, and that inconsistency is new.

Either half would close it: split on \r\n|\r|\n both when deciding to box and when splitting the body, or drop \r from the _render_value test so a CR-bearing string is at least quoted and its boundaries visible.

Happy to send a patch for either or both — the reorder is a one-line move and the CR fix is a re.split plus one condition, and I can run them against the full suite.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants