Skip to content

Fix literal block scalars losing a newline as the last node - #1505

Closed
afonsojanu wants to merge 1 commit into
jbeder:masterfrom
afonsojanu:fix/literal-chomping-last-node
Closed

afonsojanu wants to merge 1 commit into
jbeder:masterfrom
afonsojanu:fix/literal-chomping-last-node

Conversation

@afonsojanu

Copy link
Copy Markdown

I was poking at the literal block scalar emitter and noticed that round-tripping a string through Emitter and back through Load doesn't always give you the same bytes back. Specifically:

YAML::Emitter out;
out << YAML::BeginMap;
out << YAML::Key << "data" << YAML::Value << YAML::Literal << std::string("hello\nworld\n\n");
out << YAML::EndMap;
auto node = YAML::Load(out.c_str());
node["data"].as<std::string>(); // "hello\nworld\n", one newline short

Tracing it back, WriteLiteralString in emitterutils.cpp picks a chomping indicator (-, nothing, or +) based on how many newlines the string ends with, and for the "plain"/"keep" cases it deliberately writes one fewer newline than the source has. The comment above it explains why: whatever gets emitted right after this scalar is expected to start with a \n of its own, and that stand-in covers the one that got dropped. That holds as long as another key, sequence entry, or anything else follows. It doesn't hold when the literal is the last thing written before the document ends, since in that case nothing ever supplies the missing character and the raw buffer is simply short one line break. A sequence version of the same scalar, with something after it, round-trips fine, which is what had me confused for a bit until I compared the two side by side.

I didn't want to just stop dropping that newline unconditionally, since one of the existing tests (LiteralWithAndWithoutTrailingEmptyLines) pins down the exact byte output for the "something follows" case, relying on precisely this borrowed newline. Changing that would mean either breaking a scenario that currently works, or touching the shared block map/sequence transition code that every other emitted node depends on (I tried the latter first, and it fixed this bug but broke two unrelated existing tests because of how those entry points unconditionally insert a line break before the next key regardless of column position).

So instead, WriteLiteralString now reports through an output parameter whether it owes the stream a trailing newline. Emitter remembers that in a small flag, clears it the moment any further node gets prepared (which is exactly the point where the existing mechanism supplies the break on its own), and c_str() writes the missing newline itself if the flag is still set when the buffer is finally read out. Everything that already worked continues to go through the same code path unchanged; only the "nothing follows" case gets the extra byte it was missing.

Added two tests that build a map and a sequence where the literal scalar is the last node written, then check the round-tripped value against the original rather than pinning exact byte output, since that's really what this needed to prove. Confirmed both fail on the prior code and pass after the fix; the rest of the suite, including the existing byte-exact chomping test, is untouched by this change.

WriteLiteralString() drops one trailing newline from 'strip' and 'keep'
chomping styles on the assumption that whatever gets emitted next will
start with the '\n' that separates it from this node, and that extra
character stands in for the one we dropped. That holds as long as
something else follows, but when the literal scalar is the last thing
written before the document ends (e.g. the last value in a map, or the
last entry in a sequence, with nothing after), no such newline ever
gets written, and the content silently comes out one line break short
on the next load.

Repro: emit a map whose only/last value is Literal << "a\nb\n" and
compare the round-tripped string against the original - it loses the
trailing break, and with two or more trailing breaks it loses one of
those too ('keep' style).

WriteLiteralString now reports whether it dropped such a newline, and
Emitter remembers that via a small flag, clearing it as soon as any
further node is prepared (which is exactly when the existing "borrow"
mechanism actually supplies the break). If nothing else is written,
c_str() adds the missing line break itself before returning the
buffer, so the scalar stands on its own either way. Added two
regression tests covering a literal as the final map value and as the
final sequence entry; the rest of the existing emitter suite, including
the sequence case that already relied on the follow-up newline, still
passes unchanged.
@afonsojanu

Copy link
Copy Markdown
Author

Closing this for now as part of a clean slate — stepping back from everything I had open across these repos.

@afonsojanu afonsojanu closed this Oct 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant