Repository navigation
Use compile-time matchers for RegEx expressions - #1500
alex-thiessen-for-siemens wants to merge 2 commits into
Conversation
6a37f27 to
87573c2
Compare
Replace scanner and emitter expressions with private C++11 compile-time matchers. Eliminate runtime AST construction, dynamic vector storage, and static destruction in YAML::RegEx. Keep stream matcher instantiation inside yaml-cpp so Windows DLL consumers do not require unexported Stream internals. Add compile-time and boundary tests and a reproducible benchmark for matching and parser/emitter throughput. Reference verification: not applicable This changes only an internal matcher representation and lifetime behavior; it does not change YAML syntax, schema resolution, or native conversion semantics.
87573c2 to
2e28734
Compare
Add an integration death test covering jbeder#1281 to verify that emitting from a static object's destructor during program termination does not wrap plain scalars in double quotes due to destroyed matchers. Add static assertions covering jbeder#1471 ensuring RegEx and CharBitSet remain trivially destructible and trivially copyable. Reference verification: not applicable This covers static deinitialization emission to prevent regressions and asserts compile-time trivial destructibility; it does not change YAML syntax, schema resolution, or native conversion semantics. Signed-off-by: Alex Thiessen <alex.thiessen@siemens.com>
SGSSGene
left a comment
SGSSGene
left a comment
There was a problem hiding this comment.
I haven't read finished reading this PR yet, but I already have a few questions that I inlined.
As quick a recap: This replaces the RegEx implementation with a new implementation that does not rely on heap allocations (no malloc). This implies that there is nothing the destructors have to do for cleanup.
This in turn has the nice property, that if we enter a 'destructed' class, it still function as expected, since nothing has practically been destroyed?
| static int Match(const Source& source) { | ||
| const int first = First::Match(source); | ||
| return first >= 0 && Tail::Match(source) >= 0 ? first : -1; | ||
| } |
There was a problem hiding this comment.
Not sure how And is dealing with different length matching? To me it seems they should all have the same length? (maybe?)
| static int Match(const Source& source) { | |
| const int first = First::Match(source); | |
| return first >= 0 && Tail::Match(source) >= 0 ? first : -1; | |
| } | |
| static int Match(const Source& source) { | |
| const int first = First::Match(source); | |
| return first >= 0 && Tail::Match(source) == first ? first : -1; | |
| } |
| static int Match(const Source& source) { | ||
| return Pattern::Match(source) >= 0 ? -1 : 1; | ||
| } |
There was a problem hiding this comment.
I thought Match is returning the number of matched characters?
Why is the return value 1 and not 0? (I am not 100% what I would expect from a Nor operator)
| template <typename Pattern> | ||
| const RegEx& Matcher() { | ||
| static constexpr RegEx expression = MakeStreamRegEx<Pattern>(); | ||
| return expression; | ||
| } |
There was a problem hiding this comment.
If I understand correct, this is the only line that has a static member. (Which of course is instantiated many many times). Is that correct?
This branch has not been deployed
2 participants
Summary
This PR modernizes the internal regular expression engine used by the
scanner and emitter, converting expressions to compile-time matchers
implemented in C++11.
(
std::vector<RegEx>), and static destruction of global regularexpression objects (
YAML::Exp::*).deinitialization bugs reported in YAML Emitter using YAML::DoubleQuoted when emitted in a destructor of a staticly created struct/class #1281 and Avoid static destructors during program exit #1471.
CharBitSet) backed bystd::array<std::uint64_t, 4>with compile-time membership tests.templates (
Or<Pattern...>,And<Pattern...>,Seq<Pattern...>).emitter call sites.
Performance & Allocation Benchmarks
A dedicated microbenchmark (
regex_benchmarkinutil/regex_benchmark.cpp) was executed on Linux x86_64 to compareagainst the baseline engine:
allocations during initialization and recursive evaluation.
construction, pattern matching, and destruction.
Exp::Word: ~288 ns / iteration (0 allocations)Exp::Chomp: ~22 ns / iteration (0 allocations)Exp::DocStart: ~15 ns / iteration (0 allocations)improved due to elimination of pointer indirection and heap lookups
in hot scanning loops.
Verification
multi-character negation tests in
test/regex_test.cpp.