Fix átomaton construction and decide reverse ε-machine finiteness - #12
Merged
Merged
Conversation
`atomaton_from_language` ended in a spurious `determinize()`, which collapsed
the átomaton back to the minimal DFA of the language -- Brzozowski's
minimization rather than the átomaton. The átomaton is the *reverse* of the
minimal DFA of the reverse language (Brzozowski & Tamm 2014, Theorem 2), so
the pipeline now stops at the reversal. On the paper's three-state example
this is the difference between 3 states and the correct 6 states with 3
initial.
Fill in the `AtomicAutomaton.validate` stub with Theorem 4: a state is atomic
iff the set of `N^{RD}` states containing it is a union of Nerode classes of
`N^{RD}`. `atomic_states` reports this per state and `is_atomic` aggregates,
which by Corollary 2 is equivalent to `N^{RD}` being minimal -- the condition
under which the subset construction yields a minimal DFA.
Extract `nerode_partition` from `minimize_moore` via a shared `_moore_refine`.
It deliberately does not trim: states with an empty right language have to
survive as their own class, or the atomicity test can report a false positive.
Tests cover Examples 5 and 6 of the paper, including all three atomic/
non-atomic combinations for the reverse.
Co-authored-by: Cursor <cursoragent@cursor.com>
A finite forward epsilon-machine does not imply a finite reverse one. When the reverse belief set never closes, from_time_reversed caught the generic StochasticValidationError that the max_states cap raises and fell back to _row_normalized_presentation, which merely row-normalizes the reversed HMM without determinizing. On a three-state witness that returns a non-unifilar machine whose state entropy equals the forward C_mu exactly, so causal_irreversibility() reported 0.0 -- 'perfectly reversible' -- for a process whose true Delta C_mu is -infinity. Give the cap its own MixedStateExplosionError (a StochasticValidationError, so existing handlers keep working) and re-raise it from from_time_reversed. Genuine UnifilarityError still falls back as before. The regression test uses a witness whose infinitude has a closed form: the block map multiplies the posterior ratio z/y by exactly 9/7, so the futures (01)^k have pairwise distinct posteriors for every k. Co-authored-by: Cursor <cursoragent@cursor.com>
Because the seed belief is uniform and the machine is unifilar, the
retrodictive causal states are exactly the normalized vectors
(Pr(x|s))_s over all words x. So the reverse machine is finite iff every
log-likelihood ratio log Pr(x|s) - log Pr(x|s') takes finitely many
values. Reading a symbol moves the pair (s,s') to (delta(s,a),
delta(s',a)) and multiplies the ratio by p(a|s)/p(a|s'), so:
the reverse epsilon-machine is finite
iff every cycle of that pair graph has weight one.
A cycle of weight g != 1 traversed k times gives ratios g^k; unit cycle
weights make the weight a potential difference, bounding the beliefs by
|S|^2. This is the twins property of weighted-automata determinization,
and since a unifilar epsilon-machine is an unambiguous weighted automaton
over a commutative cancellative semiring, the O(|Q|^2+|E|^2) test of
Allauzen & Mohri (2003) applies.
Validated against exact-rational brute force on all 432 strongly
connected 3-state binary machines (432/432), plus 4-state and ternary
samples and adversarial tuned probabilities where a cycle weight is 1 by
coincidence -- those are finite despite sharing structure with explosive
machines, so this is not a structural test.
Two implementation notes the validation caught: the pair graph must be a
MultiDiGraph, since two symbols can carry a pair to the same successor
with different ratios and collapsing them hides the inconsistency; and
numeric weights accumulate in log space, because products of ratios
underflow and make any absolute tolerance meaningless.
Co-authored-by: Cursor <cursoragent@cursor.com>
The phenomenon the explosion machinery is built around is named and worked out in Chaos 21 037107 -- Sec. VI B 3 gives a ternary process with two recurrent forward causal states and countably infinitely many reverse ones. Per the repo's literature-references rule, cite it where the behavior is introduced: MixedStateExplosionError, from_time_reversed, reverse_is_finite, and the two docs pages. Also corrects a wrong claim in those docs. An infinite reverse presentation does not make C_mu^- infinite -- C_mu^- is the entropy of the retrodictive stationary distribution, which converges when the weights decay geometrically, as they do in that example (C_mu^- is finite there). What explodes is the cardinality of the presentation, which is the actual reason there is nothing to return. Ellison2011 was already in references.bib but recorded as an arXiv preprint; upgraded to the published Chaos entry with its DOI. Co-authored-by: Cursor <cursoragent@cursor.com>
The pair-graph cycle-weight criterion is Theorem 5 of Allauzen & Mohri (2003), not a rephrasing: a trim cycle-unambiguous weighted automaton over a commutative cancellative semiring has the twins property iff every cycle of A cap A^-1 has weight one. Theorem 6 is the per-SCC potential algorithm this function implements. Corrects the hypothesis from 'unambiguous' to 'cycle-unambiguous', which is both weaker and the one that actually holds (unifilarity gives one path per state/word). Records what is not being invoked: their 'twins iff determinizable' equivalence is for trim unambiguous automata over the tropical semiring, and they note twins does not imply determinizable over the real semiring for infinitely ambiguous automata. Full-support seeding makes every state initial, so this automaton is |S|-ambiguous over the real semiring. The equivalence comes from the direct log-ratio argument instead. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Carries the five átomaton / time-reversal commits that were merged into
cursor/wheeler-automatavia #10 after #9 had already landed, so they never reachedmain.cursor/wheeler-automatahas a byte-identical tree to this branch (e636fdb) — it is just these five commits plus the merge commit from #10 — so this single PR covers all the unmerged work on both branches, and both can be deleted once it lands.What's here
Átomaton construction and atomicity (
sofic/automata/atomaton.py,algorithms.py) —atomaton_from_languageended in a spuriousdeterminize(), which collapsed the átomaton back to the minimal DFA of the language, i.e. Brzozowski's minimization rather than the átomaton. The átomaton is the reverse of the minimal DFA of the reverse language (Brzozowski & Tamm 2014, Theorem 2), so the pipeline now stops at the reversal. On the paper's three-state example that is the difference between 3 states and the correct 6 states with 3 initial.The
AtomicAutomaton.validatestub is filled in with Theorem 4: a state is atomic iff the set ofN^{RD}states containing it is a union of Nerode classes ofN^{RD}.nerode_partitionis extracted fromminimize_moorevia a shared_moore_refine, and deliberately does not trim — states with an empty right language have to survive as their own class or the atomicity test reports a false positive.Explosion is propagated, not swallowed (
sofic/generators/epsilon_machine.py,mixed_state_construction.py,sofic/exceptions.py) — a finite forward ε-machine does not imply a finite reverse one. When the reverse belief set never closed,from_time_reversedcaught the genericStochasticValidationErrorraised by themax_statescap and fell back to_row_normalized_presentation, which row-normalizes the reversed HMM without determinizing. On a three-state witness that returns a non-unifilar machine whose state entropy equals the forwardC_μexactly, socausal_irreversibility()reported0.0— "perfectly reversible" — for a process whose trueΔC_μis-∞.The cap now raises its own
MixedStateExplosionError(aStochasticValidationError, so existing handlers keep working) andfrom_time_reversedre-raises it. GenuineUnifilarityErrorstill falls back as before.Deciding reverse finiteness (
sofic/generators/reversal.py) — the newreverse_is_finite. Because the seed belief is uniform and the machine unifilar, the retrodictive causal states are exactly the normalized vectors(Pr(x|s))_s, so the reverse machine is finite iff every log-likelihood ratio takes finitely many values. Reading a symbol moves the pair(s, s')to(δ(s,a), δ(s',a))and multiplies the ratio byp(a|s)/p(a|s'), giving:That is the twins property of weighted-automata determinization, so the
O(|Q|²+|E|²)test of Allauzen & Mohri (2003) applies.Verification
1031 tests pass (21 more than
main),ruffclean, merges cleanly intomain.reverse_is_finitewas validated against exact-rational brute force on all 432 strongly connected 3-state binary machines (432/432), plus 4-state and ternary samples, and adversarial tuned probabilities where a cycle weight is 1 by coincidence — those come out finite despite sharing structure with explosive machines, so this is not a structural test in disguise.Two implementation details that validation caught are worth knowing about: the pair graph must be a
MultiDiGraph, since two symbols can carry a pair to the same successor with different ratios and collapsing them hides the inconsistency; and weights accumulate in log space, because products of ratios underflow and make any absolute tolerance meaningless.The explosion regression test uses a witness whose infinitude has a closed form — the block map multiplies the posterior ratio
z/yby exactly9/7, so the futures(01)^khave pairwise distinct posteriors for everyk.Citations
Per the repo's
literature-referencesrule, Ellison et al. (2011) is cited where the explosion behavior is introduced (MixedStateExplosionError,from_time_reversed,reverse_is_finite, and the two docs pages); Chaos 21 037107 Sec. VI B 3 gives a ternary process with two recurrent forward causal states and countably infinitely many reverse ones.Ellison2011was already inreferences.bibas an arXiv preprint and is upgraded to the published Chaos entry with its DOI.The docs also carried a wrong claim, now corrected: an infinite reverse presentation does not make
C_μ^-infinite.C_μ^-is the entropy of the retrodictive stationary distribution, which converges when the weights decay geometrically, as they do in that example. What explodes is the cardinality of the presentation, which is the actual reason there is nothing to return.The twins citation is stated precisely: the pair-graph cycle-weight criterion is Theorem 5 of Allauzen & Mohri (2003), with Theorem 6 the per-SCC potential algorithm actually implemented. The hypothesis is cycle-unambiguous, not unambiguous — weaker, and the one that actually holds, since unifilarity gives one path per state/word. Their "twins iff determinizable" equivalence is explicitly not invoked: it is for trim unambiguous automata over the tropical semiring, and full-support seeding makes every state initial, so this automaton is
|S|-ambiguous over the real semiring. The equivalence used here comes from the direct log-ratio argument instead.Made with Cursor