Add co-lexicographic (Wheeler) ordering

docs/automata/automata.rst

@@ -46,6 +46,7 @@ Canonical structures
    atomaton
    observation_table
    learning
+   wheeler
 
 Advanced
 ========

docs/automata/wheeler.rst

@@ -0,0 +1,150 @@
+.. wheeler.rst
+.. py:module:: sofic.automata.wheeler
+
+********************************
+Co-lexicographic (Wheeler) order
+********************************
+
+A labeled graph is *Wheeler* when its states admit a total order in which the
+states with no incoming edges come first and, for edges ``(u, v)`` labeled
+``a`` and ``(u', v')`` labeled ``a'``,
+
+* ``a < a'`` implies ``v < v'``, and
+* ``a == a'`` and ``u < u'`` imply ``v <= v'``
+
+:cite:`Gagie2017`. Equivalently, the words reaching each state form an
+interval of the co-lexicographically sorted prefixes of the language
+:cite:`Alanko2020`. Co-lex order compares words from the last symbol backwards,
+so a Wheeler presentation is one whose states partition history space into
+*bands of recency* rather than arbitrary sets.
+
+Wheeler orders are the width-one case of the co-lexicographic partial orders of
+:cite:`CotumaccioPrezza2021`. :func:`colex_width` measures how far a machine is
+from being Wheeler; the width bounds the cost of indexing it, storing it, and
+determinizing it.
+
+Presentations versus languages
+==============================
+
+:func:`is_wheeler` asks whether *this presentation* is Wheeler. For a
+deterministic presentation that is a sorting question, answerable in polynomial
+time; for a general labeled graph, recognizing Wheelerness is NP-complete
+:cite:`GibneyThankachan2019`. Whether the *language* is Wheeler — whether any
+equivalent automaton is — is harder still: ``O(mn)`` for a DFA
+:cite:`Becker2023`, improving the first polynomial algorithm
+:cite:`Alanko2021`, and PSPACE-complete for an NFA :cite:`DAgostino2023`.
+:func:`~sofic.shifts.wheeler.wheeler_cover` and
+:func:`~sofic.generators.wheeler_epsilon.wheeler_presentation` search over
+presentations for the process-level question.
+
+Wheeler is not a restatement of finite memory. The golden mean process is
+Wheeler; so is
+:func:`~sofic.examples.epsilon_machines.wheeler_infinite_order_process`, whose
+Markov order is infinite. Conversely the even process is not Wheeler in any
+presentation, because Wheeler languages are star-free
+:cite:`ShyrThierrin1974` :cite:`Alanko2021` and the even process counts
+``1``\ s modulo two.
+
+.. ipython::
+
+   In [1]: from sofic.examples import golden_mean, even_process, wheeler_infinite_order_process
+
+   In [2]: golden_mean().wheeler_order().states
+
+   In [3]: even_process().is_wheeler(), even_process().colex_width()
+
+   In [4]: machine = wheeler_infinite_order_process(); machine.is_wheeler(), machine.markov_order()
+
+Every graph-backed model inherits :meth:`~sofic.base.StateMachine.is_wheeler`,
+:meth:`~sofic.base.StateMachine.wheeler_order`, and
+:meth:`~sofic.base.StateMachine.colex_width`, so automata, shifts, and
+ε-machines all answer the same questions.
+
+What the order buys
+===================
+
+* A canonical state numbering, hence canonical transition matrices and
+  ``O(m)`` equality by comparing Burrows-Wheeler strings
+  (:func:`wheeler_canonical_form`, :func:`wheeler_isomorphic`).
+* Path coherence: the states reachable from an interval on a given symbol are
+  again an interval. :func:`~sofic.generators.synchronization.power_automaton`
+  uses this to replace ``2^n`` subsets with ``n(n+1)/2`` intervals, which makes
+  the Markov order, cryptic order, and reset threshold polynomial.
+* Determinization to at most ``2n - 1 - |Sigma|`` states
+  (:func:`wnfa_to_wdfa`) and a unique minimal WDFA (:func:`minimum_wdfa`),
+  both of which fail for general automata.
+* A succinct index — see below.
+
+Order
+=====
+
+.. autoclass:: WheelerOrder
+   :members: validate
+.. autoclass:: LabeledGraph
+.. autoexception:: WheelerError
+
+.. autofunction:: labeled_graph
+.. autofunction:: wheeler_order
+.. autofunction:: wheeler_order_of_graph
+.. autofunction:: is_wheeler
+.. autofunction:: is_input_consistent
+.. autofunction:: check_wheeler_axioms
+
+Width
+=====
+
+.. autofunction:: colex_width
+.. autofunction:: maximum_colex_relation
+.. autofunction:: maximum_colex_relation_of_graph
+
+Canonical forms and minimization
+================================
+
+.. autofunction:: minimum_wdfa
+.. autofunction:: wnfa_to_wdfa
+.. autofunction:: wheeler_canonical_form
+.. autofunction:: wheeler_isomorphic
+.. autofunction:: wheeler_state_index
+
+Burrows-Wheeler index
+=====================
+
+.. py:currentmodule:: sofic.automata.wheeler_index
+
+:class:`WheelerIndex` stores a Wheeler machine as the out-degree, in-degree,
+and label arrays of :cite:`Gagie2017`. Because the Wheeler axioms make the
+edges sorted by ``(label, source)`` coincide with the edges sorted by target,
+following a symbol maps one node interval onto another — FM-index backward
+search, generalized to labeled graphs. Rank is served by binary search over
+per-symbol position arrays, which costs a logarithmic factor but adds no
+dependency beyond numpy.
+
+A shift presents its factor language with every state both initial and
+accepting, the case :cite:`Gagie2017` Theorem 6 covers explicitly, so the same
+index answers membership and enumeration queries for shifts.
+
+.. ipython::
+
+   In [1]: from sofic.examples import golden_mean
+
+   In [2]: from sofic.automata.wheeler_index import WheelerIndex
+
+   In [3]: index = WheelerIndex.from_model(golden_mean()); index.bits()
+
+   In [4]: index.contains((0, 1, 0)), index.contains((1, 1))
+
+   In [5]: list(index.words_of_length(3))
+
+   In [6]: index.unrank_word(index.rank_word((0, 1, 0)), 3)
+
+Words are listed in co-lexicographic order, so :meth:`WheelerIndex.rank_word`
+and :meth:`WheelerIndex.unrank_word` invert one another and
+:meth:`WheelerIndex.sample_word` draws uniformly from the words of a length
+without enumerating them.
+
+.. autoclass:: WheelerIndex
+   :members: from_model, bits, step, forward_search, contains, count_states,
+             states_reached, count_words, words_of_length, rank_word,
+             unrank_word, sample_word
+
+.. autofunction:: wheeler_index

docs/generators/generators.rst

@@ -38,6 +38,7 @@ Computational mechanics
    generative_models
    directional_flow
    alternative_complexity
+   wheeler_epsilon
 
 Constructions and conversions
 =============================

docs/generators/wheeler_epsilon.rst

@@ -0,0 +1,103 @@
+.. wheeler_epsilon.rst
+.. py:module:: sofic.generators.wheeler_epsilon
+
+**********************************
+Co-lexicographic epsilon-machines
+**********************************
+
+.. warning::
+
+   Everything on this page is original and uncited. Wheeler automata are a
+   purely topological theory :cite:`Gagie2017` :cite:`Alanko2020`; a literature
+   search turned up no treatment of weighted, probabilistic, or
+   information-theoretic Wheeler automata, so no canonical source exists for
+   the quantities defined here. They are proposals, and every docstring says
+   so.
+
+Co-lexicographic order compares words from the last symbol backwards, which is
+the *recency* order on pasts. A presentation is Wheeler exactly when its
+partition of history space is an interval partition under recency: each state
+owns a contiguous band of pasts rather than an arbitrary set. Two consequences
+follow, one measure-theoretic and one information-theoretic.
+
+The stationary distribution becomes a genuine cumulative distribution over
+pasts. Under an arbitrary state numbering the partial sums of the stationary
+vector mean nothing; in Wheeler order they sweep history space from the most
+remote pasts to the most recent, which is what arithmetic coding over pasts
+needs. :func:`colex_cdf` returns that sweep, :func:`cylinder_measure` the mass
+of a rank interval, and :func:`word_cylinder_measure` the mass of the interval
+a word selects — pairing directly with
+:meth:`~sofic.automata.wheeler_index.WheelerIndex.forward_search`.
+
+The constraint has a price in bits. A process whose causal states are not
+recency intervals must split them to get one, and the split states cost
+entropy. :func:`wheeler_statistical_complexity` measures the result and
+:func:`wheeler_complexity_gap` the excess over :math:`C_\mu`.
+
+.. math::
+
+   C_W = \operatorname{H}[\text{Wheeler states}] \ge \max(C_\mu, \operatorname{H}[X_0]),
+
+the first bound because every Wheeler presentation refines the causal-state
+partition, with equality exactly when the ε-machine is already Wheeler; the
+second because a Wheeler presentation is input consistent, so its state
+determines the symbol that entered it. The fair coin is the extreme case of the
+second bound: nothing at all to predict, :math:`C_\mu = 0`, yet sortability
+costs a full bit of memory for a symbol the process will never reuse.
+
+.. ipython::
+
+   In [1]: from sofic.examples import golden_mean, fair_coin
+
+   In [2]: from sofic.generators.wheeler_epsilon import colex_cdf, wheeler_complexity_gap
+
+   In [3]: machine = golden_mean(); machine.is_wheeler()
+
+   In [4]: colex_cdf(machine)
+
+   In [5]: machine.wheeler_statistical_complexity(), machine.statistical_complexity()
+
+   In [6]: coin = fair_coin(); coin.is_wheeler(), coin.statistical_complexity()
+
+   In [7]: coin.wheeler_statistical_complexity(), wheeler_complexity_gap(coin)
+
+Finding a presentation
+======================
+
+:func:`wheeler_presentation` returns the machine itself when it is already
+Wheeler. Otherwise, when the Markov order ``R`` is finite, it returns the
+order-``R`` de Bruijn presentation, whose states *are* the length-``R`` words
+and so sort co-lexicographically by construction — every finite-order process
+is therefore a Wheeler *language* even when its ε-machine is not a Wheeler
+*presentation*. Failing that it tries edge-machine refinements, whose order-``k``
+states are length-``k`` transition paths and are therefore entered on a single
+symbol.
+
+The search can fail for good reason: Wheeler languages are star-free
+:cite:`Alanko2021`, so the even process has no Wheeler presentation at any
+order and :func:`wheeler_presentation` raises
+:class:`~sofic.automata.wheeler.WheelerError`.
+
+The converse separation also holds.
+:func:`~sofic.examples.epsilon_machines.wheeler_infinite_order_process` is a
+five-state Wheeler ε-machine of infinite Markov order, so Wheelerness is not a
+restatement of finite memory in either direction.
+
+API
+===
+
+.. autofunction:: wheeler_presentation
+.. autofunction:: debruijn_presentation
+.. autofunction:: wheeler_statistical_complexity
+.. autofunction:: wheeler_complexity_gap
+.. autofunction:: colex_cdf
+.. autofunction:: cylinder_measure
+.. autofunction:: word_cylinder_measure
+
+:class:`~sofic.generators.epsilon_machine.EpsilonMachine` also exposes
+:meth:`~sofic.generators.epsilon_machine.EpsilonMachine.wheeler_presentation`
+and
+:meth:`~sofic.generators.epsilon_machine.EpsilonMachine.wheeler_statistical_complexity`,
+alongside the :meth:`~sofic.base.StateMachine.is_wheeler`,
+:meth:`~sofic.base.StateMachine.wheeler_order`, and
+:meth:`~sofic.base.StateMachine.colex_width` methods every model inherits.

docs/references.bib

@@ -902,6 +902,107 @@ @inproceedings{Volkov2008
   doi       = {10.1007/978-3-540-88282-4_4},
 }
 
+@article{Gagie2017,
+  author  = {Gagie, Travis and Manzini, Giovanni and Sir{\'e}n, Jouni},
+  title   = {Wheeler Graphs: A Framework for {BWT}-Based Data Structures},
+  journal = {Theoretical Computer Science},
+  volume  = {698},
+  pages   = {67--78},
+  year    = {2017},
+  doi     = {10.1016/j.tcs.2017.06.016},
+}
+
+@inproceedings{Alanko2020,
+  author    = {Alanko, Jarno and D'Agostino, Giovanna and Policriti, Alberto and Prezza, Nicola},
+  title     = {Regular Languages Meet Prefix Sorting},
+  booktitle = {Proceedings of the Thirty-First Annual {ACM-SIAM} Symposium on Discrete Algorithms ({SODA})},
+  pages     = {911--930},
+  publisher = {SIAM},
+  year      = {2020},
+  doi       = {10.1137/1.9781611975994.55},
+}
+
+@article{Alanko2021,
+  author  = {Alanko, Jarno and D'Agostino, Giovanna and Policriti, Alberto and Prezza, Nicola},
+  title   = {Wheeler Languages},
+  journal = {Information and Computation},
+  volume  = {281},
+  pages   = {104820},
+  year    = {2021},
+  doi     = {10.1016/j.ic.2021.104820},
+}
+
+@inproceedings{CotumaccioPrezza2021,
+  author    = {Cotumaccio, Nicola and Prezza, Nicola},
+  title     = {On Indexing and Compressing Finite Automata},
+  booktitle = {Proceedings of the Thirty-Second Annual {ACM-SIAM} Symposium on Discrete Algorithms ({SODA})},
+  pages     = {2585--2599},
+  publisher = {SIAM},
+  year      = {2021},
+  doi       = {10.1137/1.9781611976465.153},
+}
+
+@article{Cotumaccio2023,
+  author  = {Cotumaccio, Nicola and D'Agostino, Giovanna and Policriti, Alberto and Prezza, Nicola},
+  title   = {Co-lexicographically Ordering Automata and Regular Languages --- Part {I}},
+  journal = {Journal of the {ACM}},
+  volume  = {70},
+  number  = {4},
+  pages   = {27:1--27:73},
+  year    = {2023},
+  doi     = {10.1145/3607471},
+  eprint  = {2208.04931},
+  archivePrefix = {arXiv},
+}
+
+@inproceedings{GibneyThankachan2019,
+  author    = {Gibney, Daniel and Thankachan, Sharma V.},
+  title     = {On the Hardness and Inapproximability of Recognizing Wheeler Graphs},
+  booktitle = {27th Annual European Symposium on Algorithms ({ESA})},
+  series    = {Leibniz International Proceedings in Informatics ({LIPIcs})},
+  volume    = {144},
+  pages     = {51:1--51:16},
+  publisher = {Schloss Dagstuhl--Leibniz-Zentrum f{\"u}r Informatik},
+  year      = {2019},
+  doi       = {10.4230/LIPIcs.ESA.2019.51},
+}
+
+@inproceedings{Becker2023,
+  author    = {Becker, Ruben and Cenzato, Davide and Kim, Sung-Hwan and Kodric, Bojana and Policriti, Alberto and Prezza, Nicola},
+  title     = {Optimal {W}heeler Language Recognition},
+  booktitle = {String Processing and Information Retrieval ({SPIRE})},
+  series    = {Lecture Notes in Computer Science},
+  volume    = {14240},
+  pages     = {62--74},
+  publisher = {Springer},
+  year      = {2023},
+  doi       = {10.1007/978-3-031-43980-3_6},
+  eprint    = {2306.04737},
+  archivePrefix = {arXiv},
+}
+
+@article{ShyrThierrin1974,
+  author  = {Shyr, H. J. and Thierrin, Gabriel},
+  title   = {Ordered Automata and Associated Languages},
+  journal = {Tamkang Journal of Mathematics},
+  volume  = {5},
+  number  = {1},
+  pages   = {9--20},
+  year    = {1974},
+}
+
+@article{DAgostino2023,
+  author  = {D'Agostino, Giovanna and Martincigh, Davide and Policriti, Alberto},
+  title   = {Ordering Regular Languages and Automata: Complexity},
+  journal = {Theoretical Computer Science},
+  volume  = {949},
+  pages   = {113709},
+  year    = {2023},
+  doi     = {10.1016/j.tcs.2023.113709},
+  eprint  = {2203.12534},
+  archivePrefix = {arXiv},
+}
+
 @misc{TetrisWikiNES,
   author       = {{Tetris Wiki}},
   title        = {Tetris ({NES})},

docs/shifts/shifts.rst

@@ -22,3 +22,4 @@ type, Sofic shifts, Dyck shifts, topological Markov chains, and covers
    textile
    dyck_enumeration
    covers
+   wheeler