Two histories deserve the same predictive description when they imply the same probabilities for the future questions we allow. Matching the present display, or even the next observation, can be too weak.
The next result does not determine the next sequence
Consider two preparations that each produce a pair of binary outputs. Preparation A produces 00 or 11, each with probability one half. Preparation B produces 01 or 10, again equally likely. The first bit is fair under both preparations. So is the second bit considered separately.
The joint records differ completely: A always produces matching bits, while B always produces different bits. A summary that preserves only the probability of the next bit merges preparations that a two-bit test distinguishes. Even knowing both one-bit marginal distributions is insufficient to recover their correlation.
This finite example establishes a limited point about prediction horizons. It is not itself a full stationary process or a derivation of its minimal causal states. It shows why a future-test family must specify sequences and correlations, not merely isolated output frequencies.
Grouping histories by future behaviour
For a stochastic process, computational mechanics defines predictive equivalence through equality of conditional distributions over future observations. Histories with the same predictive distribution belong to the same causal state. In this technical usage, causal refers to the predictive structure of the process; it does not automatically supply the answer to an intervention question.
Exact equivalence requires agreement across the admitted future tests. In finite data, one can only estimate selected probabilities with uncertainty. A model that merges histories should therefore be tested on sequences not used to choose the grouping, over relevant time scales and preparations.
Predictive sufficiency is also relative to the available observations. Adding a new detector or intervention can distinguish histories that were equivalent under an earlier restricted experiment family. The observation family is part of the claim.
A small generator that remembers a parity
Consider a standard even-parity generator with two internal states E and O. In E, it emits 0 and stays in E with probability one half, or emits 1 and enters O with probability one half. In O, it must emit 1 and return to E. Between successive zeros, the number of ones is therefore even.
After a zero, the state is known to be E. Each subsequent one switches the parity. The most recent symbol alone is insufficient: histories ending in 01 and 011 both end in one, but their next-symbol probabilities differ. After 01, another one is certain; after 011, zero and one are equally likely.
In stationarity, E has probability 2/3 and O has probability 1/3. The entropy of the retained predictive state is h₂(1/3), approximately 0.918 bits. New-symbol uncertainty is generated only in E, so the entropy rate is 2/3 bit per output. Stored predictive information and newly generated randomness are different quantities.
An update must respect the grouping
Suppose histories h and h′ are identified by a summary q. A deterministic update F produces a well-defined summary update only if q(h) = q(h′) implies q(Fh) = q(Fh′). Otherwise the same advertised current state has incompatible next states. One must retain more information, restrict the model or include a dependence on earlier history.
In a stochastic model, the corresponding requirement concerns the distribution of subsequent outputs and states, with the relevant conditioning specified. An average over one preparation does not establish sufficiency for every preparation. The earlier essay How much memory does prediction need? illustrates this issue in a deterministic machine.
Connecting predictive states to completion
The expanded book treatment of equivalent histories makes future tests explicit. Its hidden-variable memory example then shows how omitting part of a state changes the form of the observable evolution. These connections make predictive-state reasoning especially relevant to SMM/DTT.
The programme still needs a physical selection of admissible histories, tests and probabilities. A minimal predictor of a measured sequence need not be a unique microscopic mechanism. Different generators may produce the same observed law. A stronger physical claim requires new preparations, interventions or observables that separate them, rather than simply assigning a preferred ontology to an already successful predictor.
Sources and further reading
- C. R. Shalizi and J. P. Crutchfield, Computational Mechanics: Pattern and Prediction, Structure and Simplicity, Journal of Statistical Physics 104 (2001), 817–879.
- M. Haj Yousef, Completion principles, real commutants, and positive-frequency geometry for orthogonal Hilbert representations, Open Journal of Mathematical Sciences 10 (2026), 996–1012.