Theory enters observation through instrument design, calibration, selection and interpretation. Making those contributions visible allows evidence to challenge a model instead of being silently shaped to agree with it.
A detector needs a rule for an event
Imagine an instrument that stores electrical pulses. An analysis defines an event as a pulse exceeding a specified amplitude and separated from neighbouring pulses by a minimum interval. Changing either threshold can change the event count while leaving the original waveform unchanged.
The event list is therefore a processed record with a procedure behind it. This does not make the list arbitrary. The thresholds may be justified by noise measurements, response tests and the scientific question. But the list should retain enough information about those choices to support inspection and meaningful comparison with another analysis.
A proposed physical effect may also change the pulse shape or timing distribution, not just the number of selected events. If a selection rule was tuned on a different kind of signal, its efficiency for the proposed effect has to be calculated or measured.
A reconstruction can contain a preference
For an original finite example, suppose an ideal apparatus measures only the sum s = x₁ + x₂ and returns s = 10. Among nonnegative pairs, infinitely many candidates reproduce the observation. The data alone do not distinguish (5,5), (6,4) and (10,0).
A reconstruction that minimizes x₁² + x₂² selects (5,5). Substituting x₂ = 10 − x₁ gives a quadratic with its minimum at x₁ = 5. A different penalty that minimizes x₂² selects (10,0). Both choices fit the measured sum exactly; their different answers come from different preferences imposed on the reconstruction.
Such regularization can be useful. It may encode independently justified knowledge or stabilize an otherwise ill-conditioned problem. What it cannot do is turn the unmeasured split into an independently observed fact merely by displaying one preferred pair.
Ask what would test the preference
Suppose a second apparatus measures the difference d = x₁ − x₂ and returns d = 2. Together with s = 10, this determines x₁ = 6 and x₂ = 4 in the exact model. The new observable separates candidates that the first apparatus could not distinguish.
If the two channels have uncertainty, the recovered values also have uncertainty. If they share a calibration error, that dependence must be retained when the results are combined. The existence of an algebraic inverse is only one part of establishing that the inference is trustworthy.
This example clarifies a productive question about theoretical assumptions: which of them can be challenged by a new observation? An assumption that cannot be tested with the current apparatus may become testable after the measurement family is expanded.
Protect a test from hindsight
When analysts try many thresholds, filters or fitted forms, the selected result may reflect that exploration. Reporting the full analysis history helps readers distinguish an exploratory pattern from a result evaluated using a procedure fixed in advance.
For a planned test, specify the observable, selection rules, primary comparison and stopping rule before inspecting the reserved outcomes. Exploratory analyses remain useful; their role is often to generate a better model or a more informative next experiment. The distinction concerns how evidence was used, not whether exploration is legitimate.
Blinding a sensitive output during method development can also help in appropriate settings. Its value depends on the actual workflow, and it does not compensate for an inadequate instrument model. A clean separation between calibration and validation remains essential.
Separate an inner model from a chosen readout
In SMM/DTT, the family of admissible observations helps define which histories count as equivalent. That choice does real mathematical work. If a desired dimension, coupling or symmetry is already embedded in the detector rules, recovering it at the output may establish consistency without independently deriving it.
The expanded book's comparison-based distance makes the dependence explicit: the supplied comparison restrictions recover a graph distance. A stronger source-level result would explain why those restrictions and their calibration are selected.
This is not a demand for theory-free observation. It is a demand to locate theory's contribution so that a conditional theorem, a physical identification and a novel empirical restriction can each be assessed on their own terms. The essays on circular derivations and inference develop the same discipline.
Sources and further reading
- JCGM, Developing and using measurement models: guidance on specifying and assessing a measurement model.
- National Academies, Understanding Reproducibility and Replicability (2019). The two-channel reconstruction is an original teaching example.