Skip to content

REALITY & TIME · EXISTENCE AND THE WORLD WE EXPERIENCE

Fit a model then test it

Fit a line and a quadratic to declared training rows, freeze their coefficients and reproduce the held-out comparison in the existing workbench.

A fitted curve answers how a chosen model accommodates the data used to determine it. Testing asks how the fixed model performs on observations that did not influence that choice. Keeping those roles separate makes a small exercise scientifically instructive.

Declare the models and the split

Compare a straight line a + bx with a quadratic a + bx + cx². Use dimensionless x and y so units do not distract from the fitting protocol. Choose the models and reserve the test rows before looking at their residuals.

Our constructed training observations are (0, 1.1), (1, 2.9), (2, 5.1), (3, 6.9) and (4, 9.1). The reserved observations are (5, 11) and (6, 13). These numbers were invented for an exercise and do not measure a physical effect.

Fit only the training rows

Ordinary least squares minimizes the sum of squared training residuals. For the line, it gives a = 1.02 and b = 2. The training predictions are 1.02, 3.02, 5.02, 7.02 and 9.02, with residuals 0.08, −0.12, 0.08, −0.12 and 0.08. Their squared sum is 0.048.

The quadratic fit gives approximately a = 1.077143, b = 1.885714 and c = 0.0285714. Its training squared-error sum is about 0.0365714. The additional coefficient improves the training fit, as expected for this nested model pair.

Freeze the coefficients and test

At x = 5 and 6, the fixed line predicts 11.02 and 13.02. The fixed quadratic predicts 11.22 and 13.42. Subtracting predictions from the reserved observations gives the following results:

Original held-out comparison; no coefficients refitted on these rows
ModelResidual at x = 5Residual at x = 6Test root-mean-square error
Straight line−0.02−0.020.02
Quadratic−0.22−0.42Approximately 0.3353

The quadratic has lower training error but higher test error in this exercise. That is a concrete result about these rows and this protocol. Two reserved observations do not establish that straight lines are generally better models.

Reproduce the workbench calculation

Open Experimental Comparison. Choose “Straight line → quadratic” and “Use the CSV split column.” Paste the following data, then compare the coefficients and residuals with the values above:

x,y,split
0,1.1,train
1,2.9,train
2,5.1,train
3,6.9,train
4,9.1,train
5,11,test
6,13,test

No sigma column is supplied, so this is the ordinary least-squares case. The test allocation is explicit rather than inferred from row order. Preserve the complete export so the training and test roles remain visible.

Avoid consuming the test twice

If we inspect these test errors and then search through many additional formulas until one performs well on the same rows, those rows have become part of model selection. Reporting the final agreement as an untouched test would misdescribe the process.

A later independent comparison can evaluate the revised model. Likewise, a physically motivated restriction chosen before the data is different from a restriction invented after an inconvenient residual appears. Record both choices and their timing.

Connect fit quality with physical meaning

A good prediction of y does not identify what x and y represent physically. It also does not establish that an added quadratic coefficient comes from inner-time generation. The prediction-and-comparison guide asks for the source model, physical dictionary and reserved consequences.

Continue with prediction after calibration, sensitivity and provenance. The useful contribution of a fit is clearest when its inputs, scope and subsequent tests remain available for inspection.

Sources and further reading

  1. Mohamed Haj Yousef, Experimental Comparison workbench; Make a prediction survive calibration and comparison.
  2. The seven observations, training split, coefficients and residuals are an original reproducible teaching exercise. They support no claim of a new physical detection.

Continue reading

Essay · 4 min read

How AI models temporal patterns

Distinguish sequence position, elapsed time and prediction; examine temporal information, held-out records and the limits of a successful pattern.

Essay · 5 min read

Designing a fair comparison between models

Compare two relaxation models using reserved measurements and an explicit noise model, while keeping parameter choices and detector assumptions visible.

Follow a connected reading path →

SMONAD

Share this page

Share this page with its own title, summary and preview image.

More platforms