What does entropy measure?

“Entropy measures disorder” is a familiar introduction, but it leaves the important work undone. What counts as disorder? Which microscopic differences are being ignored? And are we discussing a physical system, a probability distribution or a description of what an observer knows?

A small counting problem gives a more precise starting point. It also reveals why a statistical preference for a kind of state is not yet a law about how that state changes.

Describe the system at two levels

Imagine six distinguishable switches, each either off or on. A complete microscopic description lists all six values. For example, 001101 identifies which three switches are on.

Now suppose the available display records only the number n of switches that are on. It maps many different microscopic configurations to the same coarse description. Both 001101 and 110010 produce n = 3.

There are 2⁶ = 64 configurations altogether. The number belonging to the coarse description n is Ω(n) = 6!/[n!(6−n)!]. This count depends on the description we chose. A display that retained the first switch's value as well as the total would make finer distinctions.

Count the possibilities

Why take a logarithm?

If two independent systems have Ω₁ and Ω₂ compatible configurations, their combined count is Ω₁Ω₂. Taking a logarithm turns this multiplication into addition: ln(Ω₁Ω₂) = ln Ω₁ + ln Ω₂.

This supplies an elementary reason for the form of Boltzmann's entropy, S = k_B ln Ω. In a physical application, k_B supplies entropy units and the model must specify the permitted microstates, constraints and counting measure. For continuous state spaces, an appropriate measure replaces a simple finite count.

Our six-switch example illustrates the counting structure. It does not assign a measured thermodynamic entropy to an actual device without modelling its physical degrees of freedom.

Knowledge and multiplicity

Suppose we know that the actual configuration is 001101. There is no uncertainty about its six values in this ideal description. Nevertheless, it belongs to the twenty-member class n = 3. The size of that class has not changed because someone learned the detailed state.

Alternatively, suppose we know only n = 3 and assign equal probabilities to the twenty compatible configurations. The Shannon information entropy of that specified distribution is log₂20 bits. It expresses uncertainty over microscopic configurations under those assumptions.

These two uses of entropy can be related, but the question and the probability distribution must be stated. Entropy is not simply a feeling of confusion, and a change in someone's knowledge is not automatically a physical heat transfer.

A count is not a dynamics

Twenty configurations have n = 3, while only one has n = 0. Does that prove that a system starting with every switch off must move towards n = 3? No. We have not supplied a rule that changes any switch.

A rule that does nothing leaves the initial state unchanged. A rule that flips all six switches together alternates between n = 0 and n = 6. Both remain compatible with the same counting table.

This is why explaining a thermodynamic tendency requires more than identifying a large macrostate. The dynamics, the preparation and the conditions under which a statistical description is applicable matter too. The companion essay on the direction of processes develops that question.

From counts to the research programme

In SMM/DTT, a completed record may retain fewer distinctions than the generative description. The switch display makes one aspect of that relationship explicit: a readout groups underlying states. To connect such a grouping with physical entropy production requires a further account of evolution, resources and observation.

Use this example when reading a claim that completion produces irreversibility. Identify the retained variables, count or measure the states they group together, and locate the dynamical argument. A definition of readout and a demonstrated direction of change are separate achievements.

Sources and further reading

• Lebowitz (1993), “Macroscopic laws, microscopic dynamics, time's arrow and Boltzmann's entropy,” Physica A 194, 1–27: the physical statistical-mechanical setting.
• How much memory does prediction need?: distinctions retained by a readout.
• Dual-Time Topological Geometry and the Emergence of Temporal Asymmetry in Non-Equilibrium Dynamics: the programme's related published framework.

The six-switch construction and its counts are supplied here as a teaching example. No physical entropy-production measurements are reported.