Skip to article frontmatterSkip to article content
Site not loading correctly?

This may be due to an incorrect BASE_URL configuration. See the MyST Documentation for reference.

4.3. Microscopic View of Entropy

See also: Course-wide Conventions & Notation

Run the code: Notebook for Section 4.3, in which you can change and rerun the Python cells of this section in your browser.

Overview and Learning Objectives

Section 4.1 and Section 4.2 developed entropy entirely from macroscopic reasoning: a state function defined by dS=δqrev/TdS = \delta q_{\mathrm{rev}}/T, used to derive engine efficiency limits and the direction of spontaneous heat flow. This section shifts to the microscopic side, connecting entropy to the statistical-mechanical framework from Module 2. The payoff is substantial: entropy becomes a measure of how many ways a system can realize a macrostate, and its connection to the partition function QQ provides a direct bridge between microscopic energy levels and macroscopic thermodynamic potentials.

We introduce three formulas for entropy — Boltzmann’s S=kBln⁡ΩS = k_{\mathrm{B}}\ln\Omega (isolated systems), the more general Gibbs form S=−kB∑piln⁡piS = -k_{\mathrm{B}}\sum p_i\ln p_i (any ensemble), and the canonical identity S=U/T+kBln⁡QS = U/T + k_{\mathrm{B}}\ln Q (the Gibbs form evaluated for the canonical ensemble) — and show that they are mutually consistent. The section culminates in the identification of the Helmholtz free energy A=−kBTln⁡QA = -k_{\mathrm{B}}T\ln Q, which was foreshadowed in Section 2.3 and Section 3.3.

Learning objectives:

Core Ideas and Derivations

Entropy Increases Until the System Reaches Equilibrium

The second law for an isolated system (Section 4.2) requires

dS≥0,dS \ge 0,

with equality only at equilibrium. The qualitative picture, which Figure 4.3.1 sketches, is:

Source
Loading...

Figure 4.3.1:Entropy of an isolated system during a spontaneous process (schematic). The entropy rises from SiS_i to its maximum value SfS_f and then stays constant at equilibrium.

A concrete example from Section 4.2 illustrates this. Two subsystems at different temperatures (TA>TBT_A > T_B) in an insulated box exchange heat, and we showed that

dS=dUA ⁣(1TA−1TB)>0dS = dU_A\!\left(\frac{1}{T_A} - \frac{1}{T_B}\right) > 0

as long as TA≠TBT_A \neq T_B. Energy flows from AA to BB, raising TBT_B and lowering TAT_A, until TA=TBT_A = T_B. At that point the factor (1/TA−1/TB)(1/T_A - 1/T_B) vanishes, dS=0dS = 0, and the system has reached thermal equilibrium at the entropy maximum.


The Clausius Inequality

The entropy definition dS=δqrev/TdS = \delta q_{\mathrm{rev}}/T applies specifically to reversible heat transfer. A more general statement, valid for both reversible and irreversible processes, is the Clausius inequality:

dS≥δqT.dS \ge \frac{\delta q}{T}.

Here TT is the temperature at which the heat is transferred to the system. For heat exchanged with a reservoir, it is the temperature of the reservoir.

The physical interpretation is that irreversibility produces additional entropy beyond the entropy “carried in” by heat flow. For an isolated system (δq=0\delta q = 0), the Clausius inequality reduces to dS≥0dS \ge 0, recovering the second law.

The Clausius inequality also shows why the Carnot engine (Section 4.2) sets the maximum efficiency. Consider any engine, reversible or not, that in each cycle absorbs heat qhot>0q_{\mathrm{hot}} > 0 from the hot reservoir and releases heat to the cold reservoir (qcold<0q_{\mathrm{cold}} < 0). For the Carnot engine these are qABq_{AB} and qCDq_{CD}. The working substance returns to its initial state, so its entropy change over the cycle is zero, and the Clausius inequality, summed over the cycle, gives

0=ΔScycle≥qhotThot+qcoldTcold.0 = \Delta S_{\mathrm{cycle}} \ge \frac{q_{\mathrm{hot}}}{T_{\mathrm{hot}}} + \frac{q_{\mathrm{cold}}}{T_{\mathrm{cold}}}.

Because qhot>0q_{\mathrm{hot}} > 0, this rearranges to qcold/qhot≤−Tcold/Thotq_{\mathrm{cold}}/q_{\mathrm{hot}} \le -T_{\mathrm{cold}}/T_{\mathrm{hot}}. The efficiency, defined as in Section 4.2, is therefore

η=qhot+qcoldqhot=1+qcoldqhot≤1−TcoldThot=ηCarnot.\eta = \frac{q_{\mathrm{hot}} + q_{\mathrm{cold}}}{q_{\mathrm{hot}}} = 1 + \frac{q_{\mathrm{cold}}}{q_{\mathrm{hot}}} \le 1 - \frac{T_{\mathrm{cold}}}{T_{\mathrm{hot}}} = \eta_{\mathrm{Carnot}}.

Equality holds only for a reversible engine. An irreversible engine releases more heat to the cold reservoir for the same heat absorbed, so it delivers less work.


Entropy and the Number of Microstates: Boltzmann’s Formula

We now connect entropy to the microscopic picture. For an isolated system (microcanonical ensemble), the fundamental postulate of statistical mechanics (Section 2.1) assigns equal probability pi=1/Ωp_i = 1/\Omega to each of the Ω\Omega accessible microstates. The entropy of such a system is

S=kBln⁡Ω.S = k_{\mathrm{B}} \ln \Omega.

Here Ω\Omega is the number of microstates compatible with the macroscopic constraints (fixed UU, VV, NN). You can think of Ω\Omega as the degeneracy of the macrostate: fix the total energy and other extensive quantities, then count how many distinct microscopic configurations share those values.

As an example, place NN distinguishable particles in a box and record only which half of the box each particle is in. A microstate is one assignment of the particles to the two halves, and there are 2N2^N of them. The macrostate with nLn_L particles in the left half contains

Ω(nL)=N!nL! (N−nL)!\Omega(n_L) = \frac{N!}{n_L!\,(N - n_L)!}

microstates. Figure 4.3.2 shows all microstates for N=4N = 4 and the shape of Ω(nL)\Omega(n_L) for larger NN.

Source
Loading...

Figure 4.3.2:Microstates of NN distinguishable particles in the two halves of a box. (a) The 24=162^4 = 16 microstates for N=4N = 4, sorted by the number nLn_L of particles in the left half. Each color marks one particle. (b) The number of microstates Ω(nL)\Omega(n_L), divided by its largest value Ωmax⁡\Omega_{\max}, against the fraction of the particles in the left half for three values of NN. As NN grows, almost all microstates lie close to the even split.

In Figure 4.3.3, the particles of an ideal gas start in the left half of the box, as in the free expansion of Example 2 in Section 4.1. Open the partition, and follow nLn_L and S=kBln⁡Ω(nL)S = k_{\mathrm{B}}\ln\Omega(n_L) as the gas spreads.

Figure 4.3.3:Expansion of NN particles into a vacuum. Panel (a) shows the particles, which do not interact. Panel (b) shows the fraction of the particles in the left half and the entropy S=kBln⁡Ω(nL)S = k_{\mathrm{B}}\ln\Omega(n_L) against time. The dashed lines mark the even split and the largest possible value of ln⁡Ω\ln\Omega.

Two things to try:

  1. Run the expansion with N=10N = 10 and then with N=200N = 200. After the gas has spread, compare how far nL/Nn_L/N and ln⁡Ω\ln\Omega move away from the dashed lines.

  2. For N=200N = 200, compare the largest value of ln⁡Ω\ln\Omega with Nln⁡2N\ln 2. Example 2 of Section 4.1 gives ΔS=nRln⁡2\Delta S = nR\ln 2 for a gas that doubles its volume, and nR=NkBnR = Nk_{\mathrm{B}}.

For a few particles, ln⁡Ω\ln\Omega falls below its largest value from time to time. For the 1023 particles of a macroscopic gas, such fluctuations are far too small to observe, which is why the Second Law can be stated as dS≥0dS \ge 0.

Why the Logarithm?

A key property of thermodynamic entropy is that it is extensive: for two independent subsystems AA and BB,

Stotal=SA+SB.S_{\mathrm{total}} = S_A + S_B.

But the number of microstates is multiplicative for independent systems:

Ωtotal=ΩA ΩB.\Omega_{\mathrm{total}} = \Omega_A\,\Omega_B.

The logarithm converts the product into a sum:

Stotal=kBln⁡(ΩA ΩB)=kBln⁡ΩA+kBln⁡ΩB=SA+SB.S_{\mathrm{total}} = k_{\mathrm{B}} \ln(\Omega_A\,\Omega_B) = k_{\mathrm{B}} \ln \Omega_A + k_{\mathrm{B}} \ln \Omega_B = S_A + S_B.

Without the logarithm, entropy would not be additive for independent subsystems.


Entropy in Terms of Probabilities: The Gibbs Entropy

The Gibbs (or Gibbs–Shannon) formula generalizes Boltzmann’s result from “counting equally likely microstates” to working with an arbitrary probability distribution {pi}\{p_i\} over microstates i=1,…,Mi = 1, \dots, M:

S=−kB∑i=1Mpi ln⁡pi.S = -k_{\mathrm{B}} \sum_{i=1}^{M} p_i\,\ln p_i.

A microstate with pi=0p_i = 0 contributes nothing to the sum, because pln⁡p→0p\ln p \to 0 as p→0p \to 0. For two independent subsystems AA and BB, the probability that AA is in microstate ii while BB is in microstate jj is the product pi(A) pj(B)p_i^{(A)}\,p_j^{(B)}.

Figure 4.3.4 compares three distributions over the same six microstates.

Source
Loading...

Figure 4.3.4:Three probability distributions over the same M=6M = 6 microstates, with the Gibbs entropy of each. The entropy is smallest for the distribution in (a), which is concentrated on few microstates, and largest for the uniform distribution in (c), for which S=kBln⁡6S = k_{\mathrm{B}}\ln 6.

This formula has two important features:

  1. It reduces to Boltzmann’s formula in the microcanonical case. If all Ω\Omega accessible microstates are equally likely (pi=1/Ωp_i = 1/\Omega), then

    S=−kB∑i=1Ω1Ωln⁡ ⁣(1Ω)=−kBln⁡ ⁣(1Ω)=kBln⁡Ω.S = -k_{\mathrm{B}} \sum_{i=1}^{\Omega} \frac{1}{\Omega}\ln\!\left(\frac{1}{\Omega}\right) = -k_{\mathrm{B}}\ln\!\left(\frac{1}{\Omega}\right) = k_{\mathrm{B}}\ln\Omega.
  2. It applies when microstates are not equally likely — for instance, when the system is in contact with a heat bath (canonical ensemble). In this case, pi=e−βEi/Qp_i = e^{-\beta E_i}/Q, and different microstates have different probabilities.


Canonical Ensemble: Entropy in Terms of the Partition Function

We now evaluate the Gibbs entropy for the canonical ensemble. Recall from Section 2.2 that a closed system (fixed NN, VV) in thermal contact with a reservoir at temperature TT has microstate probabilities

pi=e−βEiQ,β≡1kBT,p_i = \frac{e^{-\beta E_i}}{Q}, \qquad \beta \equiv \frac{1}{k_{\mathrm{B}} T},

where the canonical partition function is

Q≡∑i=1Me−βEi.Q \equiv \sum_{i=1}^{M} e^{-\beta E_i}.

Derivation

Starting from the Gibbs entropy,

S=−kB∑ipiln⁡pi,S = -k_{\mathrm{B}} \sum_i p_i\ln p_i,

insert ln⁡pi=−βEi−ln⁡Q\ln p_i = -\beta E_i - \ln Q:

S=−kB∑ipi (−βEi−ln⁡Q)=kBβ∑ipiEi  +  kB(ln⁡Q)∑ipi.\begin{aligned} S &= -k_{\mathrm{B}} \sum_i p_i\,(-\beta E_i - \ln Q) \\ &= k_{\mathrm{B}}\beta \sum_i p_i E_i \;+\; k_{\mathrm{B}}(\ln Q)\sum_i p_i. \end{aligned}

Using ∑ipi=1\sum_i p_i = 1 and the definition of internal energy from Section 2.3,

U≡⟨E⟩=∑ipiEi,U \equiv \langle E \rangle = \sum_i p_i E_i,

we obtain

S=kBβU+kBln⁡Q.S = k_{\mathrm{B}}\beta U + k_{\mathrm{B}}\ln Q.

Substituting β=1/(kBT)\beta = 1/(k_{\mathrm{B}} T):

S=UT+kBln⁡Q.\boxed{ S = \frac{U}{T} + k_{\mathrm{B}}\ln Q. }

This is a central result: it expresses the macroscopic state function SS directly in terms of the partition function QQ and the internal energy UU, both of which we know how to compute from microscopic energy levels.


Connection to the Helmholtz Free Energy

Rearranging the boxed result gives

U−TS=−kBTln⁡Q.U - TS = -k_{\mathrm{B}}T\ln Q.

The left-hand side is the Helmholtz free energy, A≡U−TSA \equiv U - TS — a state function whose natural variables are TT and VV. We therefore have

A=−kBTln⁡Q.\boxed{ A = -k_{\mathrm{B}}T\ln Q. }

This result was foreshadowed in Section 2.3 (where it was stated without derivation) and in Section 3.3 (where we noted that A=U−TSA = U - TS would be “convenient at constant TT and VV”). It is arguably the single most important equation in canonical statistical mechanics: if you can compute QQ, you can compute AA, and from AA you can derive all other thermodynamic quantities — SS, UU, PP, CVC_V, and chemical potentials — by taking appropriate derivatives. Section 5.1 develops this machinery.


Microscopic Interpretation of Heat (Connection to Section 3.2)

The results above connect naturally to the microscopic interpretation of the First Law developed in Section 3.2. Differentiating U=∑ipiEiU = \sum_i p_i E_i gives

dU=∑ipi dEi+∑iEi dpi.dU = \sum_i p_i\, dE_i + \sum_i E_i\, dp_i.

For a reversible process in a closed system with only PVPV work, comparing with dU=δqrev+δwrev=δqrev−P dVdU = \delta q_{\mathrm{rev}} + \delta w_{\mathrm{rev}} = \delta q_{\mathrm{rev}} - P\,dV, we identified (Section 3.2):

The Gibbs entropy formula leads to the same identification. Its differential is

dS=−kB∑i(ln⁡pi+1)dpi=−kB∑iln⁡pi dpi,dS = -k_{\mathrm{B}}\sum_i \left(\ln p_i + 1\right) dp_i = -k_{\mathrm{B}}\sum_i \ln p_i\, dp_i,

because ∑idpi=0\sum_i dp_i = 0. Along a reversible path the system stays in equilibrium, so the probabilities remain canonical, with ln⁡pi=−Ei/(kBT)−ln⁡Q\ln p_i = -E_i/(k_{\mathrm{B}}T) - \ln Q. Substituting, and using ∑idpi=0\sum_i dp_i = 0 once more,

dS=1T∑iEi dpi=δqrevT.dS = \frac{1}{T}\sum_i E_i\, dp_i = \frac{\delta q_{\mathrm{rev}}}{T}.

This is the definition of entropy from Section 4.1, now obtained from the microscopic side. In a reversible process the entropy changes only through the heat term. Reversible work alone (∑iEi dpi=0\sum_i E_i\,dp_i = 0) leaves the entropy unchanged, in agreement with the macroscopic result that dS=0dS = 0 on a reversible adiabat.

The restriction to reversible processes matters. In the free expansion of Example 2 in Section 4.1, no heat flows and no work is done, yet the distribution over microstates changes and the entropy of the gas increases.


Worked Examples

Example 1: Entropy of a Two-State Distribution

Problem. A system has two microstates with probabilities pp and 1−p1-p. Compute the Gibbs entropy and find where it is maximized.

Solution. The Gibbs entropy is

S(p)=−kB[pln⁡p+(1−p)ln⁡(1−p)].S(p) = -k_{\mathrm{B}}\left[p\ln p + (1-p)\ln(1-p)\right].

The derivative dS/dp=−kB[ln⁡p−ln⁡(1−p)]dS/dp = -k_{\mathrm{B}}\left[\ln p - \ln(1-p)\right] vanishes at p=1/2p = 1/2, where

S=−kB[12ln⁡12+12ln⁡12]=−kBln⁡12=kBln⁡2.S = -k_{\mathrm{B}}\left[\tfrac{1}{2}\ln\tfrac{1}{2} + \tfrac{1}{2}\ln\tfrac{1}{2}\right] = -k_{\mathrm{B}}\ln\tfrac{1}{2} = k_{\mathrm{B}}\ln 2.

To confirm this is a maximum, note that S(p)S(p) is concave (its second derivative d2S/dp2=−kB/[p(1−p)]<0d^2S/dp^2 = -k_{\mathrm{B}}/[p(1-p)] < 0 for 0<p<10 < p < 1) and is zero at the endpoints p=0p = 0 and p=1p = 1.

Result. The maximum entropy kBln⁡2k_{\mathrm{B}}\ln 2 occurs at p=1/2p = 1/2, where the distribution is most spread out. This is consistent with Boltzmann’s formula: two equally likely microstates give Ω=2\Omega = 2 and S=kBln⁡2S = k_{\mathrm{B}}\ln 2.


Example 2: Entropy of the Two-State System from Section 2.2

Problem. In Section 2.2, we analyzed a two-state system with energies E1=0E_1 = 0 and E2=ε=0.010 eVE_2 = \varepsilon = 0.010\ \mathrm{eV} and found p1=0.596p_1 = 0.596 and p2=0.404p_2 = 0.404 at T=300 KT = 300\ \mathrm{K}. Compute the entropy two ways: (a) from the Gibbs formula, and (b) from the canonical identity S=U/T+kBln⁡QS = U/T + k_{\mathrm{B}}\ln Q.

Solution.

(a) Gibbs formula.

S=−kB[0.596ln⁡(0.596)+0.404ln⁡(0.404)].S = -k_{\mathrm{B}}\left[0.596\ln(0.596) + 0.404\ln(0.404)\right].

Evaluating: 0.596ln⁡(0.596)=−0.30840.596\ln(0.596) = -0.3084 and 0.404ln⁡(0.404)=−0.36620.404\ln(0.404) = -0.3662, so

S=−kB(−0.3084−0.3662)=0.675 kB=9.32×10−24 J/K.S = -k_{\mathrm{B}}(-0.3084 - 0.3662) = 0.675\,k_{\mathrm{B}} = 9.32 \times 10^{-24}\ \mathrm{J/K}.

(b) Canonical identity.

From Section 2.2: Q=1.679Q = 1.679, and U=ε p2=(0.010)(0.404)=4.04×10−3 eV=6.47×10−22 JU = \varepsilon\,p_2 = (0.010)(0.404) = 4.04 \times 10^{-3}\ \mathrm{eV} = 6.47 \times 10^{-22}\ \mathrm{J}.

S=UT+kBln⁡Q=6.47×10−22300+(1.381×10−23)ln⁡(1.679).S = \frac{U}{T} + k_{\mathrm{B}}\ln Q = \frac{6.47 \times 10^{-22}}{300} + (1.381 \times 10^{-23})\ln(1.679).
S=2.16×10−24+7.16×10−24=9.32×10−24 J/K.S = 2.16 \times 10^{-24} + 7.16 \times 10^{-24} = 9.32 \times 10^{-24}\ \mathrm{J/K}.

Result. Both methods give the same answer, as they must — the canonical identity is derived from the Gibbs formula. Notice that the entropy (0.675 kB0.675\,k_{\mathrm{B}}) is less than the maximum (kBln⁡2=0.693 kBk_{\mathrm{B}}\ln 2 = 0.693\,k_{\mathrm{B}}) because the distribution is not perfectly uniform: the ground state is slightly more populated than the excited state at 300 K. At higher temperatures, the two probabilities would become more equal, and SS would approach kBln⁡2k_{\mathrm{B}}\ln 2 from below.


Concept Checks

  1. How does “entropy increases until equilibrium” translate into a statement about probability distributions over microstates?

  2. Why does S=kBln⁡ΩS = k_{\mathrm{B}}\ln\Omega require the microcanonical assumption pi=1/Ωp_i = 1/\Omega? What goes wrong if you try to apply it to a canonical ensemble?

  3. In the Gibbs entropy formula, why does S=0S = 0 when pi=1p_i = 1 for one state and pj=0p_j = 0 for all others?

  4. How can entropy be extensive (additive for independent subsystems) while probability distributions are normalized to 1?

  5. Starting from A=−kBTln⁡QA = -k_{\mathrm{B}}T\ln Q, how would you obtain SS and UU as functions of TT? (Hint: think about which partial derivatives of AA give SS and UU.)

Key Takeaways