Measure Theory and Integration

Measure theory provides the rigorous foundation for length, area, volume, probability, and integration. Before Henri Lebesgue’s 1902 thesis Intégrale, longueur, aire there was no satisfactory way to integrate badly-behaved functions, no uniform notion of “size” of subsets of the real line, and no rigorous framework for probability. The modern formulation — measurable spaces, $\sigma$-algebras, measures, Lebesgue integral, and the convergence theorems — solved these problems and reshaped twentieth-century analysis. Andrey Kolmogorov’s 1933 Grundbegriffe der Wahrscheinlichkeitsrechnung used measure theory to give probability its first axiomatic treatment, and the entire edifice of modern probability, stochastic calculus, ergodic theory, functional analysis, harmonic analysis, and partial differential equations is built on it. This note covers $\sigma$-algebras and measurable functions, construction of the Lebesgue measure, the Lebesgue integral, the three convergence theorems, $L^p$ spaces, product measures and Fubini’s theorem, Radon-Nikodym differentiation, Carathéodory’s extension theorem, Lebesgue’s differentiation theorem, absolute continuity and singularity, Hausdorff measure and fractal dimension, and applications to probability and functional analysis.

See also

• functional-analysis — $L^p$ spaces, dual spaces, Riesz representation theorem; functional analysis cannot be done without measure theory.
• probability-fundamentals — probability is measure theory with $\mu (\Omega )=1$.
• probability-distributions — distributions are measures on ${\mathbb{R}}^n$.
• stochastic-calculus — Itô calculus is built on filtered probability spaces.
• multivariate-calculus — Lebesgue integration generalises Riemann; Fubini supersedes the iterated-integral theorem.
• information-theory — Kullback-Leibler divergence is defined via Radon-Nikodym.
• fft-spectral — Fourier transforms on $L^1$ and $L^2$ require measure theory.
• algebraic-geometry-foundations — Hausdorff dimension and measure-theoretic geometry.

1. The need for measure theory

1.1 Why Riemann is not enough

The Riemann integral, taught in undergraduate calculus, partitions the domain into intervals and approximates by step functions. It works beautifully for continuous functions on compact intervals but breaks down in three crucial ways:

1. The class of Riemann-integrable functions is too small. The Dirichlet function $1_{\mathbb{Q}}$ — equal to $1$ on rationals and $0$ on irrationals — is not Riemann-integrable. Yet it is “almost everywhere $0$” and ought to integrate to $0$.

2. Limit interchange is fragile. If $f_n\to f$ pointwise with $|f_n|\le g$, we want $\int f_n\to \int f$. Riemann theory requires uniform convergence; for pointwise convergence under domination the answer requires the Lebesgue framework.

3. Function spaces are not complete. The space of continuous functions on $[0,1]$ with the $L^2$ norm $\Vert f{\Vert }_2={\left({\int }_0^1|f{|}^2\right)}^{1/2}$ is incomplete (sequences of continuous functions can converge in $L^2$ to a discontinuous limit). For Hilbert-space methods we need a complete space; Lebesgue theory provides it.

1.2 The “size” problem

Even before integration, the question “what subsets of $\mathbb{R}$ have a well-defined length?” has no naive answer. We want a set function $\mu$ on subsets of $\mathbb{R}$ satisfying:

• $\mu ([a,b])=b-a$ (consistency with length).
• $\mu (A\cup B)=\mu (A)+\mu (B)$ for disjoint $A,B$ (additivity).
• $\mu (A+t)=\mu (A)$ (translation invariance).

If we demand all of these for every subset of $\mathbb{R}$, even just for finite additivity, no such $\mu$ exists in ${\mathbb{R}}^n$ for $n\ge 3$ (Banach-Tarski paradox, 1924, requires the axiom of choice). For $\sigma$-additivity (countable additivity), no such $\mu$ exists in ${\mathbb{R}}^n$ for $n\ge 1$. The fix: restrict $\mu$ to a sufficiently rich but not universal class of subsets — the $\sigma$-algebra of measurable sets.

2. $\sigma$-algebras and measurable spaces

2.1 Definitions

A $\sigma$-algebra $\mathcal{F}$ on a set $\Omega$ is a collection of subsets of $\Omega$ such that:

1. $\Omega \in \mathcal{F}$.
2. If $A\in \mathcal{F}$, then $A^c=\Omega \setminus A\in \mathcal{F}$.
3. If $A_1,A_2,\dots \in \mathcal{F}$, then ${\bigcup }_{n=1}^{\infty }{A}_n\in \mathcal{F}$.

Properties (2) and (3) imply closure under countable intersections via De Morgan. The pair $(\Omega ,\mathcal{F})$ is a measurable space; elements of $\mathcal{F}$ are measurable sets.

2.2 The Borel $\sigma$-algebra

The smallest $\sigma$-algebra on $\mathbb{R}$ containing all open intervals is the Borel $\sigma$-algebra $\mathcal{B}(\mathbb{R})$. By symmetry it equally contains all closed intervals, half-open intervals, open sets, closed sets, $G_{\delta }$ sets (countable intersections of opens), $F_{\sigma }$ sets (countable unions of closeds), and so on. The construction generalises to any topological space.

The cardinality of $\mathcal{B}(\mathbb{R})$ is $\mathfrak{c}=2^{{\aleph }_0}$ (continuum), the same as $|\mathbb{R}|$ — far smaller than the cardinality $2^{\mathfrak{c}}$ of the power set $\mathcal{P}(\mathbb{R})$.

2.3 Generated $\sigma$-algebras and $\sigma (\mathcal{C})$

For any collection $\mathcal{C}$ of subsets of $\Omega$, the intersection of all $\sigma$-algebras containing $\mathcal{C}$ is itself a $\sigma$-algebra — the $\sigma$-algebra generated by $\mathcal{C}$, written $\sigma (\mathcal{C})$. The Borel $\sigma$-algebra is $\sigma (\text{open sets})$.

2.4 $\pi$-systems and Dynkin’s $\pi$-$\lambda$ theorem

Two measures that agree on a $\pi$-system (a family closed under finite intersections) agree on the entire $\sigma$-algebra it generates — provided one of the measures is $\sigma$-finite. This is Dynkin’s $\pi$-$\lambda$ theorem (also known as the monotone class theorem), an indispensable tool for proving measure-theoretic statements by reducing to a generating class.

2.5 Measurable functions

A function $f:(\Omega ,\mathcal{F})\to ({\Omega }^{\prime },{\mathcal{F}}^{\prime })$ is measurable if $f^{-1}(A^{\prime })\in \mathcal{F}$ for every $A^{\prime }\in {\mathcal{F}}^{\prime }$. For real-valued $f:\Omega \to \mathbb{R}$ measurability means $f^{-1}((-\infty ,a])\in \mathcal{F}$ for every $a\in \mathbb{R}$.

Composition of measurable functions is measurable. Sums, products, $\sup$, $\inf$, $\mathrm{limsup}$, $\mathrm{liminf}$, and pointwise limits of measurable functions are measurable. This last fact is what distinguishes measurable from continuous functions — continuity is not preserved under pointwise limits, but measurability is.

3. Measures

3.1 Definition

A measure on $(\Omega ,\mathcal{F})$ is a function $\mu :\mathcal{F}\to [0,\infty ]$ with:

1. $\mu (\varnothing )=0$.
2. Countable additivity ($\sigma$-additivity): for disjoint $A_1,A_2,\dots \in \mathcal{F}$, $\mu \left({\bigcup }_{n=1}^{\infty }{A}_n\right)={\sum }_{n=1}^{\infty }\mu (A_n).$

The triple $(\Omega ,\mathcal{F},\mu )$ is a measure space.

Special cases:

• Probability measure: $\mu (\Omega )=1$.
• Finite measure: $\mu (\Omega )<\infty$.
• $\sigma$-finite measure: $\Omega ={\bigcup }_n{A}_n$ with $\mu (A_n)<\infty$.
• Signed measure: takes values in $[-\infty ,\infty ]$ (not both $+\infty$ and $-\infty$).
• Complex measure: takes values in $\mathbb{C}$, with countable additivity and $|\mu |$ finite.

3.2 Properties

For any measure $\mu$:

• Monotonicity: $A\subseteq B\Rightarrow \mu (A)\le \mu (B)$.
• Subadditivity: $\mu \left({\bigcup }_n{A}_n\right)\le {\sum }_n\mu (A_n)$.
• Continuity from below: $A_n\uparrow A\Rightarrow \mu (A_n)\to \mu (A)$.
• Continuity from above: $A_n\downarrow A$ and $\mu (A_1)<\infty \Rightarrow \mu (A_n)\to \mu (A)$.

3.3 Examples

• Counting measure on $(\Omega ,\mathcal{P}(\Omega ))$: $\mu (A)=|A|$ if $A$ is finite, $\infty$ otherwise.
• Dirac measure ${\delta }_x$ at point $x$: ${\delta }_x(A)=1$ if $x\in A$, $0$ otherwise.
• Lebesgue measure on ${\mathbb{R}}^n$: the unique translation-invariant Borel measure with $\lambda ([0,1{]}^n)=1$. Constructed in §4.
• Radon measure: a Borel measure that is locally finite (finite on compact sets) and inner regular (every Borel set is approximated from inside by compact sets).
• Gaussian measure $\mathcal{N}(0,I)$ on ${\mathbb{R}}^n$: density $(2\pi {)}^{-n/2}{e}^{-|x{|}^2/2}$ with respect to Lebesgue.
• Haar measure on a locally compact topological group: the unique (up to scalar) translation-invariant Radon measure.

4. Construction of Lebesgue measure

4.1 Outer measure

Define the Lebesgue outer measure on $\mathbb{R}$: ${\lambda }^{\ast }(A)=\inf \left\{{\sum }_{n=1}^{\infty }(b_n-a_n):A\subseteq {\bigcup }_n(a_n,b_n)\right\}.$

The outer measure is defined on all subsets but is only countably subadditive in general, not additive.

4.2 Carathéodory’s criterion

Constantin Carathéodory 1914: a set $A$ is Lebesgue measurable if for every set $E\subseteq \mathbb{R}$, ${\lambda }^{\ast }(E)={\lambda }^{\ast }(E\cap A)+{\lambda }^{\ast }(E\cap A^c).$

The collection $\mathcal{L}$ of Lebesgue-measurable sets is a $\sigma$-algebra containing the Borel sets, and ${\lambda }^{\ast }{|}_{\mathcal{L}}=\lambda$ is a complete measure. Completeness: every subset of a $\lambda$-null set is $\lambda$-measurable (and has measure zero).

4.3 Carathéodory extension theorem

This is the workhorse for constructing measures. Let $\mathcal{A}$ be an algebra (closed under finite unions and complements) on $\Omega$ and ${\mu }_0:\mathcal{A}\to [0,\infty ]$ a pre-measure (countably additive on $\mathcal{A}$ insofar as countable unions remain in $\mathcal{A}$). Then ${\mu }_0$ extends uniquely (provided ${\mu }_0$ is $\sigma$-finite) to a measure on $\sigma (\mathcal{A})$.

This is the technique behind constructions of Lebesgue measure, product measures, Haar measure, and many others.

4.4 Lebesgue vs Borel

The Lebesgue $\sigma$-algebra $\mathcal{L}$ is strictly larger than $\mathcal{B}(\mathbb{R})$. In fact $|\mathcal{L}|=2^{\mathfrak{c}}$ (a Lebesgue-null set has all $2^{\mathfrak{c}}$ subsets Lebesgue-measurable). But $|\mathcal{B}(\mathbb{R})|=\mathfrak{c}$. So there are Lebesgue-measurable non-Borel sets — uncountably many.

4.5 Non-measurable sets

Without the axiom of choice (AC), every subset of $\mathbb{R}$ is Lebesgue-measurable (Solovay 1970, in ZF + dependent choice + “every set is Lebesgue-measurable” being consistent if a measurable cardinal exists). With AC, Vitali sets are non-measurable: pick one representative from each equivalence class of $\mathbb{R}/\mathbb{Q}$ in $[0,1]$. If the resulting set $V$ were measurable, translation invariance plus countability of $\mathbb{Q}\cap [-1,1]$ would force $\lambda (V)=0$ and $\lambda ([0,1])=0$, contradiction.

Banach-Tarski (1924) shows that in ${\mathbb{R}}^3$ a ball can be partitioned into finitely many pieces and reassembled (using AC) into two balls of the same size — those pieces are necessarily non-measurable.

5. The Lebesgue integral

5.1 Simple functions

A simple function is a measurable function taking finitely many values: $\phi ={\sum }_{i=1}^n{c}_i{1}_{A_i},A_i\in \mathcal{F},A_i\text{ disjoint}.$ For $c_i\ge 0$, define $\int \phi d\mu ={\sum }_{i=1}^n{c}_i\mu (A_i),$ with the convention $0\cdot \infty =0$.

5.2 Approximation theorem

Every non-negative measurable function $f$ is the pointwise limit of an increasing sequence of non-negative simple functions: ${\phi }_n(x)={\sum }_{k=0}^{n{2}^n-1}\frac{k}{2^n}{1}_{\{k/2^n\le f<(k+1)/2^n\}}+n{1}_{\{f\ge n\}}.$

5.3 Lebesgue integral of non-negative functions

For $f\ge 0$ measurable, $\int fd\mu =\sup \left\{\int \phi d\mu :0\le \phi \le f,\phi \text{ simple}\right\}.$

5.4 General Lebesgue integral

For measurable $f:\Omega \to \overline{\mathbb{R}}$, write $f=f^{+}-f^{-}$ where $f^{+}=\max (f,0)$ and $f^{-}=\max (-f,0)$. Define $\int fd\mu =\int f^{+}d\mu -\int f^{-}d\mu$ provided not both terms are $\infty$. $f$ is integrable if both terms are finite, equivalently $\int |f|d\mu <\infty$. The space of integrable functions is $L^1(\mu )$.

5.5 Properties

• Linearity: $\int (\alpha f+\beta g)d\mu =\alpha \int fd\mu +\beta \int gd\mu$.
• Monotonicity: $f\le g\Rightarrow \int fd\mu \le \int gd\mu$.
• Triangle: $\left|\int fd\mu \right|\le \int |f|d\mu$.
• a.e. equality: $f=g$ almost everywhere $\Rightarrow \int f=\int g$.

5.6 Lebesgue vs Riemann

For bounded $f$ on a compact interval $[a,b]$: $f$ is Riemann-integrable iff $f$ is continuous almost everywhere (Lebesgue’s criterion, 1902). When both integrals exist, they agree. The Lebesgue integral extends to all measurable functions whose absolute value integrates finitely.

Improper Riemann integrals (e.g., ${\int }_0^{\infty }\sin (x)/xdx$) can exist when the Lebesgue integral does not — Lebesgue requires absolute integrability.

6. The three convergence theorems

These are the central theorems of integration theory and the reason Lebesgue’s framework dominates.

6.1 Monotone Convergence Theorem (Beppo Levi 1906)

Let $0\le f_1\le f_2\le \cdots$ measurable with $f_n\uparrow f$ pointwise (a.e.). Then $\int f_{n}d\mu \to \int fd\mu .$

Proof sketch: $\int f_n$ is increasing and bounded above by $\int f$. For the reverse, fix simple $\phi \le f$ and $\alpha \in (0,1)$; the sets $E_n=\{f_n\ge \alpha \phi \}$ increase to $\Omega$, so ${\int }_{E_n}\alpha \phi \le \int f_n$, and letting $n\to \infty$ then $\alpha \to 1$ gives $\int \phi \le \lim \int f_n$.

6.2 Fatou’s Lemma (Pierre Fatou 1906)

For non-negative measurable $f_n$: $\int {\mathrm{liminf}}_{n\to \infty }{f}_{n}d\mu \le {\mathrm{liminf}}_{n\to \infty }\int f_{n}d\mu .$

Strict inequality possible: $f_n=1_{[n,n+1]}$ on $\mathbb{R}$. Then $\mathrm{liminf}{f}_n=0$ but $\int f_n=1$ for all $n$.

6.3 Dominated Convergence Theorem (Henri Lebesgue 1908)

Let $f_n$ measurable with $f_n\to f$ a.e. and $|f_n|\le g$ for some integrable $g$. Then $f\in L^1$ and $\int f_{n}d\mu \to \int fd\mu ,\int |f_n-f|d\mu \to 0.$

Proof sketch: Apply Fatou to $g+f_n\ge 0$ and to $g-f_n\ge 0$. Get $\int (g+f)\le \mathrm{liminf}\int (g+f_n)$ and $\int (g-f)\le \mathrm{liminf}\int (g-f_n)$. Combine.

This is the workhorse of analysis. Almost every limit-interchange argument — differentiating under the integral, Fourier inversion, weak convergence in PDE — uses DCT or a refinement.

6.4 Vitali’s convergence theorem

A refinement of DCT replacing the domination hypothesis with uniform integrability: ${\lim }_{c\to \infty }{\sup }_n{\int }_{\{|f_n|>c\}}|f_n|d\mu =0.$ Together with $f_n\to f$ in measure, this gives $f_n\to f$ in $L^1$. Indispensable in probability — sequences of random variables are often uniformly integrable without a uniform dominant.

7. $L^p$ spaces

7.1 Definitions

For $1\le p<\infty$ and measurable $f$: $\Vert f{\Vert }_p={\left(\int |f{|}^{p}d\mu \right)}^{1/p},L^p(\mu )=\{f:\Vert f{\Vert }_p<\infty \}/\text{a.e. equality}.$ For $p=\infty$: $\Vert f{\Vert }_{\infty }=\text{ess sup}|f|=\inf \{M:|f|\le M\text{ a.e.}\}.$

7.2 Hölder’s inequality

For $1/p+1/q=1$ (conjugate exponents): $\int |fg|d\mu \le \Vert f{\Vert }_p\Vert g{\Vert }_q.$

Special case $p=q=2$: Cauchy-Schwarz. Reverse direction with $p,q$ swapped at $\infty$ and $1$.

7.3 Minkowski’s inequality

For $1\le p\le \infty$, $\Vert \cdot {\Vert }_p$ satisfies the triangle inequality: $\Vert f+g{\Vert }_p\le \Vert f{\Vert }_p+\Vert g{\Vert }_p.$

For $p<1$ the inequality fails — $L^p$ for $p<1$ are not normed spaces (they remain metric spaces with $d(f,g)=\int |f-g{|}^{p}d\mu$ but are not locally convex).

7.4 Riesz-Fischer completeness theorem

Frigyes Riesz 1907 and Ernst Fischer 1907 (independently): $L^p(\mu )$ is complete (Banach) for $1\le p\le \infty$. $L^2(\mu )$ is Hilbert with inner product $⟨f,g⟩=\int f\bar{g}d\mu$.

Proof sketch for $1\le p<\infty$: Cauchy sequence $(f_n)$ has subsequence with $\sum \Vert f_{n_{k+1}}-f_{n_k}{\Vert }_p<\infty$. The series $\sum |f_{n_{k+1}}-f_{n_k}|$ converges a.e. by monotone convergence, so $f_{n_k}$ converges a.e. to some $f$. DCT shows $f\in L^p$ and $\Vert f_{n_k}-f{\Vert }_p\to 0$, then standard Cauchy-sequence argument gives convergence of the whole sequence.

7.5 Duality

For $1\le p<\infty$ and $\mu$ $\sigma$-finite: $L^p(\mu {)}^{\ast }\cong L^q(\mu )$ where $1/p+1/q=1$. Via the pairing $g\mapsto \left(f\mapsto \int fgd\mu \right).$

For $p=\infty$ the dual is strictly larger than $L^1$ (it contains finitely additive measures). See functional-analysis.

7.6 Density

Continuous functions of compact support $C_c({\mathbb{R}}^n)$ are dense in $L^p({\mathbb{R}}^n)$ for $1\le p<\infty$ (under Lebesgue measure). Smooth functions $C_c^{\infty }$ are dense via mollification. Step functions are dense. Simple functions are dense.

8. Product measures and Fubini’s theorem

8.1 Product $\sigma$-algebra and measure

Given measure spaces $({\Omega }_1,{\mathcal{F}}_1,{\mu }_1)$ and $({\Omega }_2,{\mathcal{F}}_2,{\mu }_2)$, the product $\sigma$-algebra ${\mathcal{F}}_1\otimes {\mathcal{F}}_2$ is generated by rectangles $A_1\times A_2$. The product measure ${\mu }_1\otimes {\mu }_2$ is the unique ($\sigma$-finite case) measure on ${\mathcal{F}}_1\otimes {\mathcal{F}}_2$ with $({\mu }_1\otimes {\mu }_2)(A_1\times A_2)={\mu }_1(A_1){\mu }_2(A_2)$. Existence by Carathéodory extension.

8.2 Fubini’s theorem (Guido Fubini 1907)

For $f\in L^1({\mu }_1\otimes {\mu }_2)$: ${\int }_{{\Omega }_1\times {\Omega }_2}fd({\mu }_1\otimes {\mu }_2)={\int }_{{\Omega }_1}\left({\int }_{{\Omega }_2}f(x,y)d{\mu }_2(y)\right)d{\mu }_1(x)={\int }_{{\Omega }_2}\left({\int }_{{\Omega }_1}f(x,y)d{\mu }_1(x)\right)d{\mu }_2(y).$

8.3 Tonelli’s theorem (Leonida Tonelli 1909)

For ${\mu }_1,{\mu }_2$ $\sigma$-finite and $f\ge 0$ measurable on the product: ${\int }_{{\Omega }_1\times {\Omega }_2}fd({\mu }_1\otimes {\mu }_2)={\int }_{{\Omega }_1}{\int }_{{\Omega }_2}fd{\mu }_{2}d{\mu }_1={\int }_{{\Omega }_2}{\int }_{{\Omega }_1}fd{\mu }_{1}d{\mu }_2,$ with all three (possibly $+\infty$) equal.

Tonelli applies without integrability — it lets you exchange order of integration for non-negative measurands regardless of whether the answer is finite. The combined Fubini-Tonelli strategy is: prove the integral of $|f|$ is finite via Tonelli, then apply Fubini to $f$ itself.

8.4 Counterexamples without $\sigma$-finiteness or sign-change

• $f(x,y)=(x^2-y^2)/(x^2+y^2{)}^2$ on $(0,1{)}^2$: not integrable; the two iterated integrals exist but differ ($\pi /4$ and $-\pi /4$).
• On $\mathbb{N}\times \mathbb{N}$ with counting measure: $f(m,n)=1$ if $m=n$, $-1$ if $n=m+1$, $0$ otherwise. The two iterated sums are $1$ and $0$.

9. Radon-Nikodym theorem

9.1 Absolute continuity and singularity

Two measures $\mu ,\nu$ on $(\Omega ,\mathcal{F})$:

• $\nu \ll \mu$ ($\nu$ is absolutely continuous w.r.t. $\mu$) if $\mu (A)=0\Rightarrow \nu (A)=0$.
• $\nu \perp \mu$ (mutually singular) if there exists a measurable partition $\Omega =A\bigsqcup A^c$ with $\nu (A)=0$ and $\mu (A^c)=0$.

9.2 Radon-Nikodym theorem (Johann Radon 1913, Otto Nikodym 1930)

Let $\mu ,\nu$ be $\sigma$-finite measures on $(\Omega ,\mathcal{F})$ with $\nu \ll \mu$. Then there is a non-negative measurable function $f$ (unique a.e.) — the Radon-Nikodym derivative $d\nu /d\mu$ — such that $\nu (A)={\int }_{A}fd\mu \text{for all }A\in \mathcal{F}.$

This is the abstract version of the fundamental theorem of calculus and the cornerstone of statistical theory: probability densities are Radon-Nikodym derivatives with respect to Lebesgue or counting measure.

9.3 Lebesgue decomposition

For $\sigma$-finite $\mu ,\nu$, there is a unique decomposition $\nu ={\nu }_{ac}+{\nu }_s$ with ${\nu }_{ac}\ll \mu$ and ${\nu }_s\perp \mu$. Combined with Radon-Nikodym: $\nu =(d{\nu }_{ac}/d\mu )d\mu +{\nu }_s$. The singular part further decomposes into a discrete (atomic) part and a continuous singular part (e.g., the Cantor measure).

9.4 Chain rule and change of variables

If $\nu \ll \mu \ll \lambda$ all $\sigma$-finite: $\frac{d\nu }{d\lambda }=\frac{d\nu }{d\mu }\cdot \frac{d\mu }{d\lambda }\lambda \text{-a.e.}$

For probability: if $X$ has density $f_X$ w.r.t. ${\mu }_X$ and $Y=g(X)$ for $g$ a $C^1$ diffeomorphism, then $f_Y(y)=f_X(g^{-1}(y))\cdot |\det (D{g}^{-1}(y))|$ (change-of-variables / Jacobian formula).

9.5 Information-theoretic application

The Kullback-Leibler divergence between $P\ll Q$: $D_{KL}(P\Vert Q)=\int \log \left(\frac{dP}{dQ}\right)dP.$ If $P/\ll Q$, define $D_{KL}(P\Vert Q)=+\infty$. See information-theory.

10. Lebesgue differentiation theorem

10.1 Statement

Henri Lebesgue 1910: for $f\in L_{\text{loc}}^1({\mathbb{R}}^n)$ (locally integrable), ${\lim }_{r\to 0}\frac{1}{\lambda (B_r(x))}{\int }_{B_r(x)}fd\lambda =f(x)\text{for a.e. }x.$ Points $x$ where this limit exists and equals $f(x)$ are Lebesgue points of $f$. The Lebesgue differentiation theorem says almost every point is a Lebesgue point.

10.2 Hardy-Littlewood maximal function

Define $Mf(x)={\sup }_{r>0}\frac{1}{\lambda (B_r(x))}{\int }_{B_r(x)}|f|d\lambda .$ Hardy-Littlewood maximal inequality (1930): for $f\in L^1({\mathbb{R}}^n)$, $\lambda (\{Mf>\alpha \})\le \frac{C_n}{\alpha }\Vert f{\Vert }_1.$ For $1<p\le \infty$, $\Vert Mf{\Vert }_p\le C_{n,p}\Vert f{\Vert }_p$ (Marcinkiewicz interpolation).

The maximal inequality and Vitali covering lemma together yield Lebesgue’s differentiation theorem and its $L^p$ generalisation. This is the entry point to real-variable harmonic analysis (Calderón-Zygmund theory).

10.3 Absolute continuity of functions

A function $F:[a,b]\to \mathbb{R}$ is absolutely continuous if for every $\epsilon >0$ there is $\delta >0$ such that for any finite disjoint family of intervals $(a_i,b_i)$ with $\sum (b_i-a_i)<\delta$, we have $\sum |F(b_i)-F(a_i)|<\epsilon$.

Theorem (FTC for Lebesgue): $F$ is absolutely continuous on $[a,b]$ iff $F$ is differentiable a.e., $F^{\prime }\in L^1([a,b])$, and $F(x)-F(a)={\int }_a^x{F}^{\prime }(t)dt.$

The Cantor function (devil’s staircase) is continuous, monotone, has $F^{\prime }=0$ a.e., yet $F(1)-F(0)=1$. It is not absolutely continuous — the assumption is essential.

10.4 Functions of bounded variation

$F$ has bounded variation if $\sup \sum |F(x_{i+1})-F(x_i)|<\infty$ over all partitions. BV functions are differences of two monotone functions (Jordan decomposition). Every BV function is differentiable a.e. (Lebesgue). The BV norm relates to the total variation of the associated signed Borel measure.

11. Signed and complex measures

11.1 Signed measures

A signed measure $\nu$ on $(\Omega ,\mathcal{F})$ takes values in $[-\infty ,\infty ]$ (never both signs of infinity) and is $\sigma$-additive.

Hahn decomposition theorem: $\Omega =P\bigsqcup N$ with $\nu \ge 0$ on $P$ and $\nu \le 0$ on $N$ (unique up to $\nu$-null sets).

Jordan decomposition: $\nu ={\nu }^{+}-{\nu }^{-}$ with ${\nu }^{+},{\nu }^{-}\ge 0$ mutually singular. Both are uniquely determined. The total variation measure is $|\nu |={\nu }^{+}+{\nu }^{-}$, and $\Vert \nu \Vert =|\nu |(\Omega )$.

11.2 Complex measures

Take values in $\mathbb{C}$ with $|\nu |$ finite. Total variation defined via the supremum over countable partitions of $|\nu (A_n)|$ summed. The space of complex measures with total variation norm is a Banach space.

11.3 Riesz-Markov-Kakutani representation

Frigyes Riesz 1909, Andrey Markov 1938, Shizuo Kakutani 1941: the dual of $C_0(X)$ for locally compact Hausdorff $X$ is isometrically isomorphic to the space of finite signed (resp. complex) regular Borel measures on $X$. Every bounded linear functional $\Lambda$ on $C_0(X)$ has the form $\Lambda (f)=\int fd\mu$ for a unique such $\mu$, with $\Vert \Lambda \Vert =\Vert \mu \Vert =|\mu |(X)$.

This is the bridge from functional analysis to measure theory and the foundation of the modern theory of distributions and weak* convergence of measures.

12. Hausdorff measure and dimension

12.1 Hausdorff outer measure

Felix Hausdorff 1918: for $s\ge 0$ and $\delta >0$, ${\mathcal{H}}_{\delta }^s(A)=\inf \left\{{\sum }_i(\text{diam }{U}_i{)}^s:A\subseteq {\bigcup }_i{U}_i,\text{diam }{U}_i\le \delta \right\}.$ Then ${\mathcal{H}}^s(A)={\lim }_{\delta \to 0}{\mathcal{H}}_{\delta }^s(A)$. This is a Borel outer measure on any metric space.

12.2 Hausdorff dimension

For each $A$ there is a critical value ${\dim }_H(A)\in [0,\infty ]$ such that ${\mathcal{H}}^s(A)=\{\begin{array}{ll}\infty & s<{\dim }_H(A),\\ 0& s>{\dim }_H(A).\end{array}$ At $s={\dim }_H(A)$ the value ${\mathcal{H}}^s(A)$ can be anything in $[0,\infty ]$.

12.3 Examples

• ${\dim }_H({\mathbb{R}}^n)=n$.
• ${\dim }_H(\text{Cantor set})=\log 2/\log 3\approx 0.6309$.
• ${\dim }_H(\text{Sierpinski triangle})=\log 3/\log 2\approx 1.585$.
• ${\dim }_H(\text{Koch snowflake boundary})=\log 4/\log 3\approx 1.262$.
• ${\dim }_H(\text{Brownian motion path})=2$ a.s. (in ${\mathbb{R}}^n$ for $n\ge 2$).
• ${\dim }_H(\text{graph of Brownian motion})=3/2$ a.s.

For self-similar sets satisfying the open-set condition (Hutchinson 1981), ${\dim }_H$ equals the similarity dimension $\log N/\log (1/r)$ for $N$ copies scaled by ratio $r$.

12.4 Hausdorff vs Lebesgue

${\mathcal{H}}^n$ on ${\mathbb{R}}^n$ equals Lebesgue measure up to a normalising constant $2^n\Gamma (n/2+1)/{\pi }^{n/2}$ (the reciprocal of the volume of the unit ball). On lower-dimensional surfaces ${\mathcal{H}}^k$ gives intrinsic $k$-dimensional area.

13. Application to probability

13.1 Kolmogorov axiomatization

Andrey Kolmogorov 1933: a probability space is $(\Omega ,\mathcal{F},P)$ with $P(\Omega )=1$. A random variable is a measurable function $X:\Omega \to \mathbb{R}$. The distribution of $X$ is the pushforward $P\circ X^{-1}$ on $(\mathbb{R},\mathcal{B}(\mathbb{R}))$. The expectation is $E[X]={\int }_{\Omega }XdP$ when this is defined.

13.2 Independence

Events $A,B$ are independent if $P(A\cap B)=P(A)P(B)$. Random variables $X,Y$ are independent if $\sigma (X)$ and $\sigma (Y)$ are independent $\sigma$-algebras, equivalently the joint distribution on ${\mathbb{R}}^2$ is the product of marginals.

13.3 Conditional expectation

For $X\in L^1(\Omega ,\mathcal{F},P)$ and sub-$\sigma$-algebra $\mathcal{G}\subseteq \mathcal{F}$, the conditional expectation $E[X|\mathcal{G}]$ is the unique (a.s.) $\mathcal{G}$-measurable random variable with ${\int }_{A}E[X|\mathcal{G}]dP={\int }_{A}XdP$ for all $A\in \mathcal{G}$. Existence by Radon-Nikodym (applied to $\nu (A)={\int }_{A}XdP$ on $\mathcal{G}$, dominated by $P{|}_{\mathcal{G}}$).

13.4 Modes of convergence

For random variables on $(\Omega ,\mathcal{F},P)$:

• Almost sure: $X_n\to X$ a.s. if $P(X_n\to X)=1$.
• In probability: $X_n\stackrel{P}{\to }X$ if $P(|X_n-X|>\epsilon )\to 0$ for all $\epsilon >0$.
• In $L^p$: $E|X_n-X{|}^p\to 0$.
• In distribution: $E[g(X_n)]\to E[g(X)]$ for all bounded continuous $g$.

Implications: a.s. $\Rightarrow$ in probability $\Rightarrow$ in distribution; $L^p$ $\Rightarrow$ in probability. Converses fail.

13.5 Strong law of large numbers

For iid $X_i$ with $E|X_1|<\infty$: $\frac{1}{n}{\sum }_{i=1}^n{X}_i\to E[X_1]\text{a.s.}$ Andrei Kolmogorov 1933.

13.6 Central limit theorem

For iid $X_i$ with mean $\mu$ and finite variance ${\sigma }^2$: $\frac{\sqrt{n}}{\sigma }\left(\frac{1}{n}{\sum }_{i=1}^n{X}_i-\mu \right)\stackrel{d}{\to }\mathcal{N}(0,1).$ Pierre-Simon Laplace 1812 (special cases), Aleksandr Lyapunov 1901, Jarl Lindeberg 1922, Paul Lévy 1925.

14. Application to functional analysis

14.1 Sobolev and BMO

Sobolev spaces $W^{k,p}(\Omega )$ require Lebesgue integration. The space BMO (bounded mean oscillation, John-Nirenberg 1961) of locally integrable functions with bounded mean oscillation $\Vert f{\Vert }_{BMO}={\sup }_B\frac{1}{|B|}{\int }_B\left|f-{\overline{f}}_B\right|dx<\infty$ is the natural endpoint for many singular-integral arguments. See functional-analysis.

14.2 Distributions and weak derivatives

The space ${\mathcal{D}}^{\prime }(\Omega )$ of distributions includes $L_{\text{loc}}^1$ via $T_f(\phi )=\int f\phi$. The weak derivative is defined via the measure-theoretic integration by parts: $\int (D^{\alpha }f)\phi =(-1{)}^{|\alpha |}\int f(D^{\alpha }\phi )$ for test $\phi \in C_c^{\infty }(\Omega )$.

14.3 Spectral measures

The spectral theorem for bounded self-adjoint operators on a Hilbert space realises every such operator as $T={\int }_{\sigma (T)}\lambda dE(\lambda )$ for a projection-valued measure $E$. Functional calculus follows by integrating against $E$.

15. Open questions and current themes

15.1 Existence of “good” measures

Banach measure problem: does there exist a finitely additive measure on $\mathcal{P}(\mathbb{R})$ extending Lebesgue and invariant under isometries? Solution: yes for $n=1,2$ (Banach 1923, via amenability of the isometry group), no for $n\ge 3$ (Banach-Tarski).

15.2 Measurable cardinals

A cardinal $\kappa$ is measurable if there is a non-trivial countably additive 2-valued measure on $\mathcal{P}(\kappa )$. Existence is independent of ZFC and stronger than ZFC (Tarski-Hanf). Connected to large cardinal axioms and inner model theory.

15.3 Geometric measure theory

Federer 1969 Geometric Measure Theory — currents, varifolds, rectifiable sets. Modern uses: minimal surface regularity (Allard 1972), optimal transport (Brenier 1991, McCann 1995, Villani’s books), Plateau’s problem, mean curvature flow weak solutions.

15.4 Optimal transport

Kantorovich-Rubinstein-Wasserstein metrics on measures. Monge problem (1781), Kantorovich relaxation (1942), modern Brenier-McCann-Villani theory. The Wasserstein-2 distance turns the space of probability measures with finite second moment into a geodesic metric space (Otto 2001, Sturm-Lott-Villani 2005-2009 for Ricci-curvature lower bounds on metric measure spaces).

15.5 Non-commutative measure theory

Von Neumann algebras and the theory of traces extend measure theory to operator algebras. A trace on a von Neumann algebra is the non-commutative analogue of integration; type II_1 factors carry a unique trace, type III factors carry none. Connes’ classification of factors (Fields Medal 1982) is the apex.

16. Worked examples and counterexamples

16.1 The Cantor function

Construct: $f_0(x)=x$ on $[0,1]$. At step $n$, replace $f_n$ on each of the $2^{n-1}$ middle-thirds of the previous step with a constant equal to the average of the values at the endpoints. The limit $f$ is continuous, monotone increasing, $f(0)=0$, $f(1)=1$, $f^{\prime }=0$ a.e. (it is constant on each removed interval, total Lebesgue measure $1$). The associated Borel measure ${\mu }_f$ defined by ${\mu }_f([0,x])=f(x)$ is singular with respect to Lebesgue measure and concentrated on the Cantor set.

16.2 Vitali-Hahn-Saks theorem

If ${\mu }_n$ are signed measures on $(\Omega ,\mathcal{F})$, all absolutely continuous w.r.t. $\mu$, with ${\lim }_n{\mu }_n(A)$ existing for every $A\in \mathcal{F}$, then the limit defines a signed measure $\nu \ll \mu$. This is the measure-theoretic Banach-Steinhaus.

16.3 Egorov’s theorem (Dmitri Egorov 1911)

If $f_n\to f$ a.e. on a finite-measure set $E$, then for every $\epsilon >0$ there is $E_{\epsilon }\subseteq E$ with $\mu (E\setminus E_{\epsilon })<\epsilon$ and $f_n\to f$ uniformly on $E_{\epsilon }$. “Almost every pointwise convergence is almost-uniform.”

16.4 Lusin’s theorem (Nikolai Luzin 1912)

For measurable $f$ on ${\mathbb{R}}^n$ and $\epsilon >0$, there is a closed set $F$ with $\lambda ({\mathbb{R}}^n\setminus F)<\epsilon$ and $f{|}_F$ continuous. “Almost every measurable function is almost continuous.”

16.5 Riesz-Thorin interpolation

Marcel Riesz 1927, Olof Thorin 1948: if a linear operator $T$ is bounded from $L^{p_0}\to L^{q_0}$ and $L^{p_1}\to L^{q_1}$, then it is bounded from $L^{p_{\theta }}\to L^{q_{\theta }}$ for all $\theta \in [0,1]$ where $1/p_{\theta }=(1-\theta )/p_0+\theta /p_1$ similarly for $q$. The bound interpolates logarithmically. Indispensable for Fourier multipliers, singular integrals, and many PDE estimates.

17. Connections to other libraries

• Probability and statistics: every modern probability text (Williams Probability with Martingales, Billingsley Probability and Measure, Durrett, Klenke, Kallenberg) is measure theory plus probability-specific constructions.
• Stochastic processes: filtered probability spaces, martingales, stochastic integrals; see stochastic-calculus.
• Functional analysis: $L^p$ spaces, duality, weak/weak* topology, Sobolev spaces; see functional-analysis.
• Harmonic analysis: Fourier transforms on $L^p$ for $1\le p\le 2$, singular integrals, Hardy-Littlewood-Sobolev inequality; see fft-spectral.
• PDE theory: weak solutions, Sobolev spaces, distributional derivatives; see pde-methods.
• Information theory: differential entropy, KL divergence, mutual information; see information-theory.
• Optimal transport and machine learning: Wasserstein-GAN, sliced Wasserstein, gradient flows in the Wasserstein space; see _index.
• Ergodic theory: measure-preserving dynamical systems, Birkhoff’s ergodic theorem (1931), entropy of dynamical systems.

Further reading

• Royden, H. L. and P. M. Fitzpatrick. 2010. Real Analysis (4th ed.). Pearson.
• Folland, G. B. 1999. Real Analysis: Modern Techniques and Their Applications (2nd ed.). Wiley.
• Rudin, W. 1987. Real and Complex Analysis (3rd ed.). McGraw-Hill.
• Bogachev, V. I. 2007. Measure Theory (two volumes). Springer.
• Halmos, P. R. 1950. Measure Theory. Springer.
• Billingsley, P. 1995. Probability and Measure (3rd ed.). Wiley.
• Durrett, R. 2019. Probability: Theory and Examples (5th ed.). Cambridge.
• Williams, D. 1991. Probability with Martingales. Cambridge.
• Stein, E. M. and R. Shakarchi. 2005. Real Analysis: Measure Theory, Integration, and Hilbert Spaces. Princeton Lectures III.
• Federer, H. 1969. Geometric Measure Theory. Springer.
• Mattila, P. 1995. Geometry of Sets and Measures in Euclidean Spaces. Cambridge.
• Tao, T. 2011. An Introduction to Measure Theory. AMS GSM 126.
• Villani, C. 2008. Optimal Transport: Old and New. Springer.
• Kallenberg, O. 2021. Foundations of Modern Probability (3rd ed.). Springer.

Adjacent

• functional-analysis
• probability-fundamentals
• probability-distributions
• stochastic-calculus
• multivariate-calculus
• information-theory
• fft-spectral
• pde-methods
• algebraic-geometry-foundations