Compendium

Python — Reference

19 min read

Python — Reference

Source: https://docs.python.org/3/

Python

  • Created: 1991 by Guido van Rossum (BDFL emeritus); now stewarded by the Python Software Foundation (PSF) and a Steering Council
  • Latest stable: 3.14.x (2025-10-07); 3.15 in development as of 2026-05
  • Paradigms: multi-paradigm — imperative, object-oriented, functional, procedural; gradual typing via typing
  • Typing: dynamic (strong); optional gradual static via type hints + external checkers (mypy, pyright, pyrefly, ty)
  • Memory: garbage collected — reference counting + cyclic garbage collector; CPython’s GC is generational (revert from incremental GC was applied in 3.14.5)
  • Compilation: interpreted via bytecode on a stack-based VM; experimental copy-and-patch JIT (PEP 744) in 3.13+; optional tail-call interpreter in 3.14
  • Primary domains: scripting, data science / ML, scientific computing, web backends, automation / DevOps, education, glue code
  • Official docs: https://docs.python.org/3/

At a glance

  • Reference implementation: CPython. Alternative implementations: PyPy (tracing JIT), GraalPy, MicroPython (embedded), Jython (legacy JVM), IronPython (.NET).
  • GIL: historically one global lock per interpreter. PEP 703 free-threaded build is officially supported as of 3.14 (PEP 779). Subinterpreters with per-interpreter GIL via concurrent.interpreters (PEP 734).
  • Versioning: annual cadence — minor X.Y in October, 2 years bugfix + 3 years security (5-year support window per PEP 602).
  • Governance: PEP (Python Enhancement Proposal) process; Steering Council elected annually since PEP 13 (2019).

Getting started

Install (official):

  • Windows / macOS: installer from https://www.python.org/downloads/
  • macOS recommended: Homebrew brew install python@3.14
  • Linux: distro packages, or apt install python3.14 on recent distros
  • Version manager (de facto standard): uv (Astral) — uv python install 3.14 — replaces pyenv for most workflows. pyenv still common.

Hello world:

print("Hello, world!")

Project layout (modern, PEP 621):

myproject/
  pyproject.toml
  src/
    mypkg/
      __init__.py
      core.py
  tests/
    test_core.py
  README.md

Package manager / build tool:

  • uv (Astral, Rust): single tool — env mgmt, lockfile, install, run. Effectively the default in 2025-26.
  • pip + venv: built-in baseline.
  • poetry, hatch, pdm: alternative project managers.
  • Build backends: hatchling, setuptools, flit-core, pdm-backend, maturin (for Rust extensions).

REPL: python — 3.14 ships a new pyrepl-based REPL with multiline editing, syntax highlighting, and tab-completion of imports. ipython is the power-user REPL.

Basics

Primitives: int (arbitrary precision), float (IEEE 754 double), complex, bool, bytes, str (Unicode), None. No fixed-width integer types in pure Python.

Variables / scope: dynamically typed; LEGB scope (Local, Enclosing, Global, Builtin). global and nonlocal keywords. No block scope — if/for do not introduce scope.

x: int = 10            # annotation is non-binding at runtime
y, z = 1, 2            # tuple unpacking
a, *rest = [1, 2, 3]   # PEP 3132 starred unpacking

Control flow: if/elif/else, for ... in, while, match/case (PEP 634, structural pattern matching, 3.10+), try/except/else/finally. 3.14 allows bracketless multi-exception: except TimeoutError, ConnectionRefusedError: (PEP 758).

Functions: first-class; lexical closures; positional-only (/) and keyword-only (*) markers; defaults; *args/**kwargs; lambdas (single expression).

def fetch(url: str, /, *, timeout: float = 5.0, **headers: str) -> bytes: ...

Strings:

  • f-strings (PEP 498, expanded in 3.12 PEP 701): f"{x = :>5.2f}"

  • t-strings (PEP 750, 3.14) — return Template objects for safe SQL/HTML/shell interpolation:

    query = t"SELECT * FROM users WHERE id = {user_id}"

  • raw r"...", byte b"...", unicode escape \N{GREEK SMALL LETTER ALPHA}.

Built-in collections: list (mutable seq), tuple (immutable seq), dict (insertion-ordered hash map since 3.7), set, frozenset. Comprehensions: list / dict / set / generator.

squares = {n: n*n for n in range(10) if n % 2}

Intermediate

Type system (PEP 484+): structural via typing.Protocol, generics, TypeVar, ParamSpec, TypeVarTuple, Concatenate, Self, Never, LiteralString, Annotated. PEP 695 (3.12) new generic syntax:

def first[T](xs: list[T]) -> T: ...
type Vector[T] = list[T]

PEP 649/749 (3.14): annotations are lazily evaluated via “annotate functions”; introspect with the new annotationlib module. Checkers: mypy, pyright (Microsoft), ty (Astral, Rust), pyrefly (Meta, Rust).

Modules / packages: every directory with __init__.py (or namespace packages, PEP 420) is a package. Imports follow sys.path. Distribution = “wheels” (.whl) on PyPI; metadata in pyproject.toml (PEP 621).

Errors: exceptions are the only mechanism. BaseException / Exception hierarchy. Exception groups (ExceptionGroup, PEP 654) and except* syntax for parallel/concurrent failure handling. raise ... from ... for chaining.

Concurrency primitives:

  • threading — bound by GIL except in free-threaded builds (3.13+ experimental, 3.14 supported).
  • multiprocessing — process per worker, IPC via pickle.
  • asyncio — single-thread cooperative; async def/await, TaskGroup (3.11), asyncio.Runner, structured concurrency patterns.
  • concurrent.futures — ThreadPoolExecutor / ProcessPoolExecutor.
  • concurrent.interpreters (3.14, PEP 734) — true multicore via subinterpreters with per-interpreter GIL.

File I/O / networking: open() (text/binary, encoding-aware), pathlib.Path (preferred over os.path), socket, http.client, urllib, ssl. High level: httpx, requests, aiohttp.

Stdlib highlights: itertools, functools, dataclasses (@dataclass(slots=True, kw_only=True)), enum, typing, pathlib, subprocess, json, sqlite3, re, collections (Counter, defaultdict, deque), contextlib, compression.zstd (3.14, PEP 784), tomllib (3.11+, read-only), zoneinfo (3.9+, IANA tz), graphlib.TopologicalSorter, statistics, secrets (cryptographically strong randomness), tomllib (read TOML, write via tomli-w 3rd party).

Async patterns in 3.11+:

import asyncio
 
async def fetch_all(urls):
    async with asyncio.TaskGroup() as tg:
        tasks = [tg.create_task(fetch(u)) for u in urls]
    # All tasks complete OR all cancel on first exception; errors aggregate into ExceptionGroup
    return [t.result() for t in tasks]
 
async def with_timeout():
    async with asyncio.timeout(5.0):     # cleaner than wait_for; nestable
        await long_op()
 
# Structured concurrency with cancellation propagation
async def race(coro1, coro2):
    async with asyncio.TaskGroup() as tg:
        t1 = tg.create_task(coro1)
        t2 = tg.create_task(coro2)
        # First to finish causes the other to cancel

Advanced

Memory model: CPython refcount + cyclic GC. Tune via gc module — gc.set_threshold(), gc.freeze() to skip pre-fork objects, gc.disable() for batch jobs. sys.getrefcount, tracemalloc for leak hunting. __slots__ to drop the per-instance __dict__ — saves ~40-60% memory per instance, gives 20-30% attribute-access speedup. PEP 683 (3.12+) introduces immortal objects — singletons (None, True, False, small ints, interned strings) skip refcount updates entirely, eliminating cache-line contention under free-threading.

Object layout reality: every CPython object pays a ~28-byte header (PyObject_HEAD: refcount + type pointer + GC list pointers). sys.getsizeof(0) is 28, sys.getsizeof("") is 49, sys.getsizeof([]) is 56. Use __slots__, array.array, numpy, or struct when you need millions of small objects.

Concurrency / parallelism deep dive:

  • Free-threaded build (python3.14t): no GIL, ~5-10% single-thread overhead in 3.14 (down from ~40% in early 3.13). Watch for C extensions that aren’t free-thread-safe — use Py_GIL_DISABLED checks. Wheel tag cp314t-cp314t for free-threaded distributables. NumPy 2.1+, Cython 3.1+, PyTorch 2.5+, scikit-learn 1.5+ already ship free-threaded wheels; ecosystem catching up through 2026.
  • Subinterpreters (PEP 734, concurrent.interpreters): each has its own GIL, isolated module state, communication via shared queues / Channel objects. Lighter-weight than processes (~5 MB vs 30+ MB) but heavier than threads. Good for CPU-bound, embarrassingly-parallel workloads when free-threading isn’t available.
  • asyncio schedulers: default selector loop on Windows/Linux, uvloop (libuv) for 2-4x throughput on POSIX. winloop (libuv port) covers Windows. asyncio.TaskGroup (3.11+) is the structured-concurrency primitive — all tasks cancel together on failure, errors aggregate into ExceptionGroup. asyncio.timeout() context manager (3.11+) supersedes wait_for. anyio + trio are stricter structured-concurrency alternatives that wrap both backends.
  • JIT (PEP 744, --enable-experimental-jit in 3.13+): copy-and-patch JIT — pre-compiled stencils stitched together at runtime. ~5% wins on benchmarks today; 3.14 added tier-2 optimizer + symbolic execution; 3.15 targets 10-15% wins.
  • Tail-call interpreter (3.14, Clang 19+, GCC 15+): rewrites the eval loop as ~200 musttail-call functions, gives the C compiler better register allocation. 3-5% gain on average, up to 10% on dispatch-heavy code.

FFI / interop:

  • ctypes — call C libs from pure Python.
  • cffi — friendlier C bindings.
  • C extension API — Python.h, PyObject*, ref counting; HPy is the portable replacement push.
  • Cython — Python-superset compiled to C.
  • PyO3 (Rust) + maturin for shipping wheels — extremely common for new native code.
  • Free-threaded wheel tag: cp314t.

Reflection: inspect, dis (bytecode), ast (parse tree), sys.settrace / sys.setprofile, sys._getframe, __class__, __dict__, __mro__.

Performance tuning:

  • Profilers: cProfile (deterministic, instrumenting, ~20% overhead), pyinstrument (sampling, low overhead, call-tree view), py-spy (sampling, no-instrument, attaches to running PID, Rust-based), scalene (CPU + GPU + memory, AI-aware), austin (frame-stack sampler, perf-compatible output), memray (Bloomberg, allocation tracker, generates flame graphs).
  • perf integration on Linux (3.12+: PYTHONPERFSUPPORT=1) — Python frames show up in Linux perf flamegraphs natively.
  • JIT: enable experimental copy-and-patch JIT with PYTHON_JIT=1 on JIT-enabled builds.
  • Tail-call interpreter (3.14, Clang 19+): 3-5% benchmark gain.
  • Specialized adaptive interpreter (PEP 659): hot bytecodes get rewritten to specialized variants (e.g., BINARY_OP → BINARY_OP_ADD_INT). Hidden, fully automatic; opcode-level inline caches.

Modern data stack (2024-2026):

ToolRoleWhy now
NumPy 2.0+n-dim arrays, BLAS/LAPACKCleaned-up API (June 2024), 30% smaller binary, free-threaded support
Polarsdataframes (Rust + Arrow)5-30x faster than pandas; lazy + streaming; Polars 1.0 May 2024
DuckDBembedded OLAP SQLSub-second analytics over Parquet/CSV; duckdb.sql("SELECT ...") directly on DataFrames
PyArrowzero-copy interchangeFoundation for pandas 2.x, Polars, DuckDB
JAXautodiff + XLA + GPU/TPUFunctional, traceable, used by DeepMind / Anthropic
PyTorch 2.5+DL with torch.compileInductor → Triton kernels; 30-100% speedups on training
Pandas 2.xtabular workhorsePyArrow backend (dtype_backend="pyarrow") for 2-5x speedups
xarraylabeled n-d arraysClimate, geo, NetCDF/Zarr
modin / daskdistributed pandasWhen data outgrows one machine

Native-code paths: Cython (Python-superset compiled to C) remains the mainstream choice for sci/ML extensions. PyO3 + maturin (Rust → wheel) is the modern alternative; production-shipping at Cloudflare, Pydantic, Polars, ruff. Mojo (Modular) is Python-superset-ish AOT; still pre-1.0 in 2026. mypyc (used by ruff, black, mypy itself) AOT-compiles type-annotated Python to C extensions — 4x typical speedup with zero source changes.

God mode

Metaclasses: class Foo(Bound, metaclass=Meta). Metaclass __new__/__init__/__call__ runs at class creation. Use sparingly — __init_subclass__ covers 90% of cases.

class PluginBase:
    registry = {}
    def __init_subclass__(cls, *, key, **kw):
        super().__init_subclass__(**kw)
        cls.registry[key] = cls
 
class JSONPlugin(PluginBase, key="json"): ...

Descriptors: the protocol behind property, classmethod, staticmethod, __slots__, ORM fields. Implement __get__/__set__/__delete__ and __set_name__.

Generic class subscript: __class_getitem__ lets Foo[int] work without metaclass tricks. typing.Generic uses it.

AST manipulation: ast.parse(src) → mutate → ast.unparse() or compile(). Use ast.NodeTransformer. Powers tools like mypy, black, ruff lints.

Bytecode / disassembly: dis.dis(fn) shows opcodes. compile(src, '<str>', 'exec') returns a code object; build them by hand with code.replace(...). The “specialized adaptive interpreter” (PEP 659) rewrites opcodes at runtime.

sys.settrace / sys.setprofile: the basis of pdb, coverage.py, sampling profilers. Per-frame Python-callable hooks.

Remote / out-of-process: sys.remote_exec(pid, "script.py") (3.14, PEP 768) — zero-overhead attach to a running interpreter. Backs python -m pdb -p PID.

C extension API: PyMethodDef, PyTypeObject, tp_traverse for GC. HPy abstracts ref counting and works on PyPy/GraalPy. The Limited API (Py_LIMITED_API) lets a single wheel target many Python versions.

Stable internals to know: Py_BuildValue, PyObject_GetAttr, freelists, the eval loop in Python/ceval.c, the MAKE_CELL/COPY_FREE_VARS opcodes for closures, LOAD_ATTR inline caches.

Free-threaded considerations: PEP 683 (immortal objects), per-object locking, biased reference counting. C extensions opt in via Py_mod_gil = Py_MOD_GIL_NOT_USED.

Typing god-tier: TypeVarTuple + Unpack (PEP 646) for variadic generics — used by NumPy/Jax for shape-typed arrays: Array[Float, Unpack[Shape]]. ParamSpec + Concatenate for typed decorators that preserve signatures. Self (PEP 673) returns the actual subclass type. override (PEP 698, 3.12) — explicit @override decorator catches typos at type-check time. LiteralString (PEP 675) for SQL/shell injection prevention. ReadOnly (PEP 705, 3.13) for read-only TypedDict fields. TypeIs (PEP 742, 3.13) — narrowing in both true and false branches. @dataclass_transform (PEP 681) lets ORMs / pydantic-like libs participate in dataclass-style inference.

Pattern matching (PEP 634, 3.10+):

match event:
    case {"type": "user", "id": int(uid)}:        # dict pattern + type check
        handle_user(uid)
    case Point(x=0, y=y):                          # class pattern
        on_y_axis(y)
    case [first, *rest] if len(rest) > 5:          # sequence + guard
        process_batch(first, rest)
    case _:
        ignore()

Patterns are not regex; they’re destructuring + type tests + guards. The __match_args__ class attribute (auto on dataclasses) drives positional matching.

Validation stack: Pydantic v2 (2023+, Rust core pydantic-core) — 5-50x faster than v1; standard for FastAPI, LangChain, Modal, Hugging Face inference. msgspec — fastest validator (Cython + msgpack), zero allocations on happy path. attrs — battle-tested, doesn’t enforce types at runtime by default. dataclasses (stdlib, no validation). Marshmallow (legacy schemas).

Idioms & style

  • PEP 8 — formatting; snake_case for functions/vars, PascalCase for classes, SCREAMING_SNAKE for constants, _protected, __name_mangled.
  • PEP 20 (Zen of Python): import this. “Flat is better than nested,” “explicit is better than implicit.”
  • Formatters: ruff format (Astral) — Black-compatible, much faster, dominant in 2025-26. black still widely used.
  • Linters: ruff (Rust, single binary, replaces flake8/pylint/isort/pydocstyle/pyupgrade for most teams). pylint for deeper checks.
  • Type checkers: mypy, pyright, ty (Astral), pyrefly (Meta).
  • Pythonic patterns:
    • “Easier to ask forgiveness than permission” (EAFP) — try/except instead of pre-checking.
    • Iterator + comprehension over manual loops.
    • Context managers (with) for resource lifetime.
    • Dataclasses or attrs over hand-written __init__.
    • @cached_property, functools.lru_cache for memoization.
    • Avoid mutable default args (def f(x=[]) is a classic bug).
  • What reviewers flag: mutable defaults, bare except:, from x import *, manual resource cleanup without with, missing type hints in public APIs, classes used where a function would do.

Ecosystem

DomainTools
Web (sync)Django 5.x, Flask 3.x, Pyramid
Web (async)FastAPI, Starlette, Litestar, Sanic, Robyn (Rust), Quart
DataNumPy 2.x, pandas 2.x, Polars, PyArrow, DuckDB, Modin, Dask
ML / DLPyTorch 2.5+, JAX, TensorFlow, scikit-learn, Hugging Face Transformers, XGBoost, LightGBM
LLM appsLangChain, LlamaIndex, DSPy, instructor, Pydantic AI, Marvin, Outlines, Guidance, Mirascope
InferencevLLM, SGLang, llama.cpp (via llama-cpp-python), TGI, MLX, Ollama bindings
ScientificSciPy, SymPy, Astropy, Biopython, Numba, CuPy, scikit-image
Async I/Ohttpx, aiohttp, anyio, trio, aiofiles
ORM / DBSQLAlchemy 2.x, Django ORM, SQLModel, asyncpg, psycopg3, Tortoise, Piccolo
ValidationPydantic v2, attrs, msgspec, marshmallow
Package mgmtuv (Astral), Poetry, Hatch, PDM, pip+venv, Rye (merged into uv), Pixi (conda-native)
Testingpytest (de facto), unittest (stdlib), hypothesis (property-based), tox/nox (matrix runners), pytest-xdist
DocsSphinx, MkDocs (+ Material), pdoc, mkdocstrings
Build / nativematurin (Rust), Cython, mypyc, scikit-build-core, meson-python, setuptools, hatchling, flit
NotebooksJupyter, JupyterLab, marimo (reactive, git-friendly), Hex, Google Colab, Deepnote
GUI / TUIQt (PyQt6/PySide6), Tkinter (stdlib), Kivy, Textual (TUI, Rich-based), Toga, Flet
Web frontendNiceGUI, Reflex, Streamlit, Gradio, Solara, Panel, Dash
ObservabilityOpenTelemetry SDK, structlog, loguru, Sentry SDK, Prometheus client
Notable usersGoogle, Instagram (Django at scale), Dropbox (built it, uses mypy), Netflix, Spotify, NASA, OpenAI, Anthropic, Stripe, JPMorgan, CERN

Modern code examples

Async with TaskGroup + timeout (3.11+)

import asyncio
import httpx
 
async def fetch_one(client: httpx.AsyncClient, url: str) -> dict:
    r = await client.get(url, timeout=10)
    r.raise_for_status()
    return r.json()
 
async def main():
    urls = ["https://api.example.com/a", "https://api.example.com/b"]
    async with httpx.AsyncClient() as client:
        async with asyncio.timeout(30):
            async with asyncio.TaskGroup() as tg:
                tasks = [tg.create_task(fetch_one(client, u)) for u in urls]
        results = [t.result() for t in tasks]
        return results
 
asyncio.run(main())

Pydantic v2 with FastAPI

from pydantic import BaseModel, EmailStr, Field, field_validator
from typing import Annotated
from fastapi import FastAPI, Depends, HTTPException
 
class UserCreate(BaseModel):
    email: EmailStr
    name: Annotated[str, Field(min_length=1, max_length=100)]
    age: Annotated[int, Field(ge=0, le=150)]
 
    @field_validator("name")
    @classmethod
    def title_case(cls, v: str) -> str:
        return v.title()
 
app = FastAPI()
 
@app.post("/users", response_model=UserCreate)
async def create_user(user: UserCreate) -> UserCreate:
    return user

Polars dataframe pipeline

import polars as pl
 
df = (
    pl.scan_csv("data.csv")                      # lazy
    .filter(pl.col("revenue") > 1000)
    .with_columns([
        (pl.col("revenue") - pl.col("cost")).alias("profit"),
        pl.col("date").str.to_date("%Y-%m-%d"),
    ])
    .group_by("region")
    .agg([
        pl.col("profit").sum(),
        pl.col("profit").mean().alias("avg_profit"),
        pl.len().alias("rows"),
    ])
    .sort("profit", descending=True)
    .collect(streaming=True)                     # only here does work happen
)

Structural pattern matching

def process(event: dict):
    match event:
        case {"type": "click", "x": int(x), "y": int(y)} if x >= 0 and y >= 0:
            click(x, y)
        case {"type": "key", "key": str(k), "modifiers": [*mods]}:
            keypress(k, mods)
        case {"type": "error", **rest}:
            log_error(rest)
        case _:
            raise ValueError(f"unknown event: {event!r}")

Gotchas

  • Mutable default arguments — evaluated once at def time. def f(x=[]): shares the list across calls.
  • Late binding closures in loops — [lambda: i for i in range(3)] all return 2. Use lambda i=i: i.
  • is vs == — is checks identity; only safe for singletons (None, True, small ints, interned strs).
  • Integer / string interning — a is b for ints in [-5, 256] may be True by accident.
  • Tuple of one: (1) is an int, (1,) is a tuple.
  • bool is a subclass of int — True == 1, isinstance(True, int) is True.
  • GIL myths — IO-bound threading still helps; CPU-bound usually wants multiprocessing or free-threaded.
  • Circular imports — split modules or import inside functions.
  • __init__.py shadowing — a package and a sibling module of the same name will conflict.
  • Pickle is unsafe — never unpickle untrusted data; arbitrary code execution.
  • subprocess.shell=True — shell injection vector; pass a list argv instead.
  • for x in dict: mutating — RuntimeError: dictionary changed size during iteration.
  • Newcomers from Java/C#: no private; convention is leading _. No method overloading; use functools.singledispatch or default args.
  • Newcomers from JS: no implicit type coercion ("1" + 1 raises). Truthiness is per-type — bool([]) is False but bool([0]) is True.

Tooling renaissance (Astral effect)

The Rust-rewrite-of-Python-tooling trend (Astral, Meta, Microsoft) has reshaped the ecosystem 2023-2026:

ReplacesModern toolSpeedupBuilt in
pip + venv + pip-tools + pyenv + pipxuv10-100xRust
black + flake8 + isort + pyupgrade + pydocstyle + autoflakeruff100-1000xRust
mypy / pyrety (Astral), pyrefly (Meta), pyright (MS)5-100xRust / Rust / TS
jupyter (browser cell editor)marimoreactivePython
poetry buildmaturin (for Rust ext)nativeRust

uv specifics: universal lock file uv.lock, single resolver across requirements/dev/build groups, native PEP 723 inline script metadata (uv run script.py reads # /// script header for deps), uv tool run X for ephemeral execution like pipx, uv python install 3.14 for managed interpreters. Effectively replaces 5+ tools for most projects.

ruff capability matrix:

  • 800+ rules covering pyflakes, pycodestyle, mccabe, pylint, isort, pydocstyle, flake8-bugbear, flake8-comprehensions, flake8-simplify, pep8-naming, pyupgrade, eradicate, perflint.
  • ruff format is Black-compatible (>99.9% match) but 30x faster.
  • Native --fix and --unsafe-fixes apply auto-fixes; SARIF output for GitHub Code Scanning.
  • [tool.ruff.lint.per-file-ignores] per-file rule customization.
  • Plugin-free architecture — ships everything in one Rust binary.

Modern packaging and distribution

Wheel publishing in 2026:

  • Pure-Python wheels (py3-none-any) — simplest, install anywhere.
  • Platform wheels (e.g., cp314-cp314-manylinux_2_28_x86_64) for native code.
  • manylinux_2_28 is the current target (CentOS 8 / RHEL 8 ABI baseline); manylinux_2_34 proposed.
  • musllinux for Alpine/Distroless containers.
  • macOS wheels: macosx_11_0_arm64 (M-series), macosx_10_15_x86_64, often a universal2 superset.
  • cibuildwheel (PyPA) — CI matrix that builds wheels for every supported triple; standard for any package shipping native code (NumPy, Pillow, Pydantic, ruff).
  • maturin for Rust extensions — builds + publishes wheels with PyO3 bindings in one step. Used by Polars, Pydantic-core, ruff, Cryptography (partially).
  • scikit-build-core for CMake-based native builds.
  • PEP 711 (PyBI) would bundle interpreters into wheels; experimental.

CPython release cadence and support

VersionReleasedBug-fix untilSecurity untilNotable
3.10Oct 2021Apr 2024Oct 2026match/case, paren-context-managers
3.11Oct 2022Apr 2025Oct 2027TaskGroup, ExceptionGroup, 10-60% speedup (Faster CPython)
3.12Oct 2023Apr 2026Oct 2028PEP 695 generics, per-interpreter GIL groundwork, f-string formal grammar
3.13Oct 2024Apr 2027Oct 2029Free-threaded experimental, JIT experimental, new REPL
3.14Oct 2025Apr 2028Oct 2030t-strings, free-threaded supported, tail-call interpreter, subinterpreters API
3.15Oct 2026 (planned)——Continued JIT tier-2 work, more typing improvements

Annual release in October; bugfix for 2 years; security for 5 years total. PEP 602.

PEP 723 inline metadata (3.11+, popularized by uv) — make a script self-contained:

# /// script
# requires-python = ">=3.12"
# dependencies = ["httpx", "rich"]
# ///
import httpx, rich

Run with uv run script.py — uv builds a temp venv, installs the deps, executes.

Free-threaded Python (PEP 703 + 779) in depth

The GIL has been Python’s biggest limitation since 1991. The free-threaded build (officially supported in 3.14 per PEP 779; experimental in 3.13) finally removes it.

How it works under the hood:

  • PEP 683 immortal objects — singletons (None, True, False, small ints, interned strings) skip refcount updates entirely; no cache contention.
  • Biased reference counting — each object has a thread-local “owning thread”; common case (single thread accessing) is a non-atomic local increment. Atomic operations only on cross-thread access.
  • Per-object locking (PyMutex) — fine-grained locks on dict, list, and other mutable containers replace global serialization.
  • Mimalloc is bundled as the default allocator for free-threaded builds — better multi-thread scaling than glibc malloc.
  • Specialized adaptive interpreter (PEP 659) safely shared across threads via versioning.

What works today (3.14):

  • Pure Python code: GIL-free, scales close to linearly on CPU-bound workloads.
  • Stdlib: thread-safe.
  • C extensions: must opt in with Py_mod_gil = Py_MOD_GIL_NOT_USED and audit for races; mainstream packages shipping free-threaded wheels in 2025-26 include NumPy 2.1+, Cython 3.1+, PyTorch 2.5+, scikit-learn 1.5+, Pillow 11+, lxml.

Performance trade-off: single-thread overhead dropped from ~40% in 3.13 to ~5-10% in 3.14; expected to be ~5% by 3.16. Most workloads benefit; CPU-bound multi-threaded Python (long requested) now possible without multiprocessing.

Modern observability stack

LayerRecommended
Loggingstructlog or loguru (configurable, JSON output)
Metricsprometheus-client, opentelemetry-instrumentation
TracingOpenTelemetry SDK + auto-instrumentation for Flask/FastAPI/SQLAlchemy/httpx/asyncpg/Celery
Error reportingSentry SDK (auto-captures unhandled exceptions, performance traces)
ProfilingContinuous: pyroscope agent or py-spy --pyspy-http; ad-hoc: scalene, memray
Debuggingdebugpy (VS Code/PyCharm), pdb/ipdb, pdbpp (better TUI), pudb (curses)

Modern Python project stack (2026)

The reference “new project” toolchain in late-2025 / 2026 has consolidated around Astral’s tooling + a few stable PEPs:

ConcernPickWhy
Interpreter mgmtuv python install 3.14Replaces pyenv; cross-platform, no shell rc-file dance
Env + lock + installuv (uv lock, uv sync, uv add, uv run)One binary, ~10-100x faster than pip+pip-tools+venv
Format + lintruff format + ruff check --fixSingle binary covers Black + isort + flake8 + pyupgrade + pydocstyle
Type-checkpyright (deep) or ty (Astral, fast)mypy still common but losing ground on perf
Testpytest + pytest-xdist (parallel) + hypothesis (property)nox / tox for matrix orchestration
Native wheelsmaturin (Rust ext) or cibuildwheel (C/C++ matrix)PyO3 + maturin is the new default for native code
Deploy targetsModal, Replicate, Fly.io, Render, AWS Lambda Powertools, Google Cloud RunAll ship first-class Python images / uv support

PEP cheatsheet worth knowing for pyproject.toml:

  • PEP 621 — [project] metadata (name, version, dependencies, optional-dependencies).
  • PEP 631 — explicit dependencies = [...] array (replaces setup.py install_requires).
  • PEP 723 — inline script metadata via # /// script header; uv run script.py resolves it on the fly.
  • PEP 735 — [dependency-groups] for dev/docs/test groups (uv + pip 25+ both support).
  • PEP 751 — pylock.toml universal lock format (proposed; uv ships its own uv.lock until standardization).

Worked gotcha examples

1. Mutable default argument — the default is evaluated once at def time:

def append_to(item, target=[]):
    target.append(item)
    return target
 
append_to(1)          # [1]
append_to(2)          # [1, 2]  ← the same list!
 
# Fix: use None sentinel
def append_to(item, target=None):
    if target is None:
        target = []
    target.append(item)
    return target

2. Late-binding closures in a loop — closures capture the variable, not its value:

fns = [lambda: i for i in range(3)]
[f() for f in fns]   # [2, 2, 2] — all return final i
 
# Fix: bind at definition time via default arg
fns = [lambda i=i: i for i in range(3)]
[f() for f in fns]   # [0, 1, 2]

3. GIL contention on CPU-bound work — threads compete for one bytecode-executing core (on non-free-threaded builds):

# Bad: 4 threads, near-zero speedup on CPU work
from threading import Thread
ts = [Thread(target=compute_pi, args=(1_000_000,)) for _ in range(4)]
for t in ts: t.start()
for t in ts: t.join()    # ~4x serial time
 
# Fix 1: ProcessPoolExecutor (forks/spawns, sidesteps GIL)
from concurrent.futures import ProcessPoolExecutor
with ProcessPoolExecutor(max_workers=4) as ex:
    list(ex.map(compute_pi, [1_000_000]*4))
 
# Fix 2 (3.14+): free-threaded build python3.14t — pure threads scale linearly
# Fix 3 (3.14+): concurrent.interpreters — subinterpreter per worker, per-interpreter GIL

Citations

Graph View