Compendium

Python — Reference

9 min read

Python — Reference

Source: https://docs.python.org/3/

Python

  • Created: 1991 by Guido van Rossum (BDFL emeritus); now stewarded by the Python Software Foundation (PSF) and a Steering Council
  • Latest stable: 3.14.x (2025-10-07); 3.15 in development as of 2026-05
  • Paradigms: multi-paradigm — imperative, object-oriented, functional, procedural; gradual typing via typing
  • Typing: dynamic (strong); optional gradual static via type hints + external checkers (mypy, pyright, pyrefly, ty)
  • Memory: garbage collected — reference counting + cyclic garbage collector; CPython’s GC is generational (revert from incremental GC was applied in 3.14.5)
  • Compilation: interpreted via bytecode on a stack-based VM; experimental copy-and-patch JIT (PEP 744) in 3.13+; optional tail-call interpreter in 3.14
  • Primary domains: scripting, data science / ML, scientific computing, web backends, automation / DevOps, education, glue code
  • Official docs: https://docs.python.org/3/

At a glance

  • Reference implementation: CPython. Alternative implementations: PyPy (tracing JIT), GraalPy, MicroPython (embedded), Jython (legacy JVM), IronPython (.NET).
  • GIL: historically one global lock per interpreter. PEP 703 free-threaded build is officially supported as of 3.14 (PEP 779). Subinterpreters with per-interpreter GIL via concurrent.interpreters (PEP 734).
  • Versioning: annual cadence — minor X.Y in October, 2 years bugfix + 3 years security (5-year support window per PEP 602).
  • Governance: PEP (Python Enhancement Proposal) process; Steering Council elected annually since PEP 13 (2019).

Getting started

Install (official):

  • Windows / macOS: installer from https://www.python.org/downloads/
  • macOS recommended: Homebrew brew install python@3.14
  • Linux: distro packages, or apt install python3.14 on recent distros
  • Version manager (de facto standard): uv (Astral) — uv python install 3.14 — replaces pyenv for most workflows. pyenv still common.

Hello world:

print("Hello, world!")

Project layout (modern, PEP 621):

myproject/
  pyproject.toml
  src/
    mypkg/
      __init__.py
      core.py
  tests/
    test_core.py
  README.md

Package manager / build tool:

  • uv (Astral, Rust): single tool — env mgmt, lockfile, install, run. Effectively the default in 2025-26.
  • pip + venv: built-in baseline.
  • poetry, hatch, pdm: alternative project managers.
  • Build backends: hatchling, setuptools, flit-core, pdm-backend, maturin (for Rust extensions).

REPL: python — 3.14 ships a new pyrepl-based REPL with multiline editing, syntax highlighting, and tab-completion of imports. ipython is the power-user REPL.

Basics

Primitives: int (arbitrary precision), float (IEEE 754 double), complex, bool, bytes, str (Unicode), None. No fixed-width integer types in pure Python.

Variables / scope: dynamically typed; LEGB scope (Local, Enclosing, Global, Builtin). global and nonlocal keywords. No block scope — if/for do not introduce scope.

x: int = 10            # annotation is non-binding at runtime
y, z = 1, 2            # tuple unpacking
a, *rest = [1, 2, 3]   # PEP 3132 starred unpacking

Control flow: if/elif/else, for ... in, while, match/case (PEP 634, structural pattern matching, 3.10+), try/except/else/finally. 3.14 allows bracketless multi-exception: except TimeoutError, ConnectionRefusedError: (PEP 758).

Functions: first-class; lexical closures; positional-only (/) and keyword-only (*) markers; defaults; *args/**kwargs; lambdas (single expression).

def fetch(url: str, /, *, timeout: float = 5.0, **headers: str) -> bytes: ...

Strings:

  • f-strings (PEP 498, expanded in 3.12 PEP 701): f"{x = :>5.2f}"

  • t-strings (PEP 750, 3.14) — return Template objects for safe SQL/HTML/shell interpolation:

    query = t"SELECT * FROM users WHERE id = {user_id}"

  • raw r"...", byte b"...", unicode escape \N{GREEK SMALL LETTER ALPHA}.

Built-in collections: list (mutable seq), tuple (immutable seq), dict (insertion-ordered hash map since 3.7), set, frozenset. Comprehensions: list / dict / set / generator.

squares = {n: n*n for n in range(10) if n % 2}

Intermediate

Type system (PEP 484+): structural via typing.Protocol, generics, TypeVar, ParamSpec, TypeVarTuple, Concatenate, Self, Never, LiteralString, Annotated. PEP 695 (3.12) new generic syntax:

def first[T](xs: list[T]) -> T: ...
type Vector[T] = list[T]

PEP 649/749 (3.14): annotations are lazily evaluated via “annotate functions”; introspect with the new annotationlib module. Checkers: mypy, pyright (Microsoft), ty (Astral, Rust), pyrefly (Meta, Rust).

Modules / packages: every directory with __init__.py (or namespace packages, PEP 420) is a package. Imports follow sys.path. Distribution = “wheels” (.whl) on PyPI; metadata in pyproject.toml (PEP 621).

Errors: exceptions are the only mechanism. BaseException / Exception hierarchy. Exception groups (ExceptionGroup, PEP 654) and except* syntax for parallel/concurrent failure handling. raise ... from ... for chaining.

Concurrency primitives:

  • threading — bound by GIL except in free-threaded builds (3.13+ experimental, 3.14 supported).
  • multiprocessing — process per worker, IPC via pickle.
  • asyncio — single-thread cooperative; async def/await, TaskGroup (3.11), asyncio.Runner, structured concurrency patterns.
  • concurrent.futures — ThreadPoolExecutor / ProcessPoolExecutor.
  • concurrent.interpreters (3.14, PEP 734) — true multicore via subinterpreters with per-interpreter GIL.

File I/O / networking: open() (text/binary, encoding-aware), pathlib.Path (preferred over os.path), socket, http.client, urllib, ssl. High level: httpx, requests, aiohttp.

Stdlib highlights: itertools, functools, dataclasses (@dataclass(slots=True, kw_only=True)), enum, typing, pathlib, subprocess, json, sqlite3, re, collections (Counter, defaultdict, deque), contextlib, compression.zstd (3.14, PEP 784), tomllib (3.11+, read-only).

Advanced

Memory model: CPython refcount + cyclic GC. Tune via gc module — gc.set_threshold(), gc.freeze() to skip pre-fork objects, gc.disable() for batch jobs. sys.getrefcount, tracemalloc for leak hunting. __slots__ to drop the per-instance __dict__.

Concurrency / parallelism deep dive:

  • Free-threaded build (python3.14t): no GIL, ~5-10% single-thread overhead in 3.14 (down from much higher in 3.13). Watch for C extensions that aren’t free-thread-safe — use Py_GIL_DISABLED checks.
  • Subinterpreters: each has its own GIL, isolated module state, communication via shared queues / channels.
  • asyncio schedulers: default selector loop on Windows/Linux, uvloop (libuv) for 2-4x throughput on POSIX.

FFI / interop:

  • ctypes — call C libs from pure Python.
  • cffi — friendlier C bindings.
  • C extension API — Python.h, PyObject*, ref counting; HPy is the portable replacement push.
  • Cython — Python-superset compiled to C.
  • PyO3 (Rust) + maturin for shipping wheels — extremely common for new native code.
  • Free-threaded wheel tag: cp314t.

Reflection: inspect, dis (bytecode), ast (parse tree), sys.settrace / sys.setprofile, sys._getframe, __class__, __dict__, __mro__.

Performance tuning:

  • Profilers: cProfile, pyinstrument, py-spy (sampling, no-instrument), scalene, austin.
  • perf integration on Linux (3.12+: PYTHONPERFSUPPORT=1).
  • JIT: enable experimental copy-and-patch JIT with PYTHON_JIT=1 on JIT-enabled builds.
  • Tail-call interpreter (3.14, Clang 19+): 3-5% benchmark gain.

God mode

Metaclasses: class Foo(Bound, metaclass=Meta). Metaclass __new__/__init__/__call__ runs at class creation. Use sparingly — __init_subclass__ covers 90% of cases.

class PluginBase:
    registry = {}
    def __init_subclass__(cls, *, key, **kw):
        super().__init_subclass__(**kw)
        cls.registry[key] = cls
 
class JSONPlugin(PluginBase, key="json"): ...

Descriptors: the protocol behind property, classmethod, staticmethod, __slots__, ORM fields. Implement __get__/__set__/__delete__ and __set_name__.

Generic class subscript: __class_getitem__ lets Foo[int] work without metaclass tricks. typing.Generic uses it.

AST manipulation: ast.parse(src) → mutate → ast.unparse() or compile(). Use ast.NodeTransformer. Powers tools like mypy, black, ruff lints.

Bytecode / disassembly: dis.dis(fn) shows opcodes. compile(src, '<str>', 'exec') returns a code object; build them by hand with code.replace(...). The “specialized adaptive interpreter” (PEP 659) rewrites opcodes at runtime.

sys.settrace / sys.setprofile: the basis of pdb, coverage.py, sampling profilers. Per-frame Python-callable hooks.

Remote / out-of-process: sys.remote_exec(pid, "script.py") (3.14, PEP 768) — zero-overhead attach to a running interpreter. Backs python -m pdb -p PID.

C extension API: PyMethodDef, PyTypeObject, tp_traverse for GC. HPy abstracts ref counting and works on PyPy/GraalPy. The Limited API (Py_LIMITED_API) lets a single wheel target many Python versions.

Stable internals to know: Py_BuildValue, PyObject_GetAttr, freelists, the eval loop in Python/ceval.c, the MAKE_CELL/COPY_FREE_VARS opcodes for closures, LOAD_ATTR inline caches.

Free-threaded considerations: PEP 683 (immortal objects), per-object locking, biased reference counting. C extensions opt in via Py_mod_gil = Py_MOD_GIL_NOT_USED.

Idioms & style

  • PEP 8 — formatting; snake_case for functions/vars, PascalCase for classes, SCREAMING_SNAKE for constants, _protected, __name_mangled.
  • PEP 20 (Zen of Python): import this. “Flat is better than nested,” “explicit is better than implicit.”
  • Formatters: ruff format (Astral) — Black-compatible, much faster, dominant in 2025-26. black still widely used.
  • Linters: ruff (Rust, single binary, replaces flake8/pylint/isort/pydocstyle/pyupgrade for most teams). pylint for deeper checks.
  • Type checkers: mypy, pyright, ty (Astral), pyrefly (Meta).
  • Pythonic patterns:
    • “Easier to ask forgiveness than permission” (EAFP) — try/except instead of pre-checking.
    • Iterator + comprehension over manual loops.
    • Context managers (with) for resource lifetime.
    • Dataclasses or attrs over hand-written __init__.
    • @cached_property, functools.lru_cache for memoization.
    • Avoid mutable default args (def f(x=[]) is a classic bug).
  • What reviewers flag: mutable defaults, bare except:, from x import *, manual resource cleanup without with, missing type hints in public APIs, classes used where a function would do.

Ecosystem

DomainTools
Web (sync)Django, Flask, Pyramid
Web (async)FastAPI, Starlette, Litestar, Sanic
DataNumPy, pandas, Polars, PyArrow, DuckDB
ML / DLPyTorch, JAX, TensorFlow, scikit-learn, Hugging Face Transformers
LLM appsLangChain, LlamaIndex, DSPy, instructor, Pydantic AI
ScientificSciPy, SymPy, Astropy, Biopython
Async I/Ohttpx, aiohttp, anyio, trio
ORM / DBSQLAlchemy, Django ORM, SQLModel, asyncpg, psycopg3
ValidationPydantic v2, attrs, msgspec, marshmallow
Testingpytest (de facto), unittest (stdlib), hypothesis (property-based), tox/nox (matrix runners)
DocsSphinx, MkDocs (+ Material), pdoc
Packaginguv, hatch, poetry, twine, build, cibuildwheel, maturin
NotebooksJupyter, JupyterLab, marimo, Hex, Google Colab
GUIQt (PyQt6/PySide6), Tkinter (stdlib), Kivy, Textual (TUI)
Notable usersGoogle, Instagram, Dropbox (built it), Netflix, Spotify, NASA, OpenAI

Gotchas

  • Mutable default arguments — evaluated once at def time. def f(x=[]): shares the list across calls.
  • Late binding closures in loops — [lambda: i for i in range(3)] all return 2. Use lambda i=i: i.
  • is vs == — is checks identity; only safe for singletons (None, True, small ints, interned strs).
  • Integer / string interning — a is b for ints in [-5, 256] may be True by accident.
  • Tuple of one: (1) is an int, (1,) is a tuple.
  • bool is a subclass of int — True == 1, isinstance(True, int) is True.
  • GIL myths — IO-bound threading still helps; CPU-bound usually wants multiprocessing or free-threaded.
  • Circular imports — split modules or import inside functions.
  • __init__.py shadowing — a package and a sibling module of the same name will conflict.
  • Pickle is unsafe — never unpickle untrusted data; arbitrary code execution.
  • subprocess.shell=True — shell injection vector; pass a list argv instead.
  • for x in dict: mutating — RuntimeError: dictionary changed size during iteration.
  • Newcomers from Java/C#: no private; convention is leading _. No method overloading; use functools.singledispatch or default args.
  • Newcomers from JS: no implicit type coercion ("1" + 1 raises). Truthiness is per-type — bool([]) is False but bool([0]) is True.

Citations

Graph View