Standard library

Python’s standard library is large — batteries included is the conventional phrase. The library covers most needs of contemporary application code: file and path handling, networking, JSON and XML, regular expressions, dates and times, cryptography, threading and multiprocessing, logging, testing, and a substantial set of utility modules. Beyond the standard library, the third-party ecosystem on PyPI is vast; modern Python projects use a small set of well-known third-party libraries for the cases the standard library does not cover gracefully (modern HTTP, data validation, async I/O, plotting).

This page surveys the principal modules. The dedicated pages cover specific topics in depth: Type hints, Iterators and generators, Decorators, Async and concurrency, I/O.

The standard library structure

Python’s standard library is organised into modules and packages. The principal areas:

Area	Principal modules
Built-ins (always available)	print, len, range, int, str, list, dict, …
Text	string, re, textwrap, unicodedata, stringprep
Binary	struct, codecs, base64, binascii
Data types	datetime, zoneinfo, calendar, collections, array, enum, dataclasses
Math	math, cmath, decimal, fractions, random, statistics
Functional	itertools, functools, operator
Files and paths	os, os.path, pathlib, shutil, glob, fileinput, tempfile
Persistence	pickle, shelve, marshal, dbm, sqlite3
Compression	zlib, gzip, bz2, lzma, tarfile, zipfile
Encodings	csv, json, xml.etree.ElementTree, html, email
Networking	socket, ssl, urllib, http.client, http.server, asyncio
Concurrency	threading, multiprocessing, concurrent.futures, subprocess, queue, asyncio
Crypto	hashlib, hmac, secrets, ssl
OS	os, sys, signal, platform, getpass
Logging and debugging	logging, pdb, traceback, warnings, inspect
Testing	unittest, doctest, unittest.mock, test
Type system	typing, types, abc
Importing	importlib, pkgutil, pkg_resources (deprecated)

The library is too large to enumerate; this page covers the principal modules a working programmer encounters.

Built-ins

The built-in functions and types are always available. The conventional ones:

# Constructors / converters:
int(...), float(...), str(...), bool(...), bytes(...), bytearray(...)
list(...), tuple(...), set(...), frozenset(...), dict(...)
type(...), isinstance(...), issubclass(...)

# I/O:
print(...), input(...)

# Iteration:
range(...), enumerate(...), zip(...), reversed(...), sorted(...)
iter(...), next(...)

# Math:
abs(...), round(...), min(...), max(...), sum(...), pow(...), divmod(...)

# Object inspection:
hasattr(...), getattr(...), setattr(...), delattr(...)
callable(...), id(...), hash(...), repr(...), str(...)
dir(...), vars(...), help(...)

# Functional:
map(...), filter(...)         # return iterators
all(...), any(...)

# Compile:
compile(...), exec(...), eval(...)

# Misc:
open(...), len(...)

The built-ins are the foundation; familiarity with them is part of fluency in the language.

os and pathlib

For file and path operations:

pathlib (the modern form)

from pathlib import Path

p = Path("/var/log/app.log")
p.exists()                              # bool
p.is_file()                              # bool
p.is_dir()                               # bool
p.stat().st_size                         # bytes

# Joining:
p = Path("/var/log") / "app.log"
p = Path.home() / "documents" / "file.txt"

# Reading and writing:
p.read_text(encoding="utf-8")
p.read_bytes()
p.write_text("contents", encoding="utf-8")
p.write_bytes(b"\x00\x01")

# Iteration:
for f in Path(".").iterdir():
    print(f)

for f in Path(".").glob("*.py"):
    print(f)

for f in Path(".").rglob("*.py"):       # recursive
    print(f)

# Manipulation:
p.parent
p.name                                   # filename with extension
p.stem                                   # filename without extension
p.suffix                                  # extension
p.with_suffix(".bak")                    # change extension
p.with_name("new_name.txt")               # change full name
p.absolute()
p.resolve()                              # absolute + resolve symlinks
p.relative_to(other)

The Path class is the conventional contemporary path API; substantially clearer than the older os.path functions.

os (system-level)

import os

os.getcwd()                              # current directory
os.chdir(path)                           # change directory
os.environ                               # the environment as a dict
os.environ.get("HOME")
os.path.exists(path)                     # legacy form
os.makedirs(path, exist_ok=True)
os.remove(path)                          # remove a file
os.rmdir(path)                            # remove an empty directory

# Listing:
os.listdir(".")

# Walking:
for root, dirs, files in os.walk("."):
    for name in files:
        process(os.path.join(root, name))

# Process info:
os.getpid()
os.getppid()
os.getuid()                               # Linux/Mac

# Spawning:
os.system("cmd")                          # rare; subprocess is preferred
os.execvp("cmd", ["cmd", "arg"])

The conventional contemporary advice: use pathlib for paths, os for system-level operations not covered by pathlib.

subprocess

For running external programs:

import subprocess

# Run and wait:
result = subprocess.run(["ls", "-la"], capture_output=True, text=True)
print(result.stdout)
print(result.returncode)

# Check status:
subprocess.run(["test", "-f", path], check=True)    # raises CalledProcessError on failure

# With pipe:
result = subprocess.run(
    ["grep", "ERROR"],
    input=text,
    capture_output=True,
    text=True
)

The subprocess.run is the conventional contemporary form (since Python 3.5); the older subprocess.call, check_call, check_output are still available but rarely the right choice.

For long-running processes, subprocess.Popen:

proc = subprocess.Popen(["server"], stdout=subprocess.PIPE)
for line in proc.stdout:
    process(line)
proc.wait()

re

Regular expressions:

import re

# Match (start of string only):
m = re.match(r"\d+", "123 abc")
if m:
    print(m.group())                     # "123"

# Search (anywhere):
m = re.search(r"\d+", "abc 123 def")
if m:
    print(m.group())                     # "123"

# Findall:
re.findall(r"\d+", "1 and 2 and 3")      # ["1", "2", "3"]

# Substitute:
re.sub(r"\d+", "X", "1 and 2 and 3")     # "X and X and X"

# Split:
re.split(r"\s+", "hello world from python")   # ["hello", "world", ...]

# Compile for repeated use:
pattern = re.compile(r"\d{4}-\d{2}-\d{2}")
matches = pattern.findall(text)

The re module is the standard regex; for substantial regex work, the third-party regex module provides additional features (Unicode handling, named character classes).

json and csv

json

import json

# Encode:
data = {"name": "alice", "age": 30, "active": True}
text = json.dumps(data)                          # '{"name": "alice", ...}'
text = json.dumps(data, indent=2)                # pretty-printed

# Decode:
data = json.loads(text)

# File:
with open(path) as f:
    data = json.load(f)
with open(path, "w") as f:
    json.dump(data, f, indent=2)

The json is the conventional Python JSON encoder/decoder. For substantial data, third-party libraries (orjson, ujson) are faster; for richer features (validation, schema), pydantic is the conventional choice.

csv

import csv

# Read:
with open("data.csv", newline="") as f:
    reader = csv.DictReader(f)
    for row in reader:
        print(row["name"], row["age"])

# Write:
with open("output.csv", "w", newline="") as f:
    writer = csv.DictWriter(f, fieldnames=["name", "age"])
    writer.writeheader()
    writer.writerow({"name": "alice", "age": 30})

The csv is the conventional CSV reader/writer. For substantial CSV work with mixed types, pandas is conventional in data-science contexts.

datetime and zoneinfo

from datetime import datetime, date, time, timedelta, timezone
from zoneinfo import ZoneInfo

# Now:
now = datetime.now(timezone.utc)             # UTC; preferable for new code
now_local = datetime.now()                    # local time; rarely the right choice

# Parsing:
dt = datetime.fromisoformat("2024-01-15T12:00:00+00:00")
dt = datetime.strptime("2024-01-15", "%Y-%m-%d")

# Formatting:
dt.isoformat()                               # "2024-01-15T12:00:00+00:00"
dt.strftime("%Y-%m-%d %H:%M:%S")

# Arithmetic:
later = now + timedelta(hours=1)
diff = later - now                           # timedelta(hours=1)

# Time zones:
tz = ZoneInfo("Europe/London")
dt_london = dt.astimezone(tz)

# Date only:
today = date.today()

The zoneinfo (since 3.9) admits time-zone-aware datetimes with the system’s IANA database; conventional contemporary form. The older pytz library is no longer necessary.

collections, itertools, functools, operator

The functional and data-structure helpers:

from collections import (
    defaultdict, Counter, deque, OrderedDict, namedtuple, ChainMap
)
import itertools
from functools import (
    lru_cache, cache, cached_property, partial, reduce, wraps,
    singledispatch, total_ordering
)
from operator import itemgetter, attrgetter, methodcaller

Treated in Data structures, Iterators and generators, Decorators, Functional.

urllib.request and http.client

The standard-library HTTP clients:

from urllib.request import urlopen
from urllib.parse import urlencode

with urlopen("https://api.example.com/data") as response:
    data = response.read()

# POST:
from urllib.request import Request
req = Request(
    url="https://api.example.com/items",
    data=json.dumps({"name": "alice"}).encode(),
    headers={"Content-Type": "application/json"},
    method="POST"
)
with urlopen(req) as response:
    result = json.load(response)

The standard-library HTTP client is workable but limited. For non-trivial HTTP, the conventional third-party choice is:

• requests — synchronous; the most popular HTTP library.
• httpx — synchronous and async; modern API; the conventional contemporary choice.
• aiohttp — async; older but mature.

# requests:
import requests

response = requests.get("https://api.example.com/data")
data = response.json()

# httpx:
import httpx

response = httpx.get("https://api.example.com/data")
data = response.json()

# httpx async:
async with httpx.AsyncClient() as client:
    response = await client.get("https://api.example.com/data")
    data = response.json()

sqlite3

A built-in SQL database:

import sqlite3

conn = sqlite3.connect("data.db")
cursor = conn.cursor()

cursor.execute("""
    CREATE TABLE IF NOT EXISTS users (
        id INTEGER PRIMARY KEY,
        name TEXT NOT NULL,
        email TEXT UNIQUE
    )
""")

cursor.execute(
    "INSERT INTO users (name, email) VALUES (?, ?)",
    ("alice", "alice@example.com")
)
conn.commit()

cursor.execute("SELECT * FROM users WHERE name = ?", ("alice",))
for row in cursor.fetchall():
    print(row)

conn.close()

The conventional uses are local-file databases, embedded storage, prototypes. For substantial database work, ORMs (SQLAlchemy, Django ORM) and database-specific libraries (psycopg for PostgreSQL, pymongo for MongoDB) are conventional.

logging

import logging

logging.basicConfig(level=logging.INFO,
                    format="%(asctime)s %(levelname)s %(name)s: %(message)s")
logger = logging.getLogger(__name__)

logger.debug("a debug message")
logger.info("an info message")
logger.warning("a warning")
logger.error("an error", exc_info=True)
logger.critical("critical")

The logging module is the conventional Python logging facility. For more elaborate setups (structured logs, JSON output, async handlers), libraries like structlog, loguru provide alternatives.

unittest and pytest

The standard unittest library:

import unittest

class TestMyFunction(unittest.TestCase):
    def test_basic(self):
        self.assertEqual(square(5), 25)

    def test_negative(self):
        with self.assertRaises(ValueError):
            square(-1)

if __name__ == "__main__":
    unittest.main()

The conventional contemporary choice is pytest:

import pytest

def test_basic():
    assert square(5) == 25

def test_negative():
    with pytest.raises(ValueError):
        square(-1)

@pytest.fixture
def sample_data():
    return [1, 2, 3, 4, 5]

def test_with_fixture(sample_data):
    assert sum(sample_data) == 15

pytest admits the conventional def test_* functions, parameterised tests, fixtures, and a substantial plugin ecosystem (pytest-cov for coverage, pytest-asyncio for async tests, pytest-xdist for parallel runs).

argparse

For command-line argument parsing:

import argparse

parser = argparse.ArgumentParser(description="Process some integers.")
parser.add_argument("integers", metavar="N", type=int, nargs="+",
                    help="an integer for the accumulator")
parser.add_argument("--sum", dest="accumulate", action="store_const",
                    const=sum, default=max,
                    help="sum the integers (default: find the max)")

args = parser.parse_args()
print(args.accumulate(args.integers))

For more elaborate CLIs, click and typer (built on click) are the conventional third-party choices:

import click

@click.command()
@click.option("--name", default="world", help="Who to greet.")
def greet(name):
    click.echo(f"Hello, {name}!")

if __name__ == "__main__":
    greet()

typing

The type-hint module:

from typing import (
    Optional, Union, List, Dict, Tuple, Set, Callable,
    Any, TypeVar, Generic, Protocol, ClassVar, Final, Literal,
    TypeAlias, ParamSpec, Concatenate, TypeGuard,
    cast, overload, runtime_checkable
)

Treated in Type hints and Duck typing and protocols.

dataclasses

from dataclasses import dataclass, field, asdict, replace

@dataclass(frozen=True)
class Point:
    x: float
    y: float

p = Point(3, 4)
asdict(p)                                    # {"x": 3, "y": 4}
p2 = replace(p, x=5)                          # new Point(x=5, y=4)

Treated in Classes and inheritance.

random and secrets

import random

random.random()                              # uniform [0.0, 1.0)
random.randint(1, 100)                       # inclusive [1, 100]
random.choice([1, 2, 3])                     # one element
random.sample([1, 2, 3, 4, 5], 3)            # k unique elements
random.shuffle(items)                        # in place

# For cryptographic randomness:
import secrets

secrets.token_hex(16)                        # 32-character hex string
secrets.token_urlsafe(16)
secrets.randbelow(1000)

The random module is for non-cryptographic randomness; the secrets module is for cryptographic uses (tokens, salts, passwords).

hashlib

Cryptographic hashes:

import hashlib

h = hashlib.sha256(b"hello world")
h.hexdigest()                                # the SHA-256 hash

# Or:
hashlib.sha256(b"hello").hexdigest()
hashlib.md5(b"data").hexdigest()             # not cryptographically secure
hashlib.blake2b(b"data").hexdigest()

For HMAC: hmac.new(key, message, digestmod="sha256").hexdigest().

For password hashing, use argon2-cffi, bcrypt, or passlib rather than the bare hashlib.

math, statistics, decimal, fractions

Numeric utilities:

import math

math.pi
math.e
math.inf
math.nan
math.sqrt(16)                                # 4.0
math.log(100, 10)                            # 2.0
math.sin(math.pi / 2)                        # 1.0
math.factorial(5)                            # 120
math.gcd(12, 8)                              # 4
math.isclose(0.1 + 0.2, 0.3)                # True
math.prod([1, 2, 3, 4])                      # 24

import statistics

statistics.mean([1, 2, 3, 4, 5])             # 3
statistics.median([1, 2, 3, 4, 5])
statistics.stdev([1, 2, 3, 4, 5])

import decimal

decimal.Decimal("0.1") + decimal.Decimal("0.2")  # Decimal("0.3"); exact

import fractions

fractions.Fraction(1, 3) + fractions.Fraction(1, 6)   # Fraction(1, 2)

For substantial numeric work, NumPy is the conventional choice; the standard-library numeric modules cover the basic cases.

A note on the broader ecosystem

Beyond the standard library, modern Python projects rely on a small set of well-known third-party libraries:

• HTTP: requests, httpx, aiohttp.
• Data validation: pydantic, attrs, marshmallow.
• Web frameworks: Flask, FastAPI, Django, Litestar.
• Data science: numpy, pandas, scipy, scikit-learn, polars.
• Plotting: matplotlib, seaborn, plotly.
• Async: asyncio (stdlib), anyio, trio.
• CLI: click, typer.
• Testing: pytest, hypothesis.
• ORMs: SQLAlchemy, Django ORM, peewee, tortoise-orm.
• Logging: structlog, loguru.
• Type checkers: mypy, pyright, basedpyright.
• Formatters/linters: ruff, black.

The conventional contemporary advice: use the standard library where it is sufficient (it usually is for application logic) and reach for third-party libraries for the cases the standard library does not cover gracefully (modern HTTP, async I/O, data validation, plotting, ORMs).

A note on the discipline

The contemporary Python standard-library advice:

• Use pathlib for paths; os.path is legacy.
• Use subprocess.run for external commands; older alternatives are deprecated.
• Use datetime with timezone.utc for new datetime code; naive datetimes are problematic.
• Use dataclasses for value-class-like records.
• Use enum.Enum for closed sets of constants.
• Use logging with a per-module logger; avoid print for application logging.
• Use pytest for testing in modern projects.
• Use type hints (typing module) and check them with mypy or pyright.

The standard library is large; the conventional advice is to know what’s there and reach for it before adding a third-party dependency.