Python § protocols

Duck typing and protocols

Python’s typing story has two complementary parts. Type hints (treated in Type hints) admit nominal typing — types are identified by name, and a value belongs to a type if its class is the type or a subclass. Duck typing admits structural typing — a value belongs to a type if it has the right operations, regardless of class. The two coexist, and modern Python uses both: nominal typing where the relationship is genuine subtyping (“a Dog is an Animal”), structural typing where the relationship is incidental (“anything that has a read() method is file-like”). The typing.Protocol class (PEP 544) bridges the two, admitting structural-typing constraints that the type checker can verify.

This page covers the duck-typing principle, the principal protocols (__iter__, __enter__, __len__, etc.), the typing.Protocol mechanism, and the conventional discipline.

The duck-typing principle

The phrase originates from the saying: “if it walks like a duck and quacks like a duck, it’s a duck”. Applied to programming: if an object has the operations the code needs, it works with that code regardless of its class.

def total_length(things):
    return sum(len(t) for t in things)

# Works for any iterable of objects with __len__:
total_length(["hello", [1, 2, 3], {"a", "b"}, b"\x00\x01"])
# 5 + 3 + 2 + 2 = 12

The function does not require its argument to inherit from a specific class. The runtime simply attempts the operations; if they succeed, the function works.

The conventional Python style is duck-typed throughout. Code reads like “this needs to be iterable” and accepts any iterable, rather than “this needs to be a Sequence subclass” and rejects anything else.

The trade-off:

Pros: Flexibility; works with any conforming type, including types from other libraries that the author cannot modify.
Cons: Errors are caught at runtime, not compile time; the function’s “interface” is implicit (in the operations it performs).

Protocols and dunders

Python’s special methods (dunders) are the foundation of duck typing. Each protocol consists of one or more dunder methods that admit the corresponding language operations:

Operation	Protocol method(s)
`len(x)`	`__len__`
`iter(x)`	`__iter__`
`next(x)`	`__next__`
`x[i]`	`__getitem__`, `__setitem__`, `__delitem__`
`i in x`	`__contains__`
`x()`	`__call__`
`with x:`	`__enter__`, `__exit__`
`async with x:`	`__aenter__`, `__aexit__`
`async for ... in x:`	`__aiter__`, `__anext__`
`bool(x)`	`__bool__`
`str(x)`	`__str__`
`repr(x)`	`__repr__`
`format(x)`	`__format__`
`hash(x)`	`__hash__`
`x == y`	`__eq__`
`x < y`	`__lt__` (and `__le__`, `__gt__`, `__ge__`)
`x + y`	`__add__`, `__radd__`
`with x:`	`__enter__`, `__exit__`
`attribute access`	`__getattr__`, `__setattr__`, `__delattr__`, `__getattribute__`

The mechanism is the foundation on which Python’s polymorphism rests. Implementing the right dunders admits a class to participate in the language’s standard idioms.

Iterators and iterables

The most pervasive protocol — iteration:

class Counter:
    def __init__(self, limit: int):
        self.n = 0
        self.limit = limit

    def __iter__(self):
        return self

    def __next__(self):
        if self.n >= self.limit:
            raise StopIteration
        self.n += 1
        return self.n

for i in Counter(5):
    print(i)         # 1, 2, 3, 4, 5

The protocol:

__iter__ returns an iterator (typically self for iterators, or a new iterator for iterables).
__next__ returns the next value or raises StopIteration to signal exhaustion.

The distinction:

Iterable — an object with __iter__. Can be passed to for. May be iterated multiple times.
Iterator — an object with __iter__ and __next__. The state of an in-progress iteration. Can be iterated only once (after exhaustion, it stays exhausted).

Most collections (list, tuple, dict, set) are iterables — calling iter() produces a fresh iterator each time. Generators are iterators — they may be iterated only once.

The full treatment is in Iterators and generators.

Context managers

The with statement uses the context-manager protocol:

class FileHolder:
    def __init__(self, path: str, mode: str):
        self.path = path
        self.mode = mode

    def __enter__(self):
        self.f = open(self.path, self.mode)
        return self.f

    def __exit__(self, exc_type, exc_value, traceback):
        self.f.close()
        return False     # don't suppress exceptions

with FileHolder("input.txt", "r") as f:
    contents = f.read()

The protocol:

__enter__ is called on with entry; returns the value bound to as.
__exit__ is called on exit (normal or exception); receives the exception info if there is one. Returns True to suppress the exception, False (the conventional default) to propagate.

The contextlib module provides utilities:

from contextlib import contextmanager

@contextmanager
def file_holder(path: str, mode: str):
    f = open(path, mode)
    try:
        yield f
    finally:
        f.close()

with file_holder("input.txt", "r") as f:
    contents = f.read()

The decorator-based form admits writing context managers as generator functions; the yield is the entry point.

For most cases the standard open() is the conventional file-context manager; user-defined context managers are appropriate for resources requiring custom acquisition/release (locks, database transactions, temporary files).

Sequence and mapping protocols

A sequence implements:

class MySequence:
    def __len__(self) -> int: ...
    def __getitem__(self, i: int): ...
    def __setitem__(self, i: int, value) -> None: ...        # for mutable
    def __delitem__(self, i: int) -> None: ...                 # for mutable
    def __contains__(self, item) -> bool: ...                   # optional; falls back to iteration

The minimum: __len__ and __getitem__. A class with these admits len(x), x[i], for item in x:, and if y in x: (with a default linear-scan implementation).

A mapping implements:

class MyMapping:
    def __getitem__(self, key): ...
    def __setitem__(self, key, value) -> None: ...
    def __delitem__(self, key) -> None: ...
    def __iter__(self): ...
    def __len__(self) -> int: ...
    def __contains__(self, key) -> bool: ...
    def keys(self): ...
    def values(self): ...
    def items(self): ...

For most use cases, inheriting from collections.abc.Sequence or collections.abc.MutableMapping provides default implementations for many of these:

from collections.abc import MutableSequence

class MyList(MutableSequence):
    def __init__(self):
        self._items = []

    def __getitem__(self, i):
        return self._items[i]

    def __setitem__(self, i, value):
        self._items[i] = value

    def __delitem__(self, i):
        del self._items[i]

    def __len__(self):
        return len(self._items)

    def insert(self, i, value):
        self._items.insert(i, value)

The MutableSequence base provides defaults for append, extend, pop, remove, clear, count, index, __contains__, __iter__, __reversed__ based on the five abstract methods.

Hashable and equality

For use as a dict key or set member, a class must be hashable:

class Point:
    def __init__(self, x: float, y: float):
        self.x = x
        self.y = y

    def __eq__(self, other):
        return (isinstance(other, Point) and
                self.x == other.x and self.y == other.y)

    def __hash__(self):
        return hash((self.x, self.y))

The contract: equal objects must have equal hashes. Defining __eq__ without __hash__ makes the class unhashable (Python sets __hash__ to None).

For value-equality types, defining both is the conventional discipline. For dataclasses, the auto-generated __eq__ and __hash__ (with frozen=True) handle this:

from dataclasses import dataclass

@dataclass(frozen=True)
class Point:
    x: float
    y: float
# Point gets __eq__ and __hash__ automatically

p = Point(3.0, 4.0)
hash(p)                 # works
points = {Point(0, 0): "origin"}    # works as dict key

`call`

A class with __call__ admits being called like a function:

class Counter:
    def __init__(self):
        self.count = 0

    def __call__(self):
        self.count += 1
        return self.count

c = Counter()
c()                # 1
c()                # 2
c()                # 3

The pattern admits stateful “function objects”. Many decorators are implemented this way; treated in Decorators.

Truthiness

The bool() function calls __bool__ (if defined) or falls back to __len__ (zero is False):

class Bag:
    def __init__(self):
        self.items = []

    def __bool__(self):
        return len(self.items) > 0

bag = Bag()
if bag:
    print("non-empty")
else:
    print("empty")

The default truthiness for objects is True. Defining __bool__ admits custom truthiness rules.

`typing.Protocol`

The typing.Protocol class admits structural typing with type-checker support:

from typing import Protocol

class HasArea(Protocol):
    def area(self) -> float: ...

class Circle:
    def __init__(self, r: float):
        self.r = r

    def area(self) -> float:
        return 3.14159 * self.r * self.r

class Square:
    def __init__(self, side: float):
        self.side = side

    def area(self) -> float:
        return self.side * self.side

def total_area(shapes: list[HasArea]) -> float:
    return sum(s.area() for s in shapes)

shapes = [Circle(1.0), Square(2.0)]
print(total_area(shapes))     # 7.14159

Circle and Square do not inherit from HasArea; they implicitly conform because they have an area() method with the right signature. The type checker accepts the call.

The mechanism is the conventional Python idiom for “the parameter must have these methods” without forcing nominal subtyping.

`runtime_checkable`

By default, Protocol is for type checking only. The @runtime_checkable decorator admits isinstance checks at runtime:

from typing import Protocol, runtime_checkable

@runtime_checkable
class HasArea(Protocol):
    def area(self) -> float: ...

isinstance(Circle(1.0), HasArea)     # True
isinstance("hello", HasArea)           # False

The runtime check is structural — it tests for the presence of the methods, not their signatures. The mechanism admits dispatching on protocol conformance at runtime.

When to use `Protocol`

The conventional choice:

Use Protocol when the relationship is structural — any type with the right methods works.
Use ABC (abc.ABC and abc.abstractmethod) when the relationship is nominal — types must explicitly opt in.

# Structural (Protocol): any type with read() works
class Readable(Protocol):
    def read(self) -> bytes: ...

# Nominal (ABC): types must explicitly inherit
from abc import ABC, abstractmethod

class Reader(ABC):
    @abstractmethod
    def read(self) -> bytes: ...

class FileReader(Reader):
    def read(self) -> bytes:
        return self.file.read()

The Protocol form admits more types (any matching shape); the ABC form is more strict (only declared subclasses). The choice depends on whether structural conformance is desirable.

Abstract base classes

The abc module admits abstract base classes — classes that declare an interface and cannot be instantiated:

from abc import ABC, abstractmethod

class Shape(ABC):
    @abstractmethod
    def area(self) -> float:
        ...

    @abstractmethod
    def perimeter(self) -> float:
        ...

    def describe(self) -> str:
        # concrete method using the abstract ones
        return f"area={self.area()}, perimeter={self.perimeter()}"

class Circle(Shape):
    def __init__(self, r: float):
        self.r = r

    def area(self) -> float:
        return 3.14159 * self.r * self.r

    def perimeter(self) -> float:
        return 2 * 3.14159 * self.r

c = Circle(5)
c.describe()         # works

# Cannot instantiate abstract:
Shape()              # TypeError: can't instantiate abstract class

The principal uses are required-interface declaration and base-class implementation. ABCs combine the contract (the abstract methods) with shared behaviour (the concrete methods).

The standard library’s collections.abc provides ABCs for the conventional protocols: Iterable, Iterator, Sequence, MutableSequence, Mapping, MutableMapping, Set, MutableSet, Hashable, Sized, Container, Callable.

EAFP vs LBYL

Python’s idiom for runtime type-checking has two principal styles:

EAFP (“Easier to Ask Forgiveness than Permission”) — try the operation; catch the failure:

try:
    value = mapping[key]
except KeyError:
    value = default

LBYL (“Look Before You Leap”) — check first, then operate:

if key in mapping:
    value = mapping[key]
else:
    value = default

The conventional Python choice is EAFP. The reasons:

Race conditions — between the check and the use, another thread may modify the state.
Performance — for the common case (the operation succeeds), the EAFP form has no overhead.
Idiomatic — Python’s exception machinery is designed for this style.

For dict access specifically, .get(key, default) is the conventional one-liner:

value = mapping.get(key, default)

`hasattr`, `getattr`, `setattr`, `delattr`

For dynamic attribute access:

hasattr(obj, "method")          # bool: does obj have method?
getattr(obj, "method")          # the attribute value
getattr(obj, "method", default) # with default if absent
setattr(obj, "method", value)   # set
delattr(obj, "method")          # delete

The mechanism admits programmatic attribute access:

def configure(obj, **settings):
    for key, value in settings.items():
        setattr(obj, key, value)

configure(widget, color="red", size=10, visible=True)

The conventional discipline is to prefer direct attribute access (obj.attr) when the attribute name is known statically; reserve getattr/setattr for the cases where the name is dynamic.

Common patterns

Iterable-friendly function

def total_length(things):
    return sum(len(t) for t in things)
# Works for any iterable of objects with __len__:
# - list of str
# - tuple of list
# - generator of dict
# - any custom type with __iter__ and __len__

The function does not name a specific type; the duck-typing accepts any conforming structure.

Custom context manager

from contextlib import contextmanager

@contextmanager
def transaction(db):
    try:
        db.begin()
        yield db
        db.commit()
    except Exception:
        db.rollback()
        raise

with transaction(database) as db:
    db.execute("INSERT INTO ...")
    db.execute("UPDATE ...")

The @contextmanager admits writing context managers as generators; the pattern is the conventional Python idiom for resource management.

Dispatch on protocol

from typing import Protocol, runtime_checkable

@runtime_checkable
class Printable(Protocol):
    def print(self) -> None: ...

def emit(x):
    if isinstance(x, Printable):
        x.print()
    else:
        print(x)

The runtime protocol check admits “if it has a print method, use it; otherwise use the default”.

Magic-method-driven DSLs

class Query:
    def __init__(self, conditions=None):
        self.conditions = conditions or []

    def __and__(self, other):
        return Query(self.conditions + other.conditions + ["AND"])

    def __or__(self, other):
        return Query(self.conditions + other.conditions + ["OR"])

q = Query(["a > 0"]) & Query(["b < 10"]) | Query(["c = 5"])

The pattern admits DSLs that look like ordinary expressions; many ORMs (SQLAlchemy, peewee) use it heavily.

A note on the discipline

The conventional contemporary Python typing discipline:

Use type hints in new code; the static checker catches many bugs.
Use Protocol for structural typing; ABC only when subtyping is genuine.
Use the dunder methods; they are the foundation of Python’s polymorphism.
Prefer EAFP — try and catch is more idiomatic than check-then-operate.
Keep duck-typing in mind; functions that work for any conforming type are the conventional Python.
Use collections.abc for abstract base classes of standard protocols.

The combination — duck typing for flexibility, type hints for static checking, Protocol for the formal middle ground — is one of the most distinctive aspects of contemporary Python.