Motivation
The main reason I care so much about the proper typing and general understandability and maintainability of the code is that for me the “real” client is as much a client of your work as are the people who are going to maintain it. You future self included. The mere fact that the code works is not enough to consider the ticket done. Creating the tests for the code is obvious requirement as well. Spending some time to make the code more readable — not so obvious. The most common problems — e.g. illogical variable or function names — are more easily caught during code review than too strict typing — the problem this post is about.
If you do not feel that readability is so crucial, let me remind you that it contributes to the speed of development and bug hunting with an appeal to authority:
Before you can change the code to add a new feature, to fix a bug, or for whatever reason caused you to fire up your editor, you have to understand what the existing code is doing. You don’t have to know the whole program, of course, but you need to load all of the relevant pieces of it into your primate brain. We tend to gloss over this step, but it’s often the most time-consuming part of programming. If you think paging some data from disk into RAM is slow, try paging it into a simian cerebrum over a pair of optical nerves. (from Robert Nystrom’s “Game Programming Patterns”)
The guideline
The general guideline I have seen and come to not only accept but believe is objectively right is:
type the parameters as loosely as possible and return values as strictly as possible.
When you think about the parameters of a function as its requirements it starts to make sense. You always require of the user exactly what is needed but not more. You do not require the exact make and model of a hammer, you require a hammer, or even “something to drive the nail in”.
Example
Worst case
I have recently been a part of a discussion on how to type such a piece of code:
from library import UltraSpecificClass
class MyClass:
def __init__(param: X | None):
if param is not None:
self.field = param
else:
self.field = UltraSpecificClass(...)
The most obvious answer seems to be that X
should be replaced with UltraSpecificClass
if the param
is
always going to be of this particular class.
But this is only a good way to type it if MyClass
uses methods that are available only to this class and
not its parent or great-grand-parent.
Typing it like that creates a bunch of questions:
- was this typing deliberate?
- what methods
UltraSpecificClass
isMyClass
using? - which methods of
UltraSpecificClass
will I need to mock in tests?
And the tests for such a class contain the dreaded (at least by me)
with patch('library.UltraSpecificClass::method', mock):
which doesn’t restrict the scope of context that I need
to think about.
Better
It might turn out that there is an ancestor class that has all the methods that MyClass
is using and X
can be replaced with it.
As this is a more standard practice I can assume that this typing was deliberate, but depending on the number of
methods in the ancestor class it may or may not make MyClass
’s requirements more clear.
It makes testing a bit more explicit if we subclass the ancestor class. But if we need only a few methods and the ancestor class has 20 abstract methods we stand before a dilemma of implementing them oneliners to please the linter and typechecker (which will “muddy the waters” and make the code far less succinct) and adding ignore annotations for the linter and typechecker.
And there is still a matter of methods that are already implemented in the ancestor class. They may or may
not be used by MyClass
or some behaviour, may happen in the constructor.
Something to drive the nail in
If MyClass
only depends on a few methods of the UltraSpecificClass
we can make it explicit with a Protocol
.
from typing import Protocol, Self
class DoesWhatINeed(Protocol):
def method1(self: Self, param: Type1) -> ReturnType:
...
def method2(self: Self) -> ReturnType2:
...
def method3(self: Self, param1: Type2, param2: Type3) -> ReturnType:
...
This approach, while a bit verbose, makes it immediately clear what MyClass
needs. There is no guessing if this
or that method of a class is used and should be mocked in test.
To test the class you do need any patching or subclassing, you can just create a tiny self-contained class implementing all the required methods.