stubtest (static vs runtime)

rather than

pyrefly coverage check (static only)

So pyrefly is a bean counter. If using mypyc, e.g. black, doesn’t have stubs, then understandable this could be a valid use case.

BUT that is an outlier. Without stubs that removes the usefulness of pyright which catches actual bugs.

static vs runtime matters

Similarly to how test coverage measures how much of a library’s source code is hit by its tests, type coverage measures what percentage of a library’s typables have type annotations.

Errrr no!

type coverage should be how much of the typing has corresponding tests. This is particularly important for stub packages. Which are focused on confirming the advertised typing accurately reflects actual typing.

Hope they didn’t cyber squat the package name.

Listen to Yoda, “Try? Either do or do not! No try there is.”

The correct answer to percentage of a libraries typables having type annotations is either zero or all. This is advertised by the py.typed file. There is no middle ground.

Wtf is wrong with pyrefly authors?

Stub package with naming starting with type- does not follow PEP561.

The convention is types-*.

So yeah, what is type-coverage?!

For anyone interested, I recently switched from using ty to Pyrefly and so far I’m very happy with the results. Mypy is more comprehensive, but painfully slow. ty delightfully fast, but lacking Django support is a deal-breaker.

Pyrefly mostly supports Django (unless you get creative with custom querysets) and for me, the trade-off is worth it for the performance.