Benchmarking#
cattrs includes a benchmarking suite to help detect performance regressions and guide performance optimizations.
The suite is based on pytest and pytest-benchmark. Benchmarks are similar to
tests, with the exception of being stored in the bench/
directory and being
used to verify performance instead of correctness.
A Sample Workflow#
First, ensure the system you’re benchmarking on is as stable as possible. For
example, the pyperf library has a system tune
command that can tweak
CPU frequency governors. You also might want to quit as many applications as
possible and run the benchmark suite on isolated CPU cores (taskset
can be
used for this purpose on Linux).
Then, generate a baseline using make bench
. This will run the benchmark suite
and save it into a file.
Following that, implement the changes you have in mind. Run the test suite to
ensure correctness. Then, compare the performance of the new code to the saved
baseline using make bench-cmp
. If the code is still correct but faster,
congratulations!