Visualize memory¶

Newer feature — less battle-tested than the find & fix core. It works, but feedback is welcome.

A number tells you whether memory grew; a chart tells you the shape — how cost scales with input size, which ids moved and were already big, how a metric drifts across versions. benchmem plot writes an interactive plotly view to standalone HTML, picking the view by run count. It reads the same saved runs as Compare two runs and takes one metric (--columns).

The numbers behind the charts¶

Before the shapes, the raw table — benchmem compare --diff, baseline (list of tuples) vs the heavier candidate (list of dicts). Every Δ% is red because the candidate carries more per row. A chart makes the trend obvious where the table only lists points:

test_rows.py::test_build_rows[10000]
 name                     peak base   candidate 
────────────────────────────────────────────────
 test_build_rows[10000]    83.1 KiB    +1231.9% 

test_rows.py::test_build_rows[200000]
 name                      peak base   candidate 
─────────────────────────────────────────────────
 test_build_rows[200000]    22.5 MiB     +102.0% 

test_rows.py::test_build_rows[500000]
 name                      peak base   candidate 
─────────────────────────────────────────────────
 test_build_rows[500000]      62 MiB      +93.6% 

test_rows.py::test_build_rows[50000]
 name                     peak base   candidate 
────────────────────────────────────────────────
 test_build_rows[50000]    4.42 MiB     +113.0% 

Scaling — how cost grows with input size¶

Scaling (one run) draws cost vs. input size — here the baseline's peak-memory curve:

benchmem plot baseline.json --columns peak -o scaling.html

Scatter — which ids moved, and were they already big¶

Scatter (two runs) puts baseline cost on x (log) and the candidate/baseline ratio on y, colour = absolute Δ. The top-right corner is "big and got bigger" — where a regression actually costs you:

benchmem plot baseline.json candidate.json --columns peak -o scatter.html

The other views¶

plot auto-selects by run count; override with --view:

Runs	Default view	Answers
1	scaling	how does cost grow with input size?
2	scatter	which ids moved, and were they already big?
2	compare (--view compare)	ranked — what moved most, in native units?
3+	sweep	fold-change across versions, one cell per (id, run)

--pivot re-points the series axis of the paired views (compare/scatter) from the run-file onto a dim, folding a single run along it (see Compare two runs → --pivot). --facet splits any view into small multiples by a dim (including node.*), --where keeps only rows matching a dim=value filter (repeatable, AND-combined), --free-axes unmatches a faceted axis from the shared default, and --label/-l names the series per run:

benchmem plot run.json --columns peak --facet node.func             # one panel per operation
benchmem plot run.json --facet node.func --free-axes y             # ...each on its own cost scale
benchmem plot run.json --where axis=n                              # one sweep at a time
benchmem plot v1.json v2.json v3.json -l 0.6 -l 0.7 -l 0.8         # name series, not file stems

Faceted panels share axes by default — right when they're commensurable (the same n grid across functions). Two cases want them unmatched:

• Different cost scales — --facet node.func where one function is far costlier flattens the cheap panels on a shared y. --free-axes y gives each its own cost range.
• Incommensurable sweeps — a run mixing sizes n (10–10⁴) and a 0–100 severity under one numeric dim squishes both onto one x. --free-axes x (or filter to one with --where axis=n).

--free-axes both frees each panel entirely. --where is the cleaner reach when you only care about one slice — it drops the rest rather than just rescaling.

Where to go next¶

• The numbers behind the charts, as a table → Compare two runs
• Fold across versions of a package → Compare across versions
• Every CLI flag → Reference