Benchmarks
Three questions decide whether this library does what it claims. How many copies of a key are left in process memory when you are done with it? Does comparing secrets leak their contents through timing? And what does the safety cost? Each has a harness in the repository, and each runs in CI.
Measured on an Apple M1 Pro, macOS 26.6.2, OCaml 5.4.1, mirage-crypto 2.4.0. Single runs, rounded. Timings on a laptop move by 10–20% between runs; the leak counts do not move at all.
Leak census
test/leak_scan.exe generates an AES-256 key in a
Secret.t, drives real AES-GCM traffic through
mirage-crypto, then scans every writable mapping of its own process for
the key: the minor heap, the C stack, and everything else. It searches
for the 24-byte tail of the key both verbatim and byte-swapped per
32-bit word, which is how the generic AES key schedule stores it.
Copies inside Secret.t payloads are excluded from the
counts, since those are the ones the library will erase.
| Case | Raw key | Expanded key |
|---|---|---|
Key held only in Secret.t, never handed out | 0 | 0 |
| Zero-copy view; schedule died young, minor heap scrubbed | 0 | 0 |
| Zero-copy view; schedule reached the major heap | 0 | 1 |
unsafe_to_string baseline; schedule reached the major heap | 1 | 1 |
The view removes the raw-key copy in every case: mirage-crypto's
of_secret holds on to its argument string, and with a view
that argument is the secret memory. What the view cannot remove
is the key schedule mirage-crypto expands into the OCaml heap. Whether
that schedule can be erased depends only on where the GC put it.
| After | View, schedule died young | View, schedule promoted | Heap-string baseline |
|---|---|---|---|
| Key live, traffic encrypted | 1 | 1 | 1 |
Key dropped, Gc.full_major | 1 | 1 | 1 |
Secret.Gc.scrub_minor_heap | 0 | 1 | 1 |
Secret.destroy | 0 | 1 | 1 |
Gc.full_major does nothing here, which is the point: the
collector reclaims memory, it does not erase it. Scrubbing the minor
heap erases a schedule that died young. A promoted schedule survives
everything — Gc.full_major,
scrub_minor_heap and Secret.destroy alike
— and stays readable until the receiving library stops building
key schedules in the heap. That is not a gap this library can close from
the outside, and the census exists so the claim is not made.
dune exec test/leak_scan.exe
Constant-time equality
bench/ct_equal.exe is a dudect-style leakage test
(Reparaz, Balasch and Verbauwhede, 2017): two classes of input,
interleaved at random, timed with a cycle counter, and compared with
Welch's t-test. One class differs from the reference secret in the
first byte, the other in the last, so a comparison that returns early
will separate the two distributions. The class contents are written into
the same working secret before each sample, so both classes see the same
addresses and the same cache state. The top 1% of samples is dropped, as
dudect does, to shed interrupts and context switches.
| Case | Equal, ticks | Differing, ticks | |t| |
|---|---|---|---|
| Differs in first byte, C primitive | 154.6 | 154.6 | 0.06 |
| Differs in last byte, C primitive | 148.1 | 148.0 | 1.73 |
Differs in first byte, Secret.equal | 148.1 | 148.2 | 0.94 |
Differs in last byte, Secret.equal | 146.8 | 146.8 | 2.51 |
String.equal, control | — | — | 1999 |
The control row is the reason to trust the other four: the same harness,
pointed at String.equal, separates the classes by a t
statistic three orders of magnitude over the threshold. A test that
cannot fail proves nothing.
Absence of evidence is not a proof of constant time. Run it on the hardware you care about, unloaded. Pull-request CI only checks that it runs; the full harness runs on the scheduled and manual workflows and remains informational because hosted runners are noisy.
dune exec bench/ct_equal.exe -- 20000 200
Allocation and access costs
| Operation | ns/op |
|---|---|
String.equal, 32 bytes, not constant time | 4 |
Bytes.create 32, for reference | 6 |
Secret.Unsafe.string_view | 7 |
Secret.equal, 32 bytes | 20 |
Secret.expose 32 bytes, scratch copy and wipe | 49 |
Secret.create 240 + destroy, default tier | 72 |
Secret.create 32 + destroy, default tier | 74 |
Secret.of_string 32 + destroy | 82 |
Secret.create 32, left to the collector | 83 |
Secret.create ~hardened 32 + destroy, pooled | 1,600 |
Secret.equal, 4,096 bytes | 1,619 |
Secret.create ~hardened 32, first use: mmap and mlock | 5,031 |
Secret.Gc.scrub_minor_heap | 17,021 |
Read it as an order-of-magnitude ladder rather than a leaderboard. A
secret in the default tier costs about a dozen times a
Bytes.create and is still well under 100 ns, which is
nothing next to any cryptographic operation you would use it for. A
zero-copy view is essentially free, because nothing is copied.
The hardened tier costs about twenty times the default tier, and its
first allocation in a size class about seventy times: that one pays for
an mmap, two guard pages and an mlock. The
pool is what separates those two rows, and it is the reason hardening is
opt in per value rather than a global switch. Harden the root key; do
not harden every nonce.
scrub_minor_heap costs a minor collection plus a
memset of the minor heap — 256k words by default, so
tens of microseconds. Call it after a handshake, not inside a loop.
dune exec bench/alloc_bench.exe
This executable measures throughput; it does not enforce allocation
budgets. bench/alloc_check.exe separately asserts that the
equality, fill, zero, and blit hot paths do not allocate in the OCaml
heap after warm-up.
dune exec bench/alloc_check.exe
What to take from this
- Pass a view, not a string, and the raw key stops appearing in process memory at all.
- Destroy secrets explicitly. Collection erases them too, but on the collector's schedule rather than yours.
- Scrub the minor heap after a key-handling phase ends; it is the only thing here that erases another library's short-lived intermediates.
- Harden long-lived keys and leave everything else on the default tier.
- Check
Secret.statusrather than assuming a hardening feature was granted, or userequire_hardeningto make a missing one fatal.