Blog
Working papers from the engine room. Every number links to a raw log in the memra repo. Subscribe via RSS.
-
Serving is a determinism problem
The batch that changed the model's mind
Under batched prefill, co-arrivals changed a MoE request's expert selection in 16% of (layer,token) pairs. How the isolation gate caught it, and the fix.
-
The compiler ate my byte
How a bit-identity gate caught an nvcc miscompile
nvcc 13.0.88 at -O3 for sm_120a zeroed one byte per quantized block. A bit-parity gate caught it on the first run. The repro, the fix, the receipts.
-
Dogfooding found three bugs in my server
Zero is a value, not unset
Dogfooding exposed three sampler bugs of one class that my entire gate battery could not see. Fixes, receipts, and the gates that now catch them.
-
Spec-decode economics on consumer-priced silicon
170 tok/s single-stream from a workstation card, token-identical to plain decode
MTP speculative decoding takes a 27B to 170.5 tok/s single-stream on an RTX PRO 6000, token-identical to plain decode — and what that does to $/token.
-
Turn 20 answers like turn 2
Session affinity for clients that rewrite their history
Agent clients rewrite history between turns, defeating prefix caching. Session affinity keeps TTFT flat across 25 turns — plus the gap it does not close.