Notes on solutions architecture, LLM agent systems, and offensive security. Written in the open, mostly while the details are still fresh.
Swapping one DSpark drafter for another on the same target — same algorithm, same block size, every server flag held constant — took decode from 158.9 to 176.8 tok/s and grew the KV pool by 61%. The 4-bit drafter accepted more tokens than the 16-bit one, not fewer.
NVIDIA's quick-start recipe for Nemotron 3.5 Lightning on a DGX Spark enables speculative decoding. On this box it costs half the throughput — 41 tok/s against 77.5 — because turning it on silently drops the engine off a fully-captured CUDA graph. The draft acceptance rate looked fine throughout.
A DGX Spark ran Qwen3.8-27B at full precision and 4.6 tok/s. An RTX 5090 couldn't hold it above 4-bit and ran it at 54.6 tok/s — then at 159.1 once I found the three things my software stack was costing me. Includes a correction to my own analysis.
A 2-bit quantised 35B MoE runs at 126 tok/s on a 16 GB consumer GPU, within 1.5% of the published figure. The tuning recipe shipped alongside those numbers was worth 3% instead of 20%, and two of its knobs couldn't be set at all.
My first time serving a model on a rented per-second GPU instead of my own card. vLLM wouldn't load the GGUF file, the CUDA build had to link against stub libraries because image builds get no GPU attached, and the context flag I thought I understood was dividing my window across concurrent request slots.
Turning on multi-token prediction in llama.cpp took a 27B model from 58 to 136 tokens per second. The same upgrade that enabled it also broke my existing config by using more VRAM — the win and the regression shipped together.
Two retrieval pipelines over the same corpus, answered by the same model with the same prompt, overlapped on 1.8 of 12 retrieved documents — and failed in opposite, predictable ways.
Fifteen years of notes that lived in Slack DMs, call recaps, and scratch files — collected somewhere they can actually be found.