coldpath: The Ollama Windows-on-Arm Build Ships with the Matrix Unit Off. One-Line Fix Filed Upstream, 5.75x on Prompt Processing.

The question I could not answer

I wanted to know something basic about my own setup: when I run a quantized LLM on an Arm machine, is it actually using the chip's matrix-multiply instructions, or silently falling back to scalar code? I could not find a way to answer it. Arm's matrix kernels (i8mm, SME2, dot-product) are selected at compile time, so a wrong build flag does not make them slow, it removes them from the binary entirely, and nothing warns you. The only advice on offer was to attach a profiler to real Arm silicon and read assembly symbol names by hand. That gap is the whole project: a question a competent engineer cannot answer today, made answerable, with no Arm hardware required.

Why a static scan is sufficient (and works on Arm but not x86)

ggml, XNNPACK and most Arm kernel libraries bake their fast paths into each binary at compile time. After llama.cpp PR #10457 (which removed runtime ISA detection to fix a ~15x regression), the i8mm/dotprod intrinsics are `#if`-guarded on `__ARM_FEATURE_MATMUL_INT8` and `__ARM_FEATURE_DOTPROD`. So if the build used the wrong `-march`, the fast path is not merely skipped: it is compiled out, physically absent from the binary. Disassembling `.text` and looking is therefore decisive. Two properties make this sound. First, AArch64 is fixed-width, 4-byte instructions on a 4-byte grid, so a data word embedded in `.text` (a literal pool, a jump table) is local to its own 4 bytes and can never cascade into the surrounding code. An x86 linear sweep would desynchronise and corrupt everything downstream; on Arm it cannot. Second, detection reads register operands (ZA tiles for SME, Z/P for SVE) because Capstone leaves `insn.groups` empty for both SVE and SME, so group-based detection silently reports zero, the exact false-negative class this tool exists to catch.

What the cold path costs, in dollars, measured live

The honest part. Every row is built `GGML_NATIVE=OFF` with a pinned `-march` to isolate the ISA effect; only `-march` changes. Cold armv8-a (the baseline a portable build falls back to, Ollama's Windows build) runs Qwen2.5-0.5B Q4_0 prefill at ~95 tok/s. The one-line fix I filed (armv8.2-a+dotprod, safe on every shipped Arm device) runs at ~545, a 5.75x speedup, about $0.08 per million prompt tokens against a $0.45 baseline. armv8.6-a+i8mm+bf16 runs at ~660, 6.9x, about $0.065. Generation improves about 2.3x. The dollars use a sample Arm-cloud on-demand rate (Graviton4 c8g, $0.0385/vCPU-hr) so only that column assumes a price; the tok/s and the 5.75x are measured on the 4-vCPU Neoverse N2 runner and regenerate from CI on every push.

The upstream fix: one line, filed as Ollama PR #17654

The cause is a single missing CMake variable in Ollama's `llama/server/CMakePresets.json`. `GGML_CPU_ALL_VARIANTS` is off (so the build cannot dlopen the best of several ISA variants at runtime), and `GGML_CPU_ARM_ARCH` is unset, so the build falls back to baseline armv8-a and every matrix intrinsic gets `#if`-ed out. Adding `"GGML_CPU_ARM_ARCH": "armv8.2-a+dotprod"` recovers dot-product on every shipped Arm device, including the Cortex-A76-class chips in Windows-on-Arm laptops that do not have i8mm. That is why the PR intentionally ships dot-product only, not the 6.9x i8mm row: dot-product is safe on every core, i8mm is not.

As a CI gate: a cold build never reaches your fleet

Point the reusable Action at your own build output, where every binary should be hot. `coldpath --require i8mm ./build/` exits non-zero if any shipped binary lacks the matrix instructions, so the pull request fails before the cold build reaches an Arm fleet. When you instead scan a whole third-party release archive, the matmul kernels usually live in one backend library (e.g. `ggml-cpu.dll`) while the `llama-*`/`ollama` executables next to it are thin launcher shims that carry no kernels, so a strict gate would fail on the shims. For that case judge the set by its best member with `--any` (CLI) or `any: true` (Action). The same flag covers a multi-variant build that dlopens the best of several single-ISA libraries at runtime, so the set is scored on the variant that actually loads, not its armv8.0 fallback.

Ground truth, and honest limitations

The tool is validated against ground truth it did not author. llama.cpp's official linux-arm64 release ships eight `ggml-cpu` backends with the ISA level in the filename, and `scripts/verify_ladder.py` reproduces that eight-row staircase exactly (armv8.0 has zero of everything, armv8.2 gets 1,184 dotprod, armv8.2_3 adds 10,735 SVE, armv8.6 adds 402 i8mm, and so on). ONNX Runtime's aarch64 wheel is a positive control proving a zero means a real absence, with 469 SME, 800 i8mm, and 1,642 dotprod. Scope is stated up front: on Apple Silicon, matmul can route through the Accelerate/AMX unit, which a disassembler cannot see, so the headline is scoped to Linux/Neoverse, Windows-on-Arm, and Android where the ISA path is the only path. coldpath reports what is present in `.text`; presence equals what executes for ggml (compile-time dispatch) and is a proven-shipped indicator for runtime-dispatch libraries.