October 26, 2026 · actual source checks, host compilation and local inference measurements.
| Check | Actual result | Scope |
|---|---|---|
| Source and device-ID inspection | RTX 3090 / GA102, 10de:2204 | Two devices; both bound to the NVIDIA driver |
| Upstream CPU/mock suite | 5 top-level tests passed | Includes 6 C component/mock programs; no physical GPU unlock or downloaded firmware |
| Compile against installed kernel headers | Failed, exit 2 | Linux 5.4.0-208: missing pci_dev_lock/pci_dev_unlock; later aperture header also absent |
| Existing local 4B inference | 9 measured requests + 3 excluded warm-ups | Three prompt sizes, real llama.cpp SSE, fixed 96-token output; shared GPU |
| Unlock / boot / driver intervention | Not performed | No resets, service changes, power/clock changes or installation |
Model: Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf, Q4_K_M; concurrency 1, temperature 0, seed 20261026, prompt caching off. Medians of three repetitions per row, excluding a separate warm-up. All measured replies generated 96 tokens with zero cached prompt tokens. Outputs matched across repeats within each row.
| Input | Prompt tokens | First content, ms | Prefill tokens/s | Decode tokens/s | Total client time, ms |
|---|---|---|---|---|---|
| short_prompt | 42 | 135.3 | 323.8 | 38.3 | 2645.6 |
| production_system | 1293 | 619.9 | 2126.6 | 38.3 | 3109.7 |
| long_prefill | 2331 | 859.9 | 2727.1 | 35.1 | 3598.3 |
Shared-GPU qualification: independent training continued on both GPUs. During measured requests GPU 0 utilization ranged from 98% to 100% (sample median 100%). The ASR/browser regression suite had finished before these requests started, but this was not a dedicated idle GPU. Do not compare these numbers with an idle after-unlock run.
First content includes local HTTP/server scheduling. Prefill and decode use llama.cpp's own timers. A small prompt's tokens/s includes large fixed overhead, so compare the same row across A/B conditions. Column medians do not necessarily add up. The 96-token cap is for throughput measurement, not an acceptable answer-length policy.
| Case | Repeat | First content, ms | Server prefill, ms | Decode tokens/s | Output tokens |
|---|---|---|---|---|---|
| short_prompt | 0 | 135.3 | 129.7 | 31.7 | 96 |
| short_prompt | 1 | 57.1 | 53.0 | 38.9 | 96 |
| short_prompt | 2 | 136.4 | 132.2 | 38.3 | 96 |
| production_system | 0 | 347.4 | 339.8 | 37.2 | 96 |
| production_system | 1 | 645.3 | 634.8 | 39.0 | 96 |
| production_system | 2 | 619.9 | 608.0 | 38.3 | 96 |
| long_prefill | 0 | 915.3 | 909.6 | 35.1 | 96 |
| long_prefill | 1 | 747.6 | 742.3 | 33.7 | 96 |
| long_prefill | 2 | 859.9 | 854.8 | 38.4 | 96 |
The upstream module changes Ampere instruction-rate selectors before the stock driver attaches and performs a PCIe function reset. It skips an already bound device. Its primary expected benefit is floating-point matrix/prompt processing, rather than an equivalent improvement to quantized serial token generation. See the pinned upstream explanation, device/driver checks and selector/reset code.
Inference for this application: benchmark Q4 prefill and decode separately. Dense-BF16 GEMM gains do not establish a Q4 assistant speedup. First obtain an uncontended baseline; after arranging maintenance, compare on the same compatible kernel, driver, exact model, clocks, thermal state and prompts with only the unlock condition changed. A kernel upgrade between conditions would confound the result.
python benchmark_local_4b.py --url http://127.0.0.1:8799 --robby-source /path/to/Humaneness-Voice/robby --llama-source /path/to/llama.cpp --upstream-source /path/to/gpu-unlock --model-file /path/to/Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf --label baseline-new-run --out baseline-new-run.json
The portable benchmark never installs gpu-unlock or starts/stops a server. It refuses to overwrite existing results. The exact executed script retains the original host defaults; the portable version adds path options and sanitizes process paths without changing requests or timing logic. Use the full A/B procedure for a future authorized maintenance window.
GPU unlock: af28cbbfd3d922a659227ecfb7369adfd924cf03
llama.cpp: 5d9e5ac30e469d44c0a5a52556de0ead03aaa5b0
GGUF SHA256: c8af2b8aea10a690a1d9491db744a9446660173cb38fc489612b556615a69612
Driver: 550.54.15 · Linux: 5.4.0-208-generic
Inspection receipt · Compile failure · CPU/mock tests · Raw baseline with GPU samples.
No private microphone audio, speaker embeddings, API credentials or unrelated user logs are included. This study directory used only a few megabytes and downloaded no model weights or firmware. The actual unlock experiment remains outstanding; no numerical hardware speedup is claimed.