- [ ] **Verify full GPU offload**: Use `--ngl 999` (or `--tensor-split 1.0`). Partial offloading forces CPU↔GPU transfers during prefill, killing bandwidth.
- [ ] **Check VRAM headroom**: Ensure 0% VRAM is used by other processes. Prefill is memory-bandwidth bound; fragmentation or shared VRAM causes thrashing.
- [ ] **Trim context window**: Set `--ctx-size` to the minimum needed. Prefill time scales linearly with prompt length; large contexts saturate memory bandwidth.
- [ ] **Tune batch/parallel**: Lower `--batch` and `--parallel` (e.g., `--batch 512 --parallel 1`). High values on a single GPU cause inefficient kernel launches and KV cache overhead.
- [ ] **Use optimal quantization**: Prefer `Q4_K_M` or `Q5_K_M` for NVIDIA. Avoid `Q8_0` or `Q2_K` unless required; they increase bandwidth demand or use slower kernels.
- [ ] **Enable Flash Attention**: Add `--flash-attn` if your GPU supports it (Ampere+ NVIDIA, RDNA3+ AMD). Reduces memory traffic during prefill.
- [ ] **Verify compute backend**: Ensure `llama.cpp` was built with `GGML_CUDA` (NVIDIA) or `GGML_HIP` (AMD). Check startup logs for `CUDA`/`HIP` detection.
- [ ] **Monitor GPU utilization**: Run `nvtop` or `nvidia-smi` during prefill. <70% utilization indicates CPU bottleneck, memory bandwidth limit, or unoptimized kernels.
- [ ] **Bypass wrappers**: Test with raw `./server -m model.gguf -c 2048 --port 8080`. Many UIs/wrappers add HTTP/queue latency that masks core inference speed.
- [ ] **Update llama.cpp**: Pull latest `main`. Prefill memory allocators and CUDA kernels are frequently optimized; older builds often underperform.