**Quick Troubleshooting Checklist – llama.cpp (single‑GPU)**  

| # | Check | What to look for | Quick fix / command |
|---|-------|------------------|---------------------|
| 1 | **Model quantization** | 8‑bit/4‑bit models use GPU memory more efficiently. 16‑bit or float32 can bottleneck CPU‑to‑GPU copy. | Re‑quantize to 4‑bit (`./quantize model.bin model.bin.q4_0 4`) or use `-m model.bin.q4_0`. |
| 2 | **GPU memory allocation** | If `-ngl` (GPU layers) is too low, many layers stay on CPU → slow prompt eval. | Increase `-ngl` to max GPU layers (`-ngl 32` or `-ngl 64`). |
| 3 | **Batch size** | Prompt eval is often run with `-n_batch 1`. Larger batches reduce per‑token overhead. | Use `-n_batch 8` or higher (but keep within GPU memory). |
| 4 | **CPU thread count** | Too many CPU threads can oversubscribe the CPU, causing context‑switch delays. | Set `-t 4` (or `-t 8` on 8‑core CPUs). |
| 5 | **Context length** | Very long prompts force many CPU‑side tokenizations. | Trim prompt or use `-n_ctx 2048` (or lower). |
| 6 | **CUDA / GPU compute** | Outdated drivers or CUDA toolkit can throttle GPU. | Update NVIDIA driver & CUDA to latest supported by llama.cpp. |
| 7 | **Pinned memory** | Unpinned host memory → slower DMA transfers. | Compile with `-DGGML_USE_CUDA=1` and `-DGGML_USE_CUDA_PINNED=1`. |
| 8 | **GPU memory fragmentation** | Repeated allocations can fragment GPU memory, slowing down. | Restart the server or use `-ngl 0` to free all GPU memory before loading. |
| 9 | **CPU‑to‑GPU copy overhead** | Prompt eval often copies the entire prompt to GPU. | Enable `-ngl 0` for prompt eval only, then switch to GPU layers for generation. |
|10 | **Profiling** | Identify the exact stage that’s slow. | Run `./main -m model.bin -t 4 -ngl 32 -n_batch 8 -n_predict 1 -l` and inspect the log. |
|11 | **Model file size** | Large models (>8 GB) may not fit in GPU memory. | Use a smaller model or split into multiple GPUs. |
|12 | **Temperature / top_k** | High sampling parameters can increase CPU work. | Reduce `-temp 0.0` or `-top_k 40` for faster eval. |

**Quick sanity test**

```bash
./main -m model.bin.q4_0 -t 4 -ngl 32 -n_batch 8 -n_predict 1 -l
```

If prompt eval still lags, try:

```bash
./main -m model.bin.q4_0 -t 4 -ngl 0 -n_batch 8 -n_predict 1 -l
```

and compare timings. The difference should highlight whether GPU layers or CPU bottlenecks are the culprit.