**Quick‑check list for slow prompt‑eval in llama.cpp (single‑GPU)**  

| # | What to check | Why it matters | Quick test / command |
|---|----------------|----------------|-----------------------|
| 1 | **GPU is actually used for generation** | Prompt‑eval is CPU‑only; if GPU isn’t active you’ll see a huge gap. | `nvidia-smi` → GPU load > 0 % during generation. |
| 2 | **Compile flag `-D GPU` is present** | Without it, the binary never talks to the GPU. | `make clean && make -j$(nproc) GPU=1` |
| 3 | **Run with `--gpu` (or `-g`)** | Enables GPU for generation; otherwise it falls back to CPU. | `./main -m model.bin --gpu 1` |
| 4 | **CPU core count / frequency** | Prompt‑eval runs on CPU; thermal throttling or low‑frequency cores will slow it. | `lscpu | grep -E 'CPU\(s\)|MHz'` |
| 5 | **CPU load during prompt‑eval** | High load from other processes can starve the thread. | `htop` or `top` while the request is in flight. |
| 6 | **Number of threads (`--threads`)** | Too few → under‑utilization; too many → context‑switch overhead. | Try `--threads 4`, `--threads 8`, `--threads 16` and compare. |
| 7 | **Context size (`--ctx-size`)** | Large context forces more memory copies and can hit CPU cache limits. | Reduce to 2048 or 4096 and see if prompt‑eval speeds up. |
| 8 | **Batch size (`--batch-size`)** | Very large batches can cause memory thrashing before the GPU kicks in. | Keep it at 1–4 for single‑GPU tests. |
| 9 | **Memory‑mapping flag (`--mmap`)** | Mapping the model file can reduce load time; unmapping can increase it. | Run with and without `--mmap` and compare. |
|10 | **Model quantization** | 4‑bit or 8‑bit models use less CPU memory bandwidth; 16‑bit can be slower on CPU. | Verify you’re using the intended quantization (`-q 4`, `-q 8`). |
|11 | **CUDA/cuBLAS version** | Older drivers or mismatched CUDA can cause CPU‑side stalls. | `nvidia-smi` → driver & CUDA version; update if needed. |
|12 | **Check for thermal throttling** | CPU or GPU throttling will show in `nvidia-smi` or `top`. | Look for “Tctl” or “Tdie” near max. |
|13 | **Validate that no `--no-gpu` flag is passed** | That flag disables GPU entirely, making everything CPU‑bound. | Ensure command line has no `--no-gpu`. |
|14 | **Profile with `perf` or `nvprof`** | Pinpoints exact CPU stalls or GPU idle periods. | `perf stat -e cycles,instructions ./main …` or `nvprof ./main …` |
|15 | **Check for disk I/O bottleneck** | If the model file is on a slow SSD, prompt‑eval may wait for data. | `iostat -xz 1` while running. |

**Bottom line:**  
Prompt‑eval is CPU‑only in llama.cpp. Make sure the GPU is actually being used for generation, keep thread count and context size reasonable, and verify CPU isn’t throttling or overloaded. If all looks good, the remaining slowdown is likely due to CPU‑bound tokenization or memory‑copy overhead, which can be mitigated by using a faster CPU, reducing context size, or enabling memory‑mapping.