## Turn 1

1.  **Understand the Goal:** The user is experiencing slow prompt evaluation (prefill) compared to generation in a llama.cpp local model server running on a single GPU. They need a concise troubleshooting checklist.

2.  **Analyze the Problem:**
    *   **Symptom:** Prompt eval (prefill) is much slower than generation.
    *   **Context:** llama.cpp, single GPU, local model server.
    *   **Why does this happen?** Prefill is memory-bound and compute-bound depending on the model size and hardware. Generation is compute-bound (matrix multiplication). If prefill is slow, it's usually due to memory bandwidth limits, suboptimal memory layout, or hardware mismatch.

3.  **Brainstorm Troubleshooting Categories:**
    *   Hardware/Driver issues
    *   llama.cpp configuration/flags
    *   Model format/quantization
    *   System/OS level issues

4.  **Flesh out specific checks within categories:**

    *   *Hardware/Drivers:*
        *   Is the GPU actually being used? (Check `nvidia-smi` or `rocminfo`).
        *   Are drivers up to date? (CUDA version mismatch is a classic issue).
        *   Is the PCIe bandwidth sufficient? (x16 vs x8, though less likely to be the *sole* cause of massive prefill slowness, still worth noting).
        *   VRAM capacity: Is it swapping to system RAM? (VRAM overflow causes massive slowdowns).

    *   *llama.cpp Configuration/Flags:*
        *   Context size (`-ngl` / `--n-gpu-layers`): Is it offloading everything?
        *   Memory layout: `--mlock` (lock memory to prevent swapping).
        *   Flash attention: `--flash-attn` (can speed up prefill significantly on newer GPUs).
        *   KV cache: `--cache-type-k` / `--cache-type-v` (fp16 vs bf16).
        *   Batch size: `--batch-size` (larger batch sizes can help prefill throughput).
        *   Num threads: `--threads` (though mostly for CPU offload, can affect memory allocation).

    *   *Model Format/Quantization:*
        *   Is the model quantized? (Q4_K_M vs Q8_0). Higher precision = more memory bandwidth needed.
        *   Is the model in GGUF format? (llama.cpp requires GGUF).
        *   Is the model too large for the VRAM? (Causes swapping).

    *   *System/OS:*
        *   Power management: Is the GPU in a low-power state? (Set to performance mode).
        *   Background processes: Is something else using the GPU?

5.  **Filter and Refine for Conciseness:** The user asked for a *concise* checklist. I need to group these logically and remove fluff.

    *   *Group 1: VRAM & Memory Swapping (Most common cause)*
        *   Check `nvidia-smi` for VRAM usage. If it's spiking or swapping, prefill will tank.
        *   Add `--mlock` to prevent system RAM swapping.
        *   Increase `--ctx-size` if the prompt is larger than the default context window.

    *   *Group 2: GPU Offload & Compute*
        *   Ensure full offload: `--n-gpu-layers -1` (or max possible).
        *   Enable Flash Attention: `--flash-attn` (huge win for prefill on Ampere+).
        *   Check CUDA version compatibility.

    *   *Group 3: KV Cache & Quantization*
        *   Optimize KV cache: `--cache-type-k bf16` and `--cache-type-v bf16` (saves bandwidth).
        *   Check quantization: Switch to a lower precision (e.g., Q4_K_M) if VRAM is tight.

    *   *Group 4: Hardware/OS*
        *   Set GPU to Performance mode (e.g., `nvidia-smi -pm 1`).
        *   Verify PCIe lane configuration (x16).

6.  **Draft the Response (Iterative refinement for tone and formatting):**
    *   *Intro:* Briefly acknowledge the issue (prefill is memory-bandwidth bound, generation is compute-bound).
    *   *Checklist:* Use bullet points, bold key terms, and provide exact flags.
    *   *Conclusion:* A quick summary of the most likely culprit.

7.  **Final Polish (Self-Correction during drafting):**
    *   *Correction:* Make sure to specify that `--n-gpu-layers -1` is the flag for full offload in llama.cpp.
    *   *Correction:* Emphasize that prefill is memory-bound, so VRAM bandwidth is the #1 suspect.
    *   *Formatting:* Use code blocks for flags to make them copy-pasteable.

8.  **Output Generation:** (This matches the provided good response).