## Turn 1

Local model server (llama.cpp) is slow.
Prompt evaluation (prefill) is much slower than generation (decoding).
Single GPU.
Concise troubleshooting checklist.

    *   *Prompt Eval (Prefill):* This is compute-bound or memory-bandwidth bound depending on sequence length. It processes many tokens at once.
    *   *Generation (Decoding):* This is almost always memory-bandwidth bound (one token at a time).
    *   *The Problem:* If prefill is slow, it usually means the GPU isn't being saturated correctly, the context window is too large for the VRAM, or there's a bottleneck in the data transfer/CPU-GPU communication.

    *   *Hardware/Driver:* Is the GPU actually being used? (CUDA/ROCm versions).
    *   *Model Loading:* Is the model fully on the GPU? (Offloading layers).
    *   *Context Window:* Is the `n_ctx` too high?
    *   *Batch Size:* Is `n_batch` too small? (This is the most common reason for slow prefill).
    *   *Flash Attention:* Is it enabled? (Crucial for prefill speed).
    *   *Quantization:* Is the format compatible? (GGUF types).
    *   *System/OS:* Is the CPU/RAM bottlenecking the feed?

    *   *Check 1: GPU Offloading.* Ensure all layers are on the GPU. If some are on CPU, prefill will crawl.
    *   *Check 2: Batch Size (`-b` or `n_batch`).* If this is low (e.g., 512), prefill is slow. Increase it (e.g., 2048 or 4096).
    *   *Check 3: Flash Attention.* Ensure it's on. It significantly speeds up prefill.
    *   *Check 4: Context Window (`-c` or `n_ctx`).* If it's huge, it eats VRAM and slows down processing.
    *   *Check 5: KV Cache.* Is it being managed correctly?
    *   *Check 6: Drivers/Toolkit.* Check CUDA/ROCm versions.

    *   *Constraint:* Concise.
    *   *Constraint:* llama.cpp specific.
    *   *Constraint:* Single GPU.

    *   *Drafting the list:*
        1.  **GPU Offloading:** Check `-ngl` (or `--n-gpu-layers`). Ensure all layers are offloaded.
        2.  **Batch Size:** Check `-b` (or `--batch-size`). Increase to 2048 or 512 (depending on VRAM).
        3.  **Flash Attention:** Ensure `--flash-attn` is enabled.
        4.  **Context Size:** Check `-c` (or `--ctx-size`). Is it unnecessarily large?
        5.  **Quantization:** Is the GGUF format optimal for the hardware?
        6.  **Driver/Library:** Verify CUDA/ROCm version compatibility.