## Turn 1

Local model server (llama.cpp) is slow.
Prompt evaluation (prefill) is much slower than generation (decoding).
Single GPU.
Concise troubleshooting checklist.

    *   *Why is prefill slow?*
        *   Memory bandwidth (GPU VRAM bandwidth).
        *   Compute (GPU cores/TFLOPS).
        *   Context length (KV cache size).
        *   Batch size (Parallel processing).
        *   Model size vs. VRAM (Offloading issues).
        *   Flash Attention (Missing optimization).
        *   Prompt length (Too long for the hardware).

    *   *Category 1: Hardware/Memory (The Basics)*
        *   Is the model fully on the GPU? (VRAM check)
        *   Is the GPU bottlenecked by memory bandwidth?

    *   *Category 2: llama.cpp Specific Parameters*
        *   `n_batch`: How many tokens are processed at once during prefill.
        *   `n_ctx`: Context window size.
        *   `flash_attn`: Is it enabled?

    *   *Category 3: Model/Quantization*
        *   Is the quantization too heavy or too light? (e.g., Q4_K_M vs Q8_0).
        *   Is the model too large for the GPU? (System RAM spillover).

    *   *Category 4: System/Driver*
        *   CUDA/ROCm versions.
        *   Power limits/Thermal throttling.

    *   *Check 1: GPU Offloading.* Ensure `n_gpu_layers` is high enough to fit the *entire* model. If it spills to CPU, prefill will tank.
    *   *Check 2: Flash Attention.* This is the biggest win for prefill speed.
    *   *Check 3: Batch Size.* `n_batch` and `n_ubatch`. If `n_batch` is too small, prefill is serial. If too large, it might hit VRAM limits.
    *   *Check 4: Context Window.* Is `n_ctx` unnecessarily large?
    *   *Check 5: Quantization.* Check if the model is actually loaded in the intended format.
    *   *Check 6: Hardware Bottlenecks.* Check VRAM bandwidth and power.

    *   Use a checklist format.
    *   Keep it concise.
    *   Focus on `llama.cpp` specific flags.