## Turn 1

Local model server (llama.cpp) is slow.
Prompt evaluation (prefill) is much slower than generation (decoding).
Single GPU.
Concise troubleshooting checklist.

    *   *Why is prefill slow?*
        *   Memory bandwidth/bottleneck (though usually, prefill is compute-bound).
        *   GPU memory (VRAM) overflow (spilling to system RAM/Disk).
        *   Context window size (too large for the hardware).
        *   Batch size (too small for the GPU's parallel processing capabilities).
        *   KV Cache management.
        *   Model quantization/format issues.
        *   Driver/CUDA issues.

    *   *Category 1: Memory & Offloading (The most common culprit)*
        *   Is the whole model on the GPU? (`-ngl` or `-n_gpu_layers`)
        *   Is it spilling to CPU? (Check VRAM usage).

    *   *Category 2: Context & Batching (The "Prefill" specific stuff)*
        *   `n_ctx`: Is it too high?
        *   `n_batch`: Is it too low? (Small batch sizes hurt prefill speed).
        *   `ubatch_size`: Related to batching.

    *   *Category 3: Model/Quantization*
        *   Is the GGUF file too big?
        *   Is the quantization level appropriate?

    *   *Category 4: Hardware/Drivers*
        *   CUDA/ROCm version.
        *   Power limits/Thermal throttling.

    *   *Check 1: GPU Offloading.* Ensure `-ngl` is high enough to fit the *entire* model + KV cache. If it spills to RAM, prefill dies.
    *   *Check 2: Batch Size.* Increase `-b` (batch size) and `-ubatch` (ubatch size). Prefill is parallel; small batches = slow prefill.
    *   *Check 3: Context Window.* Check `-c` (n_ctx). Large contexts consume massive VRAM and slow down prefill.
    *   *Check 4: KV Cache.* Check if `flash_attn` is enabled (if supported).
    *   *Check 5: Hardware.* Check for thermal throttling or low power mode.

    *   *Constraint:* Concise.
    *   *Constraint:* llama.cpp specific.
    *   *Constraint:* Single GPU.

    *   *Refining the "Prefill vs Generation" logic:*
        *   Generation is memory-bandwidth bound (sequential).
        *   Prefill is compute-bound (parallel).
        *   If prefill is slow, it means the GPU isn't being utilized effectively for parallel math OR it's being choked by memory swapping.