## Turn 1

*   Scenario: Local model server (llama.cpp) is slow.
    *   Symptom: Prompt evaluation (prefill) is significantly slower than token generation (decoding).
    *   Hardware: Single GPU.
    *   Goal: Concise troubleshooting checklist.

    *   *Prompt Evaluation (Prefill):* Processing the input tokens. This is compute-bound and benefits heavily from batching and GPU acceleration.
    *   *Generation (Decoding):* Predicting the next token. This is memory-bandwidth bound.
    *   *The Gap:* If prefill is slow but generation is okay, it usually means the GPU isn't being fully utilized for the initial batch, or there's a bottleneck in how the prompt is being fed/processed.

    *   *GPU Offloading:* Are all layers on the GPU? (`-ngl` or `--n-gpu-layers`).
    *   *Batch Size:* Is the batch size too small? (`-b` or `--batch-size`).
    *   *Context Size:* Is the context window too large, causing swapping or memory pressure? (`-c` or `--ctx-size`).
    *   *KV Cache:* Is the KV cache optimized? (Flash Attention).
    *   *Quantization:* Is the model too large for VRAM, causing system RAM fallback?
    *   *Drivers/Backend:* Is CUDA/Metal/Vulkan actually active?

    *   *Check 1: GPU Offloading.* Ensure `-ngl` is set to a value that puts all layers on the GPU. If some are on CPU, prefill (which is parallel) suffers more than generation (which is sequential).
    *   *Check 2: Batch Size.* The `-b` parameter controls how many tokens are processed at once during prefill. If it's too low (e.g., 512), large prompts take longer.
    *   *Check 3: Flash Attention.* `--flash-attn` significantly speeds up prompt processing and reduces memory usage.
    *   *Check 4: VRAM Overhead.* Check `nvidia-smi`. If VRAM is maxed, the system might be swapping to system RAM (slow).
    *   *Check 5: Backend Verification.* Confirm the logs say "BLAS = 1" or "CUDA = 1".

    *   Keep it concise.
    *   Use a checklist format.
    *   Group by "Immediate Fixes" and "Configuration Checks."