Prefill (prompt evaluation) is memory-bandwidth bound, while generation is compute-bound. If prefill is significantly slower, your GPU is likely bottlenecked by memory bandwidth, VRAM capacity, or suboptimal memory layout. 

Here is a concise troubleshooting checklist for llama.cpp on a single GPU:

### 1. Check for VRAM Swapping (Most Common)
If the model or KV cache exceeds your VRAM, the system will swap to system RAM, causing massive prefill slowdowns.
*   **Check:** Run `nvidia-smi` (or `rocm-smi`) while generating. Look for VRAM usage spiking or system RAM usage increasing.
*   **Fix:** Add `--mlock` to lock the model in VRAM and prevent swapping.
*   **Fix:** Increase `--ctx-size` if your prompt exceeds the default context window.

### 2. Optimize GPU Offload
Ensure the model is fully offloaded to the GPU.
*   **Check:** Verify the model is actually on the GPU.
*   **Fix:** Use `--n-gpu-layers -1` (or the maximum number of layers your VRAM can hold) to offload as much as possible.

### 3. Enable Flash Attention
Flash Attention drastically reduces memory bandwidth usage during prefill.
*   **Fix:** Add `--flash-attn` to your server flags. (Requires Ampere architecture or newer, e.g., RTX 30-series, 40-series, or A100).

### 4. Optimize KV Cache Memory Layout
The KV cache consumes a massive amount of VRAM during prefill. Using BF16 instead of FP16 can save bandwidth.
*   **Fix:** Add `--cache-type-k bf16` and `--cache-type-v bf16`.

### 5. Check Quantization & Model Size
Higher precision models require more memory bandwidth.
*   **Check:** Are you running a Q8_0 or FP16 model? 
*   **Fix:** Switch to a lower quantization (e.g., Q4_K_M or Q5_K_M) to reduce memory bandwidth requirements.

### 6. Server & Hardware Configuration
*   **Batch Size:** Increase `--batch-size` (e.g., `--batch-size 512` or `1024`) to improve prefill throughput.
*   **GPU Power State:** Ensure the GPU is not in a low-power state. Run `nvidia-smi -pm 1` to set persistent memory mode.
*   **PCIe Lanes:** Ensure the GPU is running at x16 PCIe bandwidth, not x8 or x4.

### 7. Verify CUDA/Driver Compatibility
*   **Check:** Ensure your CUDA toolkit version matches the version llama.cpp was compiled against. A mismatch can cause fallbacks to CPU or inefficient memory allocation.