`, `Answer:`, `Final Answer:`. Multiple stops work: `--stop "<|eot_id|>" --stop "\n\n" --stop "Final Answer:"`.
   - `--temp` & `--top-p`: For reasoning models, use `--temp 0.6` and `--top-p 0.9`. Lower temp reduces rambling but can cause loops. Higher temp increases creativity but risks divergence.
   - `--repeat-penalty`: Set to 1.1–1.2 to discourage repeating reasoning steps.
   - `--mirostat` / `--mirostat-tau`: If using Mirostat v2, set `--mirostat-tau 5.0`. It adapts temperature dynamically but can extend generation if the target isn’t met. Disable if loops persist.
   - `--flash-attn`: Enable for speed, but ensure it doesn’t alter token sampling behavior (usually safe).
   - Proxy/Reverse Proxy: Check if your proxy (Nginx, Caddy, Traefik) has `proxy_read_timeout` or `client_max_body_size` limits that truncate responses. OpenWebUI’s `LLM_API_BASE` and `LLM_API_KEY` must correctly route to llama.cpp’s `/v1/chat/completions` endpoint.

   **# Prompt Design**
   Prompt structure dictates how the model allocates its output budget. Use explicit delimiters, role separation, and termination instructions.

   *Bad Prompt (Triggers Infinite Reasoning):*
   ```
   Solve this step by step. Explain your reasoning thoroughly. What is the answer?
   ```
   *Why it fails:* No stop condition. Model interprets "explain thoroughly" as permission to generate indefinitely.

   *Good Prompt (Bounded Reasoning):*
   ```
   You are a precise assistant. Follow these rules:
   1. Think step-by-step inside <think> tags.
   2. After reasoning, output exactly one line starting with "Answer: ".
   3. Do not add explanations after "Answer: ".
   4. Stop generating immediately after the answer.

   Problem: [Insert problem]
   ```
   *Why it works:* Explicit tags, clear termination instruction, and negative constraints ("Do not add...") guide the model to a hard stop.

   *Advanced Prompt Shape (Multi-turn/Complex):*
   ```
   [System] You are a reasoning engine. Output only structured JSON with "reasoning" and "final_answer" keys.
   [User] Solve: [Problem]
   [Assistant] {
     "reasoning": "Step 1... Step 2...",
     "final_answer": "42"
   }
   ```
   *Key Principles:*
   - Use XML/JSON delimiters for machine-readable boundaries.
   - Place termination instructions at the end of the prompt (recency bias).
   - Avoid open-ended phrases like "go as deep as needed" or "explain everything".
   - For non-reasoning models, strip CoT instructions entirely. They’ll hallucinate reasoning if prompted.

   **# Benchmark Design**
   To systematically evaluate settings and prompts, design a lightweight benchmark:
   - **Test Set:** 10 diverse tasks (math, coding, logic, summarization, open-ended QA).
   - **Metrics:**
     - `tokens_consumed`: Total output tokens before stop/timeout.
     - `answer_completeness`: Binary (1 if contains expected answer, 0 otherwise).
     - `stop_sequence_hit`: 1 if model hit a configured stop token, 0 if truncated by `--n-predict`.
     - `latency_ms`: Time from request to first token.
     - `loop_detection`: Flag if >50% of tokens are repetitive or non-terminating.
   - **Workflow:**
     1. Run each prompt through llama.cpp with baseline settings.
     2. Log raw output and metrics.
     3. Vary one parameter at a time (`--temp`, `--stop`, `--ctx-size`).
     4. Compare token efficiency vs answer quality.
   - **Automation:** Use a Python script with `requests` to hit `/v1/chat/completions`, parse JSON, and calculate metrics. OpenWebUI’s built-in benchmark tab can be repurposed by exporting logs.

   **# What The Flight Recorder Should Show**
   In OpenWebUI or llama.cpp logs, look for:
   - **Token Usage Graph:** Spikes to `--n-predict` indicate budget exhaustion. Flatline at stop token = healthy.
   - **Stop Sequence Triggers:** Should show `</think>` or `Answer:` in logs. Missing triggers mean prompt/stop config mismatch.
   - **Temperature/Top-P Fluctuations:** If using Mirostat, watch for oscillating values. Stable = good. Wild swings = sampling instability.
   - **Context Window Saturation:** KV cache usage >90% often correlates with degraded reasoning or loops.
   - **Proxy Timeout Logs:** `504 Gateway Timeout` or `413 Payload Too Large` indicate network/proxy limits, not model behavior.
   - **Repetition Patterns:** Look for repeated phrases like "Let me think about this..." or recursive step numbering.

   **# Practical Defaults**
   A robust baseline for most home-lab setups:
   ```bash
   llama-server \
     --model your-model.gguf \
     --ctx-size 8192 \
     --n-predict 2048 \
     --temp 0.7 \
     --top-p 0.9 \
     --repeat-penalty 1.15 \
     --stop "<|eot_id|>" \
     --stop "\n\n" \
     --stop "Answer:" \
     --stop "</think>" \
     --flash-attn \
     --numa \
     --threads 8 \
     --batch-size 512
   ```
   OpenWebUI settings:
   - `Temperature`: 0.7
   - `Top P`: 0.9
   - `Max Tokens`: 2048
   - `Stop Sequences`: `<|eot_id|>,\n\n,Answer:,</think>`
   - `Stream`: Enabled (for real-time monitoring)
   - `System Prompt`: "You are a concise assistant. Provide step-by-step reasoning if needed, but end with a clear, direct answer. Do not continue after the answer."

   **# When To Increase Budgets**
   Increase `--n-predict` and `--ctx-size` only when:
   - Tasks require multi-step reasoning (e.g., code generation, complex math proofs).
   - Context window is saturated by long inputs (documents, logs, transcripts).
   - Model architecture explicitly supports long-context reasoning (e.g., 128k+ models).
   *Safe increase strategy:* Double `--n-predict` to 4096, monitor token efficiency. If answer completeness doesn’t improve, revert. Use `--cache-type-k q8_0` or `--cache-type-i q4_0` to mitigate KV cache bloat.

   **# When To Stop A Run**
   Implement these heuristics to abort unproductive generations:
   - **Token Budget Exceeded:** If `tokens_consumed > 0.8 * --n-predict` without a stop sequence, abort.
   - **No Progress in N Tokens:** If >100 tokens pass without new semantic content (detected via simple n-gram overlap or embedding similarity), stop.
   - **Repetition Loop:** If the last 5 tokens repeat a phrase >3 times, force stop.
   - **Timeout:** Set proxy `proxy_read_timeout 30s`. If no response in 30s, abort.
   - **User Intervention:** OpenWebUI’s stop button should be mapped to SIGINT or API cancellation. Ensure your proxy supports request cancellation.

   *Self-Correction/Refinement during drafting:*
   - Check token count: The draft is comprehensive but might be ~2,500 tokens. I need to expand each section with more technical depth, examples, and practical troubleshooting steps to reach ~4,500.
   - I'll add more detail to llama.cpp internals (KV cache, sampling, rope scaling), proxy configuration specifics, prompt engineering frameworks, benchmark automation scripts, and flight recorder log parsing.
   - I'll ensure the tone remains accessible but technical, as requested.
   - I'll structure carefully to match exact headers.
   - I'll expand each section systematically.

   Let's write it out fully, carefully monitoring depth and length. I'll aim for thorough explanations, concrete examples, and actionable advice. I'll make sure each section is substantial.
   *Structure check:*
   # Short Diagnosis
   # Why This Happens
   # Settings To Check
   # Prompt Design
   # Benchmark Design
   # What The Flight Recorder Should Show
   # Practical Defaults
   # When To Increase Budgets
   # When To Stop A Run
   All match. I'll generate the full response now. 
   I will consciously expand on technical details: autoregressive generation, KV cache dynamics, sampling algorithms (nucleus, temperature, repetition penalty), llama.cpp server architecture, proxy timeout handling, prompt templating (ChatML, Alpaca, Llama-3), benchmark metrics calculation, log parsing, and operational heuristics. This will naturally push the length toward the target while maintaining quality.
   Proceed. 
   Output matches the final response. 
   Self-Correction/Verification during thought: 
   - Check required sections: exactly as listed.
   - Check tone: accessible but technical.
   - Check examples: good/bad prompt shapes included.
   - Check length: I will deliberately expand each section with detailed explanations, technical context, and practical steps to approach ~4,500 tokens without fluff.
   - All constraints met. Generating. 
   Output generation. 
   [Done] 
   *Output Generation* (matches the final response)
   *(Note: I'll carefully count/estimate tokens during generation to ensure it's substantial. I'll use detailed technical explanations, code blocks, configuration examples, and structured troubleshooting steps.)*
   Proceeds. 
   [Output Generation] -> *See response.* 
   *(Self-Correction/Refinement during drafting process)*: I need to make sure I don't just pad, but actually provide deep, useful content. I'll expand on:
   - The exact mechanics of why llama.cpp might not stop (API vs CLI flags, stop token forwarding in OpenWebUI)
   - Detailed prompt templating for different model families (Llama 3, Qwen, Mistral, DeepSeek)
   - Benchmark script structure and metric calculation
   - Flight recorder log analysis with concrete examples
   - Operational runbook for stopping/continuing
   This will naturally reach the target length with high signal. All good.✅
   Output matches response. 
   [Proceeds]
</think>

# Short Diagnosis
Your model is trapped in an autoregressive generation loop. Instead of recognizing a natural termination point, it continues predicting tokens indefinitely, consuming its allocated output budget until it hits a hard cap or exhausts system memory. This is rarely a model defect; it is a configuration mismatch between the model’s training priors, your prompt structure, and the generation parameters passed through llama.cpp and your reverse proxy. The solution requires bounding the reasoning phase, enforcing explicit termination signals, aligning sampling settings with the model’s expected behavior, and implementing operational safeguards to abort runaway generations.

# Why This Happens
Large language models generate text token-by-token, conditioning each new token on the entire preceding sequence. When you prompt a model with open-ended reasoning tasks, it learns to output intermediate steps before a final answer. If the prompt lacks a clear delimiter or stop condition, the model treats the entire output window as valid continuation. Several architectural and operational factors compound this behavior:

1. **Missing or Misforwarded Stop Sequences:** llama.cpp respects `--stop` flags, but OpenWebUI’s API layer or your reverse proxy may strip, ignore, or misroute stop tokens. Many models expect family-specific terminators (`<|eot_id|>`, `<|end_of_text|>`, `<|reserved_special_token_...|>`). If these aren’t explicitly passed in the `/v1/chat/completions` payload, the model has no signal to halt.
2. **Aggressive Reasoning Fine-tunes:** Models like DeepSeek-R1, QwQ, or custom Chain-of-Thought fine-tunes are explicitly trained to output `<think>` blocks before answering. Without a closing tag or explicit instruction to stop, they’ll keep reasoning until the context window fills. Standard instruct models lack this prior and will hallucinate reasoning if prompted to “think step by step.”
3. **Sampling Parameter Interactions:** Low temperature combined with high top_p can cause deterministic loops. High temperature with low repetition penalty encourages rambling. Mirostat v2 adapts temperature dynamically to maintain target perplexity, but if the target isn’t met, it can extend generation indefinitely.
4. **KV Cache Saturation & Context Window Drift:** As the context window fills, the model’s attention distribution shifts. Early prompt constraints (e.g., “stop after the answer”) lose weight in the attention heads, causing the model to revert to its base training distribution, which often favors continuous generation.
5. **Proxy/Server Timeout Mismatches:** Reverse proxies (Nginx, Caddy, Traefik) often have `proxy_read_timeout` or `client_max_body_size` limits. If the model generates slowly, the proxy may drop the connection, leaving OpenWebUI in a waiting state. Conversely, if `--n-predict` is set too high, the model gets too much runway to loop.

# Settings To Check
Focus on these llama.cpp server flags and proxy configurations. Each directly influences whether the model terminates cleanly or loops.

- `--ctx-size` (or `--context`): Must accommodate prompt + reasoning + answer. Default 2048 is often too small for reasoning models. Set to 8192 or 16384. Larger contexts reduce drift but increase KV cache pressure.
- `--n-predict` (or `--max-tokens`): Hard cap on output. Set to 2048–4096 for most tasks. If it hits this, the run is cut off, which is preferable to infinite generation.
- `--stop`: Critical. Add model-specific stop tokens. Multiple stops work: `--stop "<|eot_id|>" --stop "\n\n" --stop "Answer:" --stop "</think>"`. Verify these match the model’s tokenizer.
- `--temp` & `--top-p`: For reasoning models, use `--temp 0.6` and `--top-p 0.9`. Lower temp reduces rambling but can cause loops. Higher temp increases creativity but risks divergence.
- `--repeat-penalty`: Set to 1.1–1.2 to discourage repeating reasoning steps. Default 1.0 often allows recursive loops.
- `--mirostat` / `--mirostat-tau`: If using Mirostat v2, set `--mirostat-tau 5.0`. It stabilizes temperature but can extend generation if the target isn’t met. Disable (`--mirostat 0`) if loops persist.
- `--flash-attn`: Enable for speed. It doesn’t alter sampling but improves throughput, reducing time-to-first-token.
- `--numa` / `--threads` / `--batch-size`: Optimize for your hardware. Mismatched thread counts can cause scheduling delays that manifest as slow generation, increasing proxy timeout risk.
- Proxy/Reverse Proxy: Check `proxy_read_timeout` (set to 60–120s), `proxy_buffer_size`, and `client_max_body_size`. Ensure OpenWebUI’s `LLM_API_BASE` correctly routes to llama.cpp’s `/v1/chat/completions` endpoint. Verify that `stop` arrays in the API payload are preserved.

# Prompt Design
Prompt structure dictates how the model allocates its output budget. Use explicit delimiters, role separation, and termination instructions. Poor prompts leave the model guessing where to stop; well-structured prompts enforce boundaries.

*Bad Prompt (Triggers Infinite Reasoning):*
```
Solve this step by step. Explain your reasoning thoroughly. What is the answer?
```
*Why it fails:* No stop condition. “Explain thoroughly” is interpreted as permission to generate indefinitely. The model lacks a termination signal and will keep reasoning until context saturation.

*Good Prompt (Bounded Reasoning):*
```
You are a precise assistant. Follow these rules:
1. Think step-by-step inside <think> tags.
2. After reasoning, output exactly one line starting with "Answer: ".
3. Do not add explanations after "Answer: ".
4. Stop generating immediately after the answer.

Problem: [Insert problem]
```
*Why it works:* Explicit tags, clear termination instruction, and negative constraints guide the model to a hard stop. The recency bias of the final instruction ensures compliance.

*Advanced Prompt Shape (Multi-turn/Complex):*
```
[System] You are a reasoning engine. Output only structured JSON with "reasoning" and "final_answer" keys.
[User] Solve: [Problem]
[Assistant] {
  "reasoning": "Step 1... Step 2...",
  "final_answer": "42"
}
```
*Key Principles:*
- Use XML/JSON delimiters for machine-readable boundaries.
- Place termination instructions at the end of the prompt (recency bias).
- Avoid open-ended phrases like “go as deep as needed” or “explain everything”.
- For non-reasoning models, strip CoT instructions entirely. They’ll hallucinate reasoning if prompted.
- Match the prompt template to the model’s family (ChatML for Mistral/Llama 3, Alpaca for older models, DeepSeek’s `<｜begin▁of▁sentence｜>` for R1 variants).

# Benchmark Design
To systematically evaluate settings and prompts, design a lightweight benchmark that measures token efficiency, answer quality, and termination behavior.

- **Test Set:** 10 diverse tasks (math, coding, logic, summarization, open-ended QA). Include at least 2 tasks that historically trigger loops.
- **Metrics:**
  - `tokens_consumed`: Total output tokens before stop/timeout.
  - `answer_completeness`: Binary (1 if contains expected answer, 0 otherwise).
  - `stop_sequence_hit`: 1 if model hit a configured stop token, 0 if truncated by `--n-predict`.
  - `latency_ms`: Time from request to first token.
  - `loop_detection`: Flag if >50% of tokens are repetitive or non-terminating (simple n-gram overlap or embedding cosine similarity <0.3).
- **Workflow:**
  1. Run each prompt through llama.cpp with baseline settings.
  2. Log raw output and metrics.
  3. Vary one parameter at a time (`--temp`, `--stop`, `--ctx-size`).
  4. Compare token efficiency vs answer quality.
- **Automation:** Use a Python script with `requests` to hit `/v1/chat/completions`, parse JSON, and calculate metrics. Example structure:
  ```python
  import requests, json, time
  def benchmark(prompt, settings):
      payload = {"messages": [{"role": "user", "content": prompt}], **settings}
      start = time.time()
      resp = requests.post("http://localhost:8080/v1/chat/completions", json=payload, stream=True)
      tokens = 0
      for chunk in resp.iter_lines():
          if chunk:
              data = json.loads(chunk.decode())
              tokens += len(data["choices"][0]["delta"]["content"])
      return tokens, time.time() - start
  ```
- **OpenWebUI Integration:** Export logs via the UI’s debug panel. Use the built-in benchmark tab to compare model families, but supplement with custom scripts for granular stop-sequence tracking.

# What The Flight Recorder Should Show
In OpenWebUI or llama.cpp logs, look for these indicators to diagnose runaway generation:

- **Token Usage Graph:** Spikes to `--n-predict` indicate budget exhaustion. Flatline at stop token = healthy. Gradual decline = context drift.
- **Stop Sequence Triggers:** Should show `</think>` or `Answer:` in logs. Missing triggers mean prompt/stop config mismatch.
- **Temperature/Top-P Fluctuations:** If using Mirostat, watch for oscillating values. Stable = good. Wild swings = sampling instability.
- **Context Window Saturation:** KV cache usage >90% often correlates with degraded reasoning or loops. Check `--ctx-size` vs actual prompt length.
- **Proxy Timeout Logs:** `504 Gateway Timeout` or `413 Payload Too Large` indicate network/proxy limits, not model behavior.
- **Repetition Patterns:** Look for repeated phrases like “Let me think about this...” or recursive step numbering. These indicate sampling loops.
- **API Payload Inspection:** Verify that `stop` arrays are present in the `/v1/chat/completions` request. OpenWebUI sometimes drops them if not explicitly mapped.

# Practical Defaults
A robust baseline for most home-lab setups minimizes loops while preserving reasoning capability.

```bash
llama-server \
  --model your-model.gguf \
  --ctx-size 8192 \
  --n-predict 2048 \
  --temp 0.7 \
  --top-p 0.9 \
  --repeat-penalty 1.15 \
  --stop "<|eot_id|>" \
  --stop "\n\n" \
  --stop "Answer:" \
  --stop "</think>" \
  --flash-attn \
  --numa \
  --threads 8 \
  --batch-size 512
```

OpenWebUI settings:
- `Temperature`: 0.7
- `Top P`: 0.9
- `Max Tokens`: 2048
- `Stop Sequences`: `<|eot_id|>,\n\n,Answer:,</think>`
- `Stream`: Enabled (for real-time monitoring)
- `System Prompt`: "You are a concise assistant. Provide step-by-step reasoning if needed, but end with a clear, direct answer. Do not continue after the answer."

These defaults balance creativity with termination safety. Adjust `--temp` downward for deterministic tasks, upward for creative generation. Always verify stop tokens match your model’s tokenizer.

# When To Increase Budgets
Increase `--n-predict` and `--ctx-size` only when:
- Tasks require multi-step reasoning (e.g., code generation, complex math proofs, legal analysis).
- Context window is saturated by long inputs (documents, logs, transcripts).
- Model architecture explicitly supports long-context reasoning (e.g., 128k+ models with RoPE scaling).
- Benchmark shows `answer_completeness` drops below 0.7 at current budget.

*Safe increase strategy:* Double `--n-predict` to 4096, monitor token efficiency. If answer completeness doesn’t improve, revert. Use `--cache-type-k q8_0` or `--cache-type-i q4_0` to mitigate KV cache bloat. Increase `--ctx-size` in 2048 increments, watching for memory pressure. Never exceed your GPU’s VRAM capacity; swap to CPU offloading (`--main-gpu 0 --tensor-split 0.9`) if needed.

# When To Stop A Run
Implement these heuristics to abort unproductive generations before they consume resources:

- **Token Budget Exceeded:** If `tokens_consumed > 0.8 * --n-predict` without a stop sequence, abort. The model is likely looping.
- **No Progress in N Tokens:** If >100 tokens pass without new semantic content (detected via simple n-gram overlap or embedding similarity), stop. Use OpenWebUI’s stop button or send a cancellation request to the API.
- **Repetition Loop:** If the last 5 tokens repeat a phrase >3 times, force stop. This indicates sampling instability.
- **Timeout:** Set proxy `proxy_read_timeout 30s`. If no response in 30s, abort. Adjust based on model size and hardware.
- **User Intervention:** OpenWebUI’s stop button should be mapped to SIGINT or API cancellation. Ensure your proxy supports request cancellation. Log the abort reason for benchmarking.
- **Context Saturation Warning:** If KV cache usage >90%, stop and reduce `--ctx-size` or truncate prompt history. Continuing guarantees degraded output.

These heuristics form a practical runbook for home-lab operations. Combine them with benchmark data to refine settings over time. The goal is predictable, bounded generation that delivers useful answers without exhausting resources.