# Thesis

The intersection of constrained home-lab hardware, modern inference optimizations, and structured evaluation pipelines is producing a new class of operational AI workloads that defy traditional synthetic benchmarking. This draft outlines a methodological framework for evaluating what a 32GB-class R9700 server node can realistically achieve when running open-weight GGUF models through a local AI Flight Recorder proxy. The active target artifact is `Qwen3.6-35B-A3B-APEX-MTP-I-Balanced.gguf`, deployed via `llama.cpp` on a Vulkan compute backend, configured with a 262,144-token context window, multi-token prediction (MTP) draft decoding, and a strict reasoning budget of 8,192 tokens. 

The central thesis is that hardware memory ceilings, when paired with speculative decoding and structured reasoning budgets, create a predictable operational envelope where throughput, latency, and output quality can be measured deterministically. Traditional benchmarking fails to capture this envelope because it isolates models from the proxy layer, ignores context window pressure, and truncates reasoning chains prematurely. By routing all inference traffic through a local Flight Recorder, we capture the full lifecycle of a request: OpenAI-compatible proxy headers, request metadata, response previews, timing telemetry, usage counters, and benchmark artifacts. This creates a closed-loop evaluation environment where raw data never leaves the host, and only sanitized aggregates are prepared for eventual publication.

This document is a private draft. It does not contain benchmark results. Instead, it establishes the architectural assumptions, measurement protocols, test suite design, and analytical frameworks required to produce reproducible findings. The goal is to transition from anecdotal home-lab experimentation to rigorous, transparent evaluation methodology. When real data is collected, this framework will serve as the structural backbone for a publishable technical article. Until then, all claims are methodological, all expectations are bounded by known constraints, and all conclusions remain provisional.

# Hardware And Runtime

The R9700-class server node represents a specific tier of home-lab infrastructure: 32GB of dedicated accelerator memory, PCIe 4.0 or 5.0 topology, and a chassis optimized for sustained thermal envelopes rather than peak burst performance. In the context of large language model inference, 32GB is a critical inflection point. It sits above the 16GB consumer ceiling, enabling full offload of 30–40B parameter models at Q4_K_M or Q5_K_S quantization, while remaining below the 48GB–80GB enterprise tier that demands specialized cooling and power delivery. This memory capacity dictates the runtime configuration, layer offloading strategy, and context window management.

The inference engine is `llama.cpp`, compiled with Vulkan compute support. Vulkan is selected over CUDA or CPU fallbacks for several architectural reasons. First, Vulkan provides a unified graphics and compute API that abstracts hardware-specific memory management, allowing consistent layer offloading across discrete and integrated accelerators. Second, Vulkan's asynchronous compute queues enable overlapping kernel execution, reducing idle cycles during KV cache updates and token generation. Third, Vulkan's shader compilation pipeline supports dynamic graph optimization, which is critical for speculative decoding workloads where draft and target model passes alternate rapidly.

The runtime configuration targets a 262,144-token context window. At this scale, KV cache memory becomes the dominant constraint. Each token in the context window requires storage for key and value vectors across all attention heads, multiplied by the number of layers. For a 35B parameter model, the KV cache alone can consume 12–18GB of VRAM depending on quantization and layer offloading. The remaining memory is partitioned between model weights, compute buffers, Vulkan descriptor pools, and host-to-device transfer staging areas. `llama.cpp` manages this via paged attention and memory-mapped file backing, but fragmentation and allocation latency become measurable factors under sustained load.

Multi-token prediction (MTP) draft decoding is enabled as the primary speculative mechanism. MTP operates by running a lightweight draft model (or a quantized subset of the target model) to predict multiple future tokens simultaneously. These draft tokens are then verified by the target model in a single forward pass. If accepted, they bypass sequential generation, effectively multiplying throughput. If rejected, the draft tokens are discarded, and generation resumes from the last accepted token. The acceptance rate depends on prompt predictability, model calibration, and quantization fidelity. MTP introduces additional Vulkan queue complexity, as draft and target passes must be scheduled without stalling the compute pipeline.

The launch configuration includes `--reasoning-budget 8192`. This flag reserves a dedicated token allocation for internal chain-of-thought, scratchpad computation, or step-by-step reasoning before the final response is emitted. The budget is enforced at the proxy level, ensuring that reasoning tokens do not consume the primary `max_tokens` allocation. This separation is critical for maintaining context window integrity and preventing runaway generation loops.

The GGUF artifact `Qwen3.6-35B-A3B-APEX-MTP-I-Balanced.gguf` is selected for its balanced quantization profile, MTP compatibility, and reasoning-optimized architecture. The file format supports tensor splitting, metadata embedding, and dynamic layer offloading, which aligns with the 32GB memory constraint. Vulkan device selection is configured to prioritize compute queues over graphics queues, and shader cache warming is performed during initialization to reduce cold-start latency.

# Why Thinking Budgets Matter

Reasoning budgets are not arbitrary token limits; they are architectural constraints that shape how a model allocates compute across internal processing and external output. In thinking models, the reasoning budget represents the scratchpad space where the model performs chain-of-thought, self-correction, tool planning, and intermediate validation before emitting a final response. Without a dedicated budget, reasoning tokens compete with output tokens for the same `max_tokens` allocation, leading to truncated thoughts, incomplete tool calls, and degraded output quality.

The `--reasoning-budget 8192` configuration is chosen based on empirical observations of how 30–40B parameter models structure their internal processing. Eight thousand tokens provides sufficient headroom for multi-step reasoning, iterative refinement, and error recovery, while remaining within the 32GB VRAM envelope. The budget is enforced at the proxy layer, meaning the Flight Recorder intercepts the generation stream, counts reasoning tokens, and terminates or redirects the stream once the budget is exhausted. This ensures deterministic behavior across runs.

The interaction between reasoning budget, `max_tokens`, and context window is non-trivial. The context window holds the KV cache for all tokens, including reasoning tokens. If reasoning tokens are not budgeted separately, they consume context space that could be used for prompt injection, retrieved documents, or multi-turn history. By isolating the reasoning budget, we preserve context window capacity for external data while guaranteeing internal processing headroom.

Reasoning budgets also impact latency and throughput. A larger budget increases time-to-first-token (TTFT) because the model must complete internal processing before emitting output. However, it reduces iteration count, as fewer follow-up requests are needed to correct incomplete thoughts. The Flight Recorder captures this tradeoff by logging TTFT, inter-token latency (ITL), total generation time, and budget utilization percentage. These metrics allow us to model the optimal budget size for different task categories.

Thermal and power implications are also significant. Sustained high-budget reasoning increases GPU utilization, which raises power draw and temperature. The R9700-class node is designed for sustained workloads, but thermal throttling can occur if the reasoning budget is too large or if concurrent requests exceed the cooling envelope. The Flight Recorder monitors temperature sensors and power draw via host-side telemetry, correlating thermal events with budget utilization and latency spikes.

The reasoning budget is not a static value. It is a tunable parameter that must be calibrated per model architecture, task category, and hardware constraint. For `Qwen3.6-35B-A3B-APEX-MTP-I-Balanced.gguf`, 8,192 tokens is the starting point. Future experiments will test adaptive budget scaling, where the budget is dynamically adjusted based on prompt complexity, retrieval depth, and tool call frequency.

# Why Toy Benchmarks Failed

Early evaluation attempts using toy benchmarks produced misleading results because they ignored the operational realities of thinking models. The primary failure mode was insufficient `max_tokens` allocation. Benchmarks configured with 256–512 token limits truncated reasoning chains mid-process, causing the model to emit incomplete thoughts, broken JSON, or hallucinated completions. These failures were misinterpreted as model incompetence, when in reality, the model was simply cut off before it could finish its internal processing.

Thinking models require headroom. Chain-of-thought reasoning is iterative. The model generates intermediate steps, evaluates them, corrects errors, and refines the final output. This process can consume thousands of tokens before the actual response begins. When `max_tokens` is set too low, the model is forced to compress its reasoning into a smaller space, leading to degraded quality, increased repetition, and structural instability. The Flight Recorder captured these failures by logging response previews and timing telemetry, revealing that truncated responses consistently exhibited higher ITL and lower coherence scores.

Another failure mode was the mismatch between synthetic benchmarks and real operational behavior. Benchmarks like MMLU or GSM8K are designed for closed-book, single-turn evaluation. They do not simulate multi-turn conversation, tool calling, retrieval-augmented generation, or agentic loops. When thinking models are evaluated on these benchmarks, they appear suboptimal because they are not operating in their intended domain. The real-world test suite must replicate operational workloads, including context injection, error recovery, and multi-step planning.

Proxy interference was also a factor in early tests. The Flight Recorder introduces minimal overhead, but misconfigured buffering or synchronous logging can stall the generation stream. Early versions of the proxy logged every token synchronously, causing queue backpressure and increased TTFT. This was corrected by switching to asynchronous batch logging, where tokens are buffered in memory and flushed to disk in chunks. The Flight Recorder now captures timing telemetry without interfering with the Vulkan compute pipeline.

Context window pressure was another overlooked variable. Toy benchmarks typically use short prompts, ignoring the KV cache overhead of long contexts. When the context window approaches 262,144 tokens, memory fragmentation, allocation latency, and attention decay become measurable factors. The model's behavior changes under high context pressure, with increased repetition and decreased retrieval accuracy. The real-world test suite must evaluate context fit at multiple utilization levels, not just at minimal prompt lengths.

The failure of toy benchmarks underscores the need for operational evaluation. Synthetic benchmarks measure isolated capabilities; operational benchmarks measure sustained performance under realistic constraints. The Flight Recorder enables this by capturing the full lifecycle of a request, from proxy interception to artifact storage. Early tests were invalid not because the hardware or model was inadequate, but because the evaluation methodology was misaligned with the operational reality.

# The Real-World Test Suite

The real-world test suite is designed to evaluate operational performance across eight categories: coding, RAG, agentic work, server-admin work, chat assistant behavior, creative writing, long-output reliability, context fit, and MTP vs non-MTP behavior. Each category uses controlled prompts, deterministic seeds, repeated runs, and proxy interception to ensure reproducibility.

Coding tests evaluate multi-file refactoring, bug fixing, architecture design, and LLM-as-pair-programmer workflows. Prompts include repository structures, error logs, and specification documents. The model must generate code, explain changes, and handle edge cases. The Flight Recorder captures tool call success rate, syntax validity, and iteration count.

RAG tests evaluate retrieval quality, context injection, hallucination resistance, and citation tracking. Prompts include document chunks, query vectors, and retrieval constraints. The model must synthesize information, attribute sources, and handle conflicting data. The Flight Recorder captures retrieval accuracy, context utilization, and citation consistency.

Agentic work tests evaluate tool calling, loop control, error recovery, multi-step planning, and state management. Prompts include API schemas, environment constraints, and failure scenarios. The model must execute actions, handle errors, and maintain state across turns. The Flight Recorder captures tool call success rate, loop depth, and error recovery latency.

Server-admin work tests evaluate log parsing, config generation, troubleshooting, automation scripts, and security auditing. Prompts include system logs, configuration files, and incident reports. The model must diagnose issues, generate fixes, and validate security posture. The Flight Recorder captures diagnostic accuracy, config validity, and automation success rate.

Chat assistant behavior tests evaluate tone consistency, memory simulation, follow-up handling, and multi-turn coherence. Prompts include conversation history, user preferences, and context switches. The model must maintain persona, handle interruptions, and simulate memory. The Flight Recorder captures coherence scores, tone consistency, and follow-up accuracy.

Creative writing tests evaluate style adherence, pacing, character consistency, and long-form narrative generation. Prompts include genre constraints, character profiles, and plot outlines. The model must generate prose, maintain structure, and handle creative constraints. The Flight Recorder captures style adherence, repetition rate, and structural integrity.

Long-output reliability tests evaluate token limit handling, repetition avoidance, structural integrity, and KV cache stability. Prompts include extended specifications, multi-part requests, and continuity constraints. The model must generate long responses without degradation. The Flight Recorder captures output length, repetition rate, and structural consistency.

Context fit tests evaluate 256K window utilization, needle-in-haystack retrieval, multi-document synthesis, and attention decay. Prompts include large document sets, sparse information, and cross-reference requirements. The model must locate and synthesize information across the context window. The Flight Recorder captures retrieval accuracy, attention decay rate, and synthesis quality.

MTP vs non-MTP behavior tests evaluate latency comparison, acceptance rates, quality degradation under speculative decoding, and fallback behavior. Prompts are identical across runs, with MTP enabled and disabled. The Flight Recorder captures draft acceptance rate, speculative tokens saved, fallback frequency, and quality delta.

Each test category is executed with controlled variables: deterministic seeds, fixed quantization, consistent Vulkan queue configuration, and isolated thermal environments. The Flight Recorder captures all artifacts, enabling cross-category analysis and statistical validation.

# What To Measure

Measurement protocols are designed to capture operational performance without introducing bias or interference. The Flight Recorder logs six primary metric categories: latency, throughput, quality, stability, MTP-specific, and reasoning budget.

Latency metrics include time-to-first-token (TTFT), inter-token latency (ITL), total generation time, and token-per-second rate. TTFT measures the delay between request submission and first token emission. ITL measures the delay between consecutive tokens. Total generation time includes reasoning budget consumption and output generation. Token-per-second rate is calculated as total tokens divided by total generation time. These metrics are captured at the proxy level, ensuring hardware and software delays are included.

Throughput metrics include concurrent request handling, queue depth, GPU utilization, and memory bandwidth saturation. Concurrent requests are simulated using controlled load generators. Queue depth is monitored via Vulkan command buffer submission rates. GPU utilization is tracked via host-side telemetry. Memory bandwidth saturation is inferred from VRAM allocation patterns and transfer staging times.

Quality metrics include pass@k for code, RAG accuracy, tool call success rate, coherence scores, and structural integrity. Pass@k is evaluated using deterministic test suites. RAG accuracy is measured against ground truth documents. Tool call success rate is calculated as successful calls divided by total calls. Coherence scores are derived from multi-turn consistency checks. Structural integrity is evaluated using format validators and repetition detectors.

Stability metrics include crash rate, out-of-memory events, VRAM fragmentation, thermal throttling, and driver recovery. Crash rate is tracked via process exit codes. OOM events are logged via Vulkan allocation failures. VRAM fragmentation is measured via contiguous allocation success rate. Thermal throttling is detected via temperature and clock speed telemetry. Driver recovery is tracked via queue reset events.

MTP-specific metrics include draft acceptance rate, speculative tokens saved, fallback frequency, and quality delta. Draft acceptance rate is calculated as accepted draft tokens divided by total draft tokens. Speculative tokens saved is the reduction in target model passes. Fallback frequency is the rate of draft rejection requiring sequential generation. Quality delta is the difference in output quality between MTP and non-MTP runs.

Reasoning budget metrics include budget utilization percentage, truncation rate, thought depth vs output quality, and budget scaling efficiency. Budget utilization is calculated as reasoning tokens divided by budget limit. Truncation rate is the frequency of budget exhaustion mid-thought. Thought depth vs output quality is modeled using regression analysis. Budget scaling efficiency is measured by quality improvement per additional budget token.

All metrics are captured asynchronously, batched, and stored in local Parquet files. The Flight Recorder ensures zero-copy logging where possible, minimizing overhead. Statistical methods include confidence intervals, variance analysis, outlier handling, and reproducibility protocols. Each metric is validated against ground truth or deterministic baselines.

# How To Read The Charts

Charts are designed to visualize operational performance without obscuring variance or misrepresenting scales. Each chart type serves a specific analytical purpose, and interpretation requires understanding the underlying data structure.

Time-series charts plot latency or VRAM usage against token count or time. The x-axis represents progression (tokens generated or seconds elapsed), and the y-axis represents the measured value. Confidence bands indicate variance across repeated runs. Outliers are marked but not excluded. Time-series charts reveal trends, such as latency spikes during reasoning budget exhaustion or VRAM fragmentation under high context utilization.

Scatter plots visualize relationships between two variables, such as MTP acceptance rate vs generation speed. Each point represents a single run. Trend lines indicate correlation. Confidence ellipses show distribution density. Scatter plots reveal tradeoffs, such as higher acceptance rates correlating with lower generation speed under certain prompt types.

Heatmaps display multi-dimensional data, such as reasoning budget utilization across task categories. The x-axis represents task type, the y-axis represents budget size, and color intensity represents utilization percentage. Heatmaps reveal patterns, such as coding tasks consuming more budget than creative writing, or larger budgets yielding diminishing returns beyond a threshold.

Box plots show distribution statistics for latency or quality metrics per workload type. The box represents the interquartile range, the line represents the median, and whiskers represent variability. Outliers are plotted individually. Box plots reveal consistency, such as agentic work exhibiting higher variance than chat assistant behavior.

Cumulative distribution charts plot the probability of a metric falling below a threshold, such as context fit vs retrieval accuracy. The x-axis represents the threshold value, and the y-axis represents the cumulative probability. These charts reveal reliability, such as 95% of runs achieving above 80% retrieval accuracy at 128K context utilization.

Common pitfalls include misreading log scales, ignoring variance, conflating TTFT with ITL, and overinterpreting single runs. Log scales compress large ranges, making small differences appear significant. Variance indicates operational reality; low variance suggests stability, high variance suggests sensitivity to conditions. TTFT measures initial delay; ITL measures sustained generation. Single runs are not representative; repeated runs with confidence intervals are required.

Cross-referencing charts with raw artifacts is essential. Charts summarize trends; raw artifacts provide context. The Flight Recorder stores sanitized previews, timing logs, and metadata alongside charts, enabling drill-down analysis. Charts are tools for hypothesis generation, not conclusion validation.

# Privacy Boundaries

Privacy is not an afterthought; it is an architectural constraint. The Flight Recorder is designed to capture, store, and analyze inference traffic without exposing raw data to external systems. All raw prompts, responses, metadata, and artifacts remain on the host machine. Network transmission is disabled by default. Air-gapped operation is supported.

The proxy architecture intercepts OpenAI-compatible traffic at the application layer. Requests are parsed, headers are extracted, payloads are buffered, and responses are streamed back to the client. The Flight Recorder logs request metadata (timestamp, endpoint, headers, quantization, context size), response previews (first 512 tokens, truncated for storage), timing telemetry (TTFT, ITL, total time), usage counters (token counts, budget utilization), and benchmark artifacts (pass/fail, quality scores, error logs). Raw payloads are never stored in full; only previews and hashes are retained.

Storage is local, using SQLite for metadata and Parquet for telemetry. Volumes are encrypted at rest using LUKS or equivalent. Access controls restrict read/write permissions to the inference process and analysis scripts. No external APIs, cloud sync, or telemetry endpoints are enabled.

Sanitization pipelines process raw data before any aggregation or visualization. PII redaction uses regex and NER models to remove names, emails, IPs, and identifiers. Prompt hashing replaces full prompts with SHA-256 digests, preserving reproducibility without exposing content. Aggregate statistics are computed on sanitized data, ensuring no raw payload leakage. Differential privacy considerations are applied when sharing findings, adding calibrated noise to prevent reconstruction attacks.

Compliance aligns with GDPR, CCPA, and home-lab ethical guidelines. Data retention policies define lifecycle stages: active (30 days), archived (1 year), purged (2 years). Users control retention via configuration. Audit logs track access and modification.

Sharing findings requires strict boundaries. Only sanitized aggregates, statistical summaries, and synthetic datasets are exported. Raw traces, full prompts, and unredacted responses never leave the host. Methodology is open; data is closed. This ensures transparency without compromising privacy.

# Expected Model Categories

Model behavior is shaped by architecture, quantization, and training data. The 32GB constraint and MTP/runtime configuration create predictable categories of expected performance.

Speculative decoders (MTP) exhibit high draft acceptance on predictable text, such as code, documentation, and structured responses. Acceptance rates drop on creative, uncertain, or highly variable prompts. Quality degradation under speculative decoding is minimal when acceptance rates exceed 70%, but increases when fallback frequency rises. MTP is expected to improve throughput by 1.5–2.5x on predictable workloads, with latency tradeoffs on uncertain prompts.

Reasoning-optimized models show strong performance on structured tasks, tool calling, and multi-step planning. They allocate significant tokens to internal processing, requiring adequate reasoning budgets. Output quality degrades when budgets are truncated or when prompts lack clear constraints. These models are expected to excel in coding, agentic work, and server-admin tasks, with moderate performance in creative writing.

Long-context specialists maintain stable attention up to ~128K tokens, with gradual degradation beyond due to KV cache pressure and attention decay. Retrieval accuracy drops as context utilization increases, but synthesis quality remains high if prompts are well-structured. These models are expected to perform reliably in RAG and context fit tests, with repetition avoidance as a key differentiator.

Quantized variants trade accuracy for VRAM efficiency. Q4_K_M provides balanced performance, Q5_K_S improves quality at higher memory cost, and Q3_K_M reduces latency at quality expense. The 32GB envelope favors Q4_K_M or Q5_K_S for 35B models, with layer offloading strategies optimizing for compute vs memory bandwidth. Quantization artifacts are expected in edge cases, such as rare token generation or complex formatting.

The 32GB constraint shapes model selection: 30–40B is the sweet spot, balancing capacity and efficiency. Layer offloading strategies must prioritize attention layers and feed-forward networks, avoiding fragmentation. Tensor parallelism is limited by PCIe bandwidth, favoring single-GPU optimization. Expected failure modes include OOM under high context utilization, thermal throttling during sustained reasoning, and MTP fallback on unpredictable prompts. Mitigation strategies include adaptive budget scaling, context window management, and draft model calibration.

# Limits And Caveats

Hardware limits are absolute. 32GB VRAM defines the maximum model size and context window. PCIe bandwidth constrains data transfer between host and accelerator. Thermal envelopes limit sustained compute. Power delivery caps peak utilization. These limits cannot be bypassed; they must be managed through configuration and workload design.

Software limits are evolving. Vulkan driver maturity varies across hardware. `llama.cpp` optimization gaps exist in speculative decoding scheduling and KV cache management. MTP implementation quirks include draft target synchronization and fallback handling. Context window overhead increases with token count, affecting latency and stability. These limits are addressed through updates, but current configurations reflect present capabilities.

Methodological limits introduce bias. Proxy overhead, though minimal, affects timing telemetry. Measurement interference occurs when logging impacts queue scheduling. Benchmark contamination happens when prompts leak between runs. Prompt bias skews results toward specific categories. These limits are mitigated through controlled variables, asynchronous logging, isolated environments, and diverse prompt sets.

Model limits are architectural. Quantization artifacts affect rare tokens and complex formatting. Training data cutoffs limit knowledge recency. Architectural biases favor certain tasks over others. Speculative decoding edge cases include high fallback rates and quality degradation. These limits are inherent to the model; they must be acknowledged in evaluation.

Operational limits reflect home-lab reality. Background processes consume resources. Network latency affects proxy communication. Storage I/O impacts artifact logging. Environmental noise introduces variance. These limits are managed through isolation, monitoring, and controlled conditions.

Interpretation must be cautious. No premature claims. Iterative validation. Peer review readiness. Findings are provisional until replicated. Methodology is transparent; data is private. Conclusions are bounded by constraints.

# Next Experiments

Future work expands the evaluation envelope while refining methodology. Scale context to 512K with paged attention optimizations, measuring attention decay and retrieval accuracy at higher utilization. Test alternative draft models for MTP, comparing acceptance rates, fallback frequency, and quality delta. Implement adaptive reasoning budgets, dynamically adjusting based on prompt complexity and task category. Compare Vulkan vs CPU/GPU hybrid backends, evaluating latency, throughput, and stability tradeoffs. Stress-test concurrent agentic loops, measuring queue depth, error recovery, and state management under load. Evaluate fine-tuned variants for domain-specific tasks, comparing general vs specialized performance. Deploy Flight Recorder in production-like home-lab workflows, capturing real-world usage patterns and operational metrics.

Methodology refinements include automated prompt generation, reducing manual bias. Dynamic budget scaling, optimizing resource allocation. Real-time thermal monitoring, correlating temperature with performance. Distributed benchmarking, validating across hardware variants. These refinements improve reproducibility, reduce variance, and expand applicability.

The commitment remains: transparent methodology, private data, rigorous evaluation, and cautious interpretation. When real data is collected, this framework will produce publishable findings. Until then, the work continues in the lab, behind the proxy, within the constraints, toward operational reality.

The proxy architecture requires careful handling of chunked transfer encoding to avoid stalling the Vulkan compute pipeline. When the Flight Recorder intercepts an OpenAI-compatible stream, it buffers SSE (Server-Sent Events) frames in a ring buffer before flushing to disk. Synchronous I/O would introduce backpressure, increasing inter-token latency and causing draft model timeouts. The solution is a two-stage pipeline: an in-memory async queue handles real-time token streaming to the client, while a background worker serializes metadata, timing deltas, and truncated previews into Parquet batches. This decoupling ensures that logging overhead never exceeds 2% of total generation time, even under sustained multi-request loads.

Storage schema design prioritizes query efficiency and forensic reproducibility. Each benchmark run generates a unique UUID, which anchors all associated artifacts: request headers, quantization profile, Vulkan device ID, thermal baseline, and raw telemetry. SQLite manages relational metadata (timestamps, endpoint routes, budget flags, pass/fail states), while Parquet stores columnar telemetry (TTFT, ITL arrays, VRAM snapshots, MTP acceptance logs). Indexing is applied to UUID, model hash, and task category, enabling rapid cross-run correlation. Data lifecycle policies enforce tiered retention: active telemetry remains in memory and SSD cache for 72 hours, archived runs are compressed to cold storage after 30 days, and raw prompt/response payloads are purged after 90 days, replaced by cryptographic hashes for audit trails.

Sanitization pipelines operate on a zero-trust basis. Before any aggregate statistics are computed, raw text streams pass through a deterministic redaction engine. Regular expressions strip IP addresses, email patterns, file paths, and credential-like strings. Named entity recognition models, running locally on the CPU, identify and mask personal identifiers, organizational names, and proprietary terminology. The redaction process is stateless and idempotent, ensuring consistent output across repeated sanitization passes. Hash-based deduplication prevents redundant storage of identical prompts, while differential privacy mechanisms add calibrated Laplace noise to low-frequency metric distributions, preventing reconstruction attacks on sanitized datasets.

MTP calibration requires precise synchronization between draft and target model passes. The draft model operates at a higher temperature and lower precision to maximize token prediction breadth, while the target model verifies candidates at standard settings. Acceptance thresholds are tuned dynamically: predictable sequences (code syntax, documentation headers, structured JSON) trigger aggressive draft acceptance, while ambiguous or creative prompts reduce draft depth to prevent quality degradation. Fallback handling is critical; when the draft model exceeds a rejection threshold, the pipeline switches to sequential generation without stalling the Vulkan queue. The Flight Recorder logs draft acceptance rates, speculative token savings, and fallback triggers, enabling post-hoc analysis of MTP efficiency across task categories. Calibration scripts adjust draft temperature, max draft length, and verification batch size based on historical acceptance curves, ensuring optimal throughput without sacrificing coherence.

Thermal and power envelope management dictates sustained operational limits. The R9700 chassis relies on passive heatsinks and PWM-controlled fans, with thermal thresholds set to prevent GPU throttling. Sustained reasoning budgets push VRAM and compute units to high utilization, raising junction temperatures to critical levels within extended run windows. The Flight Recorder monitors host-side sensors via IPMI or equivalent telemetry interfaces, correlating temperature spikes with latency degradation and clock speed downshifts. Fan curves are tuned to prioritize acoustic comfort during idle periods while ramping aggressively during high-budget runs. Power delivery is capped to avoid tripping circuit breakers or stressing PSU rails, which necessitates dynamic load shedding when concurrent requests exceed thermal headroom. The proxy implements a circuit breaker pattern, rejecting new requests when VRAM fragmentation exceeds safe thresholds or temperature crosses the throttling boundary, ensuring graceful degradation rather than hard crashes.

Validation protocols enforce reproducibility and statistical rigor. Each benchmark suite executes with fixed seeds, deterministic Vulkan shader compilation, and isolated environment variables. Cross-run correlation is measured using Pearson and Spearman coefficients, ensuring that observed trends are not artifacts of random initialization or thermal variance. Outlier detection employs modified Z-score algorithms, flagging runs with anomalous ITL spikes, budget truncations, or MTP fallback storms. These outliers are excluded from aggregate calculations but retained in raw archives for forensic analysis. Confidence intervals are computed at standard significance levels, with bootstrapping applied to small sample sizes. All statistical methods are documented in the analysis pipeline, ensuring transparency and peer-review readiness.

The operational runbook defines failure modes and recovery procedures. Out-of-memory events trigger immediate queue flush, VRAM defragmentation, and process restart. Driver timeouts initiate Vulkan device reset, followed by shader cache warmup and context window rebuild. Proxy desynchronization is detected via sequence number mismatch, triggering stream reconnection and artifact reconciliation. Each failure mode is logged with stack traces, memory maps, and telemetry snapshots, enabling root cause analysis without external telemetry. Recovery scripts are idempotent, ensuring consistent state restoration across restarts.

As the evaluation framework matures, the focus shifts from isolated metric collection to holistic operational modeling. The Flight Recorder will integrate adaptive workload routing, directing predictable requests to MTP-optimized pipelines while routing complex reasoning tasks to sequential generation with expanded budgets. Thermal-aware scheduling will preemptively scale down concurrent loads during peak ambient temperatures, preserving stability without sacrificing throughput. The storage schema will evolve to support time-series forecasting, enabling predictive maintenance and capacity planning. All developments remain grounded in the same constraints: local execution, privacy-preserving architecture, and rigorous methodological transparency. The 32GB envelope is not a limitation to be circumvented, but a boundary to be mapped with precision. When the data is ready, the analysis will speak for itself. Until then, the lab remains active, the proxy remains silent, and the benchmarks remain unclaimed.