Model Testing

Main Takeaways

Field notes from several weeks of testing llama.cpp, GGUF models, MTP/speculative decoding, long-context agent workloads, Vulkan vs ROCm backends, and tool-using clients on the AMD Radeon AI PRO R9700. This is not a clean lab benchmark — it is real agent sessions, long prompts, model switches, failed experiments, power draw surprises, runaway reasoning loops, and enough token burn to start seeing patterns.

Live Hardware Snapshot

Two actual machines with telemetry from their ServerTop dashboards at sample time.

Storage Context

/backup is RAID1 and /archive is a larger HDD volume on the R9700 host.

Companion Hardware Note

Model Registry Highlights

Representative GGUF profiles across both platforms, including MTP-curated and long-context candidates.

Intel Arc Pro B60 Follow-Up Test

A later follow-up run tested the same general llama.cpp/Vulkan/MTP workflow on an Intel Arc Pro B60 server — a separate box from the R9700 system. It exposed a different class of problem: not model fit or raw throughput first, but PCIe/runtime power-management stability.

B60 Test System

The first benchmark attempt made the server look idle or broken because Vulkan had fallen back to CPU/llvmpipe. The llama.cpp container logs showed:

MESA: error: Unknown kernel mode driver
warning: no usable GPU found, --gpu-layers option will be ignored

The kernel was also logging xe driver force-wake and PCIe power-state errors. The GPU and upstream PCI bridges were stuck in D3cold, and soft PCI resets failed with I/O errors. A warm reboot was not enough; after the first reboot the GPU briefly appeared and then runtime-suspended back into an unusable state.

The durable workaround was to disable the runtime power-management path for this card/driver combination with a GRUB drop-in:

GRUB_CMDLINE_LINUX_DEFAULT="$GRUB_CMDLINE_LINUX_DEFAULT pcie_port_pm=off xe.dmc_firmware_path=/dev/null xe.enable_dc=0 xe.disable_power_well=0"

After regenerating GRUB and rebooting, the card stayed in D0, /dev/dri remained present, and vulkaninfo reported the real discrete GPU:

deviceName = Intel(R) Arc(tm) Pro B60 Graphics (BMG G21)
driverName = Intel open-source Mesa driver

llama.cpp inside Docker then reported:

Vulkan0 : Intel(R) Arc(tm) Pro B60 Graphics (BMG G21) (24480 MiB, 21995 MiB free)

B60 Small Coding Benchmark

The B60 benchmark used a compact coding prompt:

Write a Python function parse_duration(text) that converts strings like "2h 30m", "45s", and "1d 2h" to seconds. Include exactly three assert tests. Return only code.

Results

MTP Effects on the B60

The B60 result is not directly comparable to the R9700 long-context runs because this was a 4096-context smoke-style coding benchmark after a driver recovery. Still, it gives a useful practical read: MTP helped both Qwen models, and it helped the 27B dense model much more than the 35B A3B MoE in this specific short-output test.

B60 Lessons

AMD R9700 Test Matrix and Live Results

Highest-confidence measured values currently available from live ServerTop metrics on the jkxeon host.

Live R9700 llama.cpp Sample

Source: ServerTop dashboard http://192.168.1.116:8090/api/metrics and /api/models

Installed R9700 Registry Highlights

The R9700 dashboard reported the curated registry as installed for the main long-context and MTP profiles.

Test Methodology

Approaches and scripts used across both platforms.

Serving Flags

Controlled 27B MTP Coding Tests

Most apples-to-apples controlled test: two Qwen3.6 27B dense MTP models on a coding prompt asking for a thread-safe EventIndex, filters, indexes, complexity notes, and pytest coverage.

Results

Conclusion: the IMAT IQ4_XS + Q8 NextN version was the better daily-driver 27B profile — faster while preserving a complete answer. The Q5_K_M version remains a useful quality baseline.

35B MTP and Opus/Qwopus Variants

The 35B A3B MTP family became the most interesting class of models. Small enough in active parameters to run comfortably on the R9700, but large enough to behave more capable than smaller dense models. MTP made them feel much less like a compromise.

Above 80% acceptance, speculative output becomes a major quality-of-life improvement. The model no longer feels like a big local compromise — it feels like an interactive coding model.

MTP in Practice

MTP is not just "more tokens per second." The draft model proposes future tokens; the main model accepts or rejects them. If acceptance is high, you get more output per expensive decode step. If acceptance is low, the speculative path can become less useful.

Acceptance Ranges

Observed MTP Acceptance Examples

The best MTP models were both fast and stable. The weaker failure mode was not usually bad MTP acceptance — it was clients giving the reasoning model too much room to think.

Results

Combines controlled small coding tests, longer 4096-token coding tests, live dashboard readings, and real agent-session observations. Treat as practical operating data, not as a standardized leaderboard.

Power, Clocks, and Idle Behavior

The R9700 behaved differently depending on model and backend.

The idle 100% issue mattered because a model that is not actively generating should not sit at high clocks and draw needless power. Tuning poll behavior allowed idle to drop back toward low usage/power, at the cost of a slightly slower wake-up.

LACT was tried for overclocking/undervolting but was not useful for increasing memory clock on this setup, so it was removed. The practical tuning path ended up being software/runtime tuning, not GPU overclocking.

Thinking Loops and Client Configuration

The most painful failures were not simple crashes. They were "the model is alive but doing the wrong thing forever."

Qwen Reasoning Loop

One OpenCode trace showed the model repeating variations of the same thought:

I see the issue now - the echo lines need to mirror the infinity symbol's actual shape...

It repeated the same geometric analysis for minutes. Root causes:

Fixes Applied

{"limit": {"context": 196608, "output": 8192}}

--reasoning-budget 8192

We also recommended removing small_model when it pointed to the same large reasoning model. A "small model" that is actually the same 35B reasoning model only makes utility calls slower and more expensive.

Cline Invalid API Responses

Cline began showing errors like:

Invalid API Response: The provider returned an empty or unparsable response.

Likely causes:

One observed state had Cline-style requests hitting Gemma 4 26B with:

That is a fragile setup for a tool client. Gemma may be useful for normal coding/chat, but the local Qwen MTP models were more promising for Cline/OpenCode tool workflows.

Screenshots

Client Settings That Matter

For local tool agents, conservative client settings are not optional. They are stability settings.

The biggest lesson: local agent stability depends on the contract between the client and the model. A model can be fast, capable, and still bad at the exact syntax a tool client expects.

Recommended Model Roles

Based on the tests and agent workloads observed.

Practical Recommendations

If rebuilding this from scratch.

Suggested Default llama.cpp Posture for Qwen MTP

--cache-ram 0 -c 196608 --parallel 1 -ngl 99 --spec-type draft-mtp --spec-draft-n-max 2 --reasoning-budget 8192

Suggested Default Client Posture

{"temperature": 0.1, "limit": {"output": 8192}}

For more fragile clients like Cline, 4096 output tokens may be a better starting point.

Practical Conclusions

Current Status and Next Steps