I Ran Qwen3-Coder 30B on an AMD RX 9070 XT — 47 tok/s, No ROCm Required
Search for how to run a coding LLM locally and the advice converges on one word: NVIDIA. CUDA is mature, every tutorial assumes it, and saying “I’ll use AMD” reliably earns you a reply telling you not to.
I used an AMD Radeon RX 9070 XT (16GB, RDNA4) anyway. Not out of principle — it was already sitting in my gaming PC, doing nothing between sessions.
Short version: Qwen3-Coder-30B-A3B runs at 41–50 tokens/sec on it, and I never installed ROCm at all.
This isn’t a victory lap. It’s a record of numbers I had to measure myself, because I couldn’t find them anywhere.
Why AMD at all
- The card was already in the machine. Extra spend: zero.
- Coding assistance is bursty. The GPU sits idle whenever I’m not gaming, and that felt like a waste.
- And honestly: “it doesn’t work” was getting repeated a lot, without anyone showing numbers.
Why I started with Vulkan
This is the original August 2026 setup, not a current ROCm support survey. I started with Vulkan and got a working local coding model. I didn’t complete a ROCm benchmark, so I can’t rank the two backends from this experiment.
The original wording went too far: trouble getting my chosen ROCm path running became a claim about RDNA4 support in general. That wasn’t a result I had measured. Support depends on the operating system, software version, and workload. For a new installation, I check AMD’s compatibility matrix and the llama.cpp build guide.
What I actually tested
My working backend reported:
ggml_vulkan: Found 1 Vulkan devices:
ggml_vulkan: 0 = AMD Radeon RX 9070 XT (AMD proprietary driver)
| fp16: 1 | bf16: 1 | warp size: 64 | matrix cores: KHR_coopmat
That identifies the device and features reported by the backend. It doesn’t isolate why a particular run was fast. Every performance figure below comes from my Vulkan run; none comes from a ROCm comparison.
The real problem: fitting 30B into 16GB
Qwen3-Coder-30B-A3B is a Mixture of Experts model. 30B total parameters, but only 3B active per token. That architecture is the whole reason this works on 16GB.
The flag that matters is --n-cpu-moe N, which pushes the expert tensors of N layers out to system RAM. Everything hinges on where you set it.
Set it too high and you’re slow. Set it too low and you run out of VRAM. So I swept it.
The n-cpu-moe sweep (measured)
--n-cpu-moe |
Prompt processing (pp256) | Generation (tg64) |
|---|---|---|
| 48 | 71 tok/s | 21 tok/s |
| 32 | 101 tok/s | 30 tok/s |
| 24 | 121 tok/s | 35 tok/s |
| 18 | 159 tok/s | 44 tok/s |
| 14 | 180 tok/s | 47 tok/s |
-ngl 99. Averages shown; the raw log keeps deviation too — at n-cpu-moe 14: pp 179.71 ± 0.42, tg 47.42 ± 3.78.14 is the sweet spot. Going from 48 down to 14 took generation from 21 to 47 tok/s — a 2.2× gain, same model, same hardware. Below 14 it runs out of memory.
I checked the opposite extreme too. With --n-cpu-moe 99 — every expert on the CPU — generation dropped to 17.9 tok/s.1 So this one flag spans roughly 2.6×.
VRAM at the sweet spot sits at 15.08GB of 15.9GB usable. A 0.8GB margin. It’s a tightrope.
1 That last figure came from an interactive session rather than llama-bench, so it isn’t strictly comparable to the table. Treat it as a rough floor.
Reproduce the short benchmark
The recorded setup was Windows, an RX 9070 XT with 16GB VRAM, a Ryzen 7 5700X3D, and 32GB system RAM. I used Qwen3-Coder-30B-A3B-Instruct UD-Q4_K_XL with llama.cpp build b9587 and the Vulkan backend.
llama-bench --version
llama-bench -m MODEL.gguf -ngl 99 -ncmoe 14 -p 256 -n 64
MODEL.gguf is a placeholder for the local model file. The command measures a short pp256/tg64 benchmark. It does not measure a full 16k prompt: the 16k setting in the next section belongs to my deployed llama-server configuration.
I didn’t retain the exact graphics-driver version or model checksum in this article’s record. That limits exact reproduction. For a new comparison, I save those identifiers and the full benchmark output, use the same binary and model for every row, close other GPU workloads, and change only ncmoe.
The official benchmark documentation explains the output. My CPU offload tuning article has the full sweep, including the second card.
Final configuration
-ngl 99 --n-cpu-moe 14 -c 16384 --parallel 1 --jinja
| Backend | llama.cpp llama-server (Vulkan) |
| Model | Qwen3-Coder-30B-A3B-Instruct UD-Q4_K_XL (16.45GB) |
| Context | 16k |
| VRAM | 15.08 / 15.9 GB |
| Generation | 41–50 tok/s |
| Prompt | 180 tok/s |
Why 16k context: the model supports far more, but context eats VRAM I don’t have. 16k covers ordinary coding work, and anything that needs a huge context goes to a different tool anyway — which brings me to how I actually use this thing.
How I actually use it: split the roles
Trying to do everything with the local model is a mistake. I split the work instead:
| Stage | Who | Why |
|---|---|---|
| Planning & architecture | Cloud model (subscription) | Needs broad context and stronger reasoning |
| Writing code | Local Qwen3-Coder | High volume, repetitive — this is where token costs explode |
| Review & integration | Cloud model | Backstop |
The editor talks to the local OpenAI-compatible endpoint through Continue.dev.
I moved code generation local because it was a large part of my interactive workload. That didn’t establish that orchestration was cheap. My later loop measurements found more output on the cloud-driver side. These were subscription sessions, not a metered API bill I eliminated.
What’s actually bad about this
An honest list, because I haven’t seen one:
- Quality sits below frontier cloud models. Give it a narrow, well-specified task and it does fine. Give it something vague and it wanders.
- 30B is too slow for inline autocomplete. A much smaller model is the right tool there.
- The GPU is exclusive. If I’m gaming, I’m not running the model. There’s only one card.
- 0.8GB of VRAM headroom. One careless setting change and it dies.
- ROCm is unmeasured here. This article doesn’t establish which backend is fastest today.
What these measurements support
The card I already owned was usable for this model and configuration. That’s the result I can defend. This isn’t a current GPU buying guide, and two machines don’t establish how every AMD or NVIDIA card behaves.
Dense 30B models can use CPU offload too. I didn’t test one here, so this MoE result doesn’t establish their speed or usability. My useful finding was the measured configuration, not a universal rule about what 16GB can run.
Closing
My short Vulkan benchmark reached 47 tok/s in this configuration. I didn’t run the ROCm comparison, so the result ends there.
I published the numbers because they didn’t exist when I went looking.
Next up: there’s a question this post doesn’t answer — which matters more, CUDA maturity or VRAM capacity? So I ran the same model on an RTX 2080 Ti (11GB): seven years old, fully mature CUDA, 5GB short on memory. The results split in a way I didn’t expect. Read that one here.
Related: the
--n-cpu-moevalue used here (14) was found by sweeping it. If you want the method rather than my numbers, see the tuning guide.
Updated October 6, 2026: added the benchmark recipe and narrowed backend-support claims to what I tested. Original measurements are unchanged.
Editorial correction, November 4, 2026: Narrowed the cost claim to my workload and linked the later result. Original measurements are unchanged.