Strata: 12GB VRAM + 32~64GB RAM to run 125B Qwen3.8-Flash-Next

[WhoCares]

Strata is very hot recently.

3-tier: VRAM -> RAM -> SSD

https://github.com/Niko1221/Strata

[chants]

Wow their practical benchmarks are in the 40 to 100 tokens per second range. Could absolutely do some useful work with a rig like that.

NVIDIA RTX 3060 12GB $280
64GB DDR5 (2x32GB) 6000MHz CL30/CL34 $180
AMD Ryzen 5 7600 or Intel i5-12400 $180
1TB NVMe M.2 SSD (Gen 4 preferred) $60
Budget B650 (AMD) or B760 (Intel) $120
650W–750W 80+ Bronze PSU & Budget Case $110

So around $930 to build a new rig and if buying the GPU used maybe even $820. That is not bad at all. I can see why it is hot.

[WhoCares]

It's surprisingly usable on my RTX 4070 + 64GB RAM + 5800X3D, though it's definitely a bit slow.

I need a 256K context window for Claude Code CLI to chew through Strata's own source repo with IQ2_XS. 128K just doesn't cut it.

[WhoCares]

There is a fork for Strata:

https://github.com/zsariboga/Strata
Strata v0.1.43 tuned for a 12 GB laptop (RTX 5070 Ti Laptop, 64 GB RAM): Swift 1.5 IQ3_XXS ~140 tok/s engine decode, vision on. Branch laptop-12gb, report in docs/LAPTOP_12GB_TUNING.md

With the above fork, I can run Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS with 60 token/s, 256k context, 80GB commited memory in Win11.

The abliterated model:
https://huggingface.co/SC117/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF/tree/main/IQ3_XXS

[Farooq Ali]

Originally Posted by WhoCares

There is a fork for Strata:

With the above fork, I can run Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS with 60 token/s, 256k context, 80GB commited memory in Win11.

The abliterated model:
https://huggingface.co/SC117/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF/tree/main/IQ3_XXS

Any updates on how this is working out in practice?
Is it good for reversing and coding? Or just as a general purpose LLM?