Thread: Strata: 12GB VRAM + 32~64GB RAM to run 125B Qwen3.8-Flash-Next
View Single Post
  #4  
Old 10-14-2026, 12:18
WhoCares's Avatar
WhoCares WhoCares is offline
who cares
  
Join Date: Jan 2002
Location: Here
Posts: 492
Rept. Given: 11
Rept. Rcvd 32 Times in 25 Posts
Thanks Given: 82
Thanks Rcvd at 280 Times in 107 Posts
WhoCares Reputation: 32
There is a fork for Strata:

https://github.com/zsariboga/Strata
Strata v0.1.43 tuned for a 12 GB laptop (RTX 5070 Ti Laptop, 64 GB RAM): Swift 1.5 IQ3_XXS ~140 tok/s engine decode, vision on. Branch laptop-12gb, report in docs/LAPTOP_12GB_TUNING.md

With the above fork, I can run Swift-Qwen3.8-Flash-Next-GSQ-RCO-IQ3_XXS with 60 token/s, 256k context, 80GB commited memory in Win11.

The abliterated model:
https://huggingface.co/SC117/Swift-1.5-Qwen3.8-Flash-Next-GSQ-RCO-abliterated-GGUF/tree/main/IQ3_XXS
__________________
AKA Solomon/blowfish.
Reply With Quote
The Following User Says Thank You to WhoCares For This Useful Post:
niculaita (10-14-2026)