Strata

Strata runs Qwen3.8-Flash-Next, a very large open-weight model, on a gaming PC. The model has 125B parameters but only 6B do work on each token, so Strata keeps the busiest parts on your graphics card and the rest in system RAM, where the CPU handles them. It serves an OpenAI- and Anthropic-compatible API on localhost. The engine is MIT and free; the model weights carry Qwen's own community license.

RAM is the real requirement, not the GPU. Plan on an NVIDIA RTX 30, 40 or 50 card with 12 GB or more (8 GB runs, slowly), 48 to 64 GB of system RAM (32 GB for the coding-only variant), and 70 to 80 GB of free disk. On the author's RTX 5070 with 64 GB of DDR5, it writes about 40 to 90 tokens per second, depending on model size and context length. It answers one request at a time.

Solo developers with the hardware: a way to run a frontier-sized open model locally at $0 per token. For general local models, llama.cpp (ggml-org/llama.cpp) and Ollama (ollama/ollama) are the mature picks.

The catch: the project is days old with one maintainer, and the Windows installer downloads a hand-built strata.exe and runs it without checking a hash. Cautious users should build from source (--build). Also read the setup prompts: the "experimental speed projection" it offers is, by its own docs, a vector that strips the model's refusals, not a speed boost.