text-generation-webui

Text-generation-webui gives you a browser-based interface to do it. Load a model, chat with it, fine-tune it, generate images. It's the Swiss Army knife for local AI.

The entire project is free under AGPL-3.0. Every feature (chat, notebook mode, model loading, LoRA training, multimodal/vision support, extensions) ships at $0. The developer sells some extension packs on Gumroad, but those are optional add-ons, not core features.

Self-hosting is the only option, and the setup complexity depends on your GPU situation. If you have an NVIDIA card with 8GB+ VRAM, the one-click installers work well. AMD and Apple Silicon support exists but can be finicky. Expect 30-60 minutes for first-time setup including downloading a model. Solo developers: this is your playground. Run models locally, experiment with fine-tuning, keep your data private. Small teams: share a beefy GPU server running the API mode. Beyond that, look at dedicated inference servers like vLLM.

The catch: GPU hardware requirements are real. You need a decent GPU to run anything useful. A 7B parameter model needs ~6GB VRAM. Anything bigger needs proportionally more. No GPU, no party.