Run and serve large language models: local inference, production serving, and model management.
Ranked by Discovery Score — our 0-100 composite of adoption, maintenance, community, and license health. How scoring works.
By Erik Loyd, SaaS CEO and former COO/CFO of an AWS Premier Partner.
Ollama is the easiest way to run open models on your own machine: one install, then ollama run and a model name, and you're chatting locally with an OpenAI-compatible API on localhost. It runs on Mac, Windows and Linux, uses NVIDIA, AMD and Intel GPUs, and Metal on Apple Silicon. The app is MIT licensed and free.
Local use needs no account. Under the hood it now reads standard GGUF files through llama.cpp, Vulkan support brings in AMD and Intel cards, and the next release makes Apple's MLX engine the default on Apple Silicon. NVIDIA cards need compute capability 5.0+ and a recent driver. Ollama Cloud, its hosted side, runs bigger open models you can't fit locally, from the same CLI.
Cloud moved to per-token pricing with monthly credits on August 31, 2026. Free gets starter credits for starter models, one request at a time. Pro is $20 a month ($200 a year) with $60 of usage credits and 3 concurrent requests. Max is $100 a month with $300 of credits and 10 concurrent requests. Team is $500 a month in early access with $1,000 of shared credits. Credits don't roll over. Solo devs: local stays free. Pay only when you need models your hardware can't run.
The catch: convenience costs control. Ollama picks quantizations and defaults for you, which is great until you need to tune something llama.cpp (ggml-org/llama.cpp) exposes directly. And now that it's venture-funded, watch how much of the roadmap tilts toward the cloud.
Local is free forever. Ollama Cloud starts at $20 a month with $60 of token credits.
**Free:** Local Ollama, MIT licensed, no account, no limits beyond your hardware. Cloud Free plan: starter credits for starter models, 1 concurrent request. **Self-hosted:** Native installers for macOS, Windows and Linux, or Docker. NVIDIA needs compute capability 5.0+ and driver 550+; AMD via ROCm v7 or Vulkan; Apple via Metal. **Paid (Ollama Cloud, per-token metering since 2026-08-31, USD):** Pro $20/month or $200/year, $60 of usage credits a month, 3 concurrent requests. Max $100/month, $300 of credits, 10 concurrent requests. Team $500/month (early access), unlimited users, $1,000 of shared credits. Enterprise custom. Credits don't roll over; overage bills at the same per-token rates, with off-peak discounts. Legacy subscribers can keep their old plans.
RAGFlow is a retrieval-augmented generation engine built around deep document understanding, and it has grown into an agent platform on top of that. It handles the messy parts of RAG other frameworks hand-wave past: PDFs with tables, slides, spreadsheets and scanned images, with MinerU and Docling parsing. Recent releases added knowledge compilation into wikis, graphs and mind maps, agent memory, and delivery into chat apps like Discord, Telegram and Line. Apache 2.0.
Self-hosting is a real project, not a docker run. You need 4+ CPU cores, 16+ GB RAM, 50+ GB disk, and x86 hardware, since there are no ARM64 images. The compose file brings up RAGFlow plus MySQL, Redis, object storage and a search backend: Elasticsearch by default, with Infinity, OpenSearch and OceanBase among the options. Template-based chunking is still the killer feature: you see and adjust how documents are split before they reach the index.
Solo developers and small teams that want full control can self-host for free. Everyone else can use RAGFlow Cloud, currently $29/mo for 5 seats on Starter or $129/mo for 20 seats on Pro, with Enterprise adding on-premises or bring-your-own-cloud deployment.
The catch is operational weight. Just chatting with a few documents? LlamaIndex or LangChain over a simple vector store ships faster. Note that v0.27 pulled the old GraphRAG and RAPTOR features from the UI, so check tutorials against your version.
Free under Apache 2.0 if you can run the stack. RAGFlow Cloud is currently $29/mo for 5 seats or $129/mo for 20.
**Free tier:** Apache 2.0 when self-hosted: deep-document parsing (tables, slides, spreadsheets, scans) with MinerU and Docling, template chunking with visual review, traceable citations, agentic RAG, knowledge compilation (wiki, graph, tree, mind map, timeline), agent memory, data connectors, chat-channel delivery, and an MCP server that is off by default. RAGFlow Cloud also has a free plan: 5 apps, 1 seat, 0.1 GB storage and 500 credits a month, without API access. **Self-hosted:** Real ops. 4+ cores, 16+ GB RAM, 50+ GB disk, Docker 24+ with Compose 2.26.1+, `vm.max_map_count` of at least 262144, and x86 hardware only. The stack runs RAGFlow with MySQL, Redis, object storage and a search engine (Elasticsearch by default; Infinity, OpenSearch, OceanBase and others supported). gVisor is needed only for the code executor sandbox. **Paid:** RAGFlow Cloud Starter is currently $29/mo (listed down from $59) for 50 apps, 5 seats, 5 GB storage, 5,000 credits and API access. Pro is currently $129/mo (down from $259) for unlimited apps, 20 seats, 50 GB and 20,000 credits. Enterprise is custom, with bring-your-own-cloud or on-premises deployment, dedicated support and a custom SLA.
DeepInfer is the inference engine behind DeepSeek's own production API, now open source under Apache-2.0. That's the software that loads a huge model onto GPUs and turns prompts into answers fast and cheap. It sits in DeepSeek's official GitHub org next to FlashMLA and DeepGEMM, so it's the real thing, and it's free. Free to download is not the same as runnable. It supports one model today, DeepSeek-V4.1-Flash, on two kinds of hardware: NVIDIA SM100 GPUs (Blackwell datacenter parts like the B200) and Huawei Ascend 950 NPUs. No H100s, no gaming cards. You build from source on Linux with CUDA 13, examples assume an 8-GPU node, and DeepSeek's own guide puts the smallest efficient production unit at 128 GPUs. There's no paid tier. The bill is the hardware. Solo and small teams: skip it and use DeepSeek's API, vLLM or SGLang. Larger teams serving DeepSeek models on Blackwell or Ascend: benchmark it against vLLM. The catch: this release is offline inference only. The README says a KV cache system, prefill/decode disaggregation and load balancing are not included, which is most of what turns an engine into a service. You get DeepSeek's engine, not DeepSeek's platform.
Free software that only runs on NVIDIA SM100 (Blackwell datacenter) GPUs or Huawei Ascend 950s, and DeepSeek sizes an efficient deployment at 128 GPUs. Almost everyone is better off with DeepSeek's API or vLLM.
**Free:** All of it. DeepInfer is Apache-2.0, and the default checkpoint it downloads (DeepSeek-V4.1-Flash-1017 on Hugging Face) is MIT-licensed. Included: optimized kernels, expert and data parallelism, DSpark speculative decoding, batch-invariant and deterministic inference (NVIDIA only) and structured outputs (regex, JSON schema, EBNF). **Self-hosted:** Linux, GCC 13+, Python 3.12, and CUDA 13 (NVIDIA) or CANN 9.2 (Ascend). Supported hardware is NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs only. Build from source with `./build.sh`; examples run with `--ep 8`. DeepSeek's published benchmarks used 4 SM100 GPUs for prefill and 64 for decode, and its deployment Q&A calls 64 prefill plus 64 decode GPUs the smallest efficient unit. Not included: a KV cache system, prefill/decode disaggregation, and load balancing across replicas. **Paid:** No paid tier. The real cost is Blackwell-class or Ascend 950 hardware, owned or rented. For most teams DeepSeek's hosted API, or vLLM/SGLang on hardware you already have, is the practical route.
llama.cpp is the engine that runs large language models on ordinary hardware: a laptop CPU, an Apple Silicon Mac, a gaming GPU, even a phone. It loads GGUF model files, the format most local-AI tools now share, and sits underneath a lot of the apps people use to run models locally. MIT licensed, and the founding ggml team joined Hugging Face in February 2026.
You can still build from source, but you don't have to: versioned releases started in August 2026, there's an installer at llama.app, a llama serve command, and a free desktop app for Mac and Windows. The built-in server speaks the OpenAI API and now Anthropic's Messages API too. Hardware support is wide, from CUDA, ROCm and Metal to Vulkan, SYCL and WebGPU.
Free at every size. Solo devs who want a local model with no middleman get the most control here; Ollama (ollama/ollama) is the friendlier wrapper. Teams serving many users at once should look at vLLM (vllm-project/vllm), which is built for throughput.
The catch: model size meets hardware fast. A 70B model at the common 4-bit quantization is a roughly 43 GB file, so it won't fit on a single 24 GB GPU and has to split between GPU and system RAM, which is slow. Expect to spend real time picking quantizations and tuning flags.
Completely free under MIT. Your cost is hardware with enough memory for the models you want.
**Free:** Everything. MIT licensed: the library, CLI, server and installer. **Self-hosted:** Runs on your own machine. Prebuilt releases and the llama.app installer, or build from source. Backends include CUDA, HIP/ROCm, Metal, Vulkan, SYCL, CANN, MUSA, OpenCL, WebGPU and others. The real cost is hardware: memory has to hold the model file plus context, so bigger models mean bigger GPUs or slow CPU offloading. **Paid:** None.
vLLM is the open source server teams use to serve language models to lots of users at once. Its PagedAttention memory management and continuous batching squeeze far more requests out of a GPU than a naive setup: the original benchmarks showed 14 to 24 times the throughput of plain Hugging Face Transformers and 2.2 to 3.5 times an optimized server like TGI. Apache 2.0, and a PyTorch Foundation project. It exposes an OpenAI-compatible API and runs on NVIDIA, AMD ROCm (with prebuilt wheels), Intel XPU, Google TPUs and Apple Silicon through a plugin. Getting the best out of it means matching CUDA versions, sizing GPU memory and tuning batch settings. A 70B model at full precision needs several data-center GPUs; quantized models shrink that. The software is free; your bill is GPUs. On-demand A100s run roughly $1.59 to $2.79 an hour on RunPod and Lambda, and an 8-GPU AWS p4d instance is about $22 an hour. Solo devs running one model for themselves should use Ollama (ollama/ollama) or llama.cpp instead. Teams serving real traffic: this is the default choice. The catch: it's a big, fast-moving codebase with a matching security record. 33 advisories landed between August and early October 2026, including remote code execution through request parameters, patched in 0.31.0. Pin versions, upgrade deliberately, and keep a gateway in front of it. Its creators also founded Inferact in January 2026, so watch for a commercial layer.
Completely free under Apache 2.0. Your bill is GPU hours.
**Free:** Everything. Apache-2.0, no paid edition. **Self-hosted:** Your cost is GPU capacity. Reference on-demand prices: A100 roughly $1.59/hr on RunPod and $1.99 to $2.79/hr on Lambda; AWS p4d (8x A100) about $21.96/hr and p5 (8x H100) about $55.04/hr. Supports NVIDIA, AMD ROCm, Intel XPU, TPU and Apple Silicon (via vllm-metal). **Paid:** None from the project. Inferact, founded by vLLM's creators, has published no pricing.
CLIProxyAPI turns coding-agent subscriptions you already pay for (ChatGPT Codex, Claude Code, Antigravity, Grok Build, Devin, Kimi, Meta's Muse Code) into an OpenAI-, Gemini- and Claude-compatible API you can point other tools at. Log in once per account and it serves those quotas as an endpoint, with round-robin load balancing across multiple accounts. MIT licensed, a Go binary or Docker. Setup is a single binary or docker compose, a config file, and OAuth logins per provider. It stores those tokens on the proxy host and exposes a management API, so lock it down if it ever leaves localhost. Releases ship almost daily, and Gemini CLI support moved into a separate plugin. There's no paid tier; the cost is the subscriptions you proxy, and the risk is those accounts. Anthropic's terms say Claude plan credentials can't be routed through third-party apps, and Google's Antigravity terms call third-party access grounds for suspension. Google already ran a ban wave in February 2026 that also locked people out of Gemini CLI. Use the official APIs for anything in production. The catch: this exists to stretch consumer plans past their intended use, and the README's sponsor wall now advertises residential proxies with one IP per account for multi-account operations. That's account-ban territory, and you're the one holding the account.
Free under MIT, but it runs on subscription accounts whose terms prohibit this use. The price is the risk of a ban.
**Free:** Everything. MIT licensed, self-hosted Go binary or Docker. **Self-hosted:** Single binary or docker compose plus a config file. Stores provider OAuth tokens on the host and exposes a management API. Usage statistics require third-party add-ons since v6.10. **Paid:** None from this project. You pay for the subscriptions you proxy. The org also publishes a separate SSPL-licensed Business edition repo with no published price. The real cost is account risk: Anthropic and Google terms prohibit routing subscription credentials through third-party tools, and Google has suspended accounts for it.
LocalAI is a self-hosted stand-in for the OpenAI API. It runs text, vision, speech, image and video models on your own hardware, CPU-only if that's all you have, and answers on OpenAI, Anthropic, Ollama and ElevenLabs-compatible endpoints, so most apps only need a new URL. MIT-licensed and free.
One docker run or the macOS app gets it going. The core stays small: each engine (llama.cpp, vLLM, whisper.cpp, MLX and more) is pulled as a separate backend only when a model needs it, and models install from a gallery, Hugging Face or the Ollama registry. It's well past a hobby server now: API keys, OIDC logins, per-user quotas and a distributed mode that spreads models across GPU machines.
There's no paid tier; hardware is the bill. Solo: free, though Ollama is simpler for one laptop. Small and larger teams: free, and the quotas and cluster mode make it a real internal AI gateway.
The catch: out of the box there's no login. Unless you set an API key or turn on user auth, anyone who can reach the port can use it. A critical unauthenticated SSRF bug was patched in July. Set LOCALAI_API_KEY, stay current, and never expose it raw.
Free with no paid tier. Hardware is the only cost, and auth is off until you configure it.
**Free:** Everything, under MIT. Text, vision, speech-to-text, text-to-speech, image, video and 3D generation, built-in agents with MCP and RAG, multi-user auth with API keys or OIDC, per-user quotas and distributed mode all ship in the open source build. **Self-hosted:** The only way to run it: Docker images for CPU, NVIDIA CUDA 12/13, AMD ROCm, Intel oneAPI, Vulkan and Jetson, or an unsigned macOS DMG. Backends download on demand when a model needs them. Your cost is hardware: CPU-only works for small models, a GPU makes larger models practical. Authentication is off unless you configure `LOCALAI_API_KEY` or `LOCALAI_AUTH=true`. **Paid:** No paid tier and no hosted offering. The project takes GitHub Sponsors and lists a business contact (contact@localai.io), with no published support plans or prices.
llmfit answers the question everyone running local AI asks first: will this model fit on my machine? It reads your CPU, RAM, and GPU, then scores every model in its catalog on memory fit, estimated speed, quality, and context length. No more downloading a huge model to find out it will not load. MIT licensed, free, one Rust binary. Install with Homebrew, Scoop, MacPorts, uv, Docker, or a curl script, on macOS, Linux, or Windows. It detects NVIDIA, Apple Silicon, AMD ROCm, and Intel GPUs, understands GGUF, AWQ, GPTQ, and EXL2 quantization, and works with Ollama, llama.cpp, MLX, Docker Model Runner, and LM Studio. No account, and the README says nothing leaves your machine unless you ask. There is also a web dashboard and a REST API through llmfit serve. Free at every size with no paid tier. Solo builders get the most out of it. Teams can run the API on shared GPU boxes to see what fits where. Once you have picked a model, ollama/ollama or ggml-org/llama.cpp does the actual running. The catch: speeds are estimates from a formula unless you run the benchmark against a live runtime, and the GPU bandwidth table only covers about 80 cards. Windows and Intel Macs get weaker detection. The model catalog is baked in at compile time, so new models only show up when you update.
Completely free and open source.
**Free:** Everything. MIT licensed CLI and terminal UI, a web dashboard, a REST API, and a built-in catalog of hundreds of models sourced from Hugging Face. **Self-hosted:** A single binary on your own machine. No account, no model download needed to get recommendations. The benchmark command needs a local runtime like Ollama already running. **Paid:** None. No hosted version, no paid tier, no sponsor-gated features.
SGLang has become one of the default engines for serving open-weight LLMs in production at scale. It runs from a single GPU up to large clusters, with prefix caching, speculative decoding, and day-zero support for new model releases. The project says it runs on hundreds of thousands of GPUs, at xAI, NVIDIA and the major clouds among others. Apache-2.0, free.
Install is a uv pip install, with builds for NVIDIA, AMD, Intel, Google TPUs and Ascend NPUs. Models span Llama, Qwen, DeepSeek, Kimi, GLM, Mistral, diffusion models, and most of Hugging Face. It is OpenAI-API compatible, so existing clients drop in without rewrites, and it doubles as a rollout backend for reinforcement learning training.
Solo developers and small teams running open-weight models: this is one of the strongest options on the shelf, especially on DeepSeek or with heavy agentic workloads. Large teams running production inference at scale: you are very likely already evaluating it.
The catch is that serious inference is still serious work. Cold starts, KV cache tuning, and multi-node setups need real engineering. SGLang gives you a faster engine, not a managed platform. RadixArk, a venture-funded company among its maintainers, is building one on top, with no public pricing yet.
Free Apache 2.0 inference engine used widely in production. Hardware and ops are the cost.
**Free:** Apache 2.0. High-throughput LLM serving from one GPU to clusters, prefix caching, speculative decoding, day-zero model support, an OpenAI-compatible API, and use as an RL rollout backend. The project is hosted under the non-profit LMSYS organization. **Self-hosted:** The way it runs. Broad hardware: NVIDIA, AMD, Intel, Google TPU and Ascend NPU. Real ops for cold starts, KV tuning, and multi-node. **Paid:** No paid tier for SGLang itself. RadixArk, a company that co-maintains SGLang, sells managed inference infrastructure built on it, with no public pricing. Hardware is otherwise the only cost.
Laya answers typed questions about text without generating any: pick one of these labels, give this a score, yes or no. Hand it a support ticket or an email and it returns probabilities in a single pass. It's an open-weight rival to TypeSafe's hosted Jev model, and it copies Jev's API, so an existing client switches by changing one URL. Apache 2.0, code and weights. It runs on CPU. The README puts one question at roughly 200 to 460 milliseconds on CPU and about 33 on a T4 GPU. There are three checkpoints of 322M to 421M parameters, one covering 100+ languages, plus a built-in HTTP server and MCP server. Jev charges $42 per billion input tokens; this costs whatever hardware you already have. mizorewww/laya-mlx ports it to Apple Silicon. Teams routing tickets, triaging email or gating agent actions at high volume: self-host it and fine-tune it. The README is blunt that out of the box it's near chance on real workflows, and says to treat it as a fast base to specialise, not a zero-shot engine. A fine-tune takes about 4-5 hours on Kaggle's free GPUs. The catch: its Jev comparisons use Jev's published numbers, not side-by-side runs, and with many options Jev wins clearly. Confidence scores behave differently too, so thresholds tuned on Jev need re-tuning.
Completely free, Apache 2.0 code and weights. Runs on CPU; the cost is your hardware and a fine-tune.
**Free:** Everything. Code and all three checkpoints (laya, laya-multilingual, laya-typed-decisions) are Apache 2.0, on top of Apache and MIT base encoders, so commercial use is clear. **Self-hosted:** A Python package with an optional HTTP server. Runs on CPU with no GPU; a GPU cuts latency roughly tenfold. Budget about 1.5-2.5 GB of disk to preload all three checkpoints. Fine-tuning fits Kaggle's free dual-T4 tier in about 4-5 hours. **Paid comparison:** TypeSafe's hosted Jev, the API Laya imitates, is $42 per billion input tokens with output free. At that price, the case for self-hosting Laya is keeping your data in-house more than saving money.
LiteLLM gives you one OpenAI-style API in front of 100+ model providers: OpenAI, Anthropic, Gemini, Bedrock, Azure, Ollama and the rest. Use it as a Python SDK, or run the proxy as a gateway with virtual keys, per-team spend tracking, budgets, rate limits, fallbacks and load balancing. The core is MIT; an enterprise folder is proprietary. Self-host the proxy with Docker and Postgres. It moves fast: a new minor version roughly every week with only the four most recent supported, so a pinned version falls out of support in about a month. A Rust rewrite is underway, targeting the end of 2026. Overhead is small for normal requests (about 8 ms at p95 in their own benchmark), but the Python proxy struggles with very large prompts under load. Everything most teams need is free, and SSO is free for up to 5 users. Enterprise adds SSO beyond that, SCIM, audit logs, secret managers, IP allowlists, team-scoped guardrails and 24/7 SLAs, priced by annual request volume with no public numbers. Small and mid-size teams: free. Large orgs with compliance needs: talk to sales. The catch: it holds every API key you own, and its security record demands attention. Two poisoned releases (1.82.7 and 1.82.8) were live on PyPI for about 40 minutes in March 2026, and critical advisories kept coming through September. Use the official Docker image, pin it, and patch fast.
Free MIT core covers most teams, with SSO free up to 5 users. Enterprise is quote-only, sized to request volume.
**Free:** The SDK and proxy under MIT (everything outside the enterprise/ directory): 100+ providers behind one API, virtual keys, users and teams, spend tracking, budgets and rate limits, fallbacks, logging, Prometheus metrics, custom guardrails and Presidio PII masking. SSO free for up to 5 users. **Self-hosted:** Docker plus Postgres. The only deployment model; LiteLLM runs no hosted gateway for you. Weekly minor releases, four supported at a time. **Paid (Enterprise):** Annual license sized to request capacity, deployment and support needs, never per token; no public price, 30-day trial key available, also via AWS Marketplace. Adds SSO beyond 5 users, SCIM, OIDC/JWT auth, audit logs, secret managers and key rotation, org and team admins, multi-region control plane, IP allowlists, tag-based budgets, built-in moderation callbacks, 24/7 support and SLAs.
Unlimited-OCR is Baidu's open model for reading documents. Point it at a scan, a screenshot or a long PDF and it returns the text with layout tags in one pass instead of stitching pages together. It builds on DeepSeek-OCR, and code and weights are both MIT, so commercial use is free.
Running it yourself means an NVIDIA GPU. It's a 3.3-billion-parameter model (about 6.7GB of weights), served through Hugging Face transformers, vLLM's prebuilt Docker image, or a custom SGLang wheel the repo ships. There's no app or UI: you write the script that feeds it pages.
Baidu Cloud now hosts it as an API, free for a limited time (200 test pages for verified individuals, 1,000 for companies), but signup means real-name verification on a Chinese cloud. Solo: free on a rented GPU, or the Hugging Face demo. Small teams: one GPU box behind vLLM. Larger teams: add a queue, or stay on AWS Textract if you need forms and tables pre-extracted.
The catch: loading it runs Baidu's own model code (trust_remote_code=True), and the hosted API sends your files to Baidu. Self-host sensitive documents, pin a model revision, and test on your own messy scans, since the benchmarks are Baidu's.
Free and MIT for both code and weights; your only cost is the GPU. Baidu Cloud's hosted API is free for now (200 test pages for individuals, 1,000 for companies) but has no published price and needs real-name verification.
**Free:** The code on GitHub and the model weights on Hugging Face are both MIT licensed (each repo carries its own MIT LICENSE file, copyright Baidu 2026). Free to download, modify and use commercially. A free demo runs on Hugging Face Spaces, and the model is also mirrored on ModelScope. **Self-hosted:** The real way to run it. You supply an NVIDIA GPU; the README's transformers path is tested on Python 3.12.3 and CUDA 12.9. The model is about 3.3B parameters, roughly 6.7GB of BF16 weights. Three serving options: plain transformers (`model.infer` for a single image, `model.infer_multi` for multi-page and PDF), vLLM with official images (`vllm/vllm-openai:unlimited-ocr` on CUDA 13.0, `unlimited-ocr-cu129` for Hopper GPUs), or SGLang via a pre-release SGLang wheel bundled in the repo, which exposes an OpenAI-compatible streaming API. `infer.py` batch-processes an image folder or a PDF with concurrent requests. No license fee; you pay for the GPU. **Paid:** Baidu Cloud sells a hosted Document Parsing (Unlimited-OCR) API that returns JSON or Markdown. It is currently free for a limited time: 200 free test pages for real-name-verified individuals, 1,000 for verified companies. It is asynchronous (submit, then poll), PDFs up to 500 pages, and Baidu has not published a per-page price yet. Your files go to Baidu's servers.
oMLX runs large language models on your Mac and manages them from the menu bar. You get a local API that speaks both the OpenAI and Anthropic formats, so Claude Code, Codex or Cursor can point at your own machine instead of a paid cloud model. Apache 2.0, no account, no paid tier. The reason to pick it over Ollama is the cache. Coding agents keep resending the same long prompt, and oMLX saves that work to your SSD, so a returning context loads from disk instead of being recomputed, even after a restart. The signed DMG auto-updates and ships the fast native kernels prebuilt. Solo developers running a local coding assistant get the most from it. Small teams can share one big Mac over the API; it refuses network binds without an API key. Larger teams belong on vLLM and real GPUs. The catch: Apple Silicon, macOS 15 or newer, and the project calls 16GB of RAM the minimum with 64GB recommended. Homebrew and source installs without full Xcode skip the native kernels, and GLM-5.2 prefill then runs roughly 30x slower. Take the DMG. It's also one person's after-hours project.
Free under Apache 2.0 with no paid tier. The real cost is the Mac: Apple Silicon, macOS 15 or newer, and the project recommends 64GB of RAM for comfortable use.
**Free:** Everything. Apache 2.0, no account, no license key, no paid tier. The DMG, the Homebrew formula and a source build are the same software; the DMG just ships the native Metal kernels precompiled. **Self-hosted:** Your Mac. Apple Silicon (M1 through M5), macOS 15.0 or newer, and Python 3.11 to 3.13 for source installs. The project's own FAQ puts the floor at 16GB of RAM and recommends 64GB or more for larger models. Weights download free from Hugging Face. Building the native kernels needs full Xcode with the Metal toolchain; Command Line Tools alone won't do it. Watch your disk: the SSD cache defaults to `auto`, which can use up to half of your free disk space. **Paid:** There is no paid tier and no donation link in the README. Spend money when you outgrow one machine: a hosted model API or rented GPUs beats buying a second Mac for occasional heavy jobs.
DwarfStar (the repo is still named ds4) is antirez's local inference engine for running a short list of big open models on hardware you own. The creator of Redis built it first for DeepSeek V4 Flash, and it now also runs DeepSeek V4.1 Flash, V4 PRO, the GLM 5.x family, and Qwen3.8 Flash Next on Metal, behind an OpenAI- and Anthropic-compatible server. It only loads the GGUF files the project itself produces. MIT, free.
The price is hardware, not software. Macs with 96 GB or more are the primary target, and smaller machines can stream weights off SSD. On Linux it runs on CUDA, including older Ada Lovelace cards like the L40S, plus DGX Spark and Strix Halo boxes. Setup is a download script and make. Weight files run to hundreds of gigabytes, so budget a big, fast SSD too.
Beyond the chat CLI it ships ds4-agent, a built-in coding agent, plus image input, disk KV caching and inference split across machines. A solo developer with a loaded Mac Studio gets a near-frontier model with nobody else in the loop. Everyone else should stay on llama.cpp or vLLM until they own the metal.
The catch is churn. antirez calls it beta quality, warns that regressions are possible and that models get dropped when better ones arrive, and says openly it was built with strong help from AI coding agents. Run it to own your inference stack, not to ship a product on it.
Free under MIT. The bill is a 96 GB-plus Mac or serious GPUs, plus hundreds of gigabytes of fast SSD.
**Free tier:** MIT licensed, with retained llama.cpp and GGML pieces also under MIT. Supports DeepSeek V4 Flash (including an experimental vision model), DeepSeek V4.1 Flash, DeepSeek V4 PRO, GLM 5.2, GLM 5.3 and GLM 5.3 Flash, and Qwen3.8 Flash Next on Metal. Includes the CLI, the `ds4-agent` coding agent, an OpenAI- and Anthropic-compatible HTTP server, SSD weight streaming, disk KV cache, and distributed inference across machines. It only runs the project's own GGUF files, not arbitrary models. **Self-hosted:** The only way it runs. Metal on Macs with 96 GB or more is the primary target; a 64 GB Mac can run Flash Q2 with SSD streaming. CUDA covers DGX Spark and multi-GPU Ada Lovelace systems such as L40S cards, and ROCm covers Strix Halo machines like the Framework Desktop. Two 128 GB Macs over RDMA can split a 4-bit Flash model with tensor parallelism. Weight files are very large (the DeepSeek V4.1 Flash Q2 download is 341 GiB), so fast storage is part of the bill. **Paid:** None. The cost is the hardware.
mlx-lm runs and fine-tunes large language models directly on a Mac. Point it at a model on Hugging Face and one command pulls it down and runs it locally, using Apple's own MLX engine instead of a cloud API or a separate GPU rig. MIT licensed, free, and built by Apple's own ml-explore team, the same group behind MLX itself. It does more than run models. You can quantize them down to 4-bit, fine-tune with LoRA or full-model training, serve with streaming and prompt caching, and even split work across multiple machines. Setup is close to trivial: pip install mlx-lm, then a single command chats with a model. The real constraint is memory. MLX uses the Mac's unified memory, so the model has to roughly fit in RAM, and pushing past that needs macOS 15 or newer plus some system tuning. And it's Apple Silicon only. No M-series chip, no mlx-lm. The honest framing on competition: this is a building block, not a finished app. llama.cpp is the closest peer and runs on more hardware; Ollama and LM Studio are more packaged and app-like, and increasingly use MLX under the hood anyway; vLLM is for datacenter GPUs, a different world. mlx-lm's edge is being the MLX-native option, which means the best raw performance on a Mac and the cleanest fine-tuning story. Solo developers and researchers on Apple Silicon: this is the fast path. Small teams can build on it; larger production serving will want something server-side. The catch is that you're trading convenience and reach for Mac-native speed. It's lower-level than Ollama, locked to Apple hardware, and capped by how much RAM you bought. Within those lines, nothing runs models on a Mac better.
Free and open source. Your only cost is owning an Apple Silicon Mac with enough RAM.
**Free:** MIT-licensed, fully open, no paid tier. Running, quantizing, fine-tuning, and serving, all free. **Cost you'll actually pay:** A Mac with enough RAM. The software is free; the hardware is the spend, and RAM is the limiting factor for model size. **The trade:** Self-hosting on a Mac you already own means no per-token cloud bill. The ceiling is your RAM, not your budget.
headroom strips the bloat out of everything an LLM agent reads before it hits the prompt, tool outputs, logs, RAG chunks, raw file dumps, conversation history. The reported cuts are large, 60 to 95 percent fewer tokens with the same answers, and the compression is reversible, so the original is cached and fetchable when the model actually needs it. Apache-2.0, local-first, and free: your content never leaves your machine. You can adopt it at whatever depth you want. Drop the Python or TypeScript SDK inline, run it as a zero-code local proxy, or wrap an agent directly, Claude Code, Cursor, Copilot, and Aider all supported, plus an MCP server. It also trims the model's output, not just its input, through verbosity steering and effort routing, and headroom learn mines your failed sessions for auto-corrections. Development is fast-moving, this is not a stale side project. For anyone running agents on metered tokens, the math is simple: less context in and out means a smaller bill, and the local proxy makes trying it nearly free of effort. Solo developers and small teams: the open Apache-2.0 build is the whole tool, use it. Larger teams wanting shared deployment get a managed enterprise offering on top, which is where the money is, cross-team memory and central management rather than a better compressor. The catch is trust in the compression. Cutting 90 percent of the tokens only works if the 10 percent kept is the right 10 percent, and reversible caching exists precisely because sometimes it is not. Watch the first few runs on real work before you wire it into production blind.
The open Apache-2.0 build is the complete tool and free; pay only for managed team deployment and shared memory.
**Free (open source, Apache-2.0):** The full compressor, local-first. SDK, proxy, agent wrapping (Claude Code, Cursor, Copilot, Aider), MCP server, reversible compression, output shaping, and headroom learn. Your data stays local. **Self-hosted:** It is local by default, nothing leaves your machine. **Paid (managed enterprise):** A hosted and managed offering for team deployments, adding shared cross-agent memory and central management. Priced for organizations.
awesome-free-llm-apis is a curated list of LLM APIs with ongoing free tiers, not one-off trial credits: Gemini, Groq, Mistral, OpenRouter, Cloudflare Workers AI, NVIDIA NIM and ten more. Each entry gives the base URL, free models, context sizes, rate limits and a link to the API key page. It's CC0. Nothing to install. It's a README maintained by Manifest, an LLM routing company that plugs its own product up top. The footnotes beat most lists: who may train on free-tier prompts, who demands real-name ID checks, which models are being retired. Our spot check of Groq, OpenRouter and Cloudflare limits matched their own docs. Solo builders and students: start here for a free prototype. Small teams: use it to pick a fallback, then pay. Larger teams: OpenRouter's free models stop at 50 requests a day until you buy $10 in credits, which tells you what these tiers are for. The catch: "permanent" is the list's word, not the providers'. Google and Mistral no longer publish numeric free limits, and some rows were never live-tested. And on October 10 someone pushed a credential-stealing workflow to this repo; it was removed the next day. Check key links before pasting a key.
The list is free and CC0. The free tiers it catalogs are prototype budgets: OpenRouter's free models stop at 50 requests a day until you buy $10 of credits, so plan the paid fallback before launch.
**Free:** The whole list, CC0. It covers 16 providers in two groups (model makers and third-party inference hosts), each with base URL, free models, context window, max output and rate limits, plus footnotes on training-on-prompts, identity verification and deprecations. **Self-hosted:** Not applicable. It's a README generated from a data file; read it on GitHub or fork it. **Paid:** None for the list. The providers it lists have paid tiers you will hit at real traffic: OpenRouter's free models allow 20 requests/minute and 50/day until you've purchased at least $10 in credits (then 1,000/day); Cloudflare Workers AI gives 10,000 Neurons/day free, then requires Workers Paid at $0.011 per 1,000 Neurons; Groq's free plan caps gpt-oss-120b at 30 requests/minute and 1,000/day.
Strata runs Qwen3.8-Flash-Next, a very large open-weight model, on a gaming PC. The model has 125B parameters but only 6B do work on each token, so Strata keeps the busiest parts on your graphics card and the rest in system RAM, where the CPU handles them. It serves an OpenAI- and Anthropic-compatible API on localhost. The engine is MIT and free; the model weights carry Qwen's own community license.
RAM is the real requirement, not the GPU. You need an NVIDIA RTX 20 to 50 series card or a recent AMD Radeon with 12 GB of VRAM or more, at least 32 GB of system RAM (enough for the coding variant only; 64 GB fits every size), and about 80 GB of free disk. On the author's RTX 5070 with 64 GB of RAM, a short chat writes roughly 50 to 95 tokens per second depending on model size. By default it answers one request at a time; you can raise that, but each answer gets slower on a 12 GB card.
Solo developers with the hardware: a way to run a frontier-sized open model locally at $0 per token. For general local models, llama.cpp (ggml-org/llama.cpp) and Ollama (ollama/ollama) are the mature picks.
The catch: the project is under three weeks old, one author writes nearly all of it, and engine releases land every day or two. The installer downloads a prebuilt engine and checks it against the SHA-256 GitHub publishes, which catches a tampered download but not a bad release, and it installs anyway if it can't get a hash. Cautious users should build from source (--build). Also read the setup prompts: the "experimental speed projection" it offers is, by its own docs, a vector that strips the model's refusals, not a speed boost.
Free engine, $0 per token, but only if you already own a 12 GB GPU and 32 to 64 GB of RAM.
**Free:** The Strata engine is MIT licensed with no paid tier, account or cloud. The model weights are not MIT: Qwen3.8-Flash-Next is under the Qwen Community License 1.0, which requires a separate license from Qwen for commercial Model-as-a-Service or AI work assistant businesses. Strata downloads third-party quantized versions (ISTA-DASLab GGUFs) from Hugging Face. **Self-hosted (your own PC):** The cost is hardware. - NVIDIA RTX 20/30/40/50 or a recent AMD Radeon (RX 7000 and 9000 series, Radeon AI PRO R9700, RX 6800/6900) with 12 GB+ VRAM and a current driver - 32 GB RAM fits the Coder variant; 48 GB fits the two smallest general sizes; 64 GB fits every size - About 80 GB free disk, SSD recommended - Windows 10/11 or Linux; two or three cards can share the model **Paid:** None. The alternative is paying per token for Qwen's hosted Qwen3.8-Flash.
Civitai is the community hub for sharing Stable Diffusion models, LoRAs, and checkpoints. Think of it as GitHub for AI image generation models: browse what others have trained, download fine-tuned versions, and share your own. Self-hosting requires Docker, Node.js 20+, PostgreSQL, and a fair bit of setup. The docker-compose gets you running but managing model storage at scale is the real operational challenge. Most people just use civitai.com directly. Useful for solo creators experimenting with image generation or teams building products on top of community models. The platform itself is free and open source. The catch: the self-hosted version is meant for development, not production. The hosted platform at civitai.com is where the community and model library actually live.
Completely free and open source. Civitai.com offers optional paid memberships for faster downloads.
## Free Tier The platform and codebase are completely free and open source. Browse, download, and share AI models at no cost on civitai.com. ## Self-Hosted Self-hosting requires Docker, Node.js 20+, PostgreSQL. The self-hosted version is intended for development, not production use. Model storage at scale is the main operational challenge. ## Paid Optional Buzz membership on civitai.com provides faster downloads and priority generation. The open source code itself has no paid tier.
LiteRT-LM runs language models directly on a device, no cloud and no internet required. The model lives on the phone, laptop, smartwatch, or even in the browser, so data never leaves the hardware, it works offline, and there's no per-query bill. This is Google's own framework, and Google uses it to power on-device AI in Chrome, Chromebook Plus, and the Pixel Watch. Apache 2.0, completely free. It's cross-platform by design, targeting Android, iOS, desktop, the web via WebGPU, and small boards like Raspberry Pi, and it taps GPU and NPU acceleration instead of grinding on the CPU. It runs open models like Gemma, Llama, Phi, and Qwen. The work isn't running a server, because there is no server. The work is on the build side: you obtain and convert models into the right format, then wire up the native SDK for each platform you ship to, and manage on-device memory per device class. Heavier than calling a cloud API, far lighter than operating an inference cluster. The real competition is other on-device runtimes. llama.cpp has broader model coverage and a bigger community; Meta's ExecuTorch is the closest vendor-backed rival; Apple's MLX wins on Macs but only on Macs. LiteRT-LM's edge is tight, official integration with Android and Google silicon. It doesn't replace a paid product so much as move certain workloads off the paid-API meter: the small and mid-size models you'd otherwise rent from a cloud. Solo and small teams shipping mobile or edge apps: this is the Google-blessed path. Larger teams already on Android get first-party support. The catch is maturity. The core runtime is production-ready and shipping in real Google products, but some bindings, Swift and JavaScript among them, are still early preview, and the project is young. And on-device models are not frontier models. If you need GPT-class quality, this isn't that. It's for when private, offline, free, and good-enough beats cloud-quality.
Free and open source. The only cost is your device's compute and the engineering to integrate it.
**Free:** Everything. Apache 2.0 framework with no paid tier, no hosted upsell, no license fee. The models it runs (Gemma and friends) are downloaded separately and are themselves free. **Cost you'll actually pay:** Your own device compute and the engineering time to integrate it per platform. That's it. **The trade:** Running inference on-device means you avoid paid cloud-API costs entirely for the workloads it can handle. The ceiling is model quality, not price.
Kev answers typed questions about a block of text and returns probabilities your code can branch on. Route this ticket to billing or shipping. Is the customer angry. How urgent is this. No prose, one forward pass, one answer per question. Three sizes (0.8B, 4B, 9B) on Qwen3.5 bases, Apache-2.0, weights and training code included. The API copies TypeSafe's System One contract, so their Python SDK points at a local server with a URL change.
Install is uv sync and one serve command. The 4B wants about 9 GB of GPU memory in bf16, the 9B about 19 GB, and both fit a 32 GB Mac. Five questions take tens of milliseconds on an H100, roughly 780 ms for the 4B on an M5, two seconds for the 9B. On Apple Silicon the older Qwen3 builds are still the quicker option.
Solo developers and small teams doing ticket routing, moderation, or intent classification can run the 4B on hardware they already own. Be honest about why, though: hosted Jev costs $42 per billion input tokens, which is cheaper than any GPU you would rent to replace it. Run Kev for data residency, offline operation, or a fine-tune on your own labels. Not to save money.
The catch is what Kev does not know. On one 900-ticket support suite the 4B loses badly to hosted Jev at spotting an angry customer, and on MMLU-Pro it scores 0.490 against Jev's 0.840, because the base model sets that ceiling. The 9B closes the tone gap to roughly a tie and the knowledge gap not at all. Narrow, well-specified decisions are the job. The server also binds to localhost with no authentication.
Free and Apache-2.0 with the weights included. But hosted Jev publishes $42 per billion input tokens, so self-host for data residency or a custom fine-tune, not to save money.
**Free: everything.** Apache-2.0. All three checkpoints are on Hugging Face and mirrored as GitHub release tarballs with SHA-256 checksums. Training code, the frozen eval suites, the research log, and a local playground ship in the repo. No account, no API key, no rate limit. A hosted demo of the 4B and 0.8B runs on Hugging Face Spaces if you want to try it before installing anything. **Self-hosted: you pay in compute, not license.** Serving the 4B in bf16 needs roughly 9 GB of GPU memory, the 9B roughly 19 GB, and both run on a 32 GB Mac. Training your own adapter is cheaper than expected: 56 minutes on a single H100 for the 4B, 91 minutes for the 9B, and a 9-minute delta fine-tune for the 4B when you start from a released checkpoint. Budget engineering time for what the project leaves out: the server ships with no authentication and binds to localhost, so a proxy, keys, and deployment are on you. **Paid: hosted Jev, and it is cheap.** Kev has no commercial tier. The paid comparison is TypeSafe's hosted Jev, the model Kev is built to imitate, and TypeSafe puts a headline rate on its homepage: $42 per billion input tokens, marketed as 238x lower than Claude Fable 5.1. There is no pricing page and no tier table, so anything beyond that number means contacting sales. Do the arithmetic before you self-host. At that rate 100 million input tokens a month runs about $4, which no rented GPU is going to beat. Self-hosting Kev buys you data residency, offline operation, and a model you can fine-tune. It does not buy you a lower bill.
TimeGPT predicts where your numbers are headed. Feed it a history of anything measured over time, daily sales, hourly server load, weekly signups, and it forecasts what comes next and flags the weird spikes. No training a model, no feature engineering, three lines of Python. The SDK is open source under Apache-2.0. Here is the important part. The SDK is open, the model is not. TimeGPT runs on Nixtla's servers, and your data goes to their API to get a forecast back. There is nothing to self-host, which is why setup is trivial, but it also means you are sending your numbers to a third party and paying per call. A free trial key lets you kick the tires. For prototyping and one-off forecasts, the free trial plus a few lines of code is impressive, you get a working forecast in minutes. For production, you are on a usage-based bill and a hard dependency on their uptime. Solo and small teams testing an idea: start free. Teams putting forecasts in front of customers: price the API against running your own model with Nixtla's open StatsForecast or Prophet. The catch is the closed model. You cannot audit it, you cannot run it offline, and if Nixtla changes pricing or pulls the service, your forecasts go with it. Open SDK, closed brain.
Free to prototype, usage-based in production; the model is closed and cloud-only, so budget for the API and the dependency.
**Free:** A trial API key and the fully open-source SDK (Apache-2.0). Enough to prototype, benchmark, and run small forecasts. **Self-hosted:** Not available for the TimeGPT model itself, it is closed and cloud-only (an Azure private deployment is listed as coming). If you need local, Nixtla's own StatsForecast and NeuralForecast libraries are fully open and run on your hardware. **Paid:** Usage-based API pricing after the trial, billed per forecast call. Snowflake integration available via stored procedures.
TensorRT-LLM squeezes maximum inference performance out of NVIDIA GPUs for large language models. It handles quantization (FP8, FP4, INT4), custom attention kernels, paged KV caching, and multi-GPU deployment through a Python API. If you are serving LLMs at scale on NVIDIA hardware, this is the optimization layer that makes the economics work. Running it yourself means you need NVIDIA GPUs, full stop. No AMD, no Apple Silicon, no CPU fallback. You will also need CUDA installed and compatible driver versions. The setup is not trivial, but NVIDIA provides containers and Docker images that smooth out the worst of it. Once running, the performance gains over naive PyTorch inference are substantial, often 2-4x throughput improvements. For teams already committed to NVIDIA hardware, TensorRT-LLM is the right call over vLLM when you need every last token per second. vLLM is easier to set up and supports more hardware. llama.cpp is better for local, single-GPU experimentation. TensorRT-LLM is for production serving where GPU cost is a real line item. The catch: you are locked to NVIDIA forever. The library only works on their GPUs, and if your cloud costs push you toward AMD or custom silicon, you are rewriting your inference stack from scratch.
Software is free. The real cost is NVIDIA GPU hardware or cloud GPU rental.
### Free Tier Free under Apache 2.0. Requires NVIDIA GPUs (no AMD/Intel support). ### Self-Hosted Heavy setup. Requires NVIDIA GPU with sufficient VRAM, CUDA toolkit, and Docker. The optimization pipeline involves model conversion and compilation steps. ### Paid None for the software. The cost is NVIDIA hardware. A single A100 80GB runs K+, though cloud GPU instances start around -3/hr.
GPT-Load puts one base URL and one access key in front of every model provider you use. Your app stops caring whether a request lands on OpenAI, Anthropic, Gemini, Bedrock, or Groq. Channels, credentials, models, and routing policy all live in a web console. MIT, Go, free. Deployment is Docker Compose with application-managed SQLite by default, MySQL or Postgres if you outgrow it. It binds to localhost, the right default for something holding every API key you own, so budget time for TLS and firewall work before exposing it. One file matters most: the encryption.key file decrypts all stored credentials, this version cannot rotate it, and losing it makes every saved credential unrecoverable. Solo devs with a single API key do not need this. Small teams running several providers who want per-key usage and cost visibility are the sweet spot. Above that, read the fine print: 2.0 runs as a single instance and shares no state, so horizontal scaling is off the table. LiteLLM is the better call at that size. The catch is the feature that draws the traffic. Alongside API keys, GPT-Load pools subscription accounts through OAuth, Codex, Claude, Antigravity, and Grok, behind one endpoint. Those flows are unofficial and can break whenever an upstream changes, and the README itself tells you to connect only accounts you are entitled to use. That is the real constraint, not boilerplate.
Free and MIT with nothing to buy. Your cost is operator time plus the upstream provider bills it routes to.
**Free tier** Everything. MIT license, no paid edition, no hosted cloud, no enterprise tier. The official site sells nothing. **Self-hosted** The only mode. Docker Compose with application-managed SQLite is the default; `DATABASE_DSN` switches it to MySQL or Postgres. Native binaries ship for Linux, macOS on amd64 and arm64, and Windows, plus a Windows installer that registers a low-privilege service. The real cost is operator time: TLS and firewall work before any remote exposure, and a backup routine that keeps the database and `encryption.key` together, because credentials are unrecoverable without the key and this version has no master key rotation. **Paid** Nothing to buy from the project. Your bill is whatever the upstream providers charge. Note that GPT-Load's own usage and cost figures are estimates derived from upstream responses, not invoices, so do not reconcile finances against them.
MOSS-Image is an open text-to-image model built for what image generators usually botch: readable text inside the picture. Posters, signs, sticky notes, Chinese included. You can also hand it a JSON layout saying where the title and each object go. It's from OpenMOSS, and code and weights are both Apache-2.0, so it's free for commercial work.
Running it locally means Python, PyTorch and an NVIDIA GPU. The weights are roughly 27 GB at bf16, over half of it the text encoder, and the Gradio app can offload to CPU for smaller cards at the cost of speed. ComfyUI supports it natively, and community trainers already do LoRA fine-tuning.
The model has no paid tier. MOSI, the team behind it, sells API access, but hasn't published an image price yet. Solo and small teams: free if you own a decent NVIDIA card. Larger teams: free, but test it against FLUX and Qwen-Image first.
The catch: it's days old and the quality claims are the lab's own. The default is 50 sampling steps, so it isn't fast, and the community is already patching vertical color banding in the SFT model. Loading it also runs custom code via trust_remote_code=True.
Free, Apache-2.0 weights included, so commercial use costs nothing beyond your own GPU. The hosted MOSI API exists but hasn't published an image price yet.
**Free:** Two checkpoints, MOSS-Image-Base and MOSS-Image-SFT, on Hugging Face and ModelScope. The pipeline code on GitHub and the model weights are both Apache-2.0, so commercial use is allowed. Features: text rendering inside images, JSON layout control with bounding boxes, prompts up to 4K tokens, and a hosted demo on Hugging Face Spaces. **Self-hosted:** `pip install -r requirements.txt`, then load it through Diffusers with `trust_remote_code=True`, or run the Gradio app. Weights total about 27 GB at bf16 (transformer about 10 GB, text encoder about 17 GB, VAE under 1 GB, measured from the Hugging Face file listing). Gradio offers GPU resident, Component and Sequential memory modes for smaller GPUs. Native ComfyUI support, day-0 support in SGLang and vLLM-Omni, and LoRA or full fine-tuning through DiffSynth-Studio and KohakuMossTrainer. **Paid:** No paid tier for the model. The MOSI API Platform hosts MOSS-Image via Playground and API with credit-based billing, but as of 2026-10-21 its pricing page lists only voice and transcription prices (for example ¥2.00, about $0.28, per 10,000 characters for TTS), with no image price published.
kimi-k3-in-c runs a 2.78-trillion-parameter model on a machine with 8 GB of RAM, and the claim holds up. It's a from-scratch inference engine in portable C99, no BLAS, no GPU, no framework. Kimi K3 is a mixture-of-experts model where only a handful of experts fire per token, so the engine streams weights from disk and caches the few it needs. Apache 2.0, free. The headline number is real but it's not the one that matters. You need roughly 1.7 TB of fast NVMe for the checkpoint, Linux on x86-64 with AVX2, and patience: generation runs seconds to tens of seconds per token depending on hardware and memory budget. More RAM buys real speed, but nothing close to interactive. Output is byte-identical at every memory budget, validated against a PyTorch reference, which is the kind of engineering discipline most projects skip. Use this to learn how MoE inference actually works, to benchmark against, or to prove a point. Don't use it to serve anything: Moonshot's hosted API is faster and cheaper for real workloads, and llama.cpp is the right tool for models that fit your hardware. The catch: it's a weeks-old, essentially solo reference implementation, and the model weights carry Moonshot's own license, not Apache. Treat it as a brilliant teaching artifact, not infrastructure.
Free to run if you have 1.7 TB of NVMe to spare. For actual work, the hosted Kimi API wins on both speed and cost.
**Free:** Everything. The engine is Apache 2.0; the Kimi K3 weights download free from Hugging Face under Moonshot's separate model license. **Self-hosted:** The only mode. Budget about 1.7 TB of fast NVMe for the 1.56 TB checkpoint plus repacked trunk, a Linux x86-64 box with AVX2, and expect tens of seconds per token. More RAM buys speed, never different output. **Paid:** Nothing to buy here. If you want Kimi K3 at usable speed, Moonshot's hosted API is the paid path, and the project itself concedes it's faster and cheaper for real work.
laya-mlx runs Laya decision models natively on Apple Silicon. Laya is an open-weights model from Convai Innovations that answers multiple choice, rating, and yes or no questions with probabilities instead of generated text. It is the open counterpart to TypeSafe AI's closed Jev API. This port drops PyTorch and runs on Apple's MLX, returning a short decision in roughly 7 to 14 milliseconds on an M3 Max by the author's own measurement. Apache 2.0. Setup is pip install laya-mlx on Python 3.11 or later and macOS 14 or later. Weights download from Hugging Face on first load, and peak memory for a short question stays under 1 GB. No cloud API and no account. It covers inference and conversion only. Training and fine-tuning stay in the upstream Laya project. Free at every size. Use it for local classification, routing, or guardrail checks on a Mac where a full LLM call is too slow. TheoLeeCJ/SemIf gets a similar interface out of general open models and also has an MLX backend. The catch: this is an unofficial port from a young pseudonymous account, at version 0.1.0 with five commits. Context tops out at 512 to 1,024 tokens. And an open issue reports that swapping the order of the options flipped the answer 27 to 33 percent of the time on a Chinese benchmark. Test option-order sensitivity before trusting it.
Completely free and open source. Runs locally on an Apple Silicon Mac.
**Free:** Everything. Apache 2.0 inference runtime and conversion tools. The Laya weights are a separate download from Hugging Face under upstream's Apache 2.0 license. **Self-hosted:** The only way it runs: locally on an Apple Silicon Mac with Python 3.11 or later and macOS 14 or later. Under 1 GB of memory for a short question. **Paid:** None. No cloud API, no account, no hosted version.
This is the 'awesome list' for AI. Models, tools, infrastructure, datasets, organized by category with brief descriptions and links to the actual projects. Awesome lists live or die by curation quality. This one focuses on 'truly open source,' not source-available, not 'open weights with a restrictive license.' That distinction matters when you're building on top of these tools. The list is maintained on GitHub and follows the awesome-re standards. The catch: awesome lists are snapshots. They go stale unless someone actively maintains them, and the growth spike suggests this was recently featured somewhere. The real question is whether it'll be maintained in 6 months. Also, 'curated' means one person's opinion of what's worth including; your needs might differ. Use it as a starting point, not a definitive source.
Free. It's a curated list on GitHub.
### Free Tier Free curated list. Reference resource, not software. ### Self-Hosted Not applicable. ### Paid None.
Rapid-MLX runs open models locally on Apple Silicon, and speed is the whole pitch. One command installs it (Homebrew, pip, or a curl script), one command serves an OpenAI-compatible API, and it also speaks the Anthropic messages format, so Claude Code, Codex CLI, and Aider can point at it as a drop-in backend. Apache 2.0, completely free. The engineering focus is on the things that make local agent workflows fail in practice. A prompt cache brings claimed time-to-first-token under a tenth of a second on repeat prompts, and tool calling is handled across 17 parser formats, which matters because local models break agent loops on malformed tool calls far more often than on reasoning. It's in Homebrew core and publishes to PyPI, with releases landing near daily. Mac-only by design. If you run agents against local models on an M-series machine, try it head-to-head with Ollama or LM Studio; the performance claims ship with benchmarks in the repo rather than vibes. On Linux or Windows, look at Ollama or vLLM instead. The catch: it's young and moves extremely fast, the headline numbers are the project's own, and independent benchmarks are still thin. And no inference engine fixes the ceiling: a fast small model is still a small model.
Completely free and open source. Your only spend is the Mac it runs on.
**Free:** Everything. Apache 2.0, installable via Homebrew, pip, or the project's install script. Chat CLI, OpenAI-compatible server, Anthropic-format endpoint, tool calling, prompt cache, and optional vision/audio extras all ship in the open package. **Self-hosted:** The only mode, and it's one binary on your Mac. No accounts, no keys, no telemetry toll. **Paid:** Nothing. There's no cloud tier or commercial edition. The spend is hardware: an Apple Silicon Mac with enough unified memory for the models you want to run.
TokenSpeed is an LLM inference engine: the server that loads an open-weight model and answers requests through an OpenAI-compatible API, the job vLLM, SGLang and TensorRT-LLM do. It's tuned for agentic workloads, MIT licensed, free, and run by the non-profit LightSeek Foundation with engineering help from NVIDIA and AMD.
This is datacenter software. Its fastest paths target NVIDIA Blackwell (B200, GB200, GB300), with recipes for Hopper, AMD MI350X and MI355X, plus early Radeon RDNA4 and Ascend NPU support. Install pip wheels (CUDA 12.9, CUDA 13, ROCm 7.2) or the Docker image, then run tokenspeed serve. Recipes cover Kimi K3, Qwen3.8, GLM 5.3, DeepSeek V4, GPT-OSS and MiniMax M3, several on launch day.
No paid tier; the GPUs are the bill. Solo: free, but on one consumer card Ollama or llama.cpp is the saner pick. Small and mid-size teams serving open models: bake it off against vLLM and SGLang on your model. Large teams serving big MoE models on Blackwell or MI355X: worth a real benchmark.
The catch: it's v0.1.1, and the headline chart (beating TensorRT-LLM on Kimi K2.5) is the team's own. The docs call the recipes templates to validate, and the getting-started container runs privileged on the host network.
Free and MIT with no paid tier. The real cost is the multi-GPU Blackwell, Hopper or MI350-class hardware its recipes assume.
**Free:** The engine, kernels and scheduler, MIT licensed (copyright LightSeek Foundation). Serves an OpenAI-compatible API under `/v1`. No usage limits and no paid edition. **Self-hosted:** Stable pip indexes for cu129, cu130 and rocm7.2 (packages `tokenspeed`, `tokenspeed-kernel`, `tokenspeed-scheduler`, `tokenspeed-mla`, Python 3.11 to 3.13), nightly wheels, or Docker `lightseekorg/tokenspeed:0.1.1` (linux/amd64, linux/arm64) and `lightseekorg/tokenspeed-amd`. Needs an NVIDIA GPU host or a ROCm 7.2 host; H100/H200 go through a CUDA 12.9 source install. Big-model recipes assume multi-GPU nodes (Qwen3.8-2.4T at TP16, Muse Spark Open at TP8 on MI350X or TP4 on B300). Component releases are biweekly. **Paid:** None. The cost is the GPU hardware or rented GPU time, which for the models it targets means multi-GPU datacenter nodes.
EdgeQuake is a GraphRAG engine: it turns a pile of documents into a knowledge graph and combines vector search with graph traversal so an AI can answer multi-hop questions, the kind where the answer is spread across several documents. It's a Rust reimplementation of the LightRAG approach with a React UI, Apache-licensed and free. Running it is a real project. You need Postgres 16 with the pgvector and Apache AGE extensions, plus Docker Compose to wire it together. It also needs an LLM and embedding provider: point it at OpenAI or Anthropic and you'll pay per call, or run it against a local Ollama model for no API cost at all. For a developer building retrieval over their own documents who specifically wants the graph-plus-vector approach, EdgeQuake is a solid, self-hostable option, mostly solo and small teams comfortable with Postgres. The Rust core is fast and the code is documented. The catch is honesty about the payoff. On accuracy it's roughly a tie with LightRAG, so you're not buying a quality leap, you're buying a Rust implementation and a UI. And the pgvector-plus-AGE Postgres setup is a genuine dependency lift. If you just need basic RAG, a simple vector store ships faster. Reach for this when the graph structure is the point.
Free and open (Apache 2.0). You supply an LLM and embedding provider: pay per call with OpenAI/Anthropic, or run it free against local Ollama.
**Free:** The entire engine and UI (Apache 2.0). **Self-hosted:** Postgres 16 with pgvector + Apache AGE, plus Docker Compose. Bring an LLM/embedding provider (Ollama for zero cost). **Paid:** None from the project. External API providers cost per call if you use them.
RikkaHub is a native Android app for chatting with any LLM through one interface. Instead of separate apps for ChatGPT, Claude and Gemini, you point it at any OpenAI, Anthropic or Google-compatible endpoint, including your own, and switch providers between conversations. It handles markdown, image, PDF and document input, web search, message branching, memory and MCP. AGPL-3.0, free. Install it from Google Play or the project site, add your API keys and go. There is nothing to self-host: the app talks directly to the providers you configure. Newer builds add a Workspace, a lightweight Linux environment built on proot that runs on the phone without root so agents can gather information and assemble documents, plus a web interface you can open in a browser. For anyone on Android juggling multiple AI providers, this is the clean way to do it. You pay each provider for tokens, or point it at a local model endpoint and pay nothing. It is a personal app, and there is no team tier. The catch is trust in the chain. A multi-provider client is only as private as the endpoints you configure, and the README itself warns that many forks exist, some leaking privacy or asking for excessive permissions. Install the official build and point it at providers you trust.
Free under AGPL-3.0. You pay the model providers you connect, or nothing with a local endpoint.
**Free tier:** Everything, under AGPL-3.0. The project moved to plain AGPL-3.0 in July 2026, replacing an earlier split license that required a commercial agreement for commercial use or more than 10 users. The Android app is free on Google Play and the project website; donations go through Patreon and Afdian. **Self-hosted:** Not applicable. The app runs on your phone and calls whatever LLM and search endpoints you configure. The Workspace feature runs a proot Linux environment locally without root. **Paid:** None for the app. You pay each LLM or search provider you connect, or nothing with a local endpoint.
claude-real-video lets an LLM actually watch a video instead of guessing from a title. It pulls the frames that matter, using scene-change detection so it grabs a new frame when the picture changes, not every few seconds, deduplicates the near-identical ones, and optionally transcribes the audio with Whisper. Out comes a set of frames, a transcript, and a manifest you feed to Claude or any model. The core is MIT and free, and it all runs on your machine. Setup is a notch above pip-install because it leans on ffmpeg, the video-processing workhorse, which you install once at the system level. After that it is Python 3.10+ and a package install, with Whisper optional if you want transcription. Nothing leaves your laptop, which matters if the footage is sensitive. The free tier is the useful 90 percent: frame extraction, dedup, transcript. A one-time Pro unlock (founder price around $19, not a subscription) adds analysis on top, camera-movement classification, editing-rhythm detection, and a perception timeline of gestures and sound events. For most people wanting to hand a video to an LLM, free is plenty. Pay the $19 only if you are doing real video teardowns and want the extra structure. The catch is scope. This prepares video for a model, it does not understand it, the understanding still happens in whatever LLM you pipe the output into. It is a very good front door, not the whole house.
Free core covers frame extraction and transcription; the one-time $19 Pro is only worth it for detailed video teardowns.
**Free (core, MIT):** Scene-aware keyframe extraction, deduplication, Whisper transcription, and a local HTML viewer. Enough to feed any LLM a real look at a video. **Self-hosted:** It is local by default, ffmpeg plus Python on your machine, no cloud step. **Paid (Pro, one-time ~$19):** Adds camera-movement classification, editing-rhythm analysis, a perception timeline (gestures, emotion, sound events), and teardown reports. One-time founder price, not a subscription.
MOSS-TTS-Nano is a multilingual text-to-speech model with only 100 million parameters that runs in real time on a CPU. No GPU required. It now handles 20 languages, including English, Chinese, German, Spanish, French, Japanese, and Korean, with solid quality for its size. Small enough to embed in desktop apps, local demos, or lightweight web services. Apache 2.0. Runs via Python with standard ML dependencies, and there is now an ONNX CPU build that nearly doubles throughput. Models are on HuggingFace and ModelScope, finetuning code is published for custom voices, and recent additions include mlx-audio support for Apple Silicon and a browser reader extension. The deployment stack stays simple: no CUDA, no heavy inference server, a basic machine handles it. Free for everyone under a permissive license. Solo developers building voice features get real-time TTS without paying per-character API fees. Teams shipping products can embed it directly without usage limits. The catch: 100M parameters means tradeoffs in naturalness and expressiveness. If you need the best possible voice quality, larger models or paid APIs like ElevenLabs will sound better. This is the right call when you want speed, low cost, and local execution over peak fidelity.
Completely free. Trade peak voice quality for zero cost and local execution.
**Free tier:** Everything. Full model, finetuning code, all languages. **Self-hosted:** Python, CPU-only. No GPU needed. HuggingFace model downloads. **Paid:** No paid tier. Open source with permissive licensing.
Mike is an open-source AI legal platform for document processing and review. Built on Next.js with an Express API, Supabase for auth and storage, S3-compatible object storage for files, and your choice of LLM provider for the analysis. AGPL-3.0. Setup is significant. You need a Supabase project, an S3-compatible bucket (Cloudflare R2 works), API keys for at least one model provider, and LibreOffice for DOC/DOCX-to-PDF conversion. The README is sparse, so plan for some reverse-engineering. Pick this if you run a small practice or a legal-ops team and want AI document review without sending client files to Harvey, Spellbook, or Casetext. Solo: free, you pay your model bill, the setup curve is real. Small teams: same plus shared infra. Large firms: they pay for Harvey because the integrations matter and they don't want to maintain this themselves. The catch: legal AI is a confidence and accuracy game, and Mike is early. Docs are thin, the feature set is whatever the README implies, and there is no SOC 2 posture out of the box. Use this because you want to own the stack, not because it's a turnkey product.
Open source. Real cost is your Supabase plan, R2 storage, and LLM usage.
**Free tier:** AGPL-3.0, no licensing cost. **Self-hosted:** Self-host the Next.js + Express stack. Supabase free tier covers small usage; Cloudflare R2 is cheap; LibreOffice is free. Real cost is the LLM provider bill plus engineering time to operate it. **Paid:** No paid tier. Cost equals your Supabase plan, R2 storage, and model API usage.
H3.c generates video with synchronized audio locally on a Mac. It's a from-scratch C inference engine for MiniMax's H3 model, written by the creator of Redis, with an interactive REPL that previews output right in your terminal. The code is MIT and a pleasure to read; local, private video generation with no cloud API is the pitch. The hardware is the gate. You want Apple Silicon with serious unified memory: peak usage runs into the tens of gigabytes depending on canvas size, plus about 37 GB of disk for the checkpoint. An SSD-streaming mode drops memory use to around 2 GB, but at a steep speed penalty at small canvas sizes. Build is a simple make with FFmpeg on PATH. This is for Mac owners with serious unified memory who want local generation, and for anyone who wants to read a modern diffusion pipeline in C. ComfyUI is the mainstream local workbench; Runway and Pika are the hosted alternatives. The catch: the MIT badge covers only the code. The model weights ship under MiniMax's custom community license, non-OSI, with an application process for US, EU, UK, and South Korea users, and the engine is useless without them. The author also frames the repo as working vertical slices, not a finished product, and canvas sizes and frame counts have sharp constraints.
Free MIT code, but the required model weights carry MiniMax's non-OSI community license, and the hardware bar is high.
**Free (the code):** MIT, no commercial offering, builds with make. **Not open (the weights):** MiniMax-H3 ships under a custom community license, not OSI-approved, with a separate application process for users in the US, EU, UK, and South Korea. Commercial evaluation means reading that agreement. **The real cost:** Hardware. Peak memory runs into the tens of gigabytes depending on canvas size (an SSD-streaming mode runs in ~2 GB at a steep speed cost), ~37 GB storage, and thermal headroom on laptops. **The trade:** No per-generation API fees, in exchange for owning serious silicon and accepting a vendor model license.
WARP (formerly WASTE) runs AI models far bigger than your computer's memory. It keeps the always-used core of a mixture-of-experts model in RAM and streams the rest off your SSD per token. The headline: the full 2.78-trillion-parameter Kimi K3 on a 64 GB MacBook Pro. It's a dependency-free C engine from the SQLite Cloud team, free under Apache 2.0. Kimi K3 is the stunt. The useful part is the mid-size models: GLM-5.3-Flash runs at about 3.9 tokens per second on that same MacBook and still works on a 16 GB machine, and DeepSeek-V4.1-Flash manages about 3.8. You get a CLI, an embeddable C library and an optional OpenAI-compatible server. Setup is real work: you download and convert the weights yourself. No paid tier. Solo tinkerers: free, start with the small Kimi-Linear. Small teams: free for offline experiments. Larger teams: this is not a serving stack, use vLLM or llama.cpp on GPUs. The catch is speed and disk. K3 runs under one token per second and needs about 1 TB of NVMe, and prefill is as slow as decode, so long prompts take minutes. The format and API aren't frozen, and the README says LLMs write the code while humans steer.
Free and Apache 2.0. The real cost is hundreds of GB of fast internal SSD and patience: about 3 to 4 tokens per second on the mid-size models, under 1 on Kimi K3.
**Free:** Apache 2.0, and the README promises it 'will always remain open source under a permissive license'. Engine, `waste` CLI, `libwaste.a` C library and the OpenAI-compatible server are all in the repo. Builds with a C11 compiler and `make` on macOS, Linux (arm64, x86_64) and Windows via MinGW-w64. **Self-hosted:** Local only, and the bill is disk. GLM-5.3-Flash: 16 GB RAM is enough, 112 GB container plus 306 GiB of temporary space to convert. DeepSeek-V4.1-Flash: 16 GB RAM, 299 GiB container plus 475 GiB temporary. Kimi K3: 64 GB RAM recommended (29.19 GB floor), about 1 TB container plus 1.42 TB temporary. Python and PyTorch are needed only for conversion. **Paid:** None. Model weights carry their own licenses: Kimi K3 ships under the custom Kimi K3 License, while GLM-5.3-Flash and DeepSeek-V4.1-Flash are MIT on Hugging Face.
Text-generation-webui gives you a browser-based interface to do it. Load a model, chat with it, fine-tune it, generate images. It's the Swiss Army knife for local AI. The entire project is free under AGPL-3.0. Every feature (chat, notebook mode, model loading, LoRA training, multimodal/vision support, extensions) ships at $0. The developer sells some extension packs on Gumroad, but those are optional add-ons, not core features. Self-hosting is the only option, and the setup complexity depends on your GPU situation. If you have an NVIDIA card with 8GB+ VRAM, the one-click installers work well. AMD and Apple Silicon support exists but can be finicky. Expect 30-60 minutes for first-time setup including downloading a model. Solo developers: this is your playground. Run models locally, experiment with fine-tuning, keep your data private. Small teams: share a beefy GPU server running the API mode. Beyond that, look at dedicated inference servers like vLLM. The catch: GPU hardware requirements are real. You need a decent GPU to run anything useful. A 7B parameter model needs ~6GB VRAM. Anything bigger needs proportionally more. No GPU, no party.
Software is completely free. The real cost is GPU hardware: $300-900 for a capable NVIDIA card.
### Free Everything core is free. Chat interface, notebook mode, model management, LoRA/QLoRA fine-tuning, multimodal support, API server mode, extensions framework, all AGPL-3.0. ### Optional Paid Extensions The developer sells extension packs on Gumroad (e.g., "Deep Reason" for enhanced reasoning). These are optional convenience add-ons, not gated features. The core project doesn't depend on them. ### Self-Hosted (Only Option) This runs on your hardware. Minimum useful setup: NVIDIA GPU with 8GB VRAM (~$200-400 used for an RTX 3060 12GB). Ideal setup: 24GB VRAM card ($400-900 used for RTX 3090/4090). Running costs are electricity. Expect $5-15/month for regular use. ### Hardware Cost Reality Check - 7B models (Mistral, Llama 3): 6-8GB VRAM. Runs on an RTX 3060. - 13B models: 10-16GB VRAM. Needs RTX 3090 or better. - 70B models: 40GB+ VRAM. Multi-GPU territory or heavy quantization. - CPU-only: possible with llama.cpp backend but painfully slow for anything useful. ### Verdict The software is free. The hardware isn't. Budget $300-900 for a capable GPU if you don't have one already.
Bespoke Nimble answers typed questions about a block of text. It returns the answer plus a probability for every option and never writes prose, so there is no JSON to parse. Bespoke Labs, the team behind Curator, fine-tuned Qwen3.5-9B on 2,676 curated examples and published the weights, the training data, and the full curation pipeline. It is an open take on TypeSafe's Jev, and jaredpalmer/kev is the other one to compare it with. Running it yourself is real work. The 9B weights alone are about 18 GB, so you need a big Apple Silicon Mac or an NVIDIA GPU with BF16, plus a merge step first. Bespoke measured roughly 106 ms per question on an H100 and 444 ms on an M5 Pro. A free public demo endpoint exists for trying it, not for production. The interesting part is the recipe, not the model. Their contrastive data curation writes pairs of examples that differ by one fact so the label flips, which teaches the model which evidence matters. Read that before building your own classifier. The catch: the repo has no license file. The weights are Apache-2.0 on Hugging Face, but the code has no stated terms, and the 90% agreement headline comes from 324 synthetic examples drawn from six source families. Bespoke built it in a day and says so.
Free weights under Apache-2.0, but the repo code ships with no license, and hosted Jev is cheaper than the GPU you would rent to replace it.
**Free tier:** Everything published is free. The Bespoke-Nimble-9B weights are Apache-2.0 on Hugging Face, and the training data, held-out set, and curation pipeline are in the repo. **Self-hosted:** You supply the hardware. The 9B weights are about 18 GB before quantization, so plan on an Apple Silicon Mac with plenty of memory or an NVIDIA GPU that supports BF16. The repo also includes a Modal deployment guide if you want to host it on rented GPUs. **Paid:** Nothing to buy from Bespoke Labs. Their free public demo endpoint needs no key but carries no service promise. The paid comparison is TypeSafe's hosted Jev, which publishes $42 per billion input tokens, so running Nimble yourself is about control and fine-tuning, not saving money. **License note:** The GitHub repository has no license file, so the code has no stated reuse terms. Only the model weights carry Apache-2.0.
Flash-moe runs Qwen3.5-397B, a 397-billion-parameter model, on a MacBook Pro with 48GB of RAM. It's a Mixture of Experts model, so only a few expert slices fire per token, and flash-moe streams those slices off the SSD instead of holding all 209GB in memory. Pure C, Objective-C and hand-written Metal shaders. Free to read, with a big license caveat.
This is a research demo, not a product. The author measured about 4.4 tokens per second at 4-bit on an M3 Max with tool calling working; the faster 2-bit mode breaks JSON output. Apple Silicon only. You download the weights, repack the experts with the included Python scripts and build with make; one user's issue describes getting it working "with a lot of pain." No paid tier. Solo tinkerers with a big Mac: worth an afternoon and the paper. Teams: no. For real local inference, use llama.cpp, Ollama or MLX.
The catch: there is no LICENSE file, so legally it's all rights reserved and you can't ship anything built on it. A request to add one has sat open since March. The last commit was March 19, 2026, and community ports, including a CUDA version, sit unmerged.
No price, but no license either: it's all rights reserved, so treat it as something to study, not something to build on.
**Free:** The full source is public on GitHub at no cost: the C/Objective-C inference engine, Metal shaders, chat TUI, weight-packing scripts and the paper. There is no LICENSE file, so no rights to reuse or redistribute are granted. **Self-hosted:** Local only, on Apple Silicon. You download the Qwen3.5-397B-A17B weights, repack the experts (about 209GB on disk at 4-bit, 120GB at 2-bit) with the included Python scripts, and build with `make`. The reference machine is an M3 Max MacBook Pro with 48GB of unified memory and a fast internal SSD. **Paid:** No paid tier and no hosted service. The only cost is the hardware and disk space.