latitude-llm

Latitude watches your AI agents the way Datadog watches your servers. When you ship an app built on LLMs, you lose visibility the moment a prompt leaves your code, you can't see why an agent went off the rails or which step failed. Latitude captures every trace, groups failures into issues you can actually act on, and runs evaluations so you know when output quality drifts. The core is open source under LGPL-3.0.

Self-hosting is real work but well-trodden: it's a TypeScript app you run yourself, bring your own database and infrastructure. The payoff is that all your prompt and trace data stays on your hardware, which matters if you're handling anything sensitive. If you'd rather skip the plumbing, the hosted cloud has a free Starter tier with 20,000 credits a month.

Solo devs and small teams testing an AI feature should start on the free cloud tier and self-host once data control or volume matters. The paid Pro plan runs $99/mo for 100,000 credits, which is the call once you're past hobby scale. Larger teams comparing this to LangSmith or Langfuse get an actually open core instead of an open-core tease.

The catch: LLM observability is a crowded space and Latitude is younger than LangSmith. The trace and eval features are strong, but you're betting on a newer project. Self-hosting also means you own the upgrade treadmill and the storage bill as your trace volume grows.