phoenix

Phoenix is open source observability for AI apps. When an LLM feature misbehaves in production, this is how you see why. It traces every model call your app makes, captures the prompts and responses, and lets you run evals to score output quality over time. Arize built it on OpenTelemetry, and it runs in a notebook, a container, or on your own server.

Self-hosting is the point, and it delivers. Run the container, point your app's tracing at it, and you get traces, datasets, experiments, and prompt management in one UI with nothing leaving your infrastructure. That matters when your prompts carry customer data you can't ship to a vendor. The ops burden is moderate. This is a real service to keep running, not a library you import.

Solo developers and small teams should self-host this and skip the LLM-observability SaaS bills from the LangSmith and Datadog tier. You get tracing and evals for the cost of a container. Larger teams already paying for Arize's hosted platform get the managed version; the open release is the same engine minus the ops work.

The catch is the license. Phoenix is Elastic License 2.0, not true open source. You can self-host and use it freely, but you cannot stand it up as a competing hosted service. For anyone using it to debug their own app, that line never gets crossed. Just know it is source-available, not MIT.