Haystack

Haystack is an open-source Python framework for building LLM applications: retrieval-augmented generation, semantic search, and increasingly, autonomous agents. In plain terms, it is the plumbing that connects your documents and data to a language model and back out as an answer, with explicit control over each step. Apache 2.0 licensed, free via pip install haystack-ai, built by deepset.

The design is modular pipelines: you chain swappable components (retrievers, rankers, generators, memory, tool-calling) into a workflow you can actually reason about. Version 2.x widened the scope from pure RAG to agent workflows, tool use, and multimodal apps, so the same framework that builds a document Q&A system now builds an agent that calls tools and keeps memory. Running it yourself is moderate work: the framework is free, but you supply the vector store, the LLM, and the infrastructure around them.

Solo and small teams: pip install and go, then budget for the LLM API calls and a vector database, not for Haystack itself. Larger orgs that want managed deployment, observability, and governance can look at Haystack Enterprise, deepset's paid tier, available cloud or self-hosted. The framework gives you the control; Enterprise sells the operations layer on top.

The catch is that Haystack's flexibility is also its learning curve. The pipeline model is powerful, but you have to design it, and you are on the hook for the costs it orchestrates: every LLM call and every vector query is your bill, not deepset's. Compare it against LlamaIndex and LangChain before committing, since all three cover overlapping ground with different opinions.