Five separate repos that actually call each other: an agent that retrieves, a gateway governing its model calls, an eval harness grading its answers, a dashboard showing the grades, and a database that remembers every run. This page installs each project's own published wheel — the exact artifact its CI built — and runs them together in your browser. The last one isn't in your browser at all: it's a live FastAPI + Postgres service, and this page really calls it. The arrows below are function calls, not a diagram.
Booting…
rag-eval-labllm-gatewayagent-grapheval-dashboard — at the endeval-history — a real server
agent-graph──retrieves via──►rag-eval-lab·agent-graph──its LLM calls──►llm-gateway·rag-eval-lab──grades──► eval_run.json ──►eval-dashboard·eval_run.json ──►eval-history──what changed?──►a real Postgres
Four questions go through all five projects, ending with an eval_run.json you can open in the dashboard — and a verdict from a real database. Watch question 4 — it isn't in the corpus, and that's the interesting one.
— or ask it your own —
What's a corpus? Just the pile of documents the system is allowed to look things up in — its entire world knowledge. That's the whole idea behind RAG: instead of letting a model answer from memory, you force it to answer from documents you chose. Which is exactly why the answer can then be checked — a model answering from memory can't be audited; one answering from your documents can.
So the stack handles any topic — you just have to hand it the documents. Ask about something that isn't in there and it won't know, which is the interesting case.
← swap the whole world it knows about
…or write your own — one id: text per line
Paste your own documents and the retriever rebuilds around them — the planets are just what the demo shipped with. Then ask a question about your corpus above.
What's real and what isn't, plainly. The retrieval, the grading, the gateway's auth/cache/cost accounting and the agent's graph are all genuinely executing — installed from each project's published wheel. The provider is llm-gateway's deterministic mock, which echoes its prompt rather than writing prose; that's what lets this run with no API key and no spend. So the grade is computed against the agent's own composed answer, not the mock's echo. Point the gateway at a real key and nothing about the wiring changes.