Most "RAG in 50 lines" projects treat the document as prose, trust raw similarity, and answer even when they shouldn't. For serious domains — clinical, legal, regulatory — an answer with no traceability isn't a feature, it's technical debt. This pipeline does the opposite, and everything — embeddings, vector store, and LLM — runs on your own hardware.
This pipeline reads the manuals and protocols you give it and answers questions with the exact section cited — it never makes one up. It runs entirely inside the clinic or hospital's own hardware, so patient records never travel to any outside server.
A psychiatrist keeps years of clinical protocols and drug-interaction manuals. Instead of scrolling through a PDF mid-consultation, they ask a question and get an answer with the exact section cited — or a flat "it's not in here" if it isn't.
Dozens of staff need the same triage and treatment guidelines. Everyone queries the same assistant instead of guessing or emailing a colleague — and no patient data ever reaches a cloud API to make that possible.
When an insurer has to justify why a treatment was approved or denied, an answer with no source is worthless. Every response here carries the citation an auditor can check against the original document.
Three sins of demo RAGs that disqualify them for serious use — and how this pipeline avoids them.
They split text blindly and lose section context. Here the manual is a hierarchy of logical units: every chunk keeps its full path (Anxiety > Panic > Assessment).
Vector similarity alone brings noise. Here candidates get re-ranked with a transparent blend of cosine + lexical overlap + heading-match boost. Auditable, in microseconds, no cross-encoder.
If the best chunk scores below MIN_SCORE, it returns "I can't find this in the manual" instead of making it up. The prompt bans outside knowledge and requires inline [S#] tags.
Indexing and answering are two separate commands over the same local store. The interesting one is the third scenario: when nothing scores high enough, the model is never called at all. Pick a path and watch it run.
Every piece exists to make retrieval defensible under audit.
Documents are split following the heading tree; each chunk keeps its section path. A chunk that would lose its parent context gets discarded, never indexed half-formed.
Source, heading, section path, ordinal — to apply surgical where filters before the LLM ever sees the query.
Below MIN_SCORE, the system answers "it's not in the manual." A refusal is a valid, audited outcome, not a failure.
OllamaEmbedder for production, a deterministic HashEmbedder for offline/CI — same interface, so the pipeline is testable with no service running at all.
MIN_SCORE and get refused anyway: fewer wrong answers, but also fewer answers. And it cuts the other way too: the gate reads only contexts[0], so the remaining chunks that fill the prompt are never gated individually — in the offline self-test the fourth cited source scored 0.00 and still reached the model. That is survivable because the score travels with every citation and gets printed alongside it, so a reader can see exactly how thin a source was; but the gate itself does not stop it. And the LLM itself runs locally by default as an 8B model (llama3.1:8b), not a frontier cloud model — it reasons less about ambiguous phrasing and sticks closer to what the text says. That ceiling is the price of never sending a document, a query, or an embedding off the machine.# indexes the synthetic sample with a deterministic offline embedder python -m src.ask selftest PASS: indexed 13 chunks PASS: top section for panic query -> ... > Panic Episodes PASS: grounded answer cited 4 source(s) PASS: off-topic query refused instead of hallucinating
The self-test verifies the contract end-to-end: chunks indexed → correct section retrieved → cited answer → off-topic question refused. To use it for real: ollama pull nomic-embed-text + llama3.1:8b, point it at your corpus, done.