guru
I built a research and reading application that answers questions with source quotations the reader can inspect.
Public Ask and Find are live. Guests get five Ask requests, then five per day. Find stays free.
- Live app
- Public source
- Independent project
- Problem
- Models asked to copy archaic English changed the wording, so a correct passage could still be misquoted.
- Engineering decision
- Replaced model-written quotations with sentence IDs resolved against the source. Kept retrieval evaluation separate from checks that the displayed wording matches the original text.
- Result
- Hallucinated quotations are ruled out by design
- My role
- Sole engineer, independent reading application
- Source access
- Public repository
In this case study Features and engineering details
What it does
- Original quotations
- The model selects sentence IDs. The application inserts the source text.
- Quote checks
- Claims that depend on a failed quotation check are removed before display.
- Scoped retrieval
- Readers can restrict a question to selected books and inspect the retrieved passages.
- Background ingestion
- A worker prepares documents while each reader's library stays in its own SQLite database.
Project view
Captured from a live Ask on September 22, 2026. The first block is the model-written synopsis. The quotation below it is the source text for a sentence ID the model selected, with the citation the application resolved. The guest allowance is five Ask requests, then five per day.
The problem with generated quotations
A retrieval system can find the right passage and still misquote it. In early tests, models asked to copy archaic English frequently changed the wording. Better retrieval alone would not fix that problem.
Let the model select the passage
I changed the answer format so the model selects sentence IDs. The application inserts the corresponding source text, then checks the quotation against the book before displaying it. Claims with a failed quote check are removed.
Quotation fidelity and answer relevance need separate checks. A quote can match the source exactly and still fail to answer the question. The model-written synopsis also remains separate from the quoted text.
Test retrieval changes before keeping them
I added source-scoped retrieval so a question about one book does not silently draw its answer from another. A saved comparison uses the same questions and ranking configuration on both sides, changing where the selected-source filter is applied.
In the September 9, 2026 run, the expected passage reached the first five results for 7 of 12 questions, compared with 1 of 12 in the baseline. This was a small set of known questions that included development examples. It supports that change without establishing broad answer accuracy. The saved evaluation records the inputs and implementation identity.
Separate the library from the engine
The September 9 evaluation used a starter library of 123 public-domain books in philosophy and spiritual thought. Each reader has a separate SQLite database. A background worker handles ingestion, and the retrieval engine can use another collection through the same quotation path.
Open the reading application or inspect the source code.