HMIS-NL, verified AI analytics
I built HMIS-NL so housing staff can ask reporting questions in plain English. A language model reads the question and Python computes the answer. A validator withholds numbers it cannot defend.
Hosted web app and Textual terminal client. Not publicly linked yet.
- Hosted app
- Public sample data
- Private source
- Problem
- Staff needed answers from HMIS data without trusting a language model to invent the reported number.
- Engineering decision
- Used a language model to interpret reporting questions and Python to calculate the answers. Web and terminal interfaces expose the definitions and evidence behind each result.
- Result
- Reporting answers staff can inspect and rerun
- Measured evidence
- 81 targeted tests passed, September 22, 2026
- My role
- Sole engineer, independent analytics application
- Source access
- Private repository. Live demo uses public HUD sample data.
In this case study Features and engineering details
What it does
- Inspectable calculations
- Reporting answers include the population definition and calculation evidence.
- Reusable reports
- Saved questions rerun through the backend and export to CSV or Word.
- Source documents
- HUD and agency-policy answers link to evidence, including highlighted PDF passages.
- Terminal access
- A Textual client shows answers with evidence and terminal charts through the same engine.
From question to checked answer
Reporting question
The user selects a period and program scope.
Structured request
The model proposes a typed query intent.
Python calculation
The engine applies the metric definition to the data.
Checked answer
Validation gates the result. Evidence explains the calculation.
Walkthrough
Recorded on September 22, 2026 against the hosted demo with public synthetic data and the live language model. The clip asks a reporting question, opens the evidence, refines by age and percent, compares two periods, saves a report, follows a HUD citation to the highlighted page, then repeats the age breakdown in the terminal client. Captions are available in the player.
Check the answer before showing it
A reporting question leaves room for interpretation. Counting people who moved into housing can produce a different answer from counting exits to permanent housing. I made the selected definition visible beside each result.
The model proposes a typed query request. Python selects the reporting population and calculates the answer. For supported metrics, a second implementation recomputes the result from a declarative specification. A disagreement withholds the answer. If the implementations share population-selection code, the verification record states that limitation.
The evidence panel explains the date range and who was counted. Here, 32,104 records become 22,142 distinct people. Calculation agreement does not establish that the model understood the question correctly, so intent evaluation remains a separate test.
The live app's evidence panel shows the reporting definition and deduplication behind the answer. All reporting images use 2022 public sample data.
View full imageCarry privacy rules into the result
Small groups can expose information about individual people. The backend masks counts below the configured threshold before sending the result to the browser.
I kept that distinction in the display and export paths. A hidden count stays labeled as hidden in the table and CSV. It is omitted from the chart. Sorting or filtering the view does not turn it into a zero or reveal the underlying value.
Detail from the live report. Two groups remain hidden under the configured threshold of 11. Zero and suppressed counts are displayed differently.
View full imageTurn a question into a report staff can reuse
I built saved reports around the same questions staff ask in chat. Each question can name its reporting dates explicitly. Running the report again recomputes the answers against the latest data through the backend validation path.
The report keeps the original question beside its result. Users can sort a breakdown and limit the visible groups while retaining the full answer. CSV and Word exports make the results usable outside the application. Dashboards reuse report blocks and render server-defined charts through ECharts.
A saved report run in the live app. The age view shows its sort and top-five steps, while the full answer remains above it.
View full imageLet staff follow an answer back to its source
Reporting definitions also need explanation. I built separate question-answering views for HUD standards and an agency's uploaded policies, so the two sources stay distinct.
Answers include citations that open the supporting document in a reference pane. For mirrored PDFs, the viewer finds the cited passage and highlights it on the original page. Staff can check the source without leaving the question, then open the full PDF for a closer read.
A live HUD Standards answer opens the original HMIS Data Dictionary at the cited passage. The highlight marks the supporting text on page 11.
View full imageUse the same engine from the terminal
I also built a full-screen Python interface with Textual. Users can ask questions from the keyboard and inspect evidence beside each answer. Results include tables and terminal charts. Reporting dates stay fixed across follow-up questions, and ambiguous requests can prompt clarification.
The terminal connects to the FastAPI service or runs locally against synthetic fixtures with a mock model. An optional voice path sends transcribed questions through the same engine. An MCP server exposes the metrics to other applications through the backend validation path.
Offline tests cover calculation disagreement and keep model-written numbers out of reporting answers. React component tests cover evidence rendering and source-document interactions.
Verification / September 22, 2026
81 targeted tests passed
- System invariants: compared calculation implementations and checked that a forced disagreement withholds the answer. Checked that model-written numbers cannot enter reporting answers. Exercised the terminal interface with synthetic fixtures.
- Router safety: in the June 2026 live-router evaluation, 88 of 90 questions that should withhold a number did so. Every emitted number was still gated by the validator.
The tests ran offline with a mock language model. The router evaluation used a live model over a fixed fixture. Neither measures general question interpretation accuracy.
Source revision c490653