Twenty years of clinical notes, finally searchable

The situation

Twenty years of clinical correspondence sat in a document store that could only be searched by patient identifier and date. Finding a prior case with a similar presentation meant asking a colleague who might remember one.

Constraints

Nothing could leave the hospital’s tenancy. De-identification had to happen before indexing, not after. And any answer shown to a clinician had to point at the document it came from, because an unsourced summary is not usable in a clinical conversation.

What we did

De-identification at ingestion, then retrieval over the resulting corpus with citations on every answer. Access follows the clinician’s existing role, so the system never surfaces a record the reader may not open.

An evaluation set of 1,200 real questions, with answers reviewed by clinicians, runs on every change and is refreshed quarterly.

What we would do differently

We underestimated how much of the corpus was duplicated. Three months in we added a supersession rule so older versions of a protocol stop competing with the current one. That should have been in the first design.

Results

Measure	Before	After
Median time to find a prior case	4 hrs	6 min
Answers with a source citation	None	100%
Records searchable	Partial	4.2M
Evaluation set size	0	1,200