Overview
| Client | SpaceSJ Lab - regulation query prototype — In-house prototype (not a client delivery) |
|---|---|
| Industry / Technology | Education / Assistant |
| Period | May-Jun 2026 (6 weeks) |
| Scope | In-house R&D - version-aware retrieval experiment |
| Stack | FastAPI, Qdrant, BGE-m3, cross-encoder |
| Disclosure grade | L3 In-house |
What was the problem?
Regulations are revised often. If the pre-revision document stays in the index, the assistant answers from a clause that has been withdrawn - and in a real organisation that causes real damage.
Why this approach?
We attached validity metadata at clause level rather than document level, and filtered the search space by the query date before retrieval ran, instead of trying to correct the answer afterwards.
What we built
- 01Clause-level chunking with effective and repeal dates as metadata
- 02Filter by query date before retrieval - withdrawn clauses never enter the candidate set
- 03A separate answer path for revision-history questions (when did this change?)
- 04Measured citations of withdrawn text against a 100-question golden set
Before to After (measured in-house)
Citations of withdrawn text
11
0
Revision-history accuracy
42%
89%
Average response latency
3.1s
1.4s
Answers carrying a source
78%
100%
Figures were measured on the same basis before the engagement and after go-live, and the calculation records are retained.