Özgür Işık Damar
Data & Artificial Intelligence

Topic intelligence2026

ContextLens

Follows what a conversation is about — and can answer “uncertain” instead of guessing.

Open source · Apache-2.0 · model v1.1.0

Source on GitHubRead the build notes
python project.py

Receipts

macro-F1 on a locked Wikipedia holdout
0.838
reports/locked/results.json
median classification latency, 4 vCPUs, no GPU
19.9 ms
reports/locked/results.json
off-topic AUROC on assistant chat
0.961
reports/locked/results.json
whole model on disk, encoder included
47 MB
README.md

Every number comes from a file in the repository — the path is under it.

The problem

A softmax classifier always returns a topic: fed “I'm going to order pizza tonight”, the model under ContextLens still says Books, 53.0%. In a conversation that is worse than silence, because a stray label steers the theme. And a model picked on encyclopedia text alone can top the leaderboard and still stumble on the short questions people actually type.

The approach

None of the nine datasets assessed covered the 8 × 28 taxonomy, so a 40,112-passage corpus was built from English Wikipedia, labelled through Wikipedia's own category graph. A fine-tuned MiniLM encoder feeds one temperature-scaled softmax over the 28 subtopics, while a Mahalanobis distance gate, a 50% confidence floor and a fastText language gate decide when the answer is “uncertain” or “not English”. Only confident turns feed the theme: it decays, expires after four turns without a confident topic and switches its dominant topic after two agreeing messages, at the cost of following a real topic change about 0.6 turns later. Every choice was made on development data, real Stack Exchange questions included, and the final scores come from a locked holdout graded after the configuration was fingerprinted.

What's built

  1. Withholds a topic from 78.1% of off-topic assistant chat and flags 96.7% of non-English sentences
  2. Encoder chosen on real questions too: mpnet-base ties it on Wikipedia but is 4.9× slower and weaker on them
  3. Locked holdout graded after a SHA-256 freeze; it had to run twice, and both runs are public with the reason
  4. In simulated chats, a one-message tangent takes over the theme 10.4% of the time, down from 48.8%

Architecture

ContextLens — The runtime from the repo's own diagram: language gate, fine-tuned MiniLM, 28-way softmax and Mahalanobis off-topic gate, then the decaying tracker, theme composer and key-less search.
The runtime from the repo's own diagram: language gate, fine-tuned MiniLM, 28-way softmax and Mahalanobis off-topic gate, then the decaying tracker, theme composer and key-less search.

A real CLI run of the README demo: a novel, the scientific method and evolution each get a topic, subtopic and confidence; their theme, “science books about biology”, seeds the search.

1 / 4
// build notesThe locked holdout I had to run twice — and why both runs are publicMy locked test set ran twice: one field hid 1,151 questions from the headline score. How I fixed the data, proved the model never moved, and published both runs.Read the build notes

More 2026 builds