Tag3 posts

#evaluation

All posts tagged evaluation.

By Özgür Işık Damar26 Sept 202611 min read
200 OK, refunded twice: grade your agent on state, not on its answer
My demo refund agent returned 200, claimed SUCCESS and refunded twice. How I catch that: tool twins that record every call, and faults that split the reply from the state.
By Özgür Işık Damar24 Sept 202612 min read
The locked holdout I had to run twice — and why both runs are public
My locked test set ran twice: one field hid 1,151 questions from the headline score. How I fixed the data, proved the model never moved, and published both runs.
By Özgür Işık Damar23 Sept 202612 min read
Delete the future, diff the features: “no leakage” as a test, not a slogan
RealPath's leakage claim is a pytest: delete everything after the anchor, rebuild, diff. What it caught when I broke the cutoff, and two leaks it still misses.

200 OK, refunded twice: grade your agent on state, not on its answer