fix(knowledge): classifier silently no-op'd every article (echoed-template + brittle parse)#209
Merged
Merged
Conversation
…emplate + brittle parse)
Found by dry-running the reclassification classifier against real prod articles
BEFORE committing the 77k migration: every call returned
{:error, :unparseable_classification}, which the worker treats as a no-op -- so a
blind commit run would have 'completed' having changed nothing.
Two causes, both fixed:
- The prompt contained a literal JSON template '{"category": "<one of: ...>"}'.
Haiku echoed it verbatim, and '<one of: ...>' is not valid JSON. Reworded the
prompt to describe the two keys without an echo-able brace template.
- parse_response required the ENTIRE trimmed text to be pure JSON. Replaced with
parse_text/1, a pure (unit-testable) function that extracts the first {...}
object, tolerating markdown fences / preamble / trailing prose, then validates
the category is active and confidence is numeric.
Tests: parse_text/1 covers clean JSON, fenced JSON, prose-wrapped, integer
confidence, the echoed-template failure mode, retired/unknown category rejection,
non-numeric confidence, and non-JSON/nil. Full local gate green.
mkreyman
added a commit
that referenced
this pull request
Jun 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
Dry-running the reclassification classifier against real prod articles before committing the 77k migration revealed every call returned
{:error, :unparseable_classification}. The worker treats an error as a no-op, so a blindcommitrun would have 'completed' having changed nothing — the exact 'does nothing at all' failure mode.Mocked unit tests passed; only a real-data dry-run caught it.
Causes + fixes
{"category": "<one of: ...>"}. Haiku echoed it verbatim, and<one of: ...>isn't valid JSON. Reworded to describe the two keys without an echo-able brace template.parse_text/1— a pure, unit-testable function that extracts the first{...}object (tolerating markdown fences / preamble / trailing prose) and validates an active category + numeric confidence.Tests
parse_text/1: clean JSON, ```json fence, prose-wrapped, integer confidence, the echoed-template failure mode, retired/unknown category rejection, non-numeric confidence, non-JSON/nil. Full local gate green (3004 tests, credo, dialyzer 0).