Fix Unicode normalization for LLM-generated simplified articles#483
Merged
mircealungu merged 1 commit intomasterfrom Feb 12, 2026
Merged
Conversation
LLM-generated content (from Anthropic) was being stored without Unicode normalization, causing diacritics like Romanian ă, â to render incorrectly. The text was in NFD (decomposed) form where diacritics are separate combining characters, causing visual spacing issues. - Add NFC normalization to create_simplified_version() for title, content, and summary - Add migration script to fix existing simplified articles with decomposed Unicode characters Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
ArchLens detected architectural changes in the following views: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
create_simplified_version()to ensure proper Unicode encodingTest plan
source ~/.venvs/z_env/bin/activate && python -m tools.migrations.26-02-12--normalize_simplified_article_unicode🤖 Generated with Claude Code