Le defaut le plus sournois du pipeline etait silencieux : de l'arabe
livre pour une cible persane (meme ecriture, mauvaise langue) passe
inapercu et part chez le lecteur. Desormais chaque lot traduit est
verifie par le detecteur d'ecritures, et chaque segment fautif est
redemande une fois au moteur avec une consigne renforcee (nom de la
langue + lettres specifiques, ex. persan پ چ ژ گ). La seconde tentative
ne remplace la premiere que si elle passe le meme controle.
- services/quality/script_detector.py : extraction d'un controle
script_issue() reutilisable ; detect_arabic_variant signale des
ormais un long texte en ecriture arabe sans aucune lettre specifique
de la langue cible (arabe pur livre pour du persan/ourdou/pachto)
- translators/segments.py : retry_wrong_script() + construction de la
consigne renforcee, sans jamais faire echouer le travail
- Word, Excel, PDF : branchement apres la memoire de traduction et les
validations humaines ; le texte inchange (chiffres, noms propres) ne
declenche jamais de retentative
- 14 tests nouveaux (tests/test_translators/test_script_retry.py)
L0 quality detection layer to catch translation failures BEFORE they
reach users. Pure Python/TypeScript, zero new dependencies, no API calls.
Backend (Python — services/quality/):
- Script detection: 145 langs mapped to 23 scripts (Latin, Cyrillic,
Greek, Arabic, Hebrew, CJK, Hangul, Kana, Devanagari, Bengali, etc.)
- Language confusion detection (e.g. Arabic text for French target)
- Arabic-script variant discrimination (Persian/Urdu/Pashto/Kurdish
confusion — e.g. Persian text returned when Arabic was requested)
- Length sanity check (with numeric/short-source exemptions)
- Prompt leak detection (Translation: / Voici la traduction: / 翻译:)
- Repetition hallucination detection (token + character level)
- File text extraction for .docx/.xlsx/.pptx/.pdf (no translator
changes needed)
- Defensive pipeline that never raises (L0 must NEVER break a job)
Frontend (TypeScript — wordly.art---traduction-de-documents/src/utils/):
- Exact 1:1 mirror of the Python module
- Zero dependencies, works in browser AND Node.js
- Native Unicode regex (\\p{L}/u) and codePoint iteration
- 63 tests using Node's built-in test runner
Integration:
- Feature-flagged: QUALITY_L0_ENABLED=false (default)
- Observation only: logs structured events, never modifies files
- try/except wrapped: impossible to break a translation job
- Lazy imports: only loaded when flag is on
- Zero impact on existing tests / behavior
Tests:
- 111 Python tests covering all paths (config, script, length, leak,
pipeline, file_extractor) — 100% pass
- 63 TypeScript tests (Node --test) — 100% pass
- 174/174 total tests for the L0 layer
Bug fixes in script mapping:
- yi (Yiddish) -> hebrew (was incorrectly mapped to arabic)
- dv (Maldivian) -> thaana (was incorrectly mapped to arabic)
- ja (Japanese) -> hiragana_katakana (distinguishes from Chinese CJK)
Phase 1 (backend) + Phase 2 (frontend) of Track A complete.
Next: Track B1 (Word/Excel format preservation quick wins).
Closes Track A phase 1+2 of the dev plan.
