sepehr
f403b2851d
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m5s
L0 quality detection layer to catch translation failures BEFORE they
reach users. Pure Python/TypeScript, zero new dependencies, no API calls.
Backend (Python — services/quality/):
- Script detection: 145 langs mapped to 23 scripts (Latin, Cyrillic,
Greek, Arabic, Hebrew, CJK, Hangul, Kana, Devanagari, Bengali, etc.)
- Language confusion detection (e.g. Arabic text for French target)
- Arabic-script variant discrimination (Persian/Urdu/Pashto/Kurdish
confusion — e.g. Persian text returned when Arabic was requested)
- Length sanity check (with numeric/short-source exemptions)
- Prompt leak detection (Translation: / Voici la traduction: / 翻译:)
- Repetition hallucination detection (token + character level)
- File text extraction for .docx/.xlsx/.pptx/.pdf (no translator
changes needed)
- Defensive pipeline that never raises (L0 must NEVER break a job)
Frontend (TypeScript — wordly.art---traduction-de-documents/src/utils/):
- Exact 1:1 mirror of the Python module
- Zero dependencies, works in browser AND Node.js
- Native Unicode regex (\\p{L}/u) and codePoint iteration
- 63 tests using Node's built-in test runner
Integration:
- Feature-flagged: QUALITY_L0_ENABLED=false (default)
- Observation only: logs structured events, never modifies files
- try/except wrapped: impossible to break a translation job
- Lazy imports: only loaded when flag is on
- Zero impact on existing tests / behavior
Tests:
- 111 Python tests covering all paths (config, script, length, leak,
pipeline, file_extractor) — 100% pass
- 63 TypeScript tests (Node --test) — 100% pass
- 174/174 total tests for the L0 layer
Bug fixes in script mapping:
- yi (Yiddish) -> hebrew (was incorrectly mapped to arabic)
- dv (Maldivian) -> thaana (was incorrectly mapped to arabic)
- ja (Japanese) -> hiragana_katakana (distinguishes from Chinese CJK)
Phase 1 (backend) + Phase 2 (frontend) of Track A complete.
Next: Track B1 (Word/Excel format preservation quick wins).
Closes Track A phase 1+2 of the dev plan.
86 lines
2.7 KiB
Python
86 lines
2.7 KiB
Python
"""
|
|
Tests for services/quality/pattern_leak.py
|
|
"""
|
|
import pytest
|
|
|
|
from services.quality.pattern_leak import check
|
|
|
|
|
|
class TestPromptLeak:
|
|
def test_english_translation_prefix(self):
|
|
r = check("Translation: Bonjour le monde")
|
|
assert r["issue"] == "prompt_leak"
|
|
|
|
def test_here_is_translation(self):
|
|
r = check("Here is the translation: Bonjour")
|
|
assert r["issue"] == "prompt_leak"
|
|
|
|
def test_french_voici_traduction(self):
|
|
r = check("Voici la traduction : Bonjour le monde")
|
|
assert r["issue"] == "prompt_leak"
|
|
|
|
def test_chinese_translation_prefix(self):
|
|
r = check("翻译：你好世界")
|
|
assert r["issue"] == "prompt_leak"
|
|
|
|
def test_sure_heres_translation(self):
|
|
r = check("Sure, here's the translation: Hello world")
|
|
assert r["issue"] == "prompt_leak"
|
|
|
|
def test_of_course_heres(self):
|
|
r = check("Of course, here you go: Bonjour")
|
|
assert r["issue"] == "prompt_leak"
|
|
|
|
def test_markdown_bold_translation(self):
|
|
r = check("**Translation** Bonjour le monde")
|
|
assert r["issue"] == "prompt_leak"
|
|
|
|
def test_normal_text_passes(self):
|
|
r = check("Bonjour le monde, comment allez-vous?")
|
|
assert r["issue"] is None
|
|
|
|
def test_text_with_translation_word_in_middle_passes(self):
|
|
r = check("Voici ce que je pense de la traduction de ce texte")
|
|
# The word "traduction" appears but not as a prefix
|
|
assert r["issue"] is None
|
|
|
|
|
|
class TestRepetitionHallucination:
|
|
def test_long_repetition_detected(self):
|
|
r = check("the the the the the the the the")
|
|
assert r["issue"] == "repetition_hallucination"
|
|
assert r["repetition_count"] >= 5
|
|
|
|
def test_short_repetition_passes(self):
|
|
# 4 times is within tolerance
|
|
r = check("I think I think I think I think")
|
|
assert r["issue"] is None
|
|
|
|
def test_normal_text_passes(self):
|
|
r = check("This is a normal sentence with no repetition issues")
|
|
assert r["issue"] is None
|
|
|
|
def test_mixed_repetition_in_long_text_passes(self):
|
|
# Repetition is local, not a hallucination
|
|
r = check("The cat sat on the mat. The cat was happy. The dog was sad.")
|
|
assert r["issue"] is None
|
|
|
|
def test_repetition_with_punctuation(self):
|
|
# "the, the, the, the, the" should still be detected
|
|
r = check("the, the, the, the, the, the")
|
|
assert r["issue"] == "repetition_hallucination"
|
|
|
|
|
|
class TestEmptyAndEdgeCases:
|
|
def test_empty_text(self):
|
|
r = check("")
|
|
assert r["issue"] is None
|
|
|
|
def test_whitespace_only(self):
|
|
r = check(" \n \t ")
|
|
assert r["issue"] is None
|
|
|
|
def test_none_input(self):
|
|
r = check(None)
|
|
assert r["issue"] is None
|