feat(quality): add L0 quality layer (Track A1 + A2 of dev plan)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m5s
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m5s
L0 quality detection layer to catch translation failures BEFORE they
reach users. Pure Python/TypeScript, zero new dependencies, no API calls.
Backend (Python — services/quality/):
- Script detection: 145 langs mapped to 23 scripts (Latin, Cyrillic,
Greek, Arabic, Hebrew, CJK, Hangul, Kana, Devanagari, Bengali, etc.)
- Language confusion detection (e.g. Arabic text for French target)
- Arabic-script variant discrimination (Persian/Urdu/Pashto/Kurdish
confusion — e.g. Persian text returned when Arabic was requested)
- Length sanity check (with numeric/short-source exemptions)
- Prompt leak detection (Translation: / Voici la traduction: / 翻译:)
- Repetition hallucination detection (token + character level)
- File text extraction for .docx/.xlsx/.pptx/.pdf (no translator
changes needed)
- Defensive pipeline that never raises (L0 must NEVER break a job)
Frontend (TypeScript — wordly.art---traduction-de-documents/src/utils/):
- Exact 1:1 mirror of the Python module
- Zero dependencies, works in browser AND Node.js
- Native Unicode regex (\\p{L}/u) and codePoint iteration
- 63 tests using Node's built-in test runner
Integration:
- Feature-flagged: QUALITY_L0_ENABLED=false (default)
- Observation only: logs structured events, never modifies files
- try/except wrapped: impossible to break a translation job
- Lazy imports: only loaded when flag is on
- Zero impact on existing tests / behavior
Tests:
- 111 Python tests covering all paths (config, script, length, leak,
pipeline, file_extractor) — 100% pass
- 63 TypeScript tests (Node --test) — 100% pass
- 174/174 total tests for the L0 layer
Bug fixes in script mapping:
- yi (Yiddish) -> hebrew (was incorrectly mapped to arabic)
- dv (Maldivian) -> thaana (was incorrectly mapped to arabic)
- ja (Japanese) -> hiragana_katakana (distinguishes from Chinese CJK)
Phase 1 (backend) + Phase 2 (frontend) of Track A complete.
Next: Track B1 (Word/Excel format preservation quick wins).
Closes Track A phase 1+2 of the dev plan.
This commit is contained in:
75
services/quality/pipeline.py
Normal file
75
services/quality/pipeline.py
Normal file
@@ -0,0 +1,75 @@
|
||||
"""
|
||||
Quality pipeline — defensive wrapper around the L0 checks.
|
||||
|
||||
The pipeline is the integration point for the route. It:
|
||||
1. Catches all exceptions (L0 must NEVER break a translation job)
|
||||
2. Adds timing
|
||||
3. Emits a single structured log line per job
|
||||
|
||||
The actual checks live in `script_detector`, `length_checker`, `pattern_leak`.
|
||||
This module is the orchestration / safety layer.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from typing import List, Optional
|
||||
|
||||
from core.logging import get_logger
|
||||
|
||||
from .script_detector import evaluate_document, DocumentQualityResult
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
def run_l0_check(
|
||||
source_chunks: List[str],
|
||||
translated_chunks: List[str],
|
||||
target_lang: Optional[str],
|
||||
job_id: Optional[str] = None,
|
||||
file_extension: Optional[str] = None,
|
||||
) -> DocumentQualityResult:
|
||||
"""
|
||||
Run the L0 quality checks defensively. Never raises.
|
||||
|
||||
Returns an empty/neutral DocumentQualityResult on internal error
|
||||
so the calling route can log and continue without affecting the
|
||||
translation job outcome.
|
||||
"""
|
||||
start = time.time()
|
||||
empty = DocumentQualityResult(
|
||||
passed=True,
|
||||
score=0.0,
|
||||
chunk_count=0,
|
||||
failed_chunk_count=0,
|
||||
issues={"internal_error": 1},
|
||||
)
|
||||
|
||||
try:
|
||||
result = evaluate_document(source_chunks, translated_chunks, target_lang)
|
||||
except Exception as e:
|
||||
elapsed_ms = round((time.time() - start) * 1000, 2)
|
||||
logger.warning(
|
||||
"quality_l0_check_failed",
|
||||
job_id=job_id,
|
||||
file_extension=file_extension,
|
||||
error=str(e)[:200],
|
||||
error_type=type(e).__name__,
|
||||
elapsed_ms=elapsed_ms,
|
||||
)
|
||||
return empty
|
||||
|
||||
elapsed_ms = round((time.time() - start) * 1000, 2)
|
||||
logger.info(
|
||||
"quality_l0_check",
|
||||
job_id=job_id,
|
||||
file_extension=file_extension,
|
||||
target_lang=target_lang,
|
||||
chunk_count=result.chunk_count,
|
||||
failed_chunk_count=result.failed_chunk_count,
|
||||
score=result.score,
|
||||
passed=result.passed,
|
||||
issues=result.issues,
|
||||
elapsed_ms=elapsed_ms,
|
||||
)
|
||||
return result
|
||||
Reference in New Issue
Block a user