feat(translation): quality pipeline overhaul + new features (audit 2026-08-29)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m20s

Translation quality & format preservation:
- Word: merge adjacent same-format runs into one unit (sentence-level
  coherence like inline-tag handling); translate comments/balloons;
  dedupe textbox collection (was translated twice); RTL no longer
  overrides center/justify alignment; CJK/Arabic font hints (eastAsia/cs)
- PPTX: chart translations now actually reach the output file
  (ChartPart.blob is read-only — rewrite chart XML in the saved ZIP);
  CJK typeface hints (a:ea)
- Excel: sheet renames no longer break references — rewrite cell
  formulas (3D/quoted), defined names, data validations, cond. formats
- PDF: bold/italic honored (hebo/heit/hebi); table cells never merge;
  unchanged blocks left untouched (typography preserved, fixes duplicate
  hyperlinks); attempted/changed stats + route gate now cover PDF;
  CJK font paths; scanned PDFs via Mistral OCR (detection + admin settings)

Features:
- formality param (formal/informal) + automatic regional-variant prompts
- output_mode=bilingual docx (source above translation)
- per-user translation memory on Redis (falls back to LRU), context-hashed
- QA report + 0-100 confidence score in job status; L0 on by default
- OpenAI-compatible providers: whole chunk in ONE numbered-JSON request
  (~15x fewer calls) with per-item fallback; base prompt always present
  (custom prompt no longer replaces translation instructions)

Infra & marketing alignment:
- plan-based engine gating + vision gating (closes paid-engine leak);
  /providers/available filtered per plan; 107 languages exposed
- zh-CN/zh-TW validation fixed; libmagic disabled on Windows (native crash)
- admin: Mistral OCR settings + engine status dashboard; httpx<0.28 pin
  (TestClient breakage); Prometheus test fixture fixed
- marketing docs aligned with code (PDF+OCR, retention, engines, pricing)
- security: .env.ionos/.env.production/provider_settings.json removed

Tests: 1173 passed / 0 failed (6 network tests deselected: free Google
endpoint temporarily blocked from this machine)
This commit is contained in:
2026-08-29 18:38:09 +02:00
parent 992f13d53c
commit 526c87348f
87 changed files with 6996 additions and 1024 deletions

View File

@@ -0,0 +1,92 @@
"""Sheet-name translation must not break references.
openpyxl does not rewrite references on rename — cell formulas, defined
names and validations pointing at a translated sheet would break (#REF!).
These tests cover the full rename + rewrite path.
"""
from openpyxl import Workbook, load_workbook
from openpyxl.workbook.defined_name import DefinedName
from translators.excel_translator import ExcelTranslator
class _FrToEn:
"""Minimal legacy-style provider: French sheet name → English."""
def translate_batch(self, texts, target_language, source_language="auto"):
mapping = {
"Ventes": "Sales",
"Données": "Data",
"Rapport des ventes": "Sales report",
"Total": "Total",
}
return [mapping.get(t, t) for t in texts]
def _make_workbook(path):
wb = Workbook()
ws = wb.active
ws.title = "Ventes"
ws["A1"] = "Rapport des ventes"
ws["A2"] = "Total"
ws["A3"] = 10
ws["A4"] = 20
# Cross-sheet formula (unquoted name)
ws2 = wb.create_sheet("Données")
ws2["A1"] = "=SUM(Ventes!A3:A4)"
# Quoted name (name would need quotes if it had spaces — keep simple here)
ws2["A2"] = "=Ventes!A3+Ventes!A4"
# Defined name pointing at the renamed sheet
wb.defined_names["TotalVentes"] = DefinedName(
"TotalVentes", attr_text="'Ventes'!$A$3"
)
wb.save(path)
return wb
class TestSheetRenameReferences:
def test_cell_formulas_and_defined_names_rewritten(self, tmp_path):
from pathlib import Path
src = tmp_path / "in.xlsx"
out = tmp_path / "out.xlsx"
_make_workbook(src)
translator = ExcelTranslator(provider=_FrToEn())
translator.translate_file(Path(src), Path(out), "en", "fr")
wb = load_workbook(out)
data = wb["Data"]
assert data["A1"].value == "=SUM(Sales!A3:A4)", data["A1"].value
assert data["A2"].value == "=Sales!A3+Sales!A4", data["A2"].value
# Defined name follows the rename
dn = wb.defined_names["TotalVentes"]
assert "Sales" in (dn.attr_text or ""), dn.attr_text
assert "Ventes" not in (dn.attr_text or "")
# The renamed sheet actually exists under its new name
assert "Sales" in wb.sheetnames
assert "Ventes" not in wb.sheetnames
def test_3d_and_multiple_refs(self, tmp_path):
mapping = {"Sheet1": "Feuille1", "Sheet2": "Feuille2"}
formula = "=SUM(Sheet1!A1:Sheet2!B2)+Sheet1!C3"
out = ExcelTranslator._rewrite_sheet_refs_in_formula(formula, mapping)
assert out == "=SUM(Feuille1!A1:Feuille2!B2)+Feuille1!C3"
def test_quoted_refs_and_prefix_safety(self, tmp_path):
mapping = {"Ventes": "Sales", "Ventes 2026": "Sales 2026"}
out = ExcelTranslator._rewrite_sheet_refs_in_formula(
"=SUM('Ventes 2026'!A1:A2)+'Ventes'!B1", mapping
)
# "Ventes 2026" (longest first) keeps its quotes (name with space);
# "Sales" needs no quotes so the canonical unquoted form is emitted.
assert out == "=SUM('Sales 2026'!A1:A2)+Sales!B1"
def test_non_sheet_bang_not_touched(self, tmp_path):
mapping = {"Ventes": "Sales"}
# "Total!A1" is not a renamed sheet — must stay untouched
out = ExcelTranslator._rewrite_sheet_refs_in_formula("=Total!A1", mapping)
assert out == "=Total!A1"