feat(rtl): rendu droite-a-gauche complet pour Word, PowerPoint, Excel et PDF
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m37s
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m37s
- source unique RTL_LANGUAGES/is_rtl dans core/languages.py (fin des 3 copies) - Word: bidi partout (corps, tableaux bidiVisual, notes/fin/commentaires, zones de texte, 6 zones d'en-tetes/pieds), insertion OOXML ordonnee, polices cs elargies aux 11 langues - PowerPoint: alignements explicites preserves, notes du presentateur, indice de police <a:cs> insert a sa place - Excel: feuilles affichees de droite a gauche, feuilles graphiques ignorees - PDF: faconnage bidi (arabic-reshaper + python-bidi), polices par ecriture (arabe/hebreu), TTF enregistree pour le PDF recompose - 46 tests nouveaux (tests/test_translators/test_rtl_layout.py), 253 au total
This commit is contained in:
33
_bmad-output/implementation-artifacts/deferred-work.md
Normal file
33
_bmad-output/implementation-artifacts/deferred-work.md
Normal file
@@ -0,0 +1,33 @@
|
||||
# Deferred Work
|
||||
|
||||
Entrées ajoutées lors du cadrage de `spec-rendu-fidele-langues-rtl.md` (2026-09-01) :
|
||||
améliorations de qualité identifiées pendant l'audit du pipeline de traduction,
|
||||
volontairement hors du périmètre de la spec RTL pour garder un objectif unique.
|
||||
|
||||
- source_spec: spec-rendu-fidele-langues-rtl.md
|
||||
summary: Fusionner les runs adjacents de même style dans PowerPoint (et les notes Word traduites run par run) pour traduire des phrases complètes au lieu de fragments.
|
||||
evidence: Word fusionne déjà les runs par signature rPr (word_translator.py l.1368-1475) ; PowerPoint traduit chaque run isolément (pptx_translator.py l.810-834) et les footnotes Word aussi (l.836-846, 909-919, 961-971) — une phrase coupée en plusieurs runs produit des traductions incohérentes. Interdit par la spec RTL (« Never : pas de refonte de la collecte des runs »).
|
||||
- source_spec: spec-rendu-fidele-langues-rtl.md
|
||||
summary: Rendre les contrôles qualité actionnables : retenter automatiquement les lots dont le script livré ne correspond pas à la langue cible (ex. arabe livré au lieu du persan).
|
||||
evidence: services/quality/ détecte déjà le mauvais script et la confusion arabe/persan (script_detector.py l.89-171, l.236-256) mais tous les contrôles sont en log-only (routes/translate_routes.py l.1644-1779) — aucune traduction n'est jamais corrigée ni retentée.
|
||||
- source_spec: spec-rendu-fidele-langues-rtl.md
|
||||
summary: Étendre le mode batch JSON (~15 textes par requête) aux providers DeepSeek et MiniMax, comme le fait déjà le provider OpenAI.
|
||||
evidence: openai_provider.py l.511-625 batch via JSON numéroté ; deepseek_provider.py et minimax_provider.py traduisent texte par texte (base.py l.60-74) — coût et latence plus élevés pour ces moteurs.
|
||||
- source_spec: spec-rendu-fidele-langues-rtl.md
|
||||
summary: Conserver le formatage intra-bloc des PDF (un mot en gras au milieu d'une phrase applique aujourd'hui le gras à tout le bloc).
|
||||
evidence: pdf_translator.py _extract_text_blocks l.541-734 calcule des flags gras/italique globaux par bloc et la réécriture utilise une police unique par bloc (l.905-922).
|
||||
|
||||
Entrées ajoutées après la revue croisée de `spec-rendu-fidele-langues-rtl.md` (2026-09-01) :
|
||||
|
||||
- source_spec: spec-rendu-fidele-langues-rtl.md
|
||||
summary: Utiliser insert_htmlbox (PyMuPDF) pour le texte RTL des PDF afin de garder un texte copiable et cherchable (ordre logique), au lieu du façonnage en formes de présentation.
|
||||
evidence: arabic_reshaper + python-bidi écrivent les lettres arabes en formes de présentation (U+FB50–U+FEFF) en ordre visuel : un copier-coller depuis le PDF donne un texte inversé et la recherche ne fonctionne plus ; PyMuPDF ≥ 1.22 (déjà dépendance) propose insert_htmlbox qui gère nativement la direction bidi.
|
||||
- source_spec: spec-rendu-fidele-langues-rtl.md
|
||||
summary: Passer en droite-à-gauche les textes des graphiques et SmartArt Word (parties chart réinjectées après traduction).
|
||||
evidence: les textes de graphiques/diagrammes sont traduits et réinjectés (word_translator.py l.1030-1186, l.1229-1356) mais aucune marque de direction n'est posée sur ces parties — étiquettes arabes affichées gauche-à-droite dans un document RTL. Exclu volontairement par la spec (« Never : pas d'inversion des graphiques »).
|
||||
- source_spec: spec-rendu-fidele-langues-rtl.md
|
||||
summary: Résoudre l'alignement hérité des masques/dispositions PowerPoint pour ne pas forcer à droite un titre centré par le modèle (sans algn explicite).
|
||||
evidence: _set_pptx_paragraph_rtl pose algn="r" quand algn est absent — or l'alignement réel peut venir du masque ; détecter l'héritage exige de remonter au layout/master (chantier de résolution de placeholder dédié).
|
||||
- source_spec: spec-rendu-fidele-langues-rtl.md
|
||||
summary: Harmoniser RTL_LOCALES du frontend (frontend/src/lib/i18n.tsx, liste ["ar","fa"]) avec la liste serveur à onze langues.
|
||||
evidence: la liste d'affichage RTL de l'interface est indépendante de core/languages.RTL_LANGUAGES ; toute nouvelle langue d'interface RTL (hébreu, ourdou) devra être ajoutée aux deux endroits sans qu'aucun contrôle ne le signale.
|
||||
@@ -0,0 +1,147 @@
|
||||
---
|
||||
title: 'Rendu fidèle des traductions pour les langues RTL (persan, arabe, hébreu…)'
|
||||
type: 'feature'
|
||||
created: '2026-09-01'
|
||||
status: 'done'
|
||||
review_loop_iteration: 0
|
||||
baseline_commit: fddd7b74282f6c1e51d20a70b1ca1e5cb89d5331
|
||||
context: []
|
||||
---
|
||||
|
||||
<frozen-after-approval reason="human-owned intent — do not modify unless human renegotiates">
|
||||
|
||||
## Intent
|
||||
|
||||
**Problem:** Un document traduit vers une langue de droite à gauche sort dégradé : Excel ignore totalement le sens, PowerPoint force l'alignement à droite même sur les titres centrés et oublie les notes du présentateur, les tableaux Word ne sont pas inversés (ni notes de bas de page, ni zones de texte), et le PDF n'applique ni le façonnage des lettres arabes ni de police adaptée (lettres déconnectées, ordre inversé, carrés). Pour un lecteur persanophone ou arabophone, le résultat est parfois illisible.
|
||||
|
||||
**Approach:** Source de vérité unique pour le RTL dans `core/languages.py`, puis corrections ciblées dans les 4 traducteurs : direction et alignement corrects sans écraser les alignements explicites, inversion des tableaux, couverture des zones oubliées (notes, zones de texte), et pour le PDF façonnage bidi (`arabic-reshaper` + `python-bidi`) avec police couvrant l'écriture arabe.
|
||||
|
||||
## Boundaries & Constraints
|
||||
|
||||
**Always:** Comportement inchangé pour toute cible non-RTL ; alignements explicites (centré, justifié) jamais écrasés ; aucun texte re-traduit ni altéré (seuls direction/alignement/indices de police sont ajoutés) ; `RTL_LANGUAGES` n'existe plus qu'une fois ; si `arabic-reshaper`/`python-bidi` manquent ou échouent, la traduction réussit quand même (texte tel quel + log d'avertissement).
|
||||
|
||||
**Ask First:** Ajouter une dépendance autre que `arabic-reshaper` et `python-bidi` ; modifier le `Dockerfile` (le paquet `fonts-noto` de Debian devrait déjà fournir Noto Naskh Arabic — vérifier dans l'image avant tout changement).
|
||||
|
||||
**Never:** Pas de refonte de la collecte des runs (fusion des runs PowerPoint / notes Word = chantier différé) ; pas de retentative automatique sur les contrôles qualité (différé) ; pas de modification des providers, de l'interface ni du frontend ; pas d'inversion des images ou graphiques vectoriels.
|
||||
|
||||
## I/O & Edge-Case Matrix
|
||||
|
||||
| Scenario | Input / State | Expected Output / Behavior | Error Handling |
|
||||
|----------|--------------|---------------------------|----------------|
|
||||
| Word cible RTL | docx : corps, tableau, note de bas de page, zone de texte | `w:bidi` sur tous les paragraphes, `w:bidiVisual` sur les tableaux, `w:sectPr/w:bidi` | N/A |
|
||||
| PPTX titre centré, cible ar | `algn="ctr"` existant | `rtl="1"` posé, `algn` reste `"ctr"` ; si `algn` absent ou `"l"` → `algn="r"` | N/A |
|
||||
| PPTX avec notes | diapositive avec notes du présentateur | notes traduites ET passées en RTL | N/A |
|
||||
| Excel cible he | xlsx multi-feuilles | `sheet_view.rightToLeft = True` sur chaque feuille | N/A |
|
||||
| PDF mode mise en page, cible fa/ar | pdf avec blocs de texte | texte façonné avant insertion, police couvrant l'arabe, alignement à droite | Police introuvable → police actuelle + warning ; bibliothèque absente → texte non façonné + warning ; jamais d'échec |
|
||||
| PDF scanné / mode texte, cible RTL | OCR ou mode text_only | reportlab : police TTF enregistrée, `wordWrap="RTL"` sur le style | Pas de TTF → Helvetica + warning (dégradation documentée) |
|
||||
| Cible non-RTL (fr, zh…) | tous formats | Aucun attribut RTL ajouté, sortie identique à aujourd'hui | N/A |
|
||||
| Code régional | `fa-IR`, `ar-EG` | `is_rtl()` normalise le préfixe → RTL appliqué | N/A |
|
||||
|
||||
</frozen-after-approval>
|
||||
|
||||
## Code Map
|
||||
|
||||
- `core/languages.py` -- `LANGUAGE_NAMES` l.12-123, `language_name()` l.126-137 : y ajouter `RTL_LANGUAGES` (frozenset) et `is_rtl(code)`.
|
||||
- `translators/word_translator.py` -- copie `RTL_LANGUAGES` l.30-32 (supprimer) ; `CS_FONTS` l.45-50 (ar/he/fa/ur seulement) ; `_set_paragraph_rtl` l.126-153 (modèle : `w:jc` touché seulement sans alignement explicite) ; `_apply_rtl_to_document` l.156-182 (corps + tableaux + en-têtes/pieds ; manque `w:bidiVisual`, zones de texte, notes) ; déclencheur l.422-424 ; notes/comments réécrits en ZIP post-save l.811-995 (parts sans RTL).
|
||||
- `translators/pptx_translator.py` -- copie l.28-30 (supprimer) ; `_set_pptx_paragraph_rtl` l.57-71 (`algn="r"` inconditionnel — bug) ; `_apply_rtl_to_presentation` l.74-78 (pas de `has_notes_slide`) ; `_apply_ea_font_hints` l.114-158 (modèle exact pour `<a:cs>`) ; notes collectées l.313-319 ; déclencheur l.399-401.
|
||||
- `translators/excel_translator.py` -- `translate_file` l.141-175 ; aucun support RTL (0 occurrence) ; sauvegarde en fin de méthode.
|
||||
- `translators/pdf_translator.py` -- copie l.63-64 (supprimer) ; `_FONT_SEARCH_PATHS` l.115-134 (aucune police arabe) ; `_get_font_path` l.187-196 (première trouvée, insensible au script) ; alignement RTL l.903 ; `insert_textbox` l.1017-1049 (pas de façonnage) ; `_generate_clean_pdf` l.1434-1516 (reportlab, aucune police enregistrée).
|
||||
- `requirements.txt` + `pyproject.toml` (dependencies l.7+) -- dépendances dupliquées : mettre les deux à jour.
|
||||
- `docker/backend/Dockerfile` l.30-40 -- `fonts-noto` installé (vérifier le chemin de Noto Naskh Arabic dans l'image avant tout ajout).
|
||||
- `tests/test_translators/` -- modèles : `test_word_translator.py`, `test_pptx_translator.py`, `test_excel_translator.py`, `test_b3_pdf_fixes.py`.
|
||||
- `services/quality/script_detector.py` -- lecture seule (distingue déjà arabe/persan — réutilisable plus tard, hors périmètre).
|
||||
|
||||
## Tasks & Acceptance
|
||||
|
||||
**Execution:**
|
||||
- [x] `core/languages.py` -- Ajouter `RTL_LANGUAGES` ({ar, he, fa, ur, ku, ps, ug, sd, yi, dv, ckb}) et `is_rtl(code) -> bool` (casse et préfixe régional ignorés) -- source unique, fin des 3 copies.
|
||||
- [x] `translators/word_translator.py` -- Importer depuis `core.languages` ; `_apply_rtl_to_document` : `w:bidiVisual` sur chaque `w:tbl` du corps et des en-têtes/pieds (XPath `.//w:tbl`, couvre l'imbriqué), bidi/rtl sur les paragraphes des `w:txbxContent` ; réécriture ZIP (l.811-995) : `w:bidi` sur les `w:pPr` de `footnotes.xml`/`endnotes.xml`/`comments.xml` quand cible RTL ; `CS_FONTS` élargi à ps, ku, sd, ug, yi, dv, ckb (Arial).
|
||||
- [x] `translators/pptx_translator.py` -- Importer ; `algn="r"` seulement si `algn` absent ou `"l"` ; `_apply_rtl_to_presentation` traite aussi `notes_text_frame` quand `has_notes_slide` ; ajouter `_apply_cs_font_hints` (modèle `_apply_ea_font_hints`) posant `<a:cs typeface="Arial">` pour les cibles RTL.
|
||||
- [x] `translators/excel_translator.py` -- Importer `is_rtl` ; avant sauvegarde : `ws.sheet_view.rightToLeft = True` par feuille quand cible RTL.
|
||||
- [x] `translators/pdf_translator.py` -- Importer ; chemins Noto Naskh Arabic / Noto Sans Arabic (Linux) + `C:/Windows/Fonts/arial.ttf` ; `_get_font_path` prend la langue et choisit une police couvrant l'arabe pour les cibles RTL ; `_shape_rtl(text)` (import protégé, fallback texte tel quel + warning) appliquée aux blocs du mode mise en page et à `_generate_clean_pdf` ; dans ce dernier : `pdfmetrics.registerFont` de la TTF si présente + `wordWrap="RTL"` + `fontName`.
|
||||
- [x] `requirements.txt` + `pyproject.toml` -- Ajouter `arabic-reshaper` et `python-bidi` (versions pinnées comme le reste).
|
||||
- [x] `tests/test_translators/test_rtl_layout.py` -- Couvrir la matrice : docx (bidi corps/note, bidiVisual), pptx (algn conservé/posé, notes, `<a:cs>`), xlsx (`rightToLeft`), pdf (façonnage, police, fallback sans bibliothèque), `is_rtl("fa-IR")`/`is_rtl("fr")`.
|
||||
|
||||
**Acceptance Criteria:**
|
||||
- Given un docx avec tableau et note de bas de page, when traduction vers `fa`, then `w:bidi` sur corps/note/zone de texte et `w:bidiVisual` sur le tableau.
|
||||
- Given un pptx avec un titre `algn="ctr"`, when traduction vers `ar`, then le titre garde `algn="ctr"` et reçoit `rtl="1"`.
|
||||
- Given un xlsx de 3 feuilles, when traduction vers `he`, then les 3 `sheet_view.rightToLeft` sont vrais.
|
||||
- Given un pdf textuel traduit vers `fa`, when mode mise en page, then texte façonné inséré et police arabe sélectionnée si présente.
|
||||
- Given une traduction vers `fr`, then aucun attribut RTL ajouté (sortie inchangée).
|
||||
|
||||
## Spec Change Log
|
||||
|
||||
## Verification
|
||||
|
||||
**Commands:**
|
||||
- `python -m pytest tests/test_translators/test_rtl_layout.py -v` -- expected: tous les tests passent.
|
||||
- `python -m pytest tests/test_translators/ tests/test_language_validation.py -q` -- expected: aucune régression.
|
||||
- `python -c "import arabic_reshaper, bidi.algorithm; print('ok')"` -- expected: ok.
|
||||
|
||||
## Suggested Review Order
|
||||
|
||||
**Source de vérité des langues**
|
||||
|
||||
- Liste unique partagée par les 4 traducteurs, fin des 3 copies locales
|
||||
[`languages.py:142`](../../core/languages.py#L142)
|
||||
|
||||
- Normalisation casse/préfixe régional (fa-IR, AR → RTL ; fr, None → non)
|
||||
[`languages.py:147`](../../core/languages.py#L147)
|
||||
|
||||
**Word — direction et ordre OOXML**
|
||||
|
||||
- Insertion positionnée selon le schéma (bidi/jc/rtl/bidiVisual jamais en fin)
|
||||
[`word_translator.py:187`](../../translators/word_translator.py#L187)
|
||||
|
||||
- Paragraphe RTL : bidi + miroir d'alignement sans toucher centré/justifié
|
||||
[`word_translator.py:202`](../../translators/word_translator.py#L202)
|
||||
|
||||
- Tableaux inversés visuellement (w:bidiVisual)
|
||||
[`word_translator.py:236`](../../translators/word_translator.py#L236)
|
||||
|
||||
- Notes/commentaires : bidi + rtl sur les runs + tableaux, pendant la réécriture ZIP
|
||||
[`word_translator.py:254`](../../translators/word_translator.py#L254)
|
||||
|
||||
- Les 6 zones d'en-têtes/pieds (défaut, première page, pages paires) couvertes
|
||||
[`word_translator.py:288`](../../translators/word_translator.py#L288)
|
||||
|
||||
**PowerPoint — alignements et polices**
|
||||
|
||||
- rtl=1 posé, algn forcé à droite seulement si absent ou « l »
|
||||
[`pptx_translator.py:71`](../../translators/pptx_translator.py#L71)
|
||||
|
||||
- Notes du présentateur incluses dans le passage RTL
|
||||
[`pptx_translator.py:91`](../../translators/pptx_translator.py#L91)
|
||||
|
||||
- Indice de police <a:ea>/<a:cs> inséré à sa place, rPr en tête du run, notes incluses
|
||||
[`pptx_translator.py:136`](../../translators/pptx_translator.py#L136)
|
||||
|
||||
**Excel — sens des feuilles**
|
||||
|
||||
- rightToLeft par feuille, protégé (feuilles graphiques sans sheet_view ignorées)
|
||||
[`excel_translator.py:369`](../../translators/excel_translator.py#L369)
|
||||
|
||||
**PDF — façonnage et polices**
|
||||
|
||||
- Façonnage bidi (lettres liées, bon ordre), repli sans échec si bibliothèques absentes
|
||||
[`pdf_translator.py:69`](../../translators/pdf_translator.py#L69)
|
||||
|
||||
- Police enregistrée une seule fois par chemin, nom unique par processus
|
||||
[`pdf_translator.py:106`](../../translators/pdf_translator.py#L106)
|
||||
|
||||
- Groupe d'écriture (arabe / hébreu / générique) avant le choix de police
|
||||
[`pdf_translator.py:185`](../../translators/pdf_translator.py#L185)
|
||||
|
||||
- Sélection de police sensible à l'écriture cible, cache par groupe
|
||||
[`pdf_translator.py:309`](../../translators/pdf_translator.py#L309)
|
||||
|
||||
- PDF recomposé : TTF enregistrée, wordWrap RTL, texte façonné
|
||||
[`pdf_translator.py:1597`](../../translators/pdf_translator.py#L1597)
|
||||
|
||||
**Périphérie**
|
||||
|
||||
- 46 tests couvrant la matrice (docx/pptx/xlsx/pdf + endnotes/comments/chartsheet)
|
||||
[`test_rtl_layout.py:1`](../../tests/test_translators/test_rtl_layout.py#L1)
|
||||
|
||||
- Dépendances façonnage (requirements.txt + pyproject.toml + uv.lock alignés)
|
||||
[`requirements.txt:11`](../../requirements.txt#L11)
|
||||
@@ -7,7 +7,7 @@ prompts say "Translate to Tagalog" instead of "Translate to tl" — LLMs
|
||||
translate noticeably better with full language names.
|
||||
"""
|
||||
|
||||
from typing import Dict
|
||||
from typing import Dict, Optional
|
||||
|
||||
LANGUAGE_NAMES: Dict[str, str] = {
|
||||
"af": "Afrikaans",
|
||||
@@ -135,3 +135,22 @@ def language_name(code: str) -> str:
|
||||
if name is None:
|
||||
name = LANGUAGE_NAMES.get(code.split("-")[0].lower(), code)
|
||||
return name
|
||||
|
||||
|
||||
# Languages written right-to-left. Single source of truth shared by every
|
||||
# translator (Word, PowerPoint, Excel, PDF) — do not duplicate this set.
|
||||
RTL_LANGUAGES: frozenset = frozenset(
|
||||
{"ar", "he", "fa", "ur", "ku", "ps", "ug", "sd", "yi", "dv", "ckb"}
|
||||
)
|
||||
|
||||
|
||||
def is_rtl(code: Optional[str]) -> bool:
|
||||
"""True when the language is written right-to-left.
|
||||
|
||||
Case and regional suffixes are ignored: "fa-IR", "AR" and "ar-EG" are
|
||||
RTL, "fr" and "" are not. None is accepted (treated as not RTL).
|
||||
"""
|
||||
if not code:
|
||||
return False
|
||||
base = code.strip().replace("_", "-").split("-")[0].lower()
|
||||
return base in RTL_LANGUAGES
|
||||
|
||||
@@ -9,6 +9,7 @@ dependencies = [
|
||||
"aiosmtplib>=3.0.0",
|
||||
"aiosqlite>=0.19.0",
|
||||
"alembic==1.13.1",
|
||||
"arabic-reshaper==3.0.0",
|
||||
"asyncpg>=0.29.0",
|
||||
"bcrypt<4.1",
|
||||
"deep-translator==1.11.4",
|
||||
@@ -33,6 +34,7 @@ dependencies = [
|
||||
"pymupdf>=1.24.0",
|
||||
"pytest>=7.0.0",
|
||||
"pytest-asyncio>=0.21.0",
|
||||
"python-bidi==0.4.2",
|
||||
"python-docx==1.1.0",
|
||||
"python-dotenv==1.0.0",
|
||||
"python-magic>=0.4.27",
|
||||
|
||||
@@ -8,6 +8,8 @@ pdf2docx>=0.5.6
|
||||
PyMuPDF>=1.24.0
|
||||
lxml>=4.9.0
|
||||
reportlab>=4.0.0
|
||||
arabic-reshaper==3.0.0
|
||||
python-bidi==0.4.2
|
||||
deep-translator==1.11.4
|
||||
python-dotenv==1.0.0
|
||||
pydantic==2.5.3
|
||||
|
||||
@@ -11,7 +11,7 @@ spec.loader.exec_module(pdf_mod)
|
||||
|
||||
original_write = pdf_mod.PDFTranslator._write_translated_block
|
||||
|
||||
def debug_write(self, page, block, font_path, is_rtl):
|
||||
def debug_write(self, page, block, font_path, rtl_target):
|
||||
target_size = block['font_size']
|
||||
original_rect = fitz.Rect(block['bbox'])
|
||||
page_rect = page.rect
|
||||
@@ -40,7 +40,7 @@ def debug_write(self, page, block, font_path, is_rtl):
|
||||
text = block['translated'][:40]
|
||||
is_heading_str = "HEAD" if is_heading else "BODY"
|
||||
print(f" [{is_heading_str}] text={text!r} | orig={original_rect.width:.0f}x{original_rect.height:.0f} | exp_v={expanded_v.width:.0f}x{expanded_v.height:.0f} | size={target_size}", flush=True)
|
||||
result = original_write(self, page, block, font_path, is_rtl)
|
||||
result = original_write(self, page, block, font_path, rtl_target)
|
||||
print(f" result: {result}", flush=True)
|
||||
return result
|
||||
|
||||
|
||||
@@ -99,7 +99,7 @@ class TestSmartFitTiers:
|
||||
|
||||
translator = pdf_mod.PDFTranslator()
|
||||
result = translator._write_translated_block(
|
||||
page, block, font_path=None, is_rtl=False
|
||||
page, block, font_path=None, rtl_target=False
|
||||
)
|
||||
# Should have succeeded
|
||||
assert result is True
|
||||
@@ -122,7 +122,7 @@ class TestSmartFitTiers:
|
||||
block = self._make_block(font_size=12, x0=100, y0=100, x1=400, y1=120)
|
||||
translator = pdf_mod.PDFTranslator()
|
||||
result = translator._write_translated_block(
|
||||
page, block, font_path=None, is_rtl=False
|
||||
page, block, font_path=None, rtl_target=False
|
||||
)
|
||||
assert result is True
|
||||
# Should have tried multiple times
|
||||
@@ -137,7 +137,7 @@ class TestSmartFitTiers:
|
||||
block = self._make_block(font_size=20, x0=100, y0=100, x1=400, y1=120)
|
||||
translator = pdf_mod.PDFTranslator()
|
||||
result = translator._write_translated_block(
|
||||
page, block, font_path=None, is_rtl=False
|
||||
page, block, font_path=None, rtl_target=False
|
||||
)
|
||||
# Should have failed (returned False) because persistent overflow
|
||||
assert result is False
|
||||
@@ -159,7 +159,7 @@ class TestSmartFitTiers:
|
||||
block = self._make_block(font_size=22, text="X" * 1000)
|
||||
translator = pdf_mod.PDFTranslator()
|
||||
translator._write_translated_block(
|
||||
page, block, font_path=None, is_rtl=False
|
||||
page, block, font_path=None, rtl_target=False
|
||||
)
|
||||
# All attempted sizes should be >= 22 * 0.90 = 19.8
|
||||
for s in call_sizes:
|
||||
@@ -180,7 +180,7 @@ class TestSmartFitTiers:
|
||||
block = self._make_block(font_size=12, text="Y" * 1000)
|
||||
translator = pdf_mod.PDFTranslator()
|
||||
translator._write_translated_block(
|
||||
page, block, font_path=None, is_rtl=False
|
||||
page, block, font_path=None, rtl_target=False
|
||||
)
|
||||
for s in call_sizes:
|
||||
if s is not None:
|
||||
@@ -201,7 +201,7 @@ class TestSmartFitTiers:
|
||||
with patch.object(pdf_mod, "_record_format_loss_metric", fake_record):
|
||||
block = self._make_block(font_size=12, text="Z" * 1000)
|
||||
result = translator._write_translated_block(
|
||||
page, block, font_path=None, is_rtl=False
|
||||
page, block, font_path=None, rtl_target=False
|
||||
)
|
||||
assert result is False
|
||||
# A format_loss metric should have been emitted
|
||||
@@ -221,7 +221,7 @@ class TestSmartFitTiers:
|
||||
with patch.object(pdf_mod, "_record_format_loss_metric", fake_record):
|
||||
block = self._make_block(font_size=12, text="Short")
|
||||
result = translator._write_translated_block(
|
||||
page, block, font_path=None, is_rtl=False
|
||||
page, block, font_path=None, rtl_target=False
|
||||
)
|
||||
assert result is True
|
||||
# No format_loss should have been emitted
|
||||
@@ -242,7 +242,7 @@ class TestSmartFitTiers:
|
||||
block = self._make_block(font_size=3, text="Q" * 1000)
|
||||
translator = pdf_mod.PDFTranslator()
|
||||
translator._write_translated_block(
|
||||
page, block, font_path=None, is_rtl=False
|
||||
page, block, font_path=None, rtl_target=False
|
||||
)
|
||||
# Exclude the placeholder font (used in graceful failure)
|
||||
real_sizes = [s for s in call_sizes if s is not None and s >= 4.5]
|
||||
|
||||
1008
tests/test_translators/test_rtl_layout.py
Normal file
1008
tests/test_translators/test_rtl_layout.py
Normal file
File diff suppressed because it is too large
Load Diff
@@ -23,6 +23,7 @@ from openpyxl.cell.cell import Cell
|
||||
from openpyxl.utils import get_column_letter
|
||||
|
||||
from services.providers.base import TranslationProvider
|
||||
from core.languages import is_rtl
|
||||
|
||||
|
||||
from core.logging import get_logger
|
||||
@@ -354,6 +355,26 @@ class ExcelTranslator:
|
||||
except Exception as e:
|
||||
_log_error("excel_sheet_images_failed", sheet_name=sheet_name, error=str(e))
|
||||
|
||||
# RTL targets: flip the sheet view so every sheet reads
|
||||
# right-to-left (column A renders rightmost). Content and
|
||||
# formatting are untouched — only the reading direction.
|
||||
# Chartsheets have no `sheet_view` and must never break the
|
||||
# save: each sheet is guarded individually.
|
||||
if is_rtl(target_language):
|
||||
flipped = 0
|
||||
for sheet_name in workbook.sheetnames:
|
||||
ws = workbook[sheet_name]
|
||||
try:
|
||||
if getattr(ws, "sheet_view", None) is not None:
|
||||
ws.sheet_view.rightToLeft = True
|
||||
flipped += 1
|
||||
except Exception as e:
|
||||
_log_error(
|
||||
"excel_rtl_view_failed", sheet=sheet_name, error=str(e)
|
||||
)
|
||||
if flipped:
|
||||
_log_info("excel_rtl_view_applied", sheets=flipped)
|
||||
|
||||
try:
|
||||
workbook.save(output_path)
|
||||
except Exception as e:
|
||||
@@ -580,7 +601,14 @@ class ExcelTranslator:
|
||||
) -> None:
|
||||
"""Collect all translatable text from worksheet cells, cell comments,
|
||||
and cell hyperlinks (URLs are preserved as-is, but their display
|
||||
labels are translated)."""
|
||||
labels are translated).
|
||||
|
||||
Chartsheets are skipped: they have no cell grid (no iter_rows)
|
||||
— their text lives in the chart XML, collected separately from
|
||||
the ZIP.
|
||||
"""
|
||||
if not hasattr(worksheet, "iter_rows"):
|
||||
return
|
||||
for row in worksheet.iter_rows():
|
||||
for cell in row:
|
||||
if cell.value is not None:
|
||||
|
||||
@@ -33,10 +33,12 @@ Scanned PDFs:
|
||||
import time
|
||||
import shutil
|
||||
import subprocess
|
||||
import zlib
|
||||
from pathlib import Path
|
||||
from typing import Dict, Any, Optional, Callable, List
|
||||
|
||||
from core.logging import get_logger
|
||||
from core.languages import is_rtl
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
@@ -60,8 +62,70 @@ HEADING_MIN_SCALE = 0.90
|
||||
# Maximum font shrink for body text
|
||||
BODY_MIN_SCALE = 0.75
|
||||
|
||||
# RTL language codes
|
||||
RTL_LANGUAGES = frozenset({"ar", "he", "fa", "ur", "ku", "ps", "ug", "sd", "yi", "dv", "ckb"})
|
||||
# Warn at most once when the RTL shaping libraries are missing.
|
||||
_RTL_SHAPING_MISSING_WARNED = False
|
||||
|
||||
|
||||
def _shape_rtl(text: str) -> str:
|
||||
"""Shape Arabic-script text for PDF engines that do no bidi of their own.
|
||||
|
||||
arabic-reshaper joins the letters into their contextual forms and
|
||||
python-bidi reorders the string for visual (left-to-right) rendering.
|
||||
Both libraries are optional on purpose: when they are missing or
|
||||
fail, the text is returned unchanged with a warning — the
|
||||
translation itself must always succeed, only the rendering degrades.
|
||||
"""
|
||||
global _RTL_SHAPING_MISSING_WARNED
|
||||
if not text:
|
||||
return text
|
||||
try:
|
||||
import arabic_reshaper
|
||||
from bidi.algorithm import get_display
|
||||
except Exception:
|
||||
if not _RTL_SHAPING_MISSING_WARNED:
|
||||
logger.warning(
|
||||
"rtl_shaping_libs_missing",
|
||||
hint="pip install arabic-reshaper python-bidi for correct RTL rendering",
|
||||
)
|
||||
_RTL_SHAPING_MISSING_WARNED = True
|
||||
return text
|
||||
try:
|
||||
return get_display(arabic_reshaper.reshape(text))
|
||||
except Exception as e:
|
||||
logger.warning("rtl_shaping_failed", error=str(e)[:200])
|
||||
return text
|
||||
|
||||
|
||||
# Font path → reportlab font name, for RTL fonts registered in this
|
||||
# process. A fixed name would make two consecutive translations with
|
||||
# different fonts (e.g. Arabic then Hebrew) overwrite each other's
|
||||
# registration.
|
||||
_RTL_FONT_REGISTRATIONS: Dict[str, str] = {}
|
||||
|
||||
|
||||
def _register_rtl_font(font_path: str) -> str:
|
||||
"""Register a TTF under a stable per-path name, once per process.
|
||||
|
||||
Returns the reportlab font name, or "Helvetica" when registration
|
||||
fails (warning logged — a font problem must never fail the job).
|
||||
"""
|
||||
registered = _RTL_FONT_REGISTRATIONS.get(font_path)
|
||||
if registered:
|
||||
return registered
|
||||
try:
|
||||
from reportlab.pdfbase import pdfmetrics
|
||||
from reportlab.pdfbase.ttfonts import TTFont
|
||||
|
||||
digest = format(zlib.crc32(str(font_path).encode("utf-8")) & 0xFFFFFFFF, "08x")
|
||||
name = f"WordlyRTL-{digest}"
|
||||
pdfmetrics.registerFont(TTFont(name, font_path))
|
||||
_RTL_FONT_REGISTRATIONS[font_path] = name
|
||||
return name
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
"pdf_rtl_font_register_failed", error=str(e)[:200], font=font_path
|
||||
)
|
||||
return "Helvetica"
|
||||
|
||||
|
||||
def _record_format_loss_metric(element_type: str, count: int = 1) -> None:
|
||||
@@ -109,6 +173,29 @@ def _libreoffice_available() -> bool:
|
||||
_libreoffice_available._cache = None # type: ignore[attr-defined]
|
||||
|
||||
|
||||
# Script groups among the RTL languages — the preferred font depends on
|
||||
# the writing system, not just the direction: a Hebrew target would
|
||||
# render as tofu with an Arabic-only font.
|
||||
_ARABIC_SCRIPT_LANGUAGES = frozenset(
|
||||
{"ar", "fa", "ur", "ps", "ku", "sd", "ug", "ckb"}
|
||||
)
|
||||
_HEBREW_SCRIPT_LANGUAGES = frozenset({"he", "yi"})
|
||||
|
||||
|
||||
def _rtl_script(code: str) -> str:
|
||||
"""Script group of an RTL language code: "arabic", "hebrew" or ""
|
||||
|
||||
("" meaning: use the generic Unicode font list — e.g. Thaana "dv",
|
||||
for which no dedicated font is searched).
|
||||
"""
|
||||
base = (code or "").strip().replace("_", "-").split("-")[0].lower()
|
||||
if base in _ARABIC_SCRIPT_LANGUAGES:
|
||||
return "arabic"
|
||||
if base in _HEBREW_SCRIPT_LANGUAGES:
|
||||
return "hebrew"
|
||||
return ""
|
||||
|
||||
|
||||
class PDFTranslator:
|
||||
"""Translates PDF files with layout preservation using PyMuPDF."""
|
||||
|
||||
@@ -133,9 +220,37 @@ class PDFTranslator:
|
||||
"/System/Library/Fonts/Helvetica.ttc",
|
||||
]
|
||||
|
||||
# Arabic-script-capable fonts (ar, fa, ur, ps, ku, sd, ug, ckb),
|
||||
# searched FIRST for those targets. Debian's fonts-noto package
|
||||
# ships Noto Naskh Arabic and Noto Sans Arabic under
|
||||
# /usr/share/fonts/truetype/noto/; on Windows, Arial covers the
|
||||
# whole Arabic script.
|
||||
_RTL_ARABIC_FONT_SEARCH_PATHS = [
|
||||
"/usr/share/fonts/truetype/noto/NotoNaskhArabic-Regular.ttf",
|
||||
"/usr/share/fonts/truetype/noto/NotoSansArabic-Regular.ttf",
|
||||
"/usr/share/fonts/opentype/noto/NotoNaskhArabic-Regular.ttf",
|
||||
"/usr/share/fonts/opentype/noto/NotoSansArabic-Regular.ttf",
|
||||
"/app/fonts/NotoNaskhArabic-Regular.ttf",
|
||||
"/app/fonts/NotoSansArabic-Regular.ttf",
|
||||
"C:/Windows/Fonts/arial.ttf",
|
||||
]
|
||||
|
||||
# Hebrew-script-capable fonts (he, yi), searched FIRST for those
|
||||
# targets. Same Debian package provides Noto Sans Hebrew; Arial
|
||||
# covers Hebrew on Windows.
|
||||
_RTL_HEBREW_FONT_SEARCH_PATHS = [
|
||||
"/usr/share/fonts/truetype/noto/NotoSansHebrew-Regular.ttf",
|
||||
"/usr/share/fonts/opentype/noto/NotoSansHebrew-Regular.ttf",
|
||||
"/app/fonts/NotoSansHebrew-Regular.ttf",
|
||||
"C:/Windows/Fonts/arial.ttf",
|
||||
]
|
||||
|
||||
def __init__(self, provider=None):
|
||||
self._provider = provider
|
||||
self._font_path: Optional[str] = None
|
||||
# script group ("arabic"/"hebrew"/"") → resolved font path, so a
|
||||
# single instance can serve Arabic and Hebrew jobs in a row.
|
||||
self._rtl_font_paths: Dict[str, str] = {}
|
||||
self._translation_stats = {"attempted": 0, "changed": 0}
|
||||
self._custom_prompt: Optional[str] = None
|
||||
# OCR overrides (admin settings); None → fall back to config.MISTRAL_*
|
||||
@@ -184,14 +299,56 @@ class PDFTranslator:
|
||||
recorder = getattr(self, "_segment_recorder", None)
|
||||
return recorder.get_pairs() if recorder is not None else []
|
||||
|
||||
def _get_font_path(self) -> Optional[str]:
|
||||
"""Resolve a Unicode-capable TTF/OTF font file."""
|
||||
if self._font_path is not None:
|
||||
return self._font_path
|
||||
for p in self._FONT_SEARCH_PATHS:
|
||||
@staticmethod
|
||||
def _first_existing_font(paths) -> Optional[str]:
|
||||
for p in paths:
|
||||
if Path(p).exists():
|
||||
self._font_path = p
|
||||
return p
|
||||
return None
|
||||
|
||||
def _get_font_path(self, target_language: str = "") -> Optional[str]:
|
||||
"""Resolve a Unicode-capable TTF/OTF font file.
|
||||
|
||||
RTL targets first search a font covering THEIR script — Noto
|
||||
Naskh/Sans Arabic for the Arabic script (ar, fa, ur…), Noto Sans
|
||||
Hebrew for Hebrew (he, yi); other RTL scripts (Thaana "dv") use
|
||||
the generic Unicode list directly. A missing script font falls
|
||||
back to the generic list with a warning — a missing font
|
||||
degrades the rendering but never fails the job. Non-RTL targets
|
||||
use the generic list only (unchanged behavior).
|
||||
"""
|
||||
if not is_rtl(target_language):
|
||||
if self._font_path:
|
||||
return self._font_path
|
||||
found = self._first_existing_font(self._FONT_SEARCH_PATHS)
|
||||
if not found:
|
||||
logger.warning("no_unicode_font_found")
|
||||
return None
|
||||
self._font_path = found
|
||||
return found
|
||||
|
||||
script = _rtl_script(target_language)
|
||||
cached = self._rtl_font_paths.get(script)
|
||||
if cached:
|
||||
return cached
|
||||
|
||||
if script == "arabic":
|
||||
preferred = self._RTL_ARABIC_FONT_SEARCH_PATHS
|
||||
elif script == "hebrew":
|
||||
preferred = self._RTL_HEBREW_FONT_SEARCH_PATHS
|
||||
else:
|
||||
preferred = []
|
||||
|
||||
found = self._first_existing_font(preferred + self._FONT_SEARCH_PATHS)
|
||||
if found:
|
||||
if preferred and found not in preferred:
|
||||
logger.warning(
|
||||
"rtl_script_font_not_found_current_font_used",
|
||||
script=script,
|
||||
font=found,
|
||||
)
|
||||
self._rtl_font_paths[script] = found
|
||||
return found
|
||||
logger.warning("no_unicode_font_found")
|
||||
return None
|
||||
|
||||
@@ -259,7 +416,7 @@ class PDFTranslator:
|
||||
doc.close()
|
||||
raise RuntimeError("PDF has no pages.")
|
||||
|
||||
font_path = self._get_font_path()
|
||||
font_path = self._get_font_path(target_language)
|
||||
logger.info(
|
||||
"pdf_layout_start",
|
||||
pages=total_pages,
|
||||
@@ -311,7 +468,7 @@ class PDFTranslator:
|
||||
"""Core PyMuPDF in-place processing — one page at a time."""
|
||||
import fitz
|
||||
|
||||
is_rtl = target_language.lower() in RTL_LANGUAGES
|
||||
rtl_target = is_rtl(target_language)
|
||||
total_blocks = 0
|
||||
translated_blocks = 0
|
||||
|
||||
@@ -514,7 +671,7 @@ class PDFTranslator:
|
||||
for block in blocks:
|
||||
if block.get("translated"):
|
||||
self._write_translated_block(
|
||||
page, block, font_path, is_rtl
|
||||
page, block, font_path, rtl_target
|
||||
)
|
||||
|
||||
if progress_callback:
|
||||
@@ -871,7 +1028,7 @@ class PDFTranslator:
|
||||
page,
|
||||
block: Dict,
|
||||
font_path: Optional[str],
|
||||
is_rtl: bool,
|
||||
rtl_target: bool,
|
||||
) -> bool:
|
||||
"""Write translated text into the block's bounding box.
|
||||
|
||||
@@ -897,6 +1054,12 @@ class PDFTranslator:
|
||||
|
||||
original_rect = fitz.Rect(block["bbox"])
|
||||
translated = block["translated"]
|
||||
if rtl_target:
|
||||
# Shape Arabic-script text (contextual letter forms + visual
|
||||
# order) before insertion — insert_textbox does no bidi
|
||||
# processing of its own. Missing/failed libs degrade to the
|
||||
# raw text (warning logged inside _shape_rtl).
|
||||
translated = _shape_rtl(translated)
|
||||
target_size = block["font_size"]
|
||||
|
||||
color = self._int_to_rgb(block["color"])
|
||||
@@ -1443,8 +1606,20 @@ class PDFTranslator:
|
||||
from reportlab.platypus import SimpleDocTemplate, Paragraph, Spacer, PageBreak
|
||||
from reportlab.lib.styles import getSampleStyleSheet
|
||||
|
||||
is_rtl = target_language.lower() in RTL_LANGUAGES
|
||||
alignment = TA_RIGHT if is_rtl else TA_JUSTIFY
|
||||
rtl_target = is_rtl(target_language)
|
||||
alignment = TA_RIGHT if rtl_target else TA_JUSTIFY
|
||||
|
||||
# RTL targets need a TTF font covering the target script (the
|
||||
# built-in Helvetica has no Arabic/Hebrew glyphs). Registration
|
||||
# failures degrade to Helvetica with a warning — never a failed
|
||||
# job.
|
||||
font_name = "Helvetica"
|
||||
if rtl_target:
|
||||
rtl_font = self._get_font_path(target_language)
|
||||
if rtl_font:
|
||||
font_name = _register_rtl_font(rtl_font)
|
||||
else:
|
||||
logger.warning("pdf_rtl_font_missing_helvetica_fallback")
|
||||
|
||||
styles = getSampleStyleSheet()
|
||||
|
||||
@@ -1464,6 +1639,8 @@ class PDFTranslator:
|
||||
leading=16,
|
||||
spaceAfter=6,
|
||||
alignment=alignment,
|
||||
fontName=font_name,
|
||||
wordWrap="RTL" if rtl_target else None,
|
||||
textColor=colors.HexColor("#1a1a1a"),
|
||||
)
|
||||
|
||||
@@ -1490,6 +1667,11 @@ class PDFTranslator:
|
||||
elements.append(Spacer(1, 4))
|
||||
continue
|
||||
|
||||
if rtl_target:
|
||||
# Shape Arabic-script text before rendering —
|
||||
# reportlab does no bidi processing of its own.
|
||||
para_text = _shape_rtl(para_text)
|
||||
|
||||
safe = (
|
||||
para_text
|
||||
.replace("&", "&")
|
||||
|
||||
@@ -20,16 +20,30 @@ from pptx.shapes.group import GroupShape
|
||||
from pptx.enum.shapes import MSO_SHAPE_TYPE
|
||||
|
||||
from services.providers.base import TranslationProvider
|
||||
from core.languages import is_rtl
|
||||
|
||||
# DrawingML namespace used by pptx XML
|
||||
_NS_A = "http://schemas.openxmlformats.org/drawingml/2006/main"
|
||||
|
||||
# Languages written right-to-left
|
||||
RTL_LANGUAGES: frozenset = frozenset(
|
||||
{"ar", "he", "fa", "ur", "ku", "ps", "ug", "sd", "yi", "dv", "ckb"}
|
||||
# a:rPr children that must come AFTER <a:latin>/<a:ea>/<a:cs> per the
|
||||
# DrawingML CT_TextCharacterProperties sequence. Inserting <a:ea>/<a:cs>
|
||||
# before them keeps the XML schema-valid (PowerPoint repairs otherwise).
|
||||
_A_RPR_HINT_SUCCESSORS = tuple(
|
||||
f"{{{_NS_A}}}{tag}"
|
||||
for tag in ("sym", "hlinkClick", "hlinkMouseOver", "rtl", "extLst")
|
||||
)
|
||||
|
||||
|
||||
def _insert_a_rpr_child(rPr, child) -> None:
|
||||
"""Insert child into a:rPr at its schema position (before sym,
|
||||
hlink*, rtl and extLst — or appended at the end when none exist)."""
|
||||
for existing in rPr:
|
||||
if existing.tag in _A_RPR_HINT_SUCCESSORS:
|
||||
existing.addprevious(child)
|
||||
return
|
||||
rPr.append(child)
|
||||
|
||||
|
||||
from core.logging import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
@@ -58,8 +72,10 @@ def _set_pptx_paragraph_rtl(paragraph) -> None:
|
||||
"""
|
||||
Enable RTL mode on a PowerPoint paragraph.
|
||||
|
||||
Sets rtl="1" and algn="r" on the <a:pPr> element, which controls
|
||||
both text direction and horizontal alignment in DrawingML.
|
||||
Sets rtl="1" on the <a:pPr> element, which controls the paragraph
|
||||
text direction. The horizontal alignment (algn) is only forced to
|
||||
"r" when the paragraph has no explicit alignment or is aligned
|
||||
"l" (left) — a centered or justified title keeps its alignment.
|
||||
"""
|
||||
p_elem = paragraph._p
|
||||
tag_pPr = f"{{{_NS_A}}}pPr"
|
||||
@@ -68,14 +84,20 @@ def _set_pptx_paragraph_rtl(paragraph) -> None:
|
||||
pPr = etree.Element(tag_pPr)
|
||||
p_elem.insert(0, pPr)
|
||||
pPr.set("rtl", "1")
|
||||
pPr.set("algn", "r")
|
||||
if pPr.get("algn") in (None, "", "l"):
|
||||
pPr.set("algn", "r")
|
||||
|
||||
|
||||
def _apply_rtl_to_presentation(presentation: Presentation) -> None:
|
||||
"""Apply RTL direction to every paragraph in all slides."""
|
||||
"""Apply RTL direction to every paragraph in all slides and notes."""
|
||||
for slide in presentation.slides:
|
||||
for shape in slide.shapes:
|
||||
_apply_rtl_to_shape(shape)
|
||||
# Speaker notes are translated like any other text — they read
|
||||
# right-to-left too when the target language does.
|
||||
if slide.has_notes_slide and slide.notes_slide.notes_text_frame is not None:
|
||||
for paragraph in slide.notes_slide.notes_text_frame.paragraphs:
|
||||
_set_pptx_paragraph_rtl(paragraph)
|
||||
|
||||
|
||||
def _apply_rtl_to_shape(shape) -> None:
|
||||
@@ -111,11 +133,10 @@ def _ea_typeface_for_target(target_language: str):
|
||||
return _EA_TYPEFACES.get(code) or _EA_TYPEFACES.get(base)
|
||||
|
||||
|
||||
def _apply_ea_font_hints(presentation: Presentation, target_language: str) -> None:
|
||||
"""Set the <a:ea> (east-asian) typeface on every run for CJK targets.
|
||||
|
||||
Blanket application is safe — the hint only affects CJK glyphs.
|
||||
"""
|
||||
def _apply_run_typeface_hint(
|
||||
presentation: Presentation, tag: str, typeface: str, log_event: str
|
||||
) -> None:
|
||||
"""Set the <a:{tag}> typeface hint on every run of every slide shape."""
|
||||
|
||||
def _hint_shape(shape) -> int:
|
||||
hinted = 0
|
||||
@@ -133,29 +154,59 @@ def _apply_ea_font_hints(presentation: Presentation, target_language: str) -> No
|
||||
def _hint_text_frame(text_frame) -> int:
|
||||
hinted = 0
|
||||
tag_rPr = f"{{{_NS_A}}}rPr"
|
||||
tag_ea = f"{{{_NS_A}}}ea"
|
||||
tag_hint = f"{{{_NS_A}}}{tag}"
|
||||
for paragraph in text_frame.paragraphs:
|
||||
for run in paragraph.runs:
|
||||
rPr = run._r.find(tag_rPr)
|
||||
if rPr is None:
|
||||
rPr = etree.SubElement(run._r, tag_rPr)
|
||||
ea = rPr.find(tag_ea)
|
||||
if ea is None:
|
||||
ea = etree.SubElement(rPr, tag_ea)
|
||||
if not ea.get("typeface"):
|
||||
ea.set("typeface", typeface)
|
||||
# a:rPr must be the FIRST child of a:r (before a:t).
|
||||
rPr = etree.Element(tag_rPr)
|
||||
run._r.insert(0, rPr)
|
||||
hint = rPr.find(tag_hint)
|
||||
if hint is None:
|
||||
hint = etree.Element(tag_hint)
|
||||
_insert_a_rpr_child(rPr, hint)
|
||||
if not hint.get("typeface"):
|
||||
hint.set("typeface", typeface)
|
||||
hinted += 1
|
||||
return hinted
|
||||
|
||||
typeface = _ea_typeface_for_target(target_language)
|
||||
if not typeface:
|
||||
return
|
||||
total = 0
|
||||
for slide in presentation.slides:
|
||||
for shape in slide.shapes:
|
||||
total += _hint_shape(shape)
|
||||
# Speaker notes get the RTL direction — they need the typeface
|
||||
# hint too.
|
||||
if slide.has_notes_slide and slide.notes_slide.notes_text_frame is not None:
|
||||
total += _hint_text_frame(slide.notes_slide.notes_text_frame)
|
||||
if total:
|
||||
_log_info("pptx_ea_font_hints_applied", runs=total, typeface=typeface)
|
||||
_log_info(log_event, runs=total, typeface=typeface)
|
||||
|
||||
|
||||
def _apply_ea_font_hints(presentation: Presentation, target_language: str) -> None:
|
||||
"""Set the <a:ea> (east-asian) typeface on every run for CJK targets.
|
||||
|
||||
Blanket application is safe — the hint only affects CJK glyphs.
|
||||
"""
|
||||
typeface = _ea_typeface_for_target(target_language)
|
||||
if not typeface:
|
||||
return
|
||||
_apply_run_typeface_hint(
|
||||
presentation, "ea", typeface, "pptx_ea_font_hints_applied"
|
||||
)
|
||||
|
||||
|
||||
def _apply_cs_font_hints(presentation: Presentation, target_language: str) -> None:
|
||||
"""Set the <a:cs> (complex-script) typeface on every run for RTL targets.
|
||||
|
||||
Blanket application is safe — the hint only affects Arabic-script
|
||||
glyphs, and Arial covers the whole Arabic script on every platform.
|
||||
"""
|
||||
if not is_rtl(target_language):
|
||||
return
|
||||
_apply_run_typeface_hint(
|
||||
presentation, "cs", "Arial", "pptx_cs_font_hints_applied"
|
||||
)
|
||||
|
||||
|
||||
class PptxProcessorError(Exception):
|
||||
@@ -397,13 +448,17 @@ class PowerPointTranslator:
|
||||
)
|
||||
|
||||
# Apply RTL layout when the target language is written right-to-left.
|
||||
if target_language.lower() in RTL_LANGUAGES:
|
||||
if is_rtl(target_language):
|
||||
_apply_rtl_to_presentation(presentation)
|
||||
|
||||
# CJK font hint so the target script renders with a proper
|
||||
# typeface instead of shape-dependent fallbacks.
|
||||
_apply_ea_font_hints(presentation, target_language)
|
||||
|
||||
# Arabic-script font hint for RTL targets (same idea as the
|
||||
# CJK hint, via the <a:cs> typeface).
|
||||
_apply_cs_font_hints(presentation, target_language)
|
||||
|
||||
if translate_images:
|
||||
try:
|
||||
self._translate_images(presentation, target_language)
|
||||
|
||||
@@ -25,11 +25,7 @@ from docx.section import Section
|
||||
from lxml import etree
|
||||
|
||||
from services.providers.base import TranslationProvider
|
||||
|
||||
# Languages written right-to-left
|
||||
RTL_LANGUAGES: frozenset = frozenset(
|
||||
{"ar", "he", "fa", "ur", "ku", "ps", "ug", "sd", "yi", "dv", "ckb"}
|
||||
)
|
||||
from core.languages import is_rtl
|
||||
|
||||
# East-Asian / complex-script font hints: when the target language uses
|
||||
# glyphs a Latin theme font lacks, Word falls back to a substitute —
|
||||
@@ -47,6 +43,13 @@ CS_FONTS: dict = {
|
||||
"he": "Arial",
|
||||
"fa": "Arial",
|
||||
"ur": "Arial",
|
||||
"ps": "Arial",
|
||||
"ku": "Arial",
|
||||
"sd": "Arial",
|
||||
"ug": "Arial",
|
||||
"yi": "Arial",
|
||||
"dv": "Arial",
|
||||
"ckb": "Arial",
|
||||
}
|
||||
|
||||
|
||||
@@ -123,6 +126,79 @@ def _log_error(event: str, **kwargs):
|
||||
logger.error(msg)
|
||||
|
||||
|
||||
# --- OOXML schema order ---------------------------------------------------
|
||||
# w:pPr, w:rPr and w:tblPr require their children in a FIXED order.
|
||||
# Appending at the end produces out-of-order XML that Word "repairs" on
|
||||
# open (with a repair prompt in some versions). Each RTL element is
|
||||
# therefore inserted just before its first successor element, per the
|
||||
# CT_PPr / CT_RPr / CT_TblPr sequences of the schema.
|
||||
|
||||
# Elements that must come AFTER w:bidi inside w:pPr.
|
||||
_PPR_BIDI_SUCCESSORS = (
|
||||
qn("w:adjustRightInd"),
|
||||
qn("w:snapToGrid"),
|
||||
qn("w:spacing"),
|
||||
qn("w:ind"),
|
||||
qn("w:contextualSpacing"),
|
||||
qn("w:mirrorIndents"),
|
||||
qn("w:suppressOverlap"),
|
||||
qn("w:jc"),
|
||||
qn("w:textDirection"),
|
||||
qn("w:textAlignment"),
|
||||
qn("w:textboxTightWrap"),
|
||||
qn("w:outlineLvl"),
|
||||
qn("w:divId"),
|
||||
qn("w:cnfStyle"),
|
||||
qn("w:rPr"),
|
||||
qn("w:sectPr"),
|
||||
qn("w:pPrChange"),
|
||||
)
|
||||
# Elements that must come AFTER w:jc inside w:pPr (w:jc itself excluded).
|
||||
_PPR_JC_SUCCESSORS = _PPR_BIDI_SUCCESSORS[_PPR_BIDI_SUCCESSORS.index(qn("w:jc")) + 1 :]
|
||||
# Elements that must come AFTER w:rtl inside w:rPr.
|
||||
_RPR_RTL_SUCCESSORS = (
|
||||
qn("w:cs"),
|
||||
qn("w:em"),
|
||||
qn("w:lang"),
|
||||
qn("w:eastAsianLayout"),
|
||||
qn("w:specVanish"),
|
||||
qn("w:oMath"),
|
||||
qn("w:rPrChange"),
|
||||
)
|
||||
# Elements that must come AFTER w:bidiVisual inside w:tblPr.
|
||||
_TBLPR_BIDIVISUAL_SUCCESSORS = (
|
||||
qn("w:tblStyleRowBandSize"),
|
||||
qn("w:tblStyleColBandSize"),
|
||||
qn("w:tblW"),
|
||||
qn("w:jc"),
|
||||
qn("w:tblCellSpacing"),
|
||||
qn("w:tblInd"),
|
||||
qn("w:tblBorders"),
|
||||
qn("w:shd"),
|
||||
qn("w:tblLayout"),
|
||||
qn("w:tblCellMar"),
|
||||
qn("w:tblLook"),
|
||||
qn("w:tblCaption"),
|
||||
qn("w:tblDescription"),
|
||||
qn("w:tblPrChange"),
|
||||
)
|
||||
|
||||
|
||||
def _insert_ordered(parent, child, successors) -> None:
|
||||
"""Insert child into parent at its schema position.
|
||||
|
||||
The child is placed just before the first existing element whose tag
|
||||
is listed in ``successors`` (the elements that must come after it in
|
||||
the OOXML sequence); appended at the end when no successor exists.
|
||||
Works on python-docx oxml elements and plain lxml elements alike.
|
||||
"""
|
||||
for existing in parent:
|
||||
if existing.tag in successors:
|
||||
existing.addprevious(child)
|
||||
return
|
||||
parent.append(child)
|
||||
|
||||
|
||||
def _set_paragraph_rtl(paragraph: Paragraph) -> None:
|
||||
"""
|
||||
Enable RTL mode on a paragraph and all its runs.
|
||||
@@ -133,37 +209,97 @@ def _set_paragraph_rtl(paragraph: Paragraph) -> None:
|
||||
paragraph has no explicit alignment — centered/justified titles
|
||||
must not be forced right-aligned.
|
||||
- w:rPr/w:rtl → run-level RTL marker for each run
|
||||
|
||||
Every element is inserted at its schema position inside w:pPr/w:rPr
|
||||
(OOXML requires a fixed child order; appended-at-the-end elements
|
||||
make Word flag the file for repair).
|
||||
"""
|
||||
pPr = paragraph._p.get_or_add_pPr()
|
||||
|
||||
if pPr.find(qn("w:bidi")) is None:
|
||||
pPr.append(OxmlElement("w:bidi"))
|
||||
_insert_ordered(pPr, OxmlElement("w:bidi"), _PPR_BIDI_SUCCESSORS)
|
||||
|
||||
jc = pPr.find(qn("w:jc"))
|
||||
explicit_alignment = jc is not None and jc.get(qn("w:val")) not in (None, "", "left")
|
||||
if not explicit_alignment:
|
||||
if jc is None:
|
||||
jc = OxmlElement("w:jc")
|
||||
pPr.append(jc)
|
||||
_insert_ordered(pPr, jc, _PPR_JC_SUCCESSORS)
|
||||
jc.set(qn("w:val"), "right")
|
||||
|
||||
for run in paragraph.runs:
|
||||
rPr = run._r.get_or_add_rPr()
|
||||
if rPr.find(qn("w:rtl")) is None:
|
||||
rPr.append(OxmlElement("w:rtl"))
|
||||
_insert_ordered(rPr, OxmlElement("w:rtl"), _RPR_RTL_SUCCESSORS)
|
||||
|
||||
|
||||
def _set_table_rtl(tbl_element) -> None:
|
||||
"""
|
||||
Mark a table as visually right-to-left.
|
||||
|
||||
w:bidiVisual inside w:tblPr flips the column order at render time:
|
||||
the first column displays rightmost. Works on python-docx CT_Tbl
|
||||
elements and on plain lxml elements (notes/comments trees) alike.
|
||||
"""
|
||||
tblPr = tbl_element.find(qn("w:tblPr"))
|
||||
if tblPr is None:
|
||||
tblPr = OxmlElement("w:tblPr")
|
||||
tbl_element.insert(0, tblPr)
|
||||
if tblPr.find(qn("w:bidiVisual")) is None:
|
||||
_insert_ordered(
|
||||
tblPr, OxmlElement("w:bidiVisual"), _TBLPR_BIDIVISUAL_SUCCESSORS
|
||||
)
|
||||
|
||||
|
||||
def _apply_bidi_to_part_tree(root) -> int:
|
||||
"""
|
||||
Switch every paragraph, run and table of a footnotes/endnotes/comments
|
||||
tree to RTL.
|
||||
|
||||
These parts are rewritten as raw XML after the document save (they
|
||||
live outside python-docx's object model), so the RTL direction is
|
||||
injected into the same in-memory tree just before write-back:
|
||||
w:bidi on every paragraph, w:rtl on every run, w:bidiVisual on the
|
||||
tables these parts may contain.
|
||||
|
||||
Returns the number of paragraphs switched to RTL.
|
||||
"""
|
||||
switched = 0
|
||||
for p in root.iter(qn("w:p")):
|
||||
pPr = p.find(qn("w:pPr"))
|
||||
if pPr is None:
|
||||
pPr = OxmlElement("w:pPr")
|
||||
p.insert(0, pPr)
|
||||
if pPr.find(qn("w:bidi")) is None:
|
||||
_insert_ordered(pPr, OxmlElement("w:bidi"), _PPR_BIDI_SUCCESSORS)
|
||||
switched += 1
|
||||
for r in p.iter(qn("w:r")):
|
||||
rPr = r.find(qn("w:rPr"))
|
||||
if rPr is None:
|
||||
rPr = OxmlElement("w:rPr")
|
||||
r.insert(0, rPr)
|
||||
if rPr.find(qn("w:rtl")) is None:
|
||||
_insert_ordered(rPr, OxmlElement("w:rtl"), _RPR_RTL_SUCCESSORS)
|
||||
for tbl in root.iter(qn("w:tbl")):
|
||||
_set_table_rtl(tbl)
|
||||
return switched
|
||||
|
||||
|
||||
def _apply_rtl_to_document(document: Document) -> None:
|
||||
"""Apply RTL direction to every paragraph and section in the document."""
|
||||
# Body paragraphs
|
||||
for para in document.paragraphs:
|
||||
_set_paragraph_rtl(para)
|
||||
# Body tables
|
||||
for table in document.tables:
|
||||
for row in table.rows:
|
||||
for cell in row.cells:
|
||||
for para in cell.paragraphs:
|
||||
_set_paragraph_rtl(para)
|
||||
body = document.element.body
|
||||
|
||||
# Every paragraph of the body tree in ONE pass: top-level paragraphs,
|
||||
# table cells (nested tables included), text boxes (w:txbxContent)
|
||||
# and SDT content all receive w:bidi + run-level w:rtl.
|
||||
for p in body.iter(qn("w:p")):
|
||||
_set_paragraph_rtl(Paragraph(p, document))
|
||||
|
||||
# Tables: w:bidiVisual flips the visual column order. iter() covers
|
||||
# nested tables too.
|
||||
for tbl in body.iter(qn("w:tbl")):
|
||||
_set_table_rtl(tbl)
|
||||
|
||||
# Headers, footers, and section-level RTL (page layout direction)
|
||||
for section in document.sections:
|
||||
# Set the section (page) direction to RTL so Word renders margins,
|
||||
@@ -172,14 +308,37 @@ def _apply_rtl_to_document(document: Document) -> None:
|
||||
if sectPr.find(qn("w:bidi")) is None:
|
||||
sectPr.append(OxmlElement("w:bidi"))
|
||||
|
||||
for hf in (section.header, section.footer):
|
||||
for para in hf.paragraphs:
|
||||
_set_paragraph_rtl(para)
|
||||
for table in hf.tables:
|
||||
for row in table.rows:
|
||||
for cell in row.cells:
|
||||
for para in cell.paragraphs:
|
||||
_set_paragraph_rtl(para)
|
||||
# Same six header/footer zones the text collection translates
|
||||
# (default, first page, even pages) — a translated first-page
|
||||
# header must not stay left-to-right. Some may be missing
|
||||
# depending on the python-docx version: getattr defensively.
|
||||
headers_footers = (
|
||||
section.header,
|
||||
section.footer,
|
||||
getattr(section, "first_page_header", None),
|
||||
getattr(section, "first_page_footer", None),
|
||||
getattr(section, "even_page_header", None),
|
||||
getattr(section, "even_page_footer", None),
|
||||
)
|
||||
for hf in headers_footers:
|
||||
if hf is None:
|
||||
continue
|
||||
# Touch paragraphs first so a linked/missing definition is
|
||||
# created exactly like the paragraph walk below would.
|
||||
paras = hf.paragraphs
|
||||
root = getattr(hf, "_element", None)
|
||||
if root is None and paras:
|
||||
root = paras[0]._p.getroottree().getroot()
|
||||
if root is not None:
|
||||
# Full story tree: paragraphs, tables (bidiVisual) and
|
||||
# text boxes inside the header/footer.
|
||||
for p in root.iter(qn("w:p")):
|
||||
_set_paragraph_rtl(Paragraph(p, document))
|
||||
for tbl in root.iter(qn("w:tbl")):
|
||||
_set_table_rtl(tbl)
|
||||
else:
|
||||
for para in paras:
|
||||
_set_paragraph_rtl(para)
|
||||
|
||||
|
||||
class WordProcessorError(Exception):
|
||||
@@ -324,7 +483,9 @@ class WordTranslator:
|
||||
# that python-docx doesn't manage (footnotes, endnotes).
|
||||
post_save_callbacks: List[Callable[[Path], None]] = []
|
||||
|
||||
self._collect_from_body(document, text_elements, post_save_callbacks)
|
||||
self._collect_from_body(
|
||||
document, text_elements, post_save_callbacks, rtl=is_rtl(target_language)
|
||||
)
|
||||
|
||||
# Collect chart text from ZIP (chart titles, axis labels, series names)
|
||||
self._collect_charts_from_zip(input_path, text_elements, chart_translations)
|
||||
@@ -420,7 +581,7 @@ class WordTranslator:
|
||||
)
|
||||
|
||||
# Apply RTL layout when the target language is written right-to-left.
|
||||
if target_language.lower() in RTL_LANGUAGES:
|
||||
if is_rtl(target_language):
|
||||
_apply_rtl_to_document(document)
|
||||
|
||||
# CJK / Arabic-script font hints so Word renders the target
|
||||
@@ -677,6 +838,7 @@ class WordTranslator:
|
||||
def _collect_from_body(
|
||||
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]],
|
||||
post_save_callbacks: List[Callable[[Path], None]] = None,
|
||||
rtl: bool = False,
|
||||
) -> None:
|
||||
"""Collect all text elements from document body.
|
||||
|
||||
@@ -711,9 +873,9 @@ class WordTranslator:
|
||||
# Pass 3: footnotes, endnotes and comments (live in separate parts)
|
||||
if post_save_callbacks is None:
|
||||
post_save_callbacks = []
|
||||
self._collect_from_footnotes(document, text_elements, post_save_callbacks)
|
||||
self._collect_from_endnotes(document, text_elements, post_save_callbacks)
|
||||
self._collect_from_comments(document, text_elements, post_save_callbacks)
|
||||
self._collect_from_footnotes(document, text_elements, post_save_callbacks, rtl=rtl)
|
||||
self._collect_from_endnotes(document, text_elements, post_save_callbacks, rtl=rtl)
|
||||
self._collect_from_comments(document, text_elements, post_save_callbacks, rtl=rtl)
|
||||
|
||||
total = len(text_elements) - count_before
|
||||
_log_info(
|
||||
@@ -811,6 +973,7 @@ class WordTranslator:
|
||||
def _collect_from_footnotes(
|
||||
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]],
|
||||
post_save_callbacks: List[Callable[[Path], None]] = None,
|
||||
rtl: bool = False,
|
||||
) -> None:
|
||||
"""Collect text from footnotes.
|
||||
|
||||
@@ -856,6 +1019,8 @@ class WordTranslator:
|
||||
# setters hold references to t_elems inside this tree.
|
||||
def write_footnotes_back(output_path: Path) -> None:
|
||||
try:
|
||||
if rtl:
|
||||
_apply_bidi_to_part_tree(footnotes_xml)
|
||||
new_blob = etree.tostring(
|
||||
footnotes_xml,
|
||||
xml_declaration=True,
|
||||
@@ -894,6 +1059,7 @@ class WordTranslator:
|
||||
def _collect_from_endnotes(
|
||||
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]],
|
||||
post_save_callbacks: List[Callable[[Path], None]] = None,
|
||||
rtl: bool = False,
|
||||
) -> None:
|
||||
"""Collect text from endnotes (python-docx 1.x compatible).
|
||||
|
||||
@@ -921,6 +1087,8 @@ class WordTranslator:
|
||||
if text_elements and post_save_callbacks is not None:
|
||||
def write_endnotes_back(output_path: Path) -> None:
|
||||
try:
|
||||
if rtl:
|
||||
_apply_bidi_to_part_tree(endnotes_xml)
|
||||
new_blob = etree.tostring(
|
||||
endnotes_xml,
|
||||
xml_declaration=True,
|
||||
@@ -944,6 +1112,7 @@ class WordTranslator:
|
||||
def _collect_from_comments(
|
||||
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]],
|
||||
post_save_callbacks: List[Callable[[Path], None]] = None,
|
||||
rtl: bool = False,
|
||||
) -> None:
|
||||
"""Collect text from comments/balloons (word/comments.xml part).
|
||||
|
||||
@@ -974,6 +1143,8 @@ class WordTranslator:
|
||||
if collected and post_save_callbacks is not None:
|
||||
def write_comments_back(output_path: Path) -> None:
|
||||
try:
|
||||
if rtl:
|
||||
_apply_bidi_to_part_tree(comments_xml)
|
||||
new_blob = etree.tostring(
|
||||
comments_xml,
|
||||
xml_declaration=True,
|
||||
|
||||
Reference in New Issue
Block a user