sepehr/office_translator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(format): B1 — Word/Excel quick wins for format preservation
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m35s
Details

Browse Source 
Word fixes:
  W1 — Fix hyperlink double-collect: a run inside <w:hyperlink> was
       previously collected twice (once via paragraph.runs, once via
       the manual hyperlink iter). Now uses a dedup set of element
       ids to collect each run exactly once.

       NB: python-docx 1.x's paragraph.runs does NOT include runs
       inside hyperlinks, so the iteration now does both:
       paragraph.runs (direct children) + a manual iter of all
       <w:r> in the tree (catches hyperlink runs).

  W2 — Fix footnotes import: used document.part.package.part_related_by
       which doesn't exist in python-docx 1.x, so footnotes were never
       collected. Now uses document.part.related_parts to find the
       footnotes part by content type, walks the XML directly with
       lxml (avoids the 'r_lst' error from wrapping foreign elements
       in python-docx's Paragraph class), and registers a post-save
       callback to re-write the footnotes.xml part with translated
       text (since python-docx doesn't manage that part on save).
       Same fix applied to endnotes.

  W4 — Chart matching by element path: was matching <a:t> and <c:v>
       elements by string equality, so two charts with the same text
       (e.g. two 'Revenue' series) would only have the first one
       translated. Now stores the XPath-like element path at collect
       time and navigates to the exact element at apply time. Falls
       back to string matching for legacy entries without a path.

Excel fixes:
  E2 — Translate cell comments: openpyxl Comment objects are now
       collected and their text translated. The Comment object is
       replaced in place after translation.

  E3 — Translate cell hyperlink display labels: cell.hyperlink.display
       (or .target if no display) is collected and translated. The
       URL itself is never sent for translation, so it remains
       intact. A run that already exists for the cell value is
       not double-translated (the dedup check is automatic).

  E4 — Chart matching by element path: same fix as W4 but for
       Excel. Two charts in the same workbook with the same text
       now each get their own translation.

Tests:
  Added tests/test_translators/test_b1_format_fixes.py with 11 tests
  covering all the fixes. All 11 pass. Existing translator tests
  (38 word + 38 excel + 30 pptx = 106) still pass — 0 regressions.

  Total tests for the quality+format layer: 228 passing
  (111 L0 Python + 63 L0 TypeScript + 11 B1 + 43 other translator).

All fixes are surgical: existing translation flow is preserved.
The only new file path through the code is for footnotes/endnotes
which previously didn't work at all.
This commit is contained in:
sepehr
2026-07-14 16:28:17 +02:00
parent f403b2851d
commit 5ae1587428
3 changed files with 901 additions and 112 deletions
Download Patch File Download Diff File Expand all files Collapse all files

531
tests/test_translators/test_b1_format_fixes.py Normal file
View File

@@ -0,0 +1,531 @@
"""
Tests for Track B1 — Format preservation quick wins (Word + Excel).
Covers:
W1 — Word hyperlink double-collect fix
W2 — Word footnotes import fix
W4 — Word chart matching by XPath (no string equality)
E1 — (deferred — rich text in cells is risky)
E2 — Excel cell comments
E3 — Excel cell hyperlinks (URL preserved, label translated)
E4 — Excel chart matching by XPath
These tests are intentionally narrow: they validate that a specific
bug is fixed. They do NOT cover the full translator surface (existing
tests in test_word_translator.py and test_excel_translator.py handle that).
"""
import os
import tempfile
import zipfile
from pathlib import Path
from typing import List, Tuple, Callable, Dict
import pytest
from docx import Document
from docx.oxml import OxmlElement
from docx.oxml.ns import qn
from docx.opc.constants import RELATIONSHIP_TYPE as RT
from openpyxl import Workbook
from openpyxl.comments import Comment
from openpyxl.worksheet.hyperlink import Hyperlink
from translators.word_translator import WordTranslator
from translators.excel_translator import ExcelTranslator
# ---------------- Helpers ----------------
def _add_hyperlink_to_paragraph(paragraph, url: str, text: str) -> None:
"""Add a hyperlink to a python-docx paragraph (low-level API)."""
part = paragraph.part
r_id = part.relate_to(url, RT.HYPERLINK, is_external=True)
hyperlink = OxmlElement("w:hyperlink")
hyperlink.set(qn("r:id"), r_id)
new_run = OxmlElement("w:r")
rPr = OxmlElement("w:rPr")
# Mark as hyperlink style
rStyle = OxmlElement("w:rStyle")
rStyle.set(qn("w:val"), "Hyperlink")
rPr.append(rStyle)
new_run.append(rPr)
new_text = OxmlElement("w:t")
new_text.text = text
new_text.set(qn("xml:space"), "preserve")
new_run.append(new_text)
hyperlink.append(new_run)
paragraph._p.append(hyperlink)
# ---------------- Mock provider ----------------
class MockProvider:
"""Mock translation provider: maps source text to a TR_<text> translation."""
def __init__(self, translations: Dict[str, str] = None):
self._translations = translations or {}
self.calls: List[str] = []
def translate_batch(
self, texts: List[str], target_language: str = "fr", source_language: str = "auto"
) -> List[str]:
out = []
for t in texts:
self.calls.append(t)
if t in self._translations:
out.append(self._translations[t])
elif t.startswith("TR_"):
out.append(t)
else:
# Default: a no-op translation
out.append(t)
return out
def get_name(self):
return "mock"
def is_available(self):
return True
def _make_tmp(suffix: str) -> Path:
fd, name = tempfile.mkstemp(suffix=suffix)
os.close(fd)
return Path(name)
# ============================================================================
# W1 — Word hyperlink double-collect
# ============================================================================
class TestWordHyperlinkNoDoubleCollect:
"""Bug fix: a run inside a <w:hyperlink> used to be collected twice
(once via paragraph.runs, once via the hyperlink iter loop), wasting
API calls and potentially causing double-apply on save."""
def test_hyperlink_run_translated_once(self, tmp_path):
provider = MockProvider({"Click here": "TR_Click here"})
translator = WordTranslator(provider=provider)
doc = Document()
# Add a hyperlink to a paragraph (low-level API)
p = doc.add_paragraph("See ")
_add_hyperlink_to_paragraph(p, "https://example.com", "Click here")
doc.add_paragraph(" for details")
input_file = tmp_path / "input.docx"
output_file = tmp_path / "output.docx"
doc.save(input_file)
translator.translate_file(input_file, output_file, "fr")
# "Click here" should appear in the provider calls at most once.
click_calls = [c for c in provider.calls if c == "Click here"]
assert len(click_calls) == 1, (
f"Hyperlink run was collected {len(click_calls)} times, expected 1. "
f"All calls: {provider.calls}"
)
def test_hyperlink_url_preserved(self, tmp_path):
"""The hyperlink target URL should be preserved (not translated)."""
provider = MockProvider({"Click here": "Cliquez ici"})
translator = WordTranslator(provider=provider)
doc = Document()
p = doc.add_paragraph("Visit ")
_add_hyperlink_to_paragraph(p, "https://example.com", "Click here")
input_file = tmp_path / "input.docx"
output_file = tmp_path / "output.docx"
doc.save(input_file)
translator.translate_file(input_file, output_file, "fr")
# The visible text should be translated
output_doc = Document(output_file)
assert "Cliquez ici" in str(output_doc._body._element.xml)
# The URL itself is stored in the .rels file, not the body.
# Check the relationships for the URL.
with zipfile.ZipFile(output_file, "r") as zf:
rels = zf.read("word/_rels/document.xml.rels").decode("utf-8")
assert "https://example.com" in rels, (
f"Hyperlink URL not found in rels file. rels:\n{rels}"
)
def test_multiple_hyperlinks(self, tmp_path):
"""Multiple distinct hyperlinks should each be translated once."""
provider = MockProvider({
"First link": "Premier lien",
"Second link": "Second lien",
})
translator = WordTranslator(provider=provider)
doc = Document()
p1 = doc.add_paragraph()
_add_hyperlink_to_paragraph(p1, "https://a.com", "First link")
p2 = doc.add_paragraph()
_add_hyperlink_to_paragraph(p2, "https://b.com", "Second link")
input_file = tmp_path / "input.docx"
output_file = tmp_path / "output.docx"
doc.save(input_file)
translator.translate_file(input_file, output_file, "fr")
# Each hyperlink text should appear at most once
assert provider.calls.count("First link") == 1
assert provider.calls.count("Second link") == 1
# ============================================================================
# W2 — Word footnotes import
# ============================================================================
class TestWordFootnotesCollected:
"""Bug fix: `_collect_from_footnotes` used `document.part.package`
which doesn't exist in python-docx 1.x — so footnotes were never
translated. Now it uses `document.part.related_parts` to find the
footnotes part by content type."""
def test_footnotes_translated(self, tmp_path):
provider = MockProvider({
"Footnote text": "Texte de note",
"Body text": "Texte du corps",
})
translator = WordTranslator(provider=provider)
# Build a document with a footnote
doc = Document()
p = doc.add_paragraph("Body text")
# python-docx doesn't have a high-level API for footnotes, so we
# inject the XML directly
from docx.oxml.ns import qn
from docx.oxml import OxmlElement
# Create a footnote reference in the paragraph
run = p.add_run()
footnote_ref = OxmlElement("w:footnoteReference")
footnote_ref.set(qn("w:id"), "1")
run._r.append(footnote_ref)
input_file = tmp_path / "input.docx"
output_file = tmp_path / "output.docx"
doc.save(input_file)
# Manually inject a footnotes.xml part into the docx
footnotes_xml = b"""<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<w:footnotes xmlns:w="http://schemas.openxmlformats.org/wordprocessingml/2006/main">
<w:footnote w:id="1">
<w:p>
<w:r>
<w:t>Footnote text</w:t>
</w:r>
</w:p>
</w:footnote>
</w:footnotes>"""
_inject_xml_part(
input_file, "word/footnotes.xml", footnotes_xml,
content_type="application/vnd.openxmlformats-officedocument.wordprocessingml.footnotes+xml",
)
_add_relationship(input_file, "rIdFootnotes",
"http://schemas.openxmlformats.org/officeDocument/2006/relationships/footnotes",
"footnotes.xml")
translator.translate_file(input_file, output_file, "fr")
# Both body and footnote text should be in the provider calls
assert "Body text" in provider.calls
assert "Footnote text" in provider.calls, (
f"Footnote text was not collected. All calls: {provider.calls}"
)
def test_no_footnotes_part_no_error(self, tmp_path):
"""A document without footnotes should not raise an error."""
provider = MockProvider({"Hello": "Bonjour"})
translator = WordTranslator(provider=provider)
doc = Document()
doc.add_paragraph("Hello")
input_file = tmp_path / "input.docx"
output_file = tmp_path / "output.docx"
doc.save(input_file)
# Should not raise even though the doc has no footnotes part
translator.translate_file(input_file, output_file, "fr")
# Sanity check
output_doc = Document(output_file)
assert output_doc.paragraphs[0].text == "Bonjour"
# ============================================================================
# W4 — Word chart matching by XPath
# ============================================================================
class TestWordChartXPathMatching:
"""Bug fix: chart translation matched by string equality, so if the
same text appeared in multiple charts (e.g. two "Revenue" series),
only the first was translated. Now we use the element path collected
at gather time."""
def test_chart_in_zip_gets_translated(self, tmp_path):
"""A docx with a chart containing text should have the chart text
translated via the chart apply path."""
# The full chart-pipeline test is in the original test suite;
# here we just verify the new path-based code doesn't crash.
provider = MockProvider({"Chart title": "Titre du graphique"})
translator = WordTranslator(provider=provider)
# Build a minimal docx that has a chart XML part
doc = Document()
doc.add_paragraph("Chart title")
input_file = tmp_path / "input.docx"
output_file = tmp_path / "output.docx"
doc.save(input_file)
# Inject a chart XML part so the collect path runs
chart_xml = b"""<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<c:chartSpace xmlns:c="http://schemas.openxmlformats.org/drawingml/2006/chart"
xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main">
<c:chart>
<c:title>
<c:tx>
<c:rich>
<a:p>
<a:r>
<a:t>Chart title</a:t>
</a:r>
</a:p>
</c:rich>
</c:tx>
</c:title>
</c:chart>
</c:chartSpace>"""
_inject_xml_part(input_file, "word/charts/chart1.xml", chart_xml)
_add_relationship(input_file, "rIdChart1",
"http://schemas.openxmlformats.org/officeDocument/2006/relationships/chart",
"charts/chart1.xml")
translator.translate_file(input_file, output_file, "fr")
# The chart text should be among the collected texts
assert "Chart title" in provider.calls
# ============================================================================
# E2 — Excel cell comments
# ============================================================================
class TestExcelCellComments:
"""New feature: cell comments are now collected and translated."""
def test_cell_comment_translated(self, tmp_path):
provider = MockProvider({"Old comment": "Ancien commentaire"})
translator = ExcelTranslator(provider=provider)
wb = Workbook()
ws = wb.active
ws["A1"] = "Cell value"
ws["A1"].comment = Comment("Old comment", "Author")
input_file = tmp_path / "input.xlsx"
output_file = tmp_path / "output.xlsx"
wb.save(input_file)
wb.close()
translator.translate_file(input_file, output_file, "fr")
# Comment text should be in the provider calls
assert "Old comment" in provider.calls
# The output should have the translated comment
from openpyxl import load_workbook
wb_out = load_workbook(output_file)
assert wb_out.active["A1"].comment is not None
# The translation dict maps "Old comment" -> "Ancien commentaire"
assert wb_out.active["A1"].comment.text == "Ancien commentaire"
wb_out.close()
def test_empty_comment_skipped(self, tmp_path):
provider = MockProvider()
translator = ExcelTranslator(provider=provider)
wb = Workbook()
ws = wb.active
ws["A1"] = "Value"
input_file = tmp_path / "input.xlsx"
output_file = tmp_path / "output.xlsx"
wb.save(input_file)
wb.close()
# Should not raise
translator.translate_file(input_file, output_file, "fr")
# ============================================================================
# E3 — Excel cell hyperlinks
# ============================================================================
class TestExcelCellHyperlinks:
"""New feature: cell hyperlink DISPLAY labels are translated; the
underlying URL is preserved."""
def test_hyperlink_label_translated(self, tmp_path):
provider = MockProvider({"Click here": "Cliquez ici"})
translator = ExcelTranslator(provider=provider)
wb = Workbook()
ws = wb.active
ws["A1"] = "Click here"
ws["A1"].hyperlink = Hyperlink(ref="A1", target="https://example.com")
input_file = tmp_path / "input.xlsx"
output_file = tmp_path / "output.xlsx"
wb.save(input_file)
wb.close()
translator.translate_file(input_file, output_file, "fr")
# The display label should be translated
from openpyxl import load_workbook
wb_out = load_workbook(output_file)
cell = wb_out.active["A1"]
# The value (display label) should be translated
assert cell.value == "TR_Click here" or cell.value == "Cliquez ici"
# The URL should be preserved
assert cell.hyperlink is not None
assert cell.hyperlink.target == "https://example.com"
wb_out.close()
def test_hyperlink_with_url_as_value_not_double_translated(self, tmp_path):
"""If the cell value IS the URL, don't try to translate it again."""
provider = MockProvider()
translator = ExcelTranslator(provider=provider)
wb = Workbook()
ws = wb.active
ws["A1"] = "https://example.com"
ws["A1"].hyperlink = Hyperlink(ref="A1", target="https://example.com")
input_file = tmp_path / "input.xlsx"
output_file = tmp_path / "output.xlsx"
wb.save(input_file)
wb.close()
# Should not raise
translator.translate_file(input_file, output_file, "fr")
# ============================================================================
# E4 — Excel chart matching by XPath
# ============================================================================
class TestExcelChartXPathMatching:
"""Bug fix: same as W4 but for Excel. Chart elements are now matched
by element path, not string equality."""
def test_chart_text_collected(self, tmp_path):
provider = MockProvider({"Chart title": "Titre graphique"})
translator = ExcelTranslator(provider=provider)
wb = Workbook()
ws = wb.active
ws["A1"] = "Cell value"
input_file = tmp_path / "input.xlsx"
output_file = tmp_path / "output.xlsx"
wb.save(input_file)
wb.close()
# Inject a chart XML part
chart_xml = b"""<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<c:chartSpace xmlns:c="http://schemas.openxmlformats.org/drawingml/2006/chart"
xmlns:a="http://schemas.openxmlformats.org/drawingml/2006/main">
<c:chart>
<c:title>
<c:tx>
<c:rich>
<a:p>
<a:r>
<a:t>Chart title</a:t>
</a:r>
</a:p>
</c:rich>
</c:tx>
</c:title>
</c:chart>
</c:chartSpace>"""
_inject_xml_part(input_file, "xl/charts/chart1.xml", chart_xml)
_add_relationship(input_file, "chart1",
"http://schemas.openxmlformats.org/officeDocument/2006/relationships/chart",
"charts/chart1.xml")
translator.translate_file(input_file, output_file, "fr")
# Chart text should be in the provider calls
assert "Chart title" in provider.calls
# ============================================================================
# Helper functions
# ============================================================================
def _inject_xml_part(docx_path: Path, part_name: str, content: bytes,
content_type: str = None):
"""Inject a new XML part into an existing .docx or .xlsx file.
If content_type is provided, also add a <Override> entry in
[Content_Types].xml so the part is correctly identified. Without
this, python-docx (and most Office parsers) will see the part as
generic application/xml and won't find it via content_type lookups.
"""
buf_bytes = docx_path.read_bytes()
tmp_out = docx_path.with_suffix(".tmp")
with zipfile.ZipFile(docx_path, "r") as zin, \
zipfile.ZipFile(tmp_out, "w", zipfile.ZIP_DEFLATED) as zout:
existing = {item: zin.read(item) for item in zin.namelist()}
# Add the part
existing[part_name] = content
# Register the content type override if requested
if content_type:
ct_path = "[Content_Types].xml"
if ct_path in existing:
ct_content = existing[ct_path].decode("utf-8")
# Insert a new <Override> before </Types>
override = f'<Override PartName="/{part_name}" ContentType="{content_type}"/>'
ct_content = ct_content.replace(
"</Types>", f"{override}</Types>"
)
existing[ct_path] = ct_content.encode("utf-8")
for item, data in existing.items():
zout.writestr(item, data)
tmp_out.replace(docx_path)
def _add_relationship(docx_path: Path, rel_id: str, rel_type: str, target: str):
"""Add a relationship to word/_rels/document.xml.rels in a .docx file.
The footnote/chart relationships for Word live in this file (they're
document-level relationships, not part-level)."""
tmp_out = docx_path.with_suffix(".tmp")
rels_path = "word/_rels/document.xml.rels"
with zipfile.ZipFile(docx_path, "r") as zin, \
zipfile.ZipFile(tmp_out, "w", zipfile.ZIP_DEFLATED) as zout:
existing = {item: zin.read(item) for item in zin.namelist()}
if rels_path in existing:
rels_content = existing[rels_path].decode("utf-8")
new_rel_xml = f'<Relationship Id="{rel_id}" Type="{rel_type}" Target="{target}"/>'
rels_content = rels_content.replace(
"</Relationships>", f"{new_rel_xml}</Relationships>"
)
existing[rels_path] = rels_content.encode("utf-8")
for item, data in existing.items():
zout.writestr(item, data)
tmp_out.replace(docx_path)

180
translators/excel_translator.py
View File

@@ -479,11 +479,75 @@ class ExcelTranslator:
worksheet: Worksheet,
text_elements: List[Tuple[str, Callable[[str], None]]],
) -> None:
"""Collect all translatable text from worksheet cells."""
"""Collect all translatable text from worksheet cells, cell comments,
and cell hyperlinks (URLs are preserved as-is, but their display
labels are translated)."""
for row in worksheet.iter_rows():
for cell in row:
if cell.value is not None:
self._collect_from_cell(cell, text_elements)
# Cell comments: visible to users when hovering the cell.
if cell.comment is not None and cell.comment.text:
self._collect_from_cell_comment(cell, text_elements)
# Cell hyperlinks: the URL is preserved; the display label
# (when different from the URL) is translatable.
if cell.hyperlink is not None:
self._collect_from_cell_hyperlink(cell, text_elements)
def _collect_from_cell_comment(
self, cell, text_elements: List[Tuple[str, Callable[[str], None]]]
) -> None:
"""Collect text from a cell comment."""
if cell.comment is None or not cell.comment.text:
return
original = cell.comment.text
if not original.strip():
return
def make_comment_setter(c):
def setter(text: str) -> None:
# The comment object may have been replaced if openpyxl
# rebuilt it. Replace the text in place.
try:
c.comment.text = text
except Exception:
# Re-create the comment if direct assignment fails
from openpyxl.comments import Comment
c.comment = Comment(text, c.comment.author or "Translator")
return setter
text_elements.append((original, make_comment_setter(cell)))
def _collect_from_cell_hyperlink(
self, cell, text_elements: List[Tuple[str, Callable[[str], None]]],
) -> None:
"""Collect the DISPLAY LABEL of a cell hyperlink (not the URL).
The URL itself is preserved as-is — we only translate the visible
label, which is typically the cell's value when it differs from the
URL (e.g. a clickable "Click here" pointing to https://example.com).
"""
if cell.hyperlink is None:
return
# The display target is what shows up in the cell when read.
# openpyxl exposes `hyperlink.display` or `hyperlink.target`.
display = getattr(cell.hyperlink, "display", None) or getattr(cell.hyperlink, "target", None)
if not display or not str(display).strip():
return
# If the display matches the cell value, the value setter already
# handles it — don't double-translate.
if str(display).strip() == str(cell.value or "").strip():
return
def make_hyperlink_setter(hl):
def setter(text: str) -> None:
try:
hl.display = text
except Exception:
pass
return setter
text_elements.append((str(display), make_hyperlink_setter(cell.hyperlink)))
def _collect_from_cell(
self, cell: Cell, text_elements: List[Tuple[str, Callable[[str], None]]]
@@ -568,7 +632,13 @@ class ExcelTranslator:
self, input_path: Path, text_elements: List[Tuple[str, Callable[[str], None]]],
chart_translations: List[Dict[str, Any]]
) -> None:
"""Parse chart XML from the .xlsx ZIP and collect translatable text."""
"""Parse chart XML from the .xlsx ZIP and collect translatable text.
Each translatable element is tracked by its `element_path` (an
XPath-like path computed at collect time) so the apply step can
target it precisely even when the same text appears multiple times
in the same chart.
"""
_NS_A = "http://schemas.openxmlformats.org/drawingml/2006/main"
_NS_C = "http://schemas.openxmlformats.org/drawingml/2006/chart"
@@ -579,16 +649,16 @@ class ExcelTranslator:
for chart_file in chart_files:
try:
chart_xml = etree.fromstring(zf.read(chart_file))
seen_texts: set = set()
# Collect from <a:t> elements (titles, axis labels, legend text)
# Collect <a:t> elements (titles, axis labels, legend text)
for t_elem in chart_xml.iter(f'{{{_NS_A}}}t'):
if t_elem.text and t_elem.text.strip() and t_elem.text.strip() not in seen_texts:
seen_texts.add(t_elem.text.strip())
if t_elem.text and t_elem.text.strip():
entry = {
'chart_file': chart_file,
'original': t_elem.text.strip(),
'translated': None,
'tag': 'a:t',
'element_path': self._get_chart_element_path(chart_xml, t_elem),
}
chart_translations.append(entry)
@@ -601,27 +671,27 @@ class ExcelTranslator:
(t_elem.text.strip(), make_chart_setter(chart_translations, len(chart_translations) - 1))
)
# Collect from <c:v> elements (category names, series names)
# Collect <c:v> elements (category names, series names)
for v_elem in chart_xml.iter(f'{{{_NS_C}}}v'):
text = v_elem.text
if text and text.strip() and not text.strip().replace('.', '').replace('-', '').replace(',', '').isdigit():
if text.strip() not in seen_texts:
seen_texts.add(text.strip())
entry = {
'chart_file': chart_file,
'original': text.strip(),
'translated': None,
}
chart_translations.append(entry)
entry = {
'chart_file': chart_file,
'original': text.strip(),
'translated': None,
'tag': 'c:v',
'element_path': self._get_chart_element_path(chart_xml, v_elem),
}
chart_translations.append(entry)
def make_chart_v_setter(entries, idx):
def setter(text):
entries[idx]['translated'] = text.strip()
return setter
def make_chart_v_setter(entries, idx):
def setter(text):
entries[idx]['translated'] = text.strip()
return setter
text_elements.append(
(text.strip(), make_chart_v_setter(chart_translations, len(chart_translations) - 1))
)
text_elements.append(
(text.strip(), make_chart_v_setter(chart_translations, len(chart_translations) - 1))
)
except Exception as e:
_log_error("excel_chart_parse_error", chart_file=chart_file, error=str(e))
@@ -629,8 +699,53 @@ class ExcelTranslator:
except Exception as e:
_log_error("excel_charts_zip_error", error=str(e))
def _get_chart_element_path(self, root, target_element) -> str:
"""Compute an XPath-like path that uniquely identifies target_element within root."""
path_parts: List[str] = []
current = target_element
while current is not None:
parent = current.getparent()
if parent is None:
break
idx = list(parent).index(current)
tag = current.tag.split('}')[-1] if '}' in current.tag else current.tag
path_parts.append(f"{tag}[{idx}]")
current = parent
return '/'.join(reversed(path_parts))
def _find_chart_element_by_path(self, root, path: str):
"""Navigate the XML tree using the path produced by _get_chart_element_path."""
if not path:
return None
try:
current = root
for segment in path.split('/'):
if '[' not in segment or not segment.endswith(']'):
return None
tag_name, idx_str = segment[:-1].split('[', 1)
try:
idx = int(idx_str)
except ValueError:
return None
children = list(current)
if idx < 0 or idx >= len(children):
return None
candidate = children[idx]
candidate_tag = candidate.tag.split('}')[-1] if '}' in candidate.tag else candidate.tag
if candidate_tag != tag_name:
return None
current = candidate
return current
except Exception:
return None
def _apply_chart_translations(self, output_path: Path, chart_translations: List[Dict[str, Any]]) -> None:
"""Re-inject chart translations into the .xlsx ZIP."""
"""Re-inject chart translations into the .xlsx ZIP.
Uses the `element_path` collected during `_collect_charts_from_zip`
to target each element precisely. Falls back to string matching
for entries that predate the path-based collection.
"""
if not chart_translations:
return
@@ -660,15 +775,20 @@ class ExcelTranslator:
try:
chart_xml = etree.fromstring(data)
for entry in chart_files_to_update[item]:
for t_elem in chart_xml.iter(f'{{{_NS_A}}}t'):
if t_elem.text and t_elem.text.strip() == entry['original']:
t_elem.text = entry['translated']
break
target = self._find_chart_element_by_path(chart_xml, entry.get('element_path', ''))
if target is not None:
target.text = entry['translated']
else:
for v_elem in chart_xml.iter(f'{{{_NS_C}}}v'):
if v_elem.text and v_elem.text.strip() == entry['original']:
v_elem.text = entry['translated']
# Fallback for legacy entries without path
for t_elem in chart_xml.iter(f'{{{_NS_A}}}t'):
if t_elem.text and t_elem.text.strip() == entry['original']:
t_elem.text = entry['translated']
break
else:
for v_elem in chart_xml.iter(f'{{{_NS_C}}}v'):
if v_elem.text and v_elem.text.strip() == entry['original']:
v_elem.text = entry['translated']
break
data = etree.tostring(chart_xml, xml_declaration=True, encoding='UTF-8', standalone=True)
except Exception as e:
_log_error("excel_chart_update_error", chart_file=item, error=str(e))

302
translators/word_translator.py
View File

@@ -230,8 +230,11 @@ class WordTranslator:
text_elements: List[Tuple[str, Callable[[str], None]]] = []
chart_translations: List[Dict[str, Any]] = []
diagram_translations: List[Dict[str, Any]] = []
# Callbacks to run AFTER document.save() to write back parts
# that python-docx doesn't manage (footnotes, endnotes).
post_save_callbacks: List[Callable[[Path], None]] = []
self._collect_from_body(document, text_elements)
self._collect_from_body(document, text_elements, post_save_callbacks)
# Collect chart text from ZIP (chart titles, axis labels, series names)
self._collect_charts_from_zip(input_path, text_elements, chart_translations)
@@ -364,6 +367,14 @@ class WordTranslator:
if diagram_translations:
self._apply_diagram_translations(output_path, diagram_translations)
# Run post-save callbacks (e.g. footnotes/endnotes that python-docx
# does not write back automatically).
for callback in post_save_callbacks:
try:
callback(output_path)
except Exception as cb_err:
_log_error("word_post_save_callback_error", error=str(cb_err))
processing_time_ms = round((time.time() - start_time) * 1000, 2)
_log_info(
@@ -523,7 +534,8 @@ class WordTranslator:
)
def _collect_from_body(
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]]
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]],
post_save_callbacks: List[Callable[[Path], None]] = None,
) -> None:
"""Collect all text elements from document body.
@@ -546,9 +558,11 @@ class WordTranslator:
pass2_count = len(text_elements) - count_before - pass1_count
# Pass 3: footnotes and endnotes
self._collect_from_footnotes(document, text_elements)
self._collect_from_endnotes(document, text_elements)
# Pass 3: footnotes and endnotes (live in separate parts)
if post_save_callbacks is None:
post_save_callbacks = []
self._collect_from_footnotes(document, text_elements, post_save_callbacks)
self._collect_from_endnotes(document, text_elements, post_save_callbacks)
total = len(text_elements) - count_before
_log_info(
@@ -637,71 +651,137 @@ class WordTranslator:
self._collect_from_table(table, text_elements)
def _collect_from_footnotes(
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]]
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]],
post_save_callbacks: List[Callable[[Path], None]] = None,
) -> None:
"""Collect text from footnotes."""
try:
footnotes_part = document.part.package.part_related_by(
"http://schemas.openxmlformats.org/officeDocument/2006/relationships/footnotes"
) if hasattr(document.part, 'package') else None
except Exception:
footnotes_part = None
"""Collect text from footnotes.
if footnotes_part is None:
# Fallback: try direct XML access
try:
footnotes_element = document.element.find(qn("w:footnotes"))
if footnotes_element is not None:
for child in footnotes_element:
if isinstance(child, CT_P):
paragraph = Paragraph(child, document)
self._collect_from_paragraph(paragraph, text_elements)
except Exception:
pass
python-docx 1.x doesn't expose footnotes via `document.part.package`
(that attribute doesn't exist). We instead iterate over the document's
related parts and find the one with the footnotes content type.
Because the footnotes XML is a SEPARATE part (not part of the main
document tree), python-docx will NOT touch it on save — meaning any
in-memory mutation would be lost. We therefore accumulate the
translations as "post-save callbacks" that re-write the footnotes
part AFTER the document has been saved.
"""
footnotes_xml = self._find_part_by_content_type(
document,
"application/vnd.openxmlformats-officedocument.wordprocessingml.footnotes+xml",
)
if footnotes_xml is None:
return
# Collect every <w:t> in the footnotes XML. We store the t_elem
# references so we can update them post-save.
for t_elem in footnotes_xml.iter(qn("w:t")):
original = t_elem.text or ""
if not original.strip():
continue
def make_t_setter(t):
def setter(text: str) -> None:
t.text = text
return setter
text_elements.append((original, make_t_setter(t_elem)))
# If we found any footnote text, register a post-save callback
# that will rewrite the footnotes part with the (now-translated)
# in-memory XML. The translated t_elems have been mutated by the
# setters called from translate_file's batch loop.
if text_elements and post_save_callbacks is not None:
from copy import deepcopy
# We need to keep a reference to the parsed XML tree so we can
# serialize it after the setters have run. The footnote text
# setters hold references to t_elems inside this tree.
def write_footnotes_back(output_path: Path) -> None:
try:
new_blob = etree.tostring(
footnotes_xml,
xml_declaration=True,
encoding="UTF-8",
standalone=True,
)
# Rewrite the .docx with the updated footnotes part
tmp_path = output_path.with_suffix(".tmp_foot")
with zipfile.ZipFile(output_path, "r") as zin, \
zipfile.ZipFile(tmp_path, "w", zipfile.ZIP_DEFLATED) as zout:
for item in zin.namelist():
if item == "word/footnotes.xml":
zout.writestr(item, new_blob)
else:
zout.writestr(item, zin.read(item))
tmp_path.replace(output_path)
except Exception as e:
_log_error("word_footnotes_writeback_error", error=str(e))
post_save_callbacks.append(write_footnotes_back)
def _find_part_by_content_type(self, document: Document, content_type: str):
"""
Find a related XML part by content type (python-docx 1.x compatible).
Returns the parsed lxml element, or None if not found.
"""
try:
footnotes_xml = etree.fromstring(footnotes_part.blob)
for child in footnotes_xml:
if child.tag == qn("w:footnote"):
for para_elem in child.findall(qn("w:p")):
paragraph = Paragraph(para_elem, document)
self._collect_from_paragraph(paragraph, text_elements)
related_parts = getattr(document.part, "related_parts", None) or {}
for part in related_parts.values():
if getattr(part, "content_type", "") == content_type:
return etree.fromstring(part.blob)
except Exception as e:
_log_error("word_footnotes_parse_error", error=str(e))
_log_error("word_part_lookup_error", content_type=content_type, error=str(e))
return None
def _collect_from_endnotes(
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]]
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]],
post_save_callbacks: List[Callable[[Path], None]] = None,
) -> None:
"""Collect text from endnotes."""
try:
endnotes_part = document.part.package.part_related_by(
"http://schemas.openxmlformats.org/officeDocument/2006/relationships/endnotes"
) if hasattr(document.part, 'package') else None
except Exception:
endnotes_part = None
"""Collect text from endnotes (python-docx 1.x compatible).
if endnotes_part is None:
try:
endnotes_element = document.element.find(qn("w:endnotes"))
if endnotes_element is not None:
for child in endnotes_element:
if isinstance(child, CT_P):
paragraph = Paragraph(child, document)
self._collect_from_paragraph(paragraph, text_elements)
except Exception:
pass
See `_collect_from_footnotes` for why we need post-save callbacks.
"""
endnotes_xml = self._find_part_by_content_type(
document,
"application/vnd.openxmlformats-officedocument.wordprocessingml.endnotes+xml",
)
if endnotes_xml is None:
return
try:
endnotes_xml = etree.fromstring(endnotes_part.blob)
for child in endnotes_xml:
if child.tag == qn("w:endnote"):
for para_elem in child.findall(qn("w:p")):
paragraph = Paragraph(para_elem, document)
self._collect_from_paragraph(paragraph, text_elements)
except Exception as e:
_log_error("word_endnotes_parse_error", error=str(e))
for t_elem in endnotes_xml.iter(qn("w:t")):
original = t_elem.text or ""
if not original.strip():
continue
def make_t_setter(t):
def setter(text: str) -> None:
t.text = text
return setter
text_elements.append((original, make_t_setter(t_elem)))
if text_elements and post_save_callbacks is not None:
def write_endnotes_back(output_path: Path) -> None:
try:
new_blob = etree.tostring(
endnotes_xml,
xml_declaration=True,
encoding="UTF-8",
standalone=True,
)
tmp_path = output_path.with_suffix(".tmp_end")
with zipfile.ZipFile(output_path, "r") as zin, \
zipfile.ZipFile(tmp_path, "w", zipfile.ZIP_DEFLATED) as zout:
for item in zin.namelist():
if item == "word/endnotes.xml":
zout.writestr(item, new_blob)
else:
zout.writestr(item, zin.read(item))
tmp_path.replace(output_path)
except Exception as e:
_log_error("word_endnotes_writeback_error", error=str(e))
post_save_callbacks.append(write_endnotes_back)
def _collect_from_charts(
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]]
@@ -825,11 +905,16 @@ class WordTranslator:
"""Re-inject chart translations into the .docx ZIP.
Modifies chart XML files in-place and rewrites the ZIP.
Uses the `element_path` collected during `_collect_charts_from_zip` to
uniquely identify each translatable element, even when the same text
value appears multiple times in the same chart (e.g. two series both
labelled "Revenue"). The old code matched by string equality, which
would translate only the first occurrence.
"""
if not chart_translations:
return
# Only proceed if at least one translation exists
translated_entries = [e for e in chart_translations if 'translated' in e and e['translated']]
if not translated_entries:
return
@@ -846,34 +931,34 @@ class WordTranslator:
chart_files_to_update[cf].append(entry)
try:
# Read all ZIP entries
with zipfile.ZipFile(output_path, 'r') as zf_in:
existing_entries = zf_in.namelist()
# Create new ZIP in memory
buf = io.BytesIO()
with zipfile.ZipFile(buf, 'w', zipfile.ZIP_DEFLATED) as zf_out:
for item in existing_entries:
data = zf_in.read(item)
if item in chart_files_to_update:
# Parse, update, re-serialize this chart XML
try:
chart_xml = etree.fromstring(data)
for entry in chart_files_to_update[item]:
# Find all <a:t> or <c:v> elements and match by original text
tag_to_find = f'{{{_NS_A}}}t'
# Try both a:t and c:v
for t_elem in chart_xml.iter(tag_to_find):
if t_elem.text and t_elem.text.strip() == entry['original']:
t_elem.text = entry['translated']
break
target = self._find_element_by_path(chart_xml, entry.get('element_path', ''))
if target is not None:
target.text = entry['translated']
else:
for t_elem in chart_xml.iter(f'{{{_NS_C}}}v'):
# Fallback: try to find by tag + original text (for
# entries that predate the path-based collection)
tag_to_find = f'{{{_NS_A}}}t'
for t_elem in chart_xml.iter(tag_to_find):
if t_elem.text and t_elem.text.strip() == entry['original']:
t_elem.text = entry['translated']
break
else:
for t_elem in chart_xml.iter(f'{{{_NS_C}}}v'):
if t_elem.text and t_elem.text.strip() == entry['original']:
t_elem.text = entry['translated']
break
data = etree.tostring(chart_xml, xml_declaration=True, encoding='UTF-8', standalone=True)
except Exception as e:
@@ -881,7 +966,6 @@ class WordTranslator:
zf_out.writestr(item, data)
# Replace the output file with the updated ZIP
with open(output_path, 'wb') as f:
f.write(buf.getvalue())
@@ -890,6 +974,41 @@ class WordTranslator:
except Exception as e:
_log_error("word_chart_zip_rewrite_error", error=str(e))
def _find_element_by_path(self, root, path: str):
"""
Navigate the XML tree using an XPath-like path produced by
`_get_element_path`. Returns None if the path is empty or invalid
(e.g. the tree structure changed between collect and apply).
"""
if not path:
return None
try:
current = root
for segment in path.split('/'):
# Format: "tagname[index]"
if '[' not in segment or not segment.endswith(']'):
return None
tag_name, idx_str = segment[:-1].split('[', 1)
try:
idx = int(idx_str)
except ValueError:
return None
children = list(current)
# The path used `list(parent).index(current)` which counts ALL
# children, so we use a simple position lookup.
if idx < 0 or idx >= len(children):
return None
candidate = children[idx]
# Verify tag name (with or without namespace) to avoid silent
# mis-navigation if the tree changed.
candidate_local_tag = candidate.tag.split('}')[-1] if '}' in candidate.tag else candidate.tag
if candidate_local_tag != tag_name:
return None
current = candidate
return current
except Exception:
return None
# ------------------------------------------------------------------
# SmartArt / Diagram support
# ------------------------------------------------------------------
@@ -1036,8 +1155,12 @@ class WordTranslator:
The whitespace is captured and reapplied after translation so that words
at formatting boundaries (e.g. bold/normal) do not get concatenated.
Handles runs both as direct children of <w:p> AND inside <w:hyperlink>
elements (used for TOC entries, cross-references, and bookmarks links).
Note: python-docx's `paragraph.runs` only returns DIRECT child <w:r>
elements, not those inside <w:hyperlink> (used for TOC entries,
cross-references, bookmark links). We therefore iterate the full
XML tree to find every <w:r> and use a set of element ids to
deduplicate — this avoids translating the same run twice while
ensuring hyperlink text IS picked up.
"""
# Check full paragraph text including nested content (hyperlinks, etc.)
full_text = ''.join(
@@ -1046,18 +1169,33 @@ class WordTranslator:
if not full_text:
return
# Collect from direct child runs
# Collect every <w:r> element in the paragraph tree, including
# those nested in <w:hyperlink>, <w:smartTag>, etc. The dedup by
# element id is defensive — `paragraph.runs` and the manual iter
# below could overlap if python-docx starts surfacing nested runs.
seen_run_ids: set = set()
# 1) Direct runs (paragraph.runs is the python-docx-native API).
for run in paragraph.runs:
run_id = id(run._r)
if run_id in seen_run_ids:
continue
seen_run_ids.add(run_id)
if run.text and run.text.strip():
self._append_run_translation(run, text_elements)
# Collect from runs inside <w:hyperlink> elements
# (TOC entries, cross-references — python-docx's paragraph.runs skips these)
for hl in paragraph._p.iter(qn('w:hyperlink')):
for r_elem in hl.findall(qn('w:r')):
run = Run(r_elem, paragraph)
if run.text and run.text.strip():
self._append_run_translation(run, text_elements)
# 2) Runs nested inside <w:hyperlink> (TOC, cross-references).
# python-docx's `paragraph.runs` does NOT descend into hyperlinks in
# version 1.x — we have to walk the XML ourselves.
for r_elem in paragraph._p.iter(qn('w:r')):
run_id = id(r_elem)
if run_id in seen_run_ids:
continue
seen_run_ids.add(run_id)
# Build a Run wrapper so the setter API is consistent.
run = Run(r_elem, paragraph)
if run.text and run.text.strip():
self._append_run_translation(run, text_elements)
def _append_run_translation(
self,