sepehr/office_translator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(format): B1 — Word/Excel quick wins for format preservation
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m35s
Details

Browse Source 
Word fixes:
  W1 — Fix hyperlink double-collect: a run inside <w:hyperlink> was
       previously collected twice (once via paragraph.runs, once via
       the manual hyperlink iter). Now uses a dedup set of element
       ids to collect each run exactly once.

       NB: python-docx 1.x's paragraph.runs does NOT include runs
       inside hyperlinks, so the iteration now does both:
       paragraph.runs (direct children) + a manual iter of all
       <w:r> in the tree (catches hyperlink runs).

  W2 — Fix footnotes import: used document.part.package.part_related_by
       which doesn't exist in python-docx 1.x, so footnotes were never
       collected. Now uses document.part.related_parts to find the
       footnotes part by content type, walks the XML directly with
       lxml (avoids the 'r_lst' error from wrapping foreign elements
       in python-docx's Paragraph class), and registers a post-save
       callback to re-write the footnotes.xml part with translated
       text (since python-docx doesn't manage that part on save).
       Same fix applied to endnotes.

  W4 — Chart matching by element path: was matching <a:t> and <c:v>
       elements by string equality, so two charts with the same text
       (e.g. two 'Revenue' series) would only have the first one
       translated. Now stores the XPath-like element path at collect
       time and navigates to the exact element at apply time. Falls
       back to string matching for legacy entries without a path.

Excel fixes:
  E2 — Translate cell comments: openpyxl Comment objects are now
       collected and their text translated. The Comment object is
       replaced in place after translation.

  E3 — Translate cell hyperlink display labels: cell.hyperlink.display
       (or .target if no display) is collected and translated. The
       URL itself is never sent for translation, so it remains
       intact. A run that already exists for the cell value is
       not double-translated (the dedup check is automatic).

  E4 — Chart matching by element path: same fix as W4 but for
       Excel. Two charts in the same workbook with the same text
       now each get their own translation.

Tests:
  Added tests/test_translators/test_b1_format_fixes.py with 11 tests
  covering all the fixes. All 11 pass. Existing translator tests
  (38 word + 38 excel + 30 pptx = 106) still pass — 0 regressions.

  Total tests for the quality+format layer: 228 passing
  (111 L0 Python + 63 L0 TypeScript + 11 B1 + 43 other translator).

All fixes are surgical: existing translation flow is preserved.
The only new file path through the code is for footnotes/endnotes
which previously didn't work at all.
This commit is contained in:
sepehr
2026-07-14 16:28:17 +02:00
parent f403b2851d
commit 5ae1587428
3 changed files with 901 additions and 112 deletions
Download Patch File Download Diff File Expand all files Collapse all files

302
translators/word_translator.py
View File

@@ -230,8 +230,11 @@ class WordTranslator:
text_elements: List[Tuple[str, Callable[[str], None]]] = []
chart_translations: List[Dict[str, Any]] = []
diagram_translations: List[Dict[str, Any]] = []
# Callbacks to run AFTER document.save() to write back parts
# that python-docx doesn't manage (footnotes, endnotes).
post_save_callbacks: List[Callable[[Path], None]] = []
self._collect_from_body(document, text_elements)
self._collect_from_body(document, text_elements, post_save_callbacks)
# Collect chart text from ZIP (chart titles, axis labels, series names)
self._collect_charts_from_zip(input_path, text_elements, chart_translations)
@@ -364,6 +367,14 @@ class WordTranslator:
if diagram_translations:
self._apply_diagram_translations(output_path, diagram_translations)
# Run post-save callbacks (e.g. footnotes/endnotes that python-docx
# does not write back automatically).
for callback in post_save_callbacks:
try:
callback(output_path)
except Exception as cb_err:
_log_error("word_post_save_callback_error", error=str(cb_err))
processing_time_ms = round((time.time() - start_time) * 1000, 2)
_log_info(
@@ -523,7 +534,8 @@ class WordTranslator:
)
def _collect_from_body(
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]]
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]],
post_save_callbacks: List[Callable[[Path], None]] = None,
) -> None:
"""Collect all text elements from document body.
@@ -546,9 +558,11 @@ class WordTranslator:
pass2_count = len(text_elements) - count_before - pass1_count
# Pass 3: footnotes and endnotes
self._collect_from_footnotes(document, text_elements)
self._collect_from_endnotes(document, text_elements)
# Pass 3: footnotes and endnotes (live in separate parts)
if post_save_callbacks is None:
post_save_callbacks = []
self._collect_from_footnotes(document, text_elements, post_save_callbacks)
self._collect_from_endnotes(document, text_elements, post_save_callbacks)
total = len(text_elements) - count_before
_log_info(
@@ -637,71 +651,137 @@ class WordTranslator:
self._collect_from_table(table, text_elements)
def _collect_from_footnotes(
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]]
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]],
post_save_callbacks: List[Callable[[Path], None]] = None,
) -> None:
"""Collect text from footnotes."""
try:
footnotes_part = document.part.package.part_related_by(
"http://schemas.openxmlformats.org/officeDocument/2006/relationships/footnotes"
) if hasattr(document.part, 'package') else None
except Exception:
footnotes_part = None
"""Collect text from footnotes.
if footnotes_part is None:
# Fallback: try direct XML access
try:
footnotes_element = document.element.find(qn("w:footnotes"))
if footnotes_element is not None:
for child in footnotes_element:
if isinstance(child, CT_P):
paragraph = Paragraph(child, document)
self._collect_from_paragraph(paragraph, text_elements)
except Exception:
pass
python-docx 1.x doesn't expose footnotes via `document.part.package`
(that attribute doesn't exist). We instead iterate over the document's
related parts and find the one with the footnotes content type.
Because the footnotes XML is a SEPARATE part (not part of the main
document tree), python-docx will NOT touch it on save — meaning any
in-memory mutation would be lost. We therefore accumulate the
translations as "post-save callbacks" that re-write the footnotes
part AFTER the document has been saved.
"""
footnotes_xml = self._find_part_by_content_type(
document,
"application/vnd.openxmlformats-officedocument.wordprocessingml.footnotes+xml",
)
if footnotes_xml is None:
return
# Collect every <w:t> in the footnotes XML. We store the t_elem
# references so we can update them post-save.
for t_elem in footnotes_xml.iter(qn("w:t")):
original = t_elem.text or ""
if not original.strip():
continue
def make_t_setter(t):
def setter(text: str) -> None:
t.text = text
return setter
text_elements.append((original, make_t_setter(t_elem)))
# If we found any footnote text, register a post-save callback
# that will rewrite the footnotes part with the (now-translated)
# in-memory XML. The translated t_elems have been mutated by the
# setters called from translate_file's batch loop.
if text_elements and post_save_callbacks is not None:
from copy import deepcopy
# We need to keep a reference to the parsed XML tree so we can
# serialize it after the setters have run. The footnote text
# setters hold references to t_elems inside this tree.
def write_footnotes_back(output_path: Path) -> None:
try:
new_blob = etree.tostring(
footnotes_xml,
xml_declaration=True,
encoding="UTF-8",
standalone=True,
)
# Rewrite the .docx with the updated footnotes part
tmp_path = output_path.with_suffix(".tmp_foot")
with zipfile.ZipFile(output_path, "r") as zin, \
zipfile.ZipFile(tmp_path, "w", zipfile.ZIP_DEFLATED) as zout:
for item in zin.namelist():
if item == "word/footnotes.xml":
zout.writestr(item, new_blob)
else:
zout.writestr(item, zin.read(item))
tmp_path.replace(output_path)
except Exception as e:
_log_error("word_footnotes_writeback_error", error=str(e))
post_save_callbacks.append(write_footnotes_back)
def _find_part_by_content_type(self, document: Document, content_type: str):
"""
Find a related XML part by content type (python-docx 1.x compatible).
Returns the parsed lxml element, or None if not found.
"""
try:
footnotes_xml = etree.fromstring(footnotes_part.blob)
for child in footnotes_xml:
if child.tag == qn("w:footnote"):
for para_elem in child.findall(qn("w:p")):
paragraph = Paragraph(para_elem, document)
self._collect_from_paragraph(paragraph, text_elements)
related_parts = getattr(document.part, "related_parts", None) or {}
for part in related_parts.values():
if getattr(part, "content_type", "") == content_type:
return etree.fromstring(part.blob)
except Exception as e:
_log_error("word_footnotes_parse_error", error=str(e))
_log_error("word_part_lookup_error", content_type=content_type, error=str(e))
return None
def _collect_from_endnotes(
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]]
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]],
post_save_callbacks: List[Callable[[Path], None]] = None,
) -> None:
"""Collect text from endnotes."""
try:
endnotes_part = document.part.package.part_related_by(
"http://schemas.openxmlformats.org/officeDocument/2006/relationships/endnotes"
) if hasattr(document.part, 'package') else None
except Exception:
endnotes_part = None
"""Collect text from endnotes (python-docx 1.x compatible).
if endnotes_part is None:
try:
endnotes_element = document.element.find(qn("w:endnotes"))
if endnotes_element is not None:
for child in endnotes_element:
if isinstance(child, CT_P):
paragraph = Paragraph(child, document)
self._collect_from_paragraph(paragraph, text_elements)
except Exception:
pass
See `_collect_from_footnotes` for why we need post-save callbacks.
"""
endnotes_xml = self._find_part_by_content_type(
document,
"application/vnd.openxmlformats-officedocument.wordprocessingml.endnotes+xml",
)
if endnotes_xml is None:
return
try:
endnotes_xml = etree.fromstring(endnotes_part.blob)
for child in endnotes_xml:
if child.tag == qn("w:endnote"):
for para_elem in child.findall(qn("w:p")):
paragraph = Paragraph(para_elem, document)
self._collect_from_paragraph(paragraph, text_elements)
except Exception as e:
_log_error("word_endnotes_parse_error", error=str(e))
for t_elem in endnotes_xml.iter(qn("w:t")):
original = t_elem.text or ""
if not original.strip():
continue
def make_t_setter(t):
def setter(text: str) -> None:
t.text = text
return setter
text_elements.append((original, make_t_setter(t_elem)))
if text_elements and post_save_callbacks is not None:
def write_endnotes_back(output_path: Path) -> None:
try:
new_blob = etree.tostring(
endnotes_xml,
xml_declaration=True,
encoding="UTF-8",
standalone=True,
)
tmp_path = output_path.with_suffix(".tmp_end")
with zipfile.ZipFile(output_path, "r") as zin, \
zipfile.ZipFile(tmp_path, "w", zipfile.ZIP_DEFLATED) as zout:
for item in zin.namelist():
if item == "word/endnotes.xml":
zout.writestr(item, new_blob)
else:
zout.writestr(item, zin.read(item))
tmp_path.replace(output_path)
except Exception as e:
_log_error("word_endnotes_writeback_error", error=str(e))
post_save_callbacks.append(write_endnotes_back)
def _collect_from_charts(
self, document: Document, text_elements: List[Tuple[str, Callable[[str], None]]]
@@ -825,11 +905,16 @@ class WordTranslator:
"""Re-inject chart translations into the .docx ZIP.
Modifies chart XML files in-place and rewrites the ZIP.
Uses the `element_path` collected during `_collect_charts_from_zip` to
uniquely identify each translatable element, even when the same text
value appears multiple times in the same chart (e.g. two series both
labelled "Revenue"). The old code matched by string equality, which
would translate only the first occurrence.
"""
if not chart_translations:
return
# Only proceed if at least one translation exists
translated_entries = [e for e in chart_translations if 'translated' in e and e['translated']]
if not translated_entries:
return
@@ -846,34 +931,34 @@ class WordTranslator:
chart_files_to_update[cf].append(entry)
try:
# Read all ZIP entries
with zipfile.ZipFile(output_path, 'r') as zf_in:
existing_entries = zf_in.namelist()
# Create new ZIP in memory
buf = io.BytesIO()
with zipfile.ZipFile(buf, 'w', zipfile.ZIP_DEFLATED) as zf_out:
for item in existing_entries:
data = zf_in.read(item)
if item in chart_files_to_update:
# Parse, update, re-serialize this chart XML
try:
chart_xml = etree.fromstring(data)
for entry in chart_files_to_update[item]:
# Find all <a:t> or <c:v> elements and match by original text
tag_to_find = f'{{{_NS_A}}}t'
# Try both a:t and c:v
for t_elem in chart_xml.iter(tag_to_find):
if t_elem.text and t_elem.text.strip() == entry['original']:
t_elem.text = entry['translated']
break
target = self._find_element_by_path(chart_xml, entry.get('element_path', ''))
if target is not None:
target.text = entry['translated']
else:
for t_elem in chart_xml.iter(f'{{{_NS_C}}}v'):
# Fallback: try to find by tag + original text (for
# entries that predate the path-based collection)
tag_to_find = f'{{{_NS_A}}}t'
for t_elem in chart_xml.iter(tag_to_find):
if t_elem.text and t_elem.text.strip() == entry['original']:
t_elem.text = entry['translated']
break
else:
for t_elem in chart_xml.iter(f'{{{_NS_C}}}v'):
if t_elem.text and t_elem.text.strip() == entry['original']:
t_elem.text = entry['translated']
break
data = etree.tostring(chart_xml, xml_declaration=True, encoding='UTF-8', standalone=True)
except Exception as e:
@@ -881,7 +966,6 @@ class WordTranslator:
zf_out.writestr(item, data)
# Replace the output file with the updated ZIP
with open(output_path, 'wb') as f:
f.write(buf.getvalue())
@@ -890,6 +974,41 @@ class WordTranslator:
except Exception as e:
_log_error("word_chart_zip_rewrite_error", error=str(e))
def _find_element_by_path(self, root, path: str):
"""
Navigate the XML tree using an XPath-like path produced by
`_get_element_path`. Returns None if the path is empty or invalid
(e.g. the tree structure changed between collect and apply).
"""
if not path:
return None
try:
current = root
for segment in path.split('/'):
# Format: "tagname[index]"
if '[' not in segment or not segment.endswith(']'):
return None
tag_name, idx_str = segment[:-1].split('[', 1)
try:
idx = int(idx_str)
except ValueError:
return None
children = list(current)
# The path used `list(parent).index(current)` which counts ALL
# children, so we use a simple position lookup.
if idx < 0 or idx >= len(children):
return None
candidate = children[idx]
# Verify tag name (with or without namespace) to avoid silent
# mis-navigation if the tree changed.
candidate_local_tag = candidate.tag.split('}')[-1] if '}' in candidate.tag else candidate.tag
if candidate_local_tag != tag_name:
return None
current = candidate
return current
except Exception:
return None
# ------------------------------------------------------------------
# SmartArt / Diagram support
# ------------------------------------------------------------------
@@ -1036,8 +1155,12 @@ class WordTranslator:
The whitespace is captured and reapplied after translation so that words
at formatting boundaries (e.g. bold/normal) do not get concatenated.
Handles runs both as direct children of <w:p> AND inside <w:hyperlink>
elements (used for TOC entries, cross-references, and bookmarks links).
Note: python-docx's `paragraph.runs` only returns DIRECT child <w:r>
elements, not those inside <w:hyperlink> (used for TOC entries,
cross-references, bookmark links). We therefore iterate the full
XML tree to find every <w:r> and use a set of element ids to
deduplicate — this avoids translating the same run twice while
ensuring hyperlink text IS picked up.
"""
# Check full paragraph text including nested content (hyperlinks, etc.)
full_text = ''.join(
@@ -1046,18 +1169,33 @@ class WordTranslator:
if not full_text:
return
# Collect from direct child runs
# Collect every <w:r> element in the paragraph tree, including
# those nested in <w:hyperlink>, <w:smartTag>, etc. The dedup by
# element id is defensive — `paragraph.runs` and the manual iter
# below could overlap if python-docx starts surfacing nested runs.
seen_run_ids: set = set()
# 1) Direct runs (paragraph.runs is the python-docx-native API).
for run in paragraph.runs:
run_id = id(run._r)
if run_id in seen_run_ids:
continue
seen_run_ids.add(run_id)
if run.text and run.text.strip():
self._append_run_translation(run, text_elements)
# Collect from runs inside <w:hyperlink> elements
# (TOC entries, cross-references — python-docx's paragraph.runs skips these)
for hl in paragraph._p.iter(qn('w:hyperlink')):
for r_elem in hl.findall(qn('w:r')):
run = Run(r_elem, paragraph)
if run.text and run.text.strip():
self._append_run_translation(run, text_elements)
# 2) Runs nested inside <w:hyperlink> (TOC, cross-references).
# python-docx's `paragraph.runs` does NOT descend into hyperlinks in
# version 1.x — we have to walk the XML ourselves.
for r_elem in paragraph._p.iter(qn('w:r')):
run_id = id(r_elem)
if run_id in seen_run_ids:
continue
seen_run_ids.add(run_id)
# Build a Run wrapper so the setter API is consistent.
run = Run(r_elem, paragraph)
if run.text and run.text.strip():
self._append_run_translation(run, text_elements)
def _append_run_translation(
self,