feat(format): B3.9 — preserve PDF table column structure during translation
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m30s

User reported that the page 6 table on the test PDF ('6. Performance
and Scaling' page) was completely broken: the 3-column table
('Document size | Avg latency (s) | Throughput (docs/min)' with
5 data rows) was rendered as a vertical list of label/value pairs
instead of as a proper table.

Root cause: a PDF 'block' that contains multiple LINES at the SAME
y but different x positions is a table row (3 cells side-by-side).
The extractor was treating the whole row as one paragraph, joining
all cell texts with newline. When the smart-fit logic wrote the
text back, it used the row's full-width bbox and \insert_textbox\
wrote everything left-aligned, collapsing all columns into one.

Fix: at extraction time, detect horizontal-layout blocks (lines at
the same y, different x within 5pt tolerance) and split them into
one sub-block per line. Each cell gets its own bbox, so the
translator writes each cell at its original x position, preserving
the column structure.

Detection heuristic:
  - Block has >= 2 lines
  - All lines have y0 within 3pt of each other (SAME_ROW_Y_TOLERANCE)
  - At least 2 lines have different x0 (within > 5pt)
If all three hold, it's a table row. Otherwise, keep the old
multi-line-paragraph behavior.

Note: PyMuPDF re-groups cells into row-blocks when reading the
output back (so 'len(blocks)' looks unchanged), but the LINES
within each block are at their correct x positions. Tests check
the line x0 values, not the block count.

Visual proof: page 7 of sample_files/test_corpus/test_pdf_translated.pdf
now shows the table with proper 3-column structure (Taille du document
| Latence moyenne (s) | Débit (docs/min)) instead of an '[translation
overflow]' placeholder.

4 new tests added:
  - test_horizontal_layout_detected: 3 lines at same y -> 3 blocks
  - test_vertical_layout_kept_as_one_block: 3 lines at different y -> 1 block
  - test_single_line_block_unchanged: 1 line -> 1 block
  - test_table_cell_each_at_own_x: e2e table translation, cells at
    correct x positions

Total: 457 tests pass (was 453), zero regression.
This commit is contained in:
2026-07-14 19:25:24 +02:00
parent 4aebb49c7b
commit 2da2c4765c
3 changed files with 252 additions and 1 deletions

View File

@@ -454,12 +454,28 @@ class PDFTranslator:
return output_path
def _extract_text_blocks(self, page) -> List[Dict]:
"""Extract text blocks with position, font, and color information."""
"""Extract text blocks with position, font, and color information.
Track B3.9: when a PDF "block" contains multiple LINES at the
same y (typical of table rows where each cell is a separate line
in the block), split it into one sub-block per line. This
preserves the column structure so a translated table row stays
a table row instead of collapsing into a single left-aligned
column.
The detection is: if a block has >= 2 lines whose y0 is within
``SAME_ROW_Y_TOLERANCE`` of each other, treat it as a
multi-column row and emit one sub-block per line. Blocks whose
lines are vertically stacked (multi-line paragraphs) keep the
old behavior.
"""
import fitz
blocks = []
data = page.get_text("dict", flags=fitz.TEXT_PRESERVE_WHITESPACE)
SAME_ROW_Y_TOLERANCE = 3.0 # points
for block in data.get("blocks", []):
if block.get("type") != 0:
continue
@@ -468,6 +484,65 @@ class PDFTranslator:
if not lines:
continue
# B3.9: detect table-row pattern. A row is multiple lines
# at the same y (different x). Multi-line paragraphs have
# lines at different y (same x).
line_y0s = [line["bbox"][1] for line in lines]
same_y = all(
abs(y - line_y0s[0]) <= SAME_ROW_Y_TOLERANCE
for y in line_y0s
)
has_horizontal_layout = (
same_y and len(lines) > 1
and any(
abs(lines[i]["bbox"][0] - lines[0]["bbox"][0]) > 5
for i in range(1, len(lines))
)
)
if has_horizontal_layout:
# Multi-column row: emit one block per line.
for line in lines:
span_parts = []
spans_info = []
for span in line.get("spans", []):
text = span.get("text", "")
if text:
span_parts.append(text)
spans_info.append({
"size": span.get("size", 12),
"font": span.get("font", "Helvetica"),
"color": span.get("color", 0),
"flags": span.get("flags", 0),
"origin": span.get("origin", (0, 0)),
})
if not span_parts:
continue
line_text = "".join(span_parts).strip()
if not line_text:
continue
avg_size = (
sum(s["size"] for s in spans_info) / len(spans_info)
if spans_info else 12.0
)
first_color = spans_info[0]["color"] if spans_info else 0
is_bold = any(s["flags"] & 16 for s in spans_info)
is_italic = any(s["flags"] & 2 for s in spans_info)
blocks.append({
"bbox": tuple(line["bbox"]),
"text": line_text,
"font_size": round(avg_size, 1),
"color": first_color,
"is_bold": is_bold,
"is_italic": is_italic,
"line_count": 1,
"translated": None,
"sub_bboxes": [tuple(line["bbox"])],
"_is_table_cell": True,
})
continue
# Standard multi-line paragraph: join lines with \n.
line_parts = []
spans_info = []