feat(format): B3.8 — column-aware next-block layout for multi-column PDFs
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m32s

User asked whether B3.6+B3.7 are generic for ALL PDFs or just for the
test_pdf.pdf. Audit found 2 genericity bugs:

1. _populate_next_block_y was sorting blocks globally by y0. In a
   multi-column PDF (journals, brochures, newspapers), the 'next
   block' of a left-column block would point to the right-column
   block at the same y, which is wrong. Fix: group blocks into
   columns by x0 proximity (15pt tolerance), then sort each column
   by y0. Each block's next_block_y is the y0 of its column-mate
   directly below it, not just the next block in y-order globally.

2. max_expand_y could go negative if next_block_y was above the
   current block (rare edge case in extracted blocks with weird
   bbox ordering). A negative max_expand_y would create an invalid
   fitz.Rect with y1 < y0, causing silent failures. Fix: clamp
   max_expand_y to >= 0.

7 new tests added:
  - test_two_columns_get_separate_next_block_y: 2-col layout,
    left and right columns get independent next_block_y mappings
  - test_centered_full_width_header_gets_own_column: full-width
    header between 2 columns is its own column
  - test_three_columns: 3-column newspaper layout
  - test_single_block_page, test_empty_block_list: edge cases
  - test_max_expand_y_clamped_to_zero: negative-expansion safety
  - test_two_column_pdf_translation_end_to_end: e2e test on a
    2-col journal PDF, 4 input blocks -> 4 output blocks preserved
    at correct positions, no cross-column overlap

Visual verification:
  scripts/verify_b3_8_multicolumn.py renders a 2-col journal PDF
  before and after translation, confirms 4 left + 4 right blocks
  preserved at exact positions.

Total tests: 453 (was 446), zero regression.
This commit is contained in:
2026-07-14 19:05:47 +02:00
parent 3ae28dd3cb
commit 4255a1a0c5
3 changed files with 411 additions and 13 deletions

View File

@@ -581,25 +581,55 @@ class PDFTranslator:
def _populate_next_block_y(self, blocks: List[Dict], page_rect) -> None:
"""For each block, set ``block['_next_block_y']`` to the y0 of the
nearest block below it (or the page bottom margin if none).
nearest block below it in the SAME COLUMN (or the page bottom
margin if none).
Track B3.7: this lets the smart-fit logic use the next block as
a ceiling for vertical expansion, so a long translated block
never overlaps its neighbour.
Track B3.8: COLUMN-AWARE. PDFs with multi-column layouts
(e.g. journal articles, brochures) would otherwise get
the wrong "next block" — sorting globally by y0 would map
a left-column block to a right-column block as its "next".
We group blocks into columns by x0 proximity (15pt tolerance),
then sort each column by y0. Blocks that don't fit any
column (e.g. full-width centered headers) are treated as
their own single-block column.
"""
margin = 18
page_bottom = page_rect.y1 - margin
COLUMN_TOLERANCE = 15.0 # points — blocks within this x0 are "same column"
# Sort by y0 ascending; for each block, the next one in this list
# is its nearest downward neighbour.
sorted_blocks = sorted(blocks, key=lambda b: b["bbox"][1])
for i, block in enumerate(sorted_blocks):
if i + 1 < len(sorted_blocks):
block["_next_block_y"] = sorted_blocks[i + 1]["bbox"][1] - 2
else:
block["_next_block_y"] = page_bottom
if not blocks:
return
# Keep a small visual gap so we don't crowd the next block's top edge.
# Group blocks into columns by x0 proximity. We iterate in
# x0-sorted order and assign each block to the first column
# whose reference x0 is within tolerance, or start a new column.
# This gives us columns ordered left-to-right.
columns: List[List[Dict]] = []
for block in sorted(blocks, key=lambda b: b["bbox"][0]):
assigned = False
for column in columns:
col_ref_x0 = column[0]["bbox"][0]
if abs(block["bbox"][0] - col_ref_x0) <= COLUMN_TOLERANCE:
column.append(block)
assigned = True
break
if not assigned:
columns.append([block])
# For each column, set next_block_y by y-order within that column.
# A block whose expanded bbox would touch the next column-mate's
# y0 will be capped to that y0 - 2 (small visual gap).
for column in columns:
column.sort(key=lambda b: b["bbox"][1])
for i, block in enumerate(column):
if i + 1 < len(column):
block["_next_block_y"] = column[i + 1]["bbox"][1] - 2
else:
block["_next_block_y"] = page_bottom
def _write_translated_block(
self,
@@ -664,10 +694,16 @@ class PDFTranslator:
# Track B3.7: cap vertical expansion at the next block's y0
# (not the page bottom), so a long translated block cannot
# overflow into its neighbour.
# Track B3.8: clamp to >= 0 to avoid an invalid (inverted) Rect
# when next_block_y sits above the current block (rare edge
# case in extracted blocks where bbox ordering is unusual).
next_block_y = block.get("_next_block_y", page_rect.y1 - margin)
max_expand_y = min(
next_block_y - original_rect.y1,
original_rect.height * MAX_VERTICAL_EXPANSION,
max_expand_y = max(
0.0,
min(
next_block_y - original_rect.y1,
original_rect.height * MAX_VERTICAL_EXPANSION,
),
)
expanded_v = fitz.Rect(
expanded_h.x0,