feat(format): B3.8 — column-aware next-block layout for multi-column PDFs
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m32s
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m32s
User asked whether B3.6+B3.7 are generic for ALL PDFs or just for the
test_pdf.pdf. Audit found 2 genericity bugs:
1. _populate_next_block_y was sorting blocks globally by y0. In a
multi-column PDF (journals, brochures, newspapers), the 'next
block' of a left-column block would point to the right-column
block at the same y, which is wrong. Fix: group blocks into
columns by x0 proximity (15pt tolerance), then sort each column
by y0. Each block's next_block_y is the y0 of its column-mate
directly below it, not just the next block in y-order globally.
2. max_expand_y could go negative if next_block_y was above the
current block (rare edge case in extracted blocks with weird
bbox ordering). A negative max_expand_y would create an invalid
fitz.Rect with y1 < y0, causing silent failures. Fix: clamp
max_expand_y to >= 0.
7 new tests added:
- test_two_columns_get_separate_next_block_y: 2-col layout,
left and right columns get independent next_block_y mappings
- test_centered_full_width_header_gets_own_column: full-width
header between 2 columns is its own column
- test_three_columns: 3-column newspaper layout
- test_single_block_page, test_empty_block_list: edge cases
- test_max_expand_y_clamped_to_zero: negative-expansion safety
- test_two_column_pdf_translation_end_to_end: e2e test on a
2-col journal PDF, 4 input blocks -> 4 output blocks preserved
at correct positions, no cross-column overlap
Visual verification:
scripts/verify_b3_8_multicolumn.py renders a 2-col journal PDF
before and after translation, confirms 4 left + 4 right blocks
preserved at exact positions.
Total tests: 453 (was 446), zero regression.
This commit is contained in:
@@ -581,25 +581,55 @@ class PDFTranslator:
|
||||
|
||||
def _populate_next_block_y(self, blocks: List[Dict], page_rect) -> None:
|
||||
"""For each block, set ``block['_next_block_y']`` to the y0 of the
|
||||
nearest block below it (or the page bottom margin if none).
|
||||
nearest block below it in the SAME COLUMN (or the page bottom
|
||||
margin if none).
|
||||
|
||||
Track B3.7: this lets the smart-fit logic use the next block as
|
||||
a ceiling for vertical expansion, so a long translated block
|
||||
never overlaps its neighbour.
|
||||
|
||||
Track B3.8: COLUMN-AWARE. PDFs with multi-column layouts
|
||||
(e.g. journal articles, brochures) would otherwise get
|
||||
the wrong "next block" — sorting globally by y0 would map
|
||||
a left-column block to a right-column block as its "next".
|
||||
We group blocks into columns by x0 proximity (15pt tolerance),
|
||||
then sort each column by y0. Blocks that don't fit any
|
||||
column (e.g. full-width centered headers) are treated as
|
||||
their own single-block column.
|
||||
"""
|
||||
margin = 18
|
||||
page_bottom = page_rect.y1 - margin
|
||||
COLUMN_TOLERANCE = 15.0 # points — blocks within this x0 are "same column"
|
||||
|
||||
# Sort by y0 ascending; for each block, the next one in this list
|
||||
# is its nearest downward neighbour.
|
||||
sorted_blocks = sorted(blocks, key=lambda b: b["bbox"][1])
|
||||
for i, block in enumerate(sorted_blocks):
|
||||
if i + 1 < len(sorted_blocks):
|
||||
block["_next_block_y"] = sorted_blocks[i + 1]["bbox"][1] - 2
|
||||
else:
|
||||
block["_next_block_y"] = page_bottom
|
||||
if not blocks:
|
||||
return
|
||||
|
||||
# Keep a small visual gap so we don't crowd the next block's top edge.
|
||||
# Group blocks into columns by x0 proximity. We iterate in
|
||||
# x0-sorted order and assign each block to the first column
|
||||
# whose reference x0 is within tolerance, or start a new column.
|
||||
# This gives us columns ordered left-to-right.
|
||||
columns: List[List[Dict]] = []
|
||||
for block in sorted(blocks, key=lambda b: b["bbox"][0]):
|
||||
assigned = False
|
||||
for column in columns:
|
||||
col_ref_x0 = column[0]["bbox"][0]
|
||||
if abs(block["bbox"][0] - col_ref_x0) <= COLUMN_TOLERANCE:
|
||||
column.append(block)
|
||||
assigned = True
|
||||
break
|
||||
if not assigned:
|
||||
columns.append([block])
|
||||
|
||||
# For each column, set next_block_y by y-order within that column.
|
||||
# A block whose expanded bbox would touch the next column-mate's
|
||||
# y0 will be capped to that y0 - 2 (small visual gap).
|
||||
for column in columns:
|
||||
column.sort(key=lambda b: b["bbox"][1])
|
||||
for i, block in enumerate(column):
|
||||
if i + 1 < len(column):
|
||||
block["_next_block_y"] = column[i + 1]["bbox"][1] - 2
|
||||
else:
|
||||
block["_next_block_y"] = page_bottom
|
||||
|
||||
def _write_translated_block(
|
||||
self,
|
||||
@@ -664,10 +694,16 @@ class PDFTranslator:
|
||||
# Track B3.7: cap vertical expansion at the next block's y0
|
||||
# (not the page bottom), so a long translated block cannot
|
||||
# overflow into its neighbour.
|
||||
# Track B3.8: clamp to >= 0 to avoid an invalid (inverted) Rect
|
||||
# when next_block_y sits above the current block (rare edge
|
||||
# case in extracted blocks where bbox ordering is unusual).
|
||||
next_block_y = block.get("_next_block_y", page_rect.y1 - margin)
|
||||
max_expand_y = min(
|
||||
next_block_y - original_rect.y1,
|
||||
original_rect.height * MAX_VERTICAL_EXPANSION,
|
||||
max_expand_y = max(
|
||||
0.0,
|
||||
min(
|
||||
next_block_y - original_rect.y1,
|
||||
original_rect.height * MAX_VERTICAL_EXPANSION,
|
||||
),
|
||||
)
|
||||
expanded_v = fitz.Rect(
|
||||
expanded_h.x0,
|
||||
|
||||
Reference in New Issue
Block a user