feat(format): B3.5 — PDF smart-fit rewrite + critical fontname=None fix
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m44s
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m44s
ROOT CAUSE FIX: PyMuPDF silently raised AttributeError when fontname=None was passed to insert_textbox. The try/except in _try_insert was swallowing the error and returning None, causing every block to be skipped via the graceful failure path. Setting fontname='helv' as the default unblocks the entire PDF translation pipeline. SMART-FIT: rewrite _write_translated_block with proper tier-fallback: - Tier 0: original bbox at original size - Tier 1: expanded horizontal - Tier 2: expanded vertical (3x original height) - Tier 3: shrink once (0.93x) - Tier 4: shrink twice (0.87x cumulative) - Tier 5: min size floor (90% for headings, 75% for body) - Tier 6: graceful skip with visible placeholder REDACTION: single redaction per block (was per sub-bbox, creating 100+ redaction rectangles per page). Now only 1 redaction per text block. FEATURE FLAG: PDF_SMART_FIT_ENABLED (default true, observation-first). METRICS: text_overflow -> format_elements_lost_total. RESULT ON REAL PDF: Before: fonts shrunk 22pt->5.6pt, hierarchy destroyed After: fonts EXACT match: [8, 11, 12, 14, 16, 22] preserved
This commit is contained in:
72
scripts/test_full_pipeline.py
Normal file
72
scripts/test_full_pipeline.py
Normal file
@@ -0,0 +1,72 @@
|
||||
"""Test if the full pipeline (extract, redact, re-insert links) affects insert_textbox."""
|
||||
import sys
|
||||
sys.path.insert(0, '.')
|
||||
import importlib.util
|
||||
import fitz
|
||||
|
||||
spec = importlib.util.spec_from_file_location('pdf_mod', 'translators/pdf_translator.py')
|
||||
pdf_mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(pdf_mod)
|
||||
|
||||
doc = fitz.open('sample_files/test_corpus/test_pdf.pdf')
|
||||
page = doc[0]
|
||||
|
||||
# Step 1: Extract blocks
|
||||
raw_blocks = pdf_mod.PDFTranslator()._extract_text_blocks(page)
|
||||
title = raw_blocks[0]
|
||||
print(f"Title bbox: {title['bbox']}")
|
||||
|
||||
# Step 2: Simulate redaction
|
||||
title_rect = fitz.Rect(title['bbox'])
|
||||
page.add_redact_annot(title_rect, fill=(1, 1, 1))
|
||||
page.apply_redactions(images=fitz.PDF_REDACT_IMAGE_NONE)
|
||||
print("After redaction")
|
||||
|
||||
# Step 3: Simulate link reinsertion (we have TOC links)
|
||||
links = list(page.get_links())
|
||||
print(f"Got {len(links)} links")
|
||||
for link in links:
|
||||
from_rect = link.get("from")
|
||||
if link.get("page") is not None:
|
||||
page.insert_link({
|
||||
"kind": fitz.LINK_GOTO,
|
||||
"from": from_rect,
|
||||
"page": link["page"],
|
||||
"to": fitz.Point(72, 72),
|
||||
})
|
||||
elif link.get("uri"):
|
||||
page.insert_link({
|
||||
"kind": fitz.LINK_URI,
|
||||
"from": from_rect,
|
||||
"uri": link["uri"],
|
||||
})
|
||||
print("After link reinsertion")
|
||||
|
||||
# Step 4: NOW try insert_textbox
|
||||
french = "Spécification technique : Office Translator v3.0"
|
||||
rect = fitz.Rect(title['bbox'])
|
||||
expanded_v = fitz.Rect(rect.x0, rect.y0, rect.x1, rect.y1 + rect.height * 3)
|
||||
|
||||
# Test on original bbox
|
||||
rc1 = page.insert_textbox(
|
||||
rect, french,
|
||||
fontsize=22,
|
||||
fontfile='C:/Windows/Fonts/arial.ttf',
|
||||
color=(0.1, 0.2, 0.5),
|
||||
align=fitz.TEXT_ALIGN_LEFT,
|
||||
overlay=True,
|
||||
)
|
||||
print(f"After full pipeline, original rect: rc={rc1}")
|
||||
|
||||
# Test on expanded_v
|
||||
rc2 = page.insert_textbox(
|
||||
expanded_v, french,
|
||||
fontsize=22,
|
||||
fontfile='C:/Windows/Fonts/arial.ttf',
|
||||
color=(0.1, 0.2, 0.5),
|
||||
align=fitz.TEXT_ALIGN_LEFT,
|
||||
overlay=True,
|
||||
)
|
||||
print(f"After full pipeline, expanded_v: rc={rc2}")
|
||||
|
||||
doc.close()
|
||||
Reference in New Issue
Block a user