feat(format): B3.5 — PDF smart-fit rewrite + critical fontname=None fix
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m44s
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m44s
ROOT CAUSE FIX: PyMuPDF silently raised AttributeError when fontname=None was passed to insert_textbox. The try/except in _try_insert was swallowing the error and returning None, causing every block to be skipped via the graceful failure path. Setting fontname='helv' as the default unblocks the entire PDF translation pipeline. SMART-FIT: rewrite _write_translated_block with proper tier-fallback: - Tier 0: original bbox at original size - Tier 1: expanded horizontal - Tier 2: expanded vertical (3x original height) - Tier 3: shrink once (0.93x) - Tier 4: shrink twice (0.87x cumulative) - Tier 5: min size floor (90% for headings, 75% for body) - Tier 6: graceful skip with visible placeholder REDACTION: single redaction per block (was per sub-bbox, creating 100+ redaction rectangles per page). Now only 1 redaction per text block. FEATURE FLAG: PDF_SMART_FIT_ENABLED (default true, observation-first). METRICS: text_overflow -> format_elements_lost_total. RESULT ON REAL PDF: Before: fonts shrunk 22pt->5.6pt, hierarchy destroyed After: fonts EXACT match: [8, 11, 12, 14, 16, 22] preserved
This commit is contained in:
33
scripts/rerun_translation.py
Normal file
33
scripts/rerun_translation.py
Normal file
@@ -0,0 +1,33 @@
|
||||
"""Re-run translation with the new B3.5 code and save to disk."""
|
||||
import sys
|
||||
sys.path.insert(0, '.')
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
import fitz
|
||||
|
||||
src = Path('sample_files/test_corpus/test_pdf.pdf')
|
||||
dst = Path('sample_files/test_corpus/test_pdf_translated.pdf')
|
||||
|
||||
from translators.pdf_translator import PDFTranslator
|
||||
PDFTranslator().translate_file(src, dst, target_language='fr', source_language='en')
|
||||
|
||||
o = fitz.open(src)
|
||||
n = fitz.open(dst)
|
||||
print(f'Original: {len(o)} pages, {src.stat().st_size} bytes')
|
||||
print(f'Translated: {len(n)} pages, {dst.stat().st_size} bytes')
|
||||
print()
|
||||
print('Page 1 font sizes:')
|
||||
o_sizes = set()
|
||||
n_sizes = set()
|
||||
for b in o[0].get_text("dict").get("blocks", []):
|
||||
for l in b.get("lines", []):
|
||||
for s in l.get("spans", []):
|
||||
o_sizes.add(round(s.get("size", 0), 1))
|
||||
for b in n[0].get_text("dict").get("blocks", []):
|
||||
for l in b.get("lines", []):
|
||||
for s in l.get("spans", []):
|
||||
n_sizes.add(round(s.get("size", 0), 1))
|
||||
print(' Original: ', sorted(o_sizes))
|
||||
print(' Translated:', sorted(n_sizes))
|
||||
o.close()
|
||||
n.close()
|
||||
Reference in New Issue
Block a user