User reported that on the page 3 of the test PDF ('2. Installation
and Setup'), the curl code block was visually broken: the 5 code
lines (curl, -H, -F, -F, -F) were split, with the first 2 lines
above the gray background box and the last 3 inside (or vice versa).
Root cause: each code line is its own PDF block. The merge logic
correctly combined them into a single block (same x0, similar font,
small gap). The smart-fit then wrote the entire 5-line text into
the merged block's bbox, shrinking the font to fit. The result
no longer aligned with the fixed-extent gray background drawing.
Fix: detect when a block is covered by a colored background drawing
(code block, callout box, info box, etc.) and:
1. Mark each line as _no_merge=True so the merge logic keeps them
as separate per-line blocks
2. Each line keeps its original y position
3. The smart-fit writes each line at its own bbox, preserving
alignment with the surrounding drawing
Detection: a block is 'covered by drawing' if >= 50% of its bbox
area intersects a filled drawing on the page. This is conservative
enough to avoid false positives from drawings that merely touch a
corner of the block.
The same logic applies to callout boxes, info boxes, and any other
visual element where the background defines a fixed extent that the
text must align with. The detection is generic — no hardcoded
patterns, no font-based heuristics.
3 new tests added:
- test_code_block_lines_marked_no_merge: 5 lines inside a
background drawing all marked _no_merge=True
- test_paragraph_not_marked_no_merge: 3 plain lines (no drawing)
still merge into 1 block (regression check)
- test_code_block_end_to_end_preserves_lines: full translate
pipeline, each line stays at its own y, all inside the drawing
Total: 460 tests pass (was 457), zero regression.
User reported that the page 6 table on the test PDF ('6. Performance
and Scaling' page) was completely broken: the 3-column table
('Document size | Avg latency (s) | Throughput (docs/min)' with
5 data rows) was rendered as a vertical list of label/value pairs
instead of as a proper table.
Root cause: a PDF 'block' that contains multiple LINES at the SAME
y but different x positions is a table row (3 cells side-by-side).
The extractor was treating the whole row as one paragraph, joining
all cell texts with newline. When the smart-fit logic wrote the
text back, it used the row's full-width bbox and \insert_textbox\
wrote everything left-aligned, collapsing all columns into one.
Fix: at extraction time, detect horizontal-layout blocks (lines at
the same y, different x within 5pt tolerance) and split them into
one sub-block per line. Each cell gets its own bbox, so the
translator writes each cell at its original x position, preserving
the column structure.
Detection heuristic:
- Block has >= 2 lines
- All lines have y0 within 3pt of each other (SAME_ROW_Y_TOLERANCE)
- At least 2 lines have different x0 (within > 5pt)
If all three hold, it's a table row. Otherwise, keep the old
multi-line-paragraph behavior.
Note: PyMuPDF re-groups cells into row-blocks when reading the
output back (so 'len(blocks)' looks unchanged), but the LINES
within each block are at their correct x positions. Tests check
the line x0 values, not the block count.
Visual proof: page 7 of sample_files/test_corpus/test_pdf_translated.pdf
now shows the table with proper 3-column structure (Taille du document
| Latence moyenne (s) | Débit (docs/min)) instead of an '[translation
overflow]' placeholder.
4 new tests added:
- test_horizontal_layout_detected: 3 lines at same y -> 3 blocks
- test_vertical_layout_kept_as_one_block: 3 lines at different y -> 1 block
- test_single_line_block_unchanged: 1 line -> 1 block
- test_table_cell_each_at_own_x: e2e table translation, cells at
correct x positions
Total: 457 tests pass (was 453), zero regression.
B3.6 — fix two visual bugs reported on the user's prod PDF:
1. Title 'Spécification technique : Office Translator v3.0' overflowed
its 2-line bbox and overlapped the 'Version du document...' block.
Root cause: MAX_VERTICAL_EXPANSION was 1.5x the original height,
way too small for a long French title. Bumped to 6.0x.
2. 'Avis important' blue background had white rectangular patches.
Root cause: redaction always used fill=(1,1,1) (opaque white),
which erased the colored drawing underneath the text.
Fix: detect when a block's bbox intersects a page drawing,
and use fill=None (transparent) for the redaction in that case.
The original drawing survives intact.
B3.7 — eliminate remaining block-vs-next-block overlap:
Computes each block's 'next_block_y' (the y0 of the nearest block
below it on the same page) and uses it as the ceiling for vertical
expansion. Previously the smart-fit logic used the page bottom as
the ceiling, which let long translated blocks flow into their
neighbour (e.g. 'Pour la dernière version...' overlapping
'8. Résolution des problèmes' in the TOC).
Also includes:
- 9 new tests (6 B3.6 + 3 B3.7) — total 446 tests pass, zero regression
- scripts/verify_b3_6_fix.py — visual+structural verification
- Updated sample_files/test_corpus/test_pdf_translated.pdf with the
clean B3.6+B3.7 output
ROOT CAUSE FIX: PyMuPDF silently raised AttributeError when fontname=None
was passed to insert_textbox. The try/except in _try_insert was swallowing
the error and returning None, causing every block to be skipped via the
graceful failure path. Setting fontname='helv' as the default unblocks
the entire PDF translation pipeline.
SMART-FIT: rewrite _write_translated_block with proper tier-fallback:
- Tier 0: original bbox at original size
- Tier 1: expanded horizontal
- Tier 2: expanded vertical (3x original height)
- Tier 3: shrink once (0.93x)
- Tier 4: shrink twice (0.87x cumulative)
- Tier 5: min size floor (90% for headings, 75% for body)
- Tier 6: graceful skip with visible placeholder
REDACTION: single redaction per block (was per sub-bbox, creating 100+
redaction rectangles per page). Now only 1 redaction per text block.
FEATURE FLAG: PDF_SMART_FIT_ENABLED (default true, observation-first).
METRICS: text_overflow -> format_elements_lost_total.
RESULT ON REAL PDF:
Before: fonts shrunk 22pt->5.6pt, hierarchy destroyed
After: fonts EXACT match: [8, 11, 12, 14, 16, 22] preserved
Major changes across backend, frontend, infrastructure:
- Provider system with model selection (Google, DeepL, OpenAI, Ollama, Google Cloud)
- Admin panel: user management, pricing, settings
- Glossary system with CSV import/export
- Subscription and tier quota management
- Security hardening (rate limiting, API key auth, path traversal fixes)
- Docker compose for dev, prod, and IONOS deployment
- Alembic migrations for new tables
- Frontend: dashboard, pricing page, landing page, i18n (en/fr)
- Test suite and verification scripts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Changes:
- Removed hardcoded admin credentials (now requires env vars)
- Added Redis session storage with in-memory fallback
- Improved CORS configuration with warnings for development mode
- Added retry_with_backoff decorator for translation API calls
- Updated pricing: Starter=, Pro=, Business=
- Stripe price IDs now loaded from environment variables
- Added redis to requirements.txt
- Updated .env.example with all new configuration options
- Created COMPREHENSIVE_REVIEW_AND_PLAN.md with deployment roadmap
- Frontend: Updated pricing page, new UI components
