Foundations:
- TranslationSegment model + migration f7e8d9c0b1a2 (segments, workspaces,
workspace_members, glossaries.workspace_id)
- SegmentRecorder injected into all 4 translators: unique (source,
translation) pairs captured per job and persisted (best-effort)
- set_segment_overrides: human-reviewed translations applied verbatim on
rebuild — top priority over TM and provider, zero API calls
Review API (routes/review_routes.py):
- GET /translations/{id}/segments (owner or job token)
- PATCH /segments/{id} edit/approve — feeds the per-user TM so approved
translations are reused in later jobs
- POST /translations/{id}/rebuild — rebuild document with reviewed text
- GET/POST /translations/{id}/xliff — XLIFF 1.2 export/import (edited
segments export their reviewed text)
Review editor (frontend /dashboard/reviews/[jobId]):
- side-by-side source/translation table, inline edit, approve (single or
all), rebuild & download (auth blob), XLIFF export/import, 13 locales
- 'Relire et corriger' link on the translation-complete screen
Team workspaces (routes/workspace_routes.py + /dashboard/teams):
- Workspace/WorkspaceMember models, roles owner/admin/member
- create (Business plan), list with seat usage, invite by email with
seat-limit enforcement (Business=5, Enterprise unlimited), removal
- shared glossaries: workspace members can use a glossary shared to their
workspace (access check extended)
Tests: 1184 passed / 0 failed (11 new: recorder, overrides, docx
capture->rebuild e2e, XLIFF structure/escaping, seats, workspace CRUD,
shared glossary access)
User reported that on the page 3 of the test PDF ('2. Installation
and Setup'), the curl code block was visually broken: the 5 code
lines (curl, -H, -F, -F, -F) were split, with the first 2 lines
above the gray background box and the last 3 inside (or vice versa).
Root cause: each code line is its own PDF block. The merge logic
correctly combined them into a single block (same x0, similar font,
small gap). The smart-fit then wrote the entire 5-line text into
the merged block's bbox, shrinking the font to fit. The result
no longer aligned with the fixed-extent gray background drawing.
Fix: detect when a block is covered by a colored background drawing
(code block, callout box, info box, etc.) and:
1. Mark each line as _no_merge=True so the merge logic keeps them
as separate per-line blocks
2. Each line keeps its original y position
3. The smart-fit writes each line at its own bbox, preserving
alignment with the surrounding drawing
Detection: a block is 'covered by drawing' if >= 50% of its bbox
area intersects a filled drawing on the page. This is conservative
enough to avoid false positives from drawings that merely touch a
corner of the block.
The same logic applies to callout boxes, info boxes, and any other
visual element where the background defines a fixed extent that the
text must align with. The detection is generic — no hardcoded
patterns, no font-based heuristics.
3 new tests added:
- test_code_block_lines_marked_no_merge: 5 lines inside a
background drawing all marked _no_merge=True
- test_paragraph_not_marked_no_merge: 3 plain lines (no drawing)
still merge into 1 block (regression check)
- test_code_block_end_to_end_preserves_lines: full translate
pipeline, each line stays at its own y, all inside the drawing
Total: 460 tests pass (was 457), zero regression.
User reported that the page 6 table on the test PDF ('6. Performance
and Scaling' page) was completely broken: the 3-column table
('Document size | Avg latency (s) | Throughput (docs/min)' with
5 data rows) was rendered as a vertical list of label/value pairs
instead of as a proper table.
Root cause: a PDF 'block' that contains multiple LINES at the SAME
y but different x positions is a table row (3 cells side-by-side).
The extractor was treating the whole row as one paragraph, joining
all cell texts with newline. When the smart-fit logic wrote the
text back, it used the row's full-width bbox and \insert_textbox\
wrote everything left-aligned, collapsing all columns into one.
Fix: at extraction time, detect horizontal-layout blocks (lines at
the same y, different x within 5pt tolerance) and split them into
one sub-block per line. Each cell gets its own bbox, so the
translator writes each cell at its original x position, preserving
the column structure.
Detection heuristic:
- Block has >= 2 lines
- All lines have y0 within 3pt of each other (SAME_ROW_Y_TOLERANCE)
- At least 2 lines have different x0 (within > 5pt)
If all three hold, it's a table row. Otherwise, keep the old
multi-line-paragraph behavior.
Note: PyMuPDF re-groups cells into row-blocks when reading the
output back (so 'len(blocks)' looks unchanged), but the LINES
within each block are at their correct x positions. Tests check
the line x0 values, not the block count.
Visual proof: page 7 of sample_files/test_corpus/test_pdf_translated.pdf
now shows the table with proper 3-column structure (Taille du document
| Latence moyenne (s) | Débit (docs/min)) instead of an '[translation
overflow]' placeholder.
4 new tests added:
- test_horizontal_layout_detected: 3 lines at same y -> 3 blocks
- test_vertical_layout_kept_as_one_block: 3 lines at different y -> 1 block
- test_single_line_block_unchanged: 1 line -> 1 block
- test_table_cell_each_at_own_x: e2e table translation, cells at
correct x positions
Total: 457 tests pass (was 453), zero regression.
User asked whether B3.6+B3.7 are generic for ALL PDFs or just for the
test_pdf.pdf. Audit found 2 genericity bugs:
1. _populate_next_block_y was sorting blocks globally by y0. In a
multi-column PDF (journals, brochures, newspapers), the 'next
block' of a left-column block would point to the right-column
block at the same y, which is wrong. Fix: group blocks into
columns by x0 proximity (15pt tolerance), then sort each column
by y0. Each block's next_block_y is the y0 of its column-mate
directly below it, not just the next block in y-order globally.
2. max_expand_y could go negative if next_block_y was above the
current block (rare edge case in extracted blocks with weird
bbox ordering). A negative max_expand_y would create an invalid
fitz.Rect with y1 < y0, causing silent failures. Fix: clamp
max_expand_y to >= 0.
7 new tests added:
- test_two_columns_get_separate_next_block_y: 2-col layout,
left and right columns get independent next_block_y mappings
- test_centered_full_width_header_gets_own_column: full-width
header between 2 columns is its own column
- test_three_columns: 3-column newspaper layout
- test_single_block_page, test_empty_block_list: edge cases
- test_max_expand_y_clamped_to_zero: negative-expansion safety
- test_two_column_pdf_translation_end_to_end: e2e test on a
2-col journal PDF, 4 input blocks -> 4 output blocks preserved
at correct positions, no cross-column overlap
Visual verification:
scripts/verify_b3_8_multicolumn.py renders a 2-col journal PDF
before and after translation, confirms 4 left + 4 right blocks
preserved at exact positions.
Total tests: 453 (was 446), zero regression.
B3.6 — fix two visual bugs reported on the user's prod PDF:
1. Title 'Spécification technique : Office Translator v3.0' overflowed
its 2-line bbox and overlapped the 'Version du document...' block.
Root cause: MAX_VERTICAL_EXPANSION was 1.5x the original height,
way too small for a long French title. Bumped to 6.0x.
2. 'Avis important' blue background had white rectangular patches.
Root cause: redaction always used fill=(1,1,1) (opaque white),
which erased the colored drawing underneath the text.
Fix: detect when a block's bbox intersects a page drawing,
and use fill=None (transparent) for the redaction in that case.
The original drawing survives intact.
B3.7 — eliminate remaining block-vs-next-block overlap:
Computes each block's 'next_block_y' (the y0 of the nearest block
below it on the same page) and uses it as the ceiling for vertical
expansion. Previously the smart-fit logic used the page bottom as
the ceiling, which let long translated blocks flow into their
neighbour (e.g. 'Pour la dernière version...' overlapping
'8. Résolution des problèmes' in the TOC).
Also includes:
- 9 new tests (6 B3.6 + 3 B3.7) — total 446 tests pass, zero regression
- scripts/verify_b3_6_fix.py — visual+structural verification
- Updated sample_files/test_corpus/test_pdf_translated.pdf with the
clean B3.6+B3.7 output
ROOT CAUSE FIX: PyMuPDF silently raised AttributeError when fontname=None
was passed to insert_textbox. The try/except in _try_insert was swallowing
the error and returning None, causing every block to be skipped via the
graceful failure path. Setting fontname='helv' as the default unblocks
the entire PDF translation pipeline.
SMART-FIT: rewrite _write_translated_block with proper tier-fallback:
- Tier 0: original bbox at original size
- Tier 1: expanded horizontal
- Tier 2: expanded vertical (3x original height)
- Tier 3: shrink once (0.93x)
- Tier 4: shrink twice (0.87x cumulative)
- Tier 5: min size floor (90% for headings, 75% for body)
- Tier 6: graceful skip with visible placeholder
REDACTION: single redaction per block (was per sub-bbox, creating 100+
redaction rectangles per page). Now only 1 redaction per text block.
FEATURE FLAG: PDF_SMART_FIT_ENABLED (default true, observation-first).
METRICS: text_overflow -> format_elements_lost_total.
RESULT ON REAL PDF:
Before: fonts shrunk 22pt->5.6pt, hierarchy destroyed
After: fonts EXACT match: [8, 11, 12, 14, 16, 22] preserved
Two critical fixes:
1. Provider "google" (default classic mode) now checks for a Google Cloud
API key (GOOGLE_CLOUD_API_KEY in env or admin settings). If present,
uses GoogleCloudTranslationProvider (official API). Previously it
always fell through to deep_translator (free scraper) which gets
blocked in production, silently returning untranslated text.
2. Added translation verification: each translator now tracks how many
texts were attempted vs actually changed. If 0 texts were translated,
the job is marked as FAILED with a clear error message instead of
returning the original file as "completed".
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Restructured docker-compose for Nginx Proxy Manager (no custom nginx)
- Added domain wordly.art configuration
- Added Prometheus + Grafana monitoring stack with pre-configured dashboards
- Added PostgreSQL backup script to NAS (daily/weekly/monthly rotation)
- Added alert rules for backend, system, and Docker metrics
- Updated deployment guide for NPM + IONOS DNS homelab setup
- Added marketing plan document
- PDF translator and watermark support
- Enhanced middleware, routes, and translator modules
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
