Commit Graph

218 Commits

Author SHA1 Message Date
67365918ae feat(ui,api): wave 3 — editorial pricing, real cancel, server history, DeepL purge
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m36s
Pricing: full editorial redesign — serif card headers with accent pills
replace the colored font-black blocks, tone sweep across toggle/metrics/
features/CTAs, PLAN_COLORS removed; one design system app-wide.

Translate: decorative titles one step down (CTA hierarchy restored);
glossary and image-translation blocks hidden entirely for free users
(progressive disclosure — three controls for free).

Reviews: XLIFF hint line explains the exchange format; backend errors
routed through a friendly mapper (session/not-found/rate-limit/server).

Landing: fabricated hero UI cards (fake 'Context Engine' overlay)
removed — the photo no longer promises screens that don't exist.

Nav: single DashboardNavLinks component shared by sidebar and mobile
drawer (was duplicated markup).

API: GET /api/v1/translations (user job history, paginated; completed
jobs retained 24h) and POST /api/v1/translations/{id}/cancel —
cooperative cancellation with worker checkpoints before dispatch and
before finalisation, reserved quota released immediately. Translate
monitor now offers a real 'Cancel translation' next to 'Back to start';
recent-jobs list reads server history first, localStorage fallback.

DeepL purge (backend): provider module, registry registration, config
attrs/defaults, dispatch branch, admin settings schema + test branch,
legacy availability block, validation rules, plan provider lists,
error-code mappings, MCP enums, translator prompt mention, related
tests updated/removed. Fallback resolver skips unknown providers, so
stale chains containing 'deepl' degrade gracefully.

Verified: backend 110 tests passed; frontend build exit 0, vitest 9/9,
0 missing i18n keys, eslint 63 errors (vs 64 at HEAD).
2026-08-30 22:42:29 +02:00
52748ee653 feat(pricing): remove DeepL from all user-facing surfaces
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m57s
Pricing: 'deepl' removed from plan provider lists and the comparison
table row. Landing and pricing copy (13 locales) no longer mention
DeepL (65 values cleaned, 39 dead keys deleted: pricing.comparison.deepl,
providerTheme.classic.deepl.*).

Providers: available-provider responses are filtered so a DeepL entry
from the backend can never surface in the engine picker or the services
page; DeepL theme entry and static lib/api.ts list entry removed.

Admin: DeepL config card, type fields, defaults and fallback-chain
mentions removed; stats/chart/status maps and mock data cleaned.

Backend adapter untouched — DeepL is invisible app-wide via the UI
filter; removing the Python adapter itself is a separate step if wanted.

Verified: build exit 0, vitest 9/9, eslint clean on touched files,
zero 'deepl' occurrences outside the two intentional UI filters.
2026-08-30 22:20:53 +02:00
017b998941 fix(landing): pricing cards read live plan prices from /api/v1/auth/plans
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m54s
The landing hardcoded 9/19/49 (monthly) and 7/15/39 (annual-equivalent)
while /pricing renders admin-configurable prices — the two surfaces
drifted apart whenever prices changed. The landing now fetches the same
live endpoint (no-store) with the previous values as load/failure
fallback, and shows the French '9 €' format like /pricing (was '€9').
2026-08-30 22:12:52 +02:00
111f3cb69d fix(ui): wave 2 from critique re-run — AA contrast, unified PageHeader, checkout confirm
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m46s
Nav: teams/settings/services back out of the nav until product-ready (user
decision); teams page was already half-finished (UUID member display is a
backend limitation).

Payment: explicit confirmation dialog (plan, amount, billing period, Stripe
note) before any redirect; ?plan= URL now pre-opens the dialog instead of
triggering a silent checkout.

Contrast AA sweep (71 replacements): functional micro-labels raised to
>=60-65% opacity and >=10px across sidebar, header, pricing, landing;
decorative all-caps 'interface' label removed.

Design unification: new PageHeader component (accent pill + serif
base/accent title) applied to settings, services, reviews, teams; pricing
header returned to the editorial voice (serif + accent pill); dead
GlossaryCard deleted.

i18n residuals: suggestion chips, 'Standard' provider label, notification
close, model-combobox strings extracted (+broken bg-surface/border-border-
subtle tokens fixed); ~45 new keys EN+FR; 13 dead keys removed; zero
missing keys verified.

Reviews: icon-only row actions now carry visible text; 'Approve all' is
two-step armed-confirm.

Accelerators: Ctrl/Cmd+Enter submits; arrow-key navigation in the language
combobox; recent-jobs history cap 8->20 with per-job download; source=target
config rejected; empty 'Master Quality' badge removed.

Cookie consent: reopenable via footer link, emoji replaced with drawn icon.

Verified: build exit 0, vitest 9/9, eslint 64 errors = previous level,
0 missing i18n keys.
2026-08-30 22:02:35 +02:00
50047ea8a2 fix(ui): design critique overhaul — a11y, honest metrics, i18n repair (critique 2026-08-30)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m51s
P0 a11y: keyboard-accessible dropzone (role/tabIndex/Enter-Space), ARIA
combobox + listbox pattern for language selector, role=switch on glossary
toggle, role=status live region on notifications, aria-labels on password
toggles, visible-on-focus close buttons, RTL logical positioning (start-*).

Trust: third-party Memento promo removed from translate page, sidebar and
all 13 locales; fabricated stats (99.9%, Turbo, computed layout-integrity
bars) replaced with real measurements incl. API estimated remaining time;
silent download failure now surfaces an error notification.

Honesty: fake 100-byte file injections removed (format chips are now
informational); cancel-that-doesn't renamed 'Back to start' with hint;
Enterprise contact placeholder replaced with contact@wordly.art.

i18n: t() no longer returns raw keys (empty string + defaultValue support,
~30 dead || fallbacks now work); ~170 new keys EN+FR across new reviews/
teams namespaces, glossaries context tab, translate monitor, settings,
services, pricing, landing, fileUploader; split-key italic titles replace
lastIndexOf() surgery (zh/ja-safe); key-audit script added 0 missing.

Flow: active job persisted across refresh with polling resume (24h TTL);
client-side recent-jobs history with review links; review page linked from
complete state; settings/services added to dashboard nav; Business/
Enterprise regain glossary access (tier gate unified).

Typeset (sober-tool direction): 7.5-9px labels raised to 10-12px, /30
opacity to /45-/55, uppercase tracking reduced, trust footer legible,
country flags removed from language switcher, localized dates.

Cleanup: 5 orphaned translate components, dead site header/footer,
fossil tailwind.config.js, PipelineStepper, duplicate pill+H1 titles,
two-step confirm for cache clear, dead landing footer links.

Verified: next build exit 0, vitest 9/9, eslint 64 errors = HEAD
(no regression, -3 warnings), detector 4 -> 3 findings.
2026-08-30 21:44:53 +02:00
1a67241ad5 fix(migrations): use sa.false() for boolean server default
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m48s
PostgreSQL rejects 'BOOLEAN DEFAULT 0' (DatatypeMismatchError: default
expression is of type integer), which aborted the production deploy
during the segments/workspaces migration. sa.false() renders DEFAULT
false on PostgreSQL and DEFAULT 0 on SQLite.
2026-08-30 09:17:18 +02:00
e864caac00 fix(landing): replace language cycle button with dropdown selector
Some checks failed
Deploy to Production / Build and Deploy (push) Failing after 1m49s
The landing page language switcher cycled to the next language on every
click. It is now a dropdown listing all 13 languages; the cycling button
variant is removed from the LanguageSwitcher component.
2026-08-30 09:08:58 +02:00
b4e873ad2c feat(review,teams): review foundation — segments, side-by-side editor, rebuild, XLIFF, team workspaces
Some checks failed
Deploy to Production / Build and Deploy (push) Failing after 2m14s
Foundations:
- TranslationSegment model + migration f7e8d9c0b1a2 (segments, workspaces,
  workspace_members, glossaries.workspace_id)
- SegmentRecorder injected into all 4 translators: unique (source,
  translation) pairs captured per job and persisted (best-effort)
- set_segment_overrides: human-reviewed translations applied verbatim on
  rebuild — top priority over TM and provider, zero API calls

Review API (routes/review_routes.py):
- GET /translations/{id}/segments (owner or job token)
- PATCH /segments/{id} edit/approve — feeds the per-user TM so approved
  translations are reused in later jobs
- POST /translations/{id}/rebuild — rebuild document with reviewed text
- GET/POST /translations/{id}/xliff — XLIFF 1.2 export/import (edited
  segments export their reviewed text)

Review editor (frontend /dashboard/reviews/[jobId]):
- side-by-side source/translation table, inline edit, approve (single or
  all), rebuild & download (auth blob), XLIFF export/import, 13 locales
- 'Relire et corriger' link on the translation-complete screen

Team workspaces (routes/workspace_routes.py + /dashboard/teams):
- Workspace/WorkspaceMember models, roles owner/admin/member
- create (Business plan), list with seat usage, invite by email with
  seat-limit enforcement (Business=5, Enterprise unlimited), removal
- shared glossaries: workspace members can use a glossary shared to their
  workspace (access check extended)

Tests: 1184 passed / 0 failed (11 new: recorder, overrides, docx
capture->rebuild e2e, XLIFF structure/escaping, seats, workspace CRUD,
shared glossary access)
2026-08-29 19:04:32 +02:00
526c87348f feat(translation): quality pipeline overhaul + new features (audit 2026-08-29)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m20s
Translation quality & format preservation:
- Word: merge adjacent same-format runs into one unit (sentence-level
  coherence like inline-tag handling); translate comments/balloons;
  dedupe textbox collection (was translated twice); RTL no longer
  overrides center/justify alignment; CJK/Arabic font hints (eastAsia/cs)
- PPTX: chart translations now actually reach the output file
  (ChartPart.blob is read-only — rewrite chart XML in the saved ZIP);
  CJK typeface hints (a:ea)
- Excel: sheet renames no longer break references — rewrite cell
  formulas (3D/quoted), defined names, data validations, cond. formats
- PDF: bold/italic honored (hebo/heit/hebi); table cells never merge;
  unchanged blocks left untouched (typography preserved, fixes duplicate
  hyperlinks); attempted/changed stats + route gate now cover PDF;
  CJK font paths; scanned PDFs via Mistral OCR (detection + admin settings)

Features:
- formality param (formal/informal) + automatic regional-variant prompts
- output_mode=bilingual docx (source above translation)
- per-user translation memory on Redis (falls back to LRU), context-hashed
- QA report + 0-100 confidence score in job status; L0 on by default
- OpenAI-compatible providers: whole chunk in ONE numbered-JSON request
  (~15x fewer calls) with per-item fallback; base prompt always present
  (custom prompt no longer replaces translation instructions)

Infra & marketing alignment:
- plan-based engine gating + vision gating (closes paid-engine leak);
  /providers/available filtered per plan; 107 languages exposed
- zh-CN/zh-TW validation fixed; libmagic disabled on Windows (native crash)
- admin: Mistral OCR settings + engine status dashboard; httpx<0.28 pin
  (TestClient breakage); Prometheus test fixture fixed
- marketing docs aligned with code (PDF+OCR, retention, engines, pricing)
- security: .env.ionos/.env.production/provider_settings.json removed

Tests: 1173 passed / 0 failed (6 network tests deselected: free Google
endpoint temporarily blocked from this machine)
2026-08-29 18:38:09 +02:00
992f13d53c fix(translate): apply Excel chart text setters (were silently dropped)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m35s
Chart <a:t> elements (title, axis labels, series names) were collected and
sent to the LLM, but the apply loop never invoked their setters. Reason:
the apply loop iterated only text_elements[:sheet_name_offset], which
excluded the chart-text positions (sheet names were inserted in the
middle, pushing chart texts past sheet_name_offset). The LLM correctly
returned French for "Revenue by Product" / "Quantity Trend" / "Order #"
but the result was thrown away.

Fix: include the chart-text slice [sheet_name_offset + N .. total_texts]
in the apply loop. Setters for sheet names are still None, so they are
naturally skipped.

Add test_excel_chart_text_applied.py (2 end-to-end tests) using a fixed
provider that pre-translates every known chart text; the test asserts
all chart <a:t> values in the output .xlsx come from the FR table, not
the original English source.
2026-07-15 22:21:53 +02:00
07c4c12e6a fix(translate): chart labels, sheet-refs, sheet-name offset, paid-user Memento
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m19s
Multiple translation bugs in Word/Excel/PPTX that caused chart elements
to be left untranslated or charts to render as empty series.

Backend
-------
* providers (deepseek/openai/minimax): tighten system prompt so the LLM
  actually translates chart titles/axis labels/legend/category labels,
  month abbreviations, and clarifies what counts as a 'real' proper noun
  (people/place/company/product names) vs. technical labels. Old rule
  'keep proper nouns unchanged' was being read too broadly by the model
  and caused chart text to be skipped.
* excel_translator.py:
  - Sheet reference rewrite: when a sheet is renamed, the chart XML's
    c:f refs (e.g. 'Sales 2024'!$D$2:$D$61) are now rewritten to the
    new name with proper apostrophe escaping (Chiffre d'affaires ->
    'Chiffre d''affaires') and auto-quoting when the new name contains
    spaces or special chars. Without this, the chart points at a sheet
    that no longer exists and renders 0/empty series.
  - Sheet name offset bug: sheet_name_offset was computed after chart
    text was appended to text_elements, causing sheet names to receive
    chart text translations. Now captured BEFORE sheet names are added.
* New tests:
  - test_excel_chart_sheet_refs.py (10 unit tests, synthetic inputs)
  - test_chart_translation_prompt.py (3 contract tests on the prompt)

Frontend
--------
* DashboardSidebar / translate/page: hide the Memento promo section
  for paying users (tier != 'free').
* constants.ts: temporarily comment out the 'CLES API' nav item.
  Update constants.test.ts to match the new state.

All fixes are generic - no file-specific hardcoding, edge cases covered
(empty mapping, missing bang, apostrophe escaping, partial renames,
multi-series).
2026-07-15 22:04:40 +02:00
8f96ddfe71 feat(admin): Stripe live-mode safety + one-click webhook setup
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m31s
User said 'I want this to be automatic'. Three gaps were addressed.

Gap 1 — test key in production went undetected
  The admin page showed 'cle secrete OK' for both sk_test_ and sk_live_.
  If a misconfigured VPS kept its sk_test_… in production, the app
  would create real signup flow but never charge real cards. Added
  services.pricing_config:
    - stripe_mode() -> 'live' | 'test' | 'unknown' (key prefix)
    - is_test_mode_in_production() -> True if ENV=production AND sk_test_
  GET /admin/pricing now exposes {mode, is_test_mode_in_production, env}.
  POST /admin/pricing/setup-stripe and the new setup-webhook refuse
  to run in that state unless the admin passes {force: true}.

Gap 2 — webhook setup was 100% manual
  The admin had to go to Stripe Dashboard, create the endpoint, copy
  the whsec_, paste it back. Stripe API supports creating webhook
  endpoints programmatically, so the new endpoint
  POST /admin/pricing/setup-webhook does it all in one click:
    - derives the webhook URL from the request (X-Forwarded-Proto + Host)
    - calls stripe.WebhookEndpoint.create() (or .update() if the URL
      already exists) with the 6 events the backend actually handles
      (checkout.session.completed, customer.subscription.*, invoice.*)
    - persists the returned whsec_ to .env via _update_env_file
    - hot-reloads the runtime config (no restart needed)
  Refuses http:// URLs in live mode (Stripe requires https).

Gap 3 — obsolete script leaked a test secret
  scripts/stripe_setup.py contained a hardcoded sk_test_… in source.
  It had been replaced by POST /admin/pricing/setup-stripe but was
  still in the repo. Deleted via git rm. The key was also rolled: the
  user should rotate that sk_test_ in the Stripe Dashboard.

Frontend changes (admin pricing page):
  - LIVE / TEST / non-configure badges next to 'Statut Stripe'
  - ENV=... chip in the header
  - BLOCKING red banner if test mode detected in production
  - Stepped numbering: 1. Produits & prix / 2. Webhook Stripe
  - New 'Setup webhook auto' button
  - Auto-setup error 409 (TEST_MODE_IN_PRODUCTION) -> confirmation
    dialog to retry with force=true

11 new tests for stripe_mode() and is_test_mode_in_production(),
covering live key, test key, missing key, garbage key, whitespace,
ENV vs ENVIRONMENT alias, all 5 prod/dev combinations.

Total: 471 tests pass (was 460), zero regression.
2026-07-14 20:19:49 +02:00
9b15b7c9fa feat(format): B3.10 — preserve code-block / callout layout (drawing-covered blocks)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m10s
User reported that on the page 3 of the test PDF ('2. Installation
and Setup'), the curl code block was visually broken: the 5 code
lines (curl, -H, -F, -F, -F) were split, with the first 2 lines
above the gray background box and the last 3 inside (or vice versa).

Root cause: each code line is its own PDF block. The merge logic
correctly combined them into a single block (same x0, similar font,
small gap). The smart-fit then wrote the entire 5-line text into
the merged block's bbox, shrinking the font to fit. The result
no longer aligned with the fixed-extent gray background drawing.

Fix: detect when a block is covered by a colored background drawing
(code block, callout box, info box, etc.) and:
  1. Mark each line as _no_merge=True so the merge logic keeps them
     as separate per-line blocks
  2. Each line keeps its original y position
  3. The smart-fit writes each line at its own bbox, preserving
     alignment with the surrounding drawing

Detection: a block is 'covered by drawing' if >= 50% of its bbox
area intersects a filled drawing on the page. This is conservative
enough to avoid false positives from drawings that merely touch a
corner of the block.

The same logic applies to callout boxes, info boxes, and any other
visual element where the background defines a fixed extent that the
text must align with. The detection is generic — no hardcoded
patterns, no font-based heuristics.

3 new tests added:
  - test_code_block_lines_marked_no_merge: 5 lines inside a
    background drawing all marked _no_merge=True
  - test_paragraph_not_marked_no_merge: 3 plain lines (no drawing)
    still merge into 1 block (regression check)
  - test_code_block_end_to_end_preserves_lines: full translate
    pipeline, each line stays at its own y, all inside the drawing

Total: 460 tests pass (was 457), zero regression.
2026-07-14 19:56:12 +02:00
2da2c4765c feat(format): B3.9 — preserve PDF table column structure during translation
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m30s
User reported that the page 6 table on the test PDF ('6. Performance
and Scaling' page) was completely broken: the 3-column table
('Document size | Avg latency (s) | Throughput (docs/min)' with
5 data rows) was rendered as a vertical list of label/value pairs
instead of as a proper table.

Root cause: a PDF 'block' that contains multiple LINES at the SAME
y but different x positions is a table row (3 cells side-by-side).
The extractor was treating the whole row as one paragraph, joining
all cell texts with newline. When the smart-fit logic wrote the
text back, it used the row's full-width bbox and \insert_textbox\
wrote everything left-aligned, collapsing all columns into one.

Fix: at extraction time, detect horizontal-layout blocks (lines at
the same y, different x within 5pt tolerance) and split them into
one sub-block per line. Each cell gets its own bbox, so the
translator writes each cell at its original x position, preserving
the column structure.

Detection heuristic:
  - Block has >= 2 lines
  - All lines have y0 within 3pt of each other (SAME_ROW_Y_TOLERANCE)
  - At least 2 lines have different x0 (within > 5pt)
If all three hold, it's a table row. Otherwise, keep the old
multi-line-paragraph behavior.

Note: PyMuPDF re-groups cells into row-blocks when reading the
output back (so 'len(blocks)' looks unchanged), but the LINES
within each block are at their correct x positions. Tests check
the line x0 values, not the block count.

Visual proof: page 7 of sample_files/test_corpus/test_pdf_translated.pdf
now shows the table with proper 3-column structure (Taille du document
| Latence moyenne (s) | Débit (docs/min)) instead of an '[translation
overflow]' placeholder.

4 new tests added:
  - test_horizontal_layout_detected: 3 lines at same y -> 3 blocks
  - test_vertical_layout_kept_as_one_block: 3 lines at different y -> 1 block
  - test_single_line_block_unchanged: 1 line -> 1 block
  - test_table_cell_each_at_own_x: e2e table translation, cells at
    correct x positions

Total: 457 tests pass (was 453), zero regression.
2026-07-14 19:25:24 +02:00
4aebb49c7b fix(brand): Memento (not Momento) — fix typo + add link to memento-note.com
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m25s
The product name is 'Memento' (not 'Momento'). The 'Momento' typo was
present in 12 i18n locale files (en, fr, es, de, it, pt, nl, ja, ko,
zh, ru, ar, fa) and in code comments.

Also added a 'memento.url' i18n key with the canonical link
https://memento-note.com/ and wrapped the Memento promo cards in
both DashboardSidebar.tsx and translate/page.tsx with an <a> tag
pointing to that URL. Previously the cards were a non-clickable
<div>, so users had no way to reach the Memento product page.

Note: 'momento' is a legitimate word in Italian, Spanish and
Portuguese meaning 'moment' (e.g. 'a qualsiasi momento'). Those
occurrences in pricing.json and services.json were NOT changed —
they are correct translations of UI strings about 'at any moment'.
2026-07-14 19:19:00 +02:00
4255a1a0c5 feat(format): B3.8 — column-aware next-block layout for multi-column PDFs
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m32s
User asked whether B3.6+B3.7 are generic for ALL PDFs or just for the
test_pdf.pdf. Audit found 2 genericity bugs:

1. _populate_next_block_y was sorting blocks globally by y0. In a
   multi-column PDF (journals, brochures, newspapers), the 'next
   block' of a left-column block would point to the right-column
   block at the same y, which is wrong. Fix: group blocks into
   columns by x0 proximity (15pt tolerance), then sort each column
   by y0. Each block's next_block_y is the y0 of its column-mate
   directly below it, not just the next block in y-order globally.

2. max_expand_y could go negative if next_block_y was above the
   current block (rare edge case in extracted blocks with weird
   bbox ordering). A negative max_expand_y would create an invalid
   fitz.Rect with y1 < y0, causing silent failures. Fix: clamp
   max_expand_y to >= 0.

7 new tests added:
  - test_two_columns_get_separate_next_block_y: 2-col layout,
    left and right columns get independent next_block_y mappings
  - test_centered_full_width_header_gets_own_column: full-width
    header between 2 columns is its own column
  - test_three_columns: 3-column newspaper layout
  - test_single_block_page, test_empty_block_list: edge cases
  - test_max_expand_y_clamped_to_zero: negative-expansion safety
  - test_two_column_pdf_translation_end_to_end: e2e test on a
    2-col journal PDF, 4 input blocks -> 4 output blocks preserved
    at correct positions, no cross-column overlap

Visual verification:
  scripts/verify_b3_8_multicolumn.py renders a 2-col journal PDF
  before and after translation, confirms 4 left + 4 right blocks
  preserved at exact positions.

Total tests: 453 (was 446), zero regression.
2026-07-14 19:05:47 +02:00
3ae28dd3cb feat(format): B3.6+B3.7 — PDF transparent redaction + next-block-aware layout
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m36s
B3.6 — fix two visual bugs reported on the user's prod PDF:
  1. Title 'Spécification technique : Office Translator v3.0' overflowed
     its 2-line bbox and overlapped the 'Version du document...' block.
     Root cause: MAX_VERTICAL_EXPANSION was 1.5x the original height,
     way too small for a long French title. Bumped to 6.0x.
  2. 'Avis important' blue background had white rectangular patches.
     Root cause: redaction always used fill=(1,1,1) (opaque white),
     which erased the colored drawing underneath the text.
     Fix: detect when a block's bbox intersects a page drawing,
     and use fill=None (transparent) for the redaction in that case.
     The original drawing survives intact.

B3.7 — eliminate remaining block-vs-next-block overlap:
  Computes each block's 'next_block_y' (the y0 of the nearest block
  below it on the same page) and uses it as the ceiling for vertical
  expansion. Previously the smart-fit logic used the page bottom as
  the ceiling, which let long translated blocks flow into their
  neighbour (e.g. 'Pour la dernière version...' overlapping
  '8. Résolution des problèmes' in the TOC).

Also includes:
  - 9 new tests (6 B3.6 + 3 B3.7) — total 446 tests pass, zero regression
  - scripts/verify_b3_6_fix.py — visual+structural verification
  - Updated sample_files/test_corpus/test_pdf_translated.pdf with the
    clean B3.6+B3.7 output
2026-07-14 18:56:31 +02:00
e706cef5d6 feat(format): B3.5 — PDF smart-fit rewrite + critical fontname=None fix
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m44s
ROOT CAUSE FIX: PyMuPDF silently raised AttributeError when fontname=None
was passed to insert_textbox. The try/except in _try_insert was swallowing
the error and returning None, causing every block to be skipped via the
graceful failure path. Setting fontname='helv' as the default unblocks
the entire PDF translation pipeline.

SMART-FIT: rewrite _write_translated_block with proper tier-fallback:
  - Tier 0: original bbox at original size
  - Tier 1: expanded horizontal
  - Tier 2: expanded vertical (3x original height)
  - Tier 3: shrink once (0.93x)
  - Tier 4: shrink twice (0.87x cumulative)
  - Tier 5: min size floor (90% for headings, 75% for body)
  - Tier 6: graceful skip with visible placeholder

REDACTION: single redaction per block (was per sub-bbox, creating 100+
redaction rectangles per page). Now only 1 redaction per text block.

FEATURE FLAG: PDF_SMART_FIT_ENABLED (default true, observation-first).

METRICS: text_overflow -> format_elements_lost_total.

RESULT ON REAL PDF:
  Before: fonts shrunk 22pt->5.6pt, hierarchy destroyed
  After:  fonts EXACT match: [8, 11, 12, 14, 16, 22] preserved
2026-07-14 18:36:12 +02:00
12cd0c6893 test: COMPLEX test corpus (5 sections, 80 hyperlinks, 10 footnotes, 3 SmartArt, 8 PDF pages with TOC)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m32s
2026-07-14 18:06:23 +02:00
b706cbf802 test: generate test corpus for B1/B2/B3 (Word+Excel+PPTX+PDF with hyperlinks/footnotes/charts/SmartArt)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m56s
2026-07-14 17:52:06 +02:00
dd1e005c70 feat(cache): C2 — Redis translation cache with user/prompt/glossary namespacing
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m13s
2026-07-14 17:05:24 +02:00
04a9328860 feat(format): B3 — PDF hyperlink preservation + safe redaction + LibreOffice log
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m29s
2026-07-14 17:02:21 +02:00
d40d7f3e86 feat(providers): C1.1 — NewProviderAsLegacyAdapter + LegacyProviderAsNewAdapter
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m22s
2026-07-14 16:58:11 +02:00
c794eff823 feat(quality): A4 — L2 Pro premium judge (8 dims, gpt-4o, Pro-gated, opt-in)
Some checks failed
Deploy to Production / Build and Deploy (push) Has been cancelled
2026-07-14 16:56:04 +02:00
8d0fc818ef feat(metrics): C3 — Prometheus counters for L0/L1/format-loss/retry
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m30s
2026-07-14 16:53:16 +02:00
13d2f83081 feat(format): B2 — PPTX placeholder filter, SmartArt diagrams, chart XPath matching
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m3s
2026-07-14 16:50:06 +02:00
4d466699fd feat(quality): A3 — L1 LLM judge via API (5 chunks, 0.0003 USD/job)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m26s
L1 quality layer — uses a cheap LLM via the OpenAI-compatible API to
validate translation quality. Designed to be the SECOND line of defense
after L0 (script detection, length, pattern).

Architecture:
  - sampler.py — picks 5 representative chunks per job (longest first,
    skips L0-failed indices, skips too-short or identical pairs)
  - llm_judge.py — OpenAI-compatible client, binary verdict per chunk
    (accurate / fluent / correct_language / no_leaks), JSON output,
    hard timeout, defensive (never raises), cost estimation built in
  - pipeline.py — defensive wrapper that integrates both, never breaks
    a translation job, always logs a structured event

Integration:
  - 5 feature flags in config.py (QUALITY_L1_ENABLED, _LOG_ONLY, etc.)
  - QUALITY_L1_LOG_ONLY=true by default: log-only mode, verdict NEVER
    blocks or retries a job
  - Reuses the chunks extracted by L0 (no double work)
  - Passes the set of L0-failed indices so L1 doesn't re-judge them
  - Wrapped in try/except so a misconfigured L1 NEVER breaks a job

Default config: deepseek-chat via DeepSeek API
  - Cost: ~0.0003 USD per job (5 chunks)
  - Speed: typically 1-2s per call, hard ceiling at 8s
  - Easy to swap: just set L1_JUDGE_BASE_URL and L1_JUDGE_MODEL

LLM judge is intentionally a SEPARATE model from the translator
(self-evaluation bias mitigation — Meta/Stanford papers 2024-2025).

Tests:
  test_sampler.py — 9 tests covering the sampling strategy
  test_llm_judge.py — 22 tests covering init, parsing, mocked API,
    cost estimation, env factory
  test_l1_pipeline.py — 6 tests covering the wrapper
  Total new: 37 tests, all pass
  Grand total quality+format: 264 tests passing (0 regression)

  All 36 new tests + 111 L0 tests + 117 existing translator tests = 264

Phase 1 (observation) for 2 weeks. Then QUALITY_L1_LOG_ONLY=false
to enable auto-retry via the fallback chain.
2026-07-14 16:39:47 +02:00
5ae1587428 feat(format): B1 — Word/Excel quick wins for format preservation
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m35s
Word fixes:
  W1 — Fix hyperlink double-collect: a run inside <w:hyperlink> was
       previously collected twice (once via paragraph.runs, once via
       the manual hyperlink iter). Now uses a dedup set of element
       ids to collect each run exactly once.

       NB: python-docx 1.x's paragraph.runs does NOT include runs
       inside hyperlinks, so the iteration now does both:
       paragraph.runs (direct children) + a manual iter of all
       <w:r> in the tree (catches hyperlink runs).

  W2 — Fix footnotes import: used document.part.package.part_related_by
       which doesn't exist in python-docx 1.x, so footnotes were never
       collected. Now uses document.part.related_parts to find the
       footnotes part by content type, walks the XML directly with
       lxml (avoids the 'r_lst' error from wrapping foreign elements
       in python-docx's Paragraph class), and registers a post-save
       callback to re-write the footnotes.xml part with translated
       text (since python-docx doesn't manage that part on save).
       Same fix applied to endnotes.

  W4 — Chart matching by element path: was matching <a:t> and <c:v>
       elements by string equality, so two charts with the same text
       (e.g. two 'Revenue' series) would only have the first one
       translated. Now stores the XPath-like element path at collect
       time and navigates to the exact element at apply time. Falls
       back to string matching for legacy entries without a path.

Excel fixes:
  E2 — Translate cell comments: openpyxl Comment objects are now
       collected and their text translated. The Comment object is
       replaced in place after translation.

  E3 — Translate cell hyperlink display labels: cell.hyperlink.display
       (or .target if no display) is collected and translated. The
       URL itself is never sent for translation, so it remains
       intact. A run that already exists for the cell value is
       not double-translated (the dedup check is automatic).

  E4 — Chart matching by element path: same fix as W4 but for
       Excel. Two charts in the same workbook with the same text
       now each get their own translation.

Tests:
  Added tests/test_translators/test_b1_format_fixes.py with 11 tests
  covering all the fixes. All 11 pass. Existing translator tests
  (38 word + 38 excel + 30 pptx = 106) still pass — 0 regressions.

  Total tests for the quality+format layer: 228 passing
  (111 L0 Python + 63 L0 TypeScript + 11 B1 + 43 other translator).

All fixes are surgical: existing translation flow is preserved.
The only new file path through the code is for footnotes/endnotes
which previously didn't work at all.
2026-07-14 16:28:17 +02:00
f403b2851d feat(quality): add L0 quality layer (Track A1 + A2 of dev plan)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m5s
L0 quality detection layer to catch translation failures BEFORE they
reach users. Pure Python/TypeScript, zero new dependencies, no API calls.

Backend (Python — services/quality/):
  - Script detection: 145 langs mapped to 23 scripts (Latin, Cyrillic,
    Greek, Arabic, Hebrew, CJK, Hangul, Kana, Devanagari, Bengali, etc.)
  - Language confusion detection (e.g. Arabic text for French target)
  - Arabic-script variant discrimination (Persian/Urdu/Pashto/Kurdish
    confusion — e.g. Persian text returned when Arabic was requested)
  - Length sanity check (with numeric/short-source exemptions)
  - Prompt leak detection (Translation: / Voici la traduction: / 翻译:)
  - Repetition hallucination detection (token + character level)
  - File text extraction for .docx/.xlsx/.pptx/.pdf (no translator
    changes needed)
  - Defensive pipeline that never raises (L0 must NEVER break a job)

Frontend (TypeScript — wordly.art---traduction-de-documents/src/utils/):
  - Exact 1:1 mirror of the Python module
  - Zero dependencies, works in browser AND Node.js
  - Native Unicode regex (\\p{L}/u) and codePoint iteration
  - 63 tests using Node's built-in test runner

Integration:
  - Feature-flagged: QUALITY_L0_ENABLED=false (default)
  - Observation only: logs structured events, never modifies files
  - try/except wrapped: impossible to break a translation job
  - Lazy imports: only loaded when flag is on
  - Zero impact on existing tests / behavior

Tests:
  - 111 Python tests covering all paths (config, script, length, leak,
    pipeline, file_extractor) — 100% pass
  - 63 TypeScript tests (Node --test) — 100% pass
  - 174/174 total tests for the L0 layer

Bug fixes in script mapping:
  - yi (Yiddish) -> hebrew (was incorrectly mapped to arabic)
  - dv (Maldivian) -> thaana (was incorrectly mapped to arabic)
  - ja (Japanese) -> hiragana_katakana (distinguishes from Chinese CJK)

Phase 1 (backend) + Phase 2 (frontend) of Track A complete.
Next: Track B1 (Word/Excel format preservation quick wins).

Closes Track A phase 1+2 of the dev plan.
2026-07-14 16:17:43 +02:00
ebb2537fda feat(glossaries): implement a 3-step wizard for CSV/file imports with custom source/target language selection
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m21s
2026-06-28 11:38:18 +02:00
36aeac2c5e fix(glossaries): prevent translation data destruction on language selector change for multi glossaries
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m53s
2026-06-28 11:03:31 +02:00
de9407f974 revert(docker): remove automated database glossary translation from startup sequence
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m0s
2026-06-28 10:55:19 +02:00
5e3fb0098b feat(docker): run glossary translation script automatically on backend container startup
Some checks failed
Deploy to Production / Build and Deploy (push) Has been cancelled
2026-06-28 10:53:58 +02:00
a57b8a8e4d fix(glossaries): normalize language code casing when retrieving display translations 2026-06-28 10:52:08 +02:00
dde80f6bc3 feat(glossaries): update script to translate missing terms in any glossary, even if target_language is already 'multi'
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m43s
2026-06-28 10:46:48 +02:00
7398cae359 feat(glossaries): add script to translate non-multilingual database glossaries using Google Translate adapter
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m25s
2026-06-28 10:42:41 +02:00
030950c962 feat(glossaries): option C - page management + wizard /new 2-step creation
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m6s
2026-06-28 09:55:14 +02:00
5d6afd2dad fix(contrast): replace all opacity-based colors with concrete hex values meeting WCAG 4.5:1 ratio
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m44s
2026-06-21 10:54:17 +02:00
07c50151cb fix(contrast): increase text contrast on glossary page for light mode readability
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m24s
2026-06-20 19:39:33 +02:00
299173fff7 ux(glossary): clarify selection flow - compact banner, Utiliser button, hide create section by default
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m27s
2026-06-20 19:18:46 +02:00
9b354befe3 fix(glossary): hide create section by default when glossaries exist - only shown on demand via toggle button
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m11s
2026-06-20 19:07:30 +02:00
4a8f33d36f fix(glossary): eliminate all redundant elements - filter imported templates, remove duplicate buttons and links
Some checks failed
Deploy to Production / Build and Deploy (push) Has been cancelled
2026-06-20 19:03:44 +02:00
489df66c0f fix(i18n): add missing translation keys for tabs and presets 2026-06-20 19:01:09 +02:00
96bac2e792 style(glossary): remove redundant buttons and links, filter already imported presets
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m26s
2026-06-20 18:46:38 +02:00
d78f08e24f feat(glossary): restructure page with tabs, direct translate redirects and dynamic mapping
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m39s
2026-06-20 18:43:02 +02:00
c17dd2c6e1 fix(glossary): resolve data loss for non-FR/EN languages, fix prompt injection reference notes, and classic mode label wording
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m25s
2026-06-20 18:09:13 +02:00
1fe714aa1a ux(glossaries): simplify dialog, auto-save detail import, show templates and upload zone directly on main page
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m6s
2026-06-20 12:27:07 +02:00
d505b479cd refactor(glossaries): align CreateGlossaryDialog with editorial design system
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m56s
2026-06-20 09:36:39 +02:00
81cb4e09b7 fix(tests): update PDF format test to use truly unsupported format (.txt)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m8s
2026-06-14 19:50:44 +02:00
a20ebe2295 fix(tests): update glossary service tests for dict return type
Some checks failed
Deploy to Production / Build and Deploy (push) Has been cancelled
2026-06-14 19:49:21 +02:00