Commit Graph

16 Commits

Author SHA1 Message Date
5f3d57b4f7 fix(direction): le sens du document traduit suit la langue cible, dans les deux sens
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m3s
Persan -> francais : le document traduit restait en lecture de droite a
gauche, heritee de la source. Trois causes :

- PDF : regression du renommage precedent — le choix d'alignement
  testait la fonction is_rtl (toujours vraie) au lieu du parametre :
  tout texte traduit s'alignait a droite, quelle que soit la cible.
  Corrige + test epinglant l'alignement gauche pour une cible latine.
- Word : les marques RTL heritees (bidi, rtl, bidiVisual, notes et
  commentaires compris) sont desormais retirees quand la cible est
  latine ; retrait fait sur une liste figee (l'iterateur lxml sautait
  des elements pendant la suppression).
- Excel : les feuilles heritees d'un affichage droite-a-gauche sont
  remises en lecture gauche-a-droite pour une cible latine.
- PowerPoint : les attributs rtl herites sont retires pour une cible
  latine (alignements visuels conserves).

3 tests de bout en bout nouveaux : document RTL traduit vers le
francais ressort en lecture gauche-a-droite dans les trois formats.
2026-09-01 21:28:46 +02:00
d92bbf0fa6 feat(qualite): retenter automatiquement les traductions livrees dans la mauvaise ecriture
Le defaut le plus sournois du pipeline etait silencieux : de l'arabe
livre pour une cible persane (meme ecriture, mauvaise langue) passe
inapercu et part chez le lecteur. Desormais chaque lot traduit est
verifie par le detecteur d'ecritures, et chaque segment fautif est
redemande une fois au moteur avec une consigne renforcee (nom de la
langue + lettres specifiques, ex. persan پ چ ژ گ). La seconde tentative
ne remplace la premiere que si elle passe le meme controle.

- services/quality/script_detector.py : extraction d'un controle
  script_issue() reutilisable ; detect_arabic_variant signale des
 ormais un long texte en ecriture arabe sans aucune lettre specifique
  de la langue cible (arabe pur livre pour du persan/ourdou/pachto)
- translators/segments.py : retry_wrong_script() + construction de la
  consigne renforcee, sans jamais faire echouer le travail
- Word, Excel, PDF : branchement apres la memoire de traduction et les
  validations humaines ; le texte inchange (chiffres, noms propres) ne
  declenche jamais de retentative
- 14 tests nouveaux (tests/test_translators/test_script_retry.py)
2026-09-01 20:52:23 +02:00
f22f645fab feat(rtl): rendu droite-a-gauche complet pour Word, PowerPoint, Excel et PDF
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m37s
- source unique RTL_LANGUAGES/is_rtl dans core/languages.py (fin des 3 copies)
- Word: bidi partout (corps, tableaux bidiVisual, notes/fin/commentaires,
  zones de texte, 6 zones d'en-tetes/pieds), insertion OOXML ordonnee,
  polices cs elargies aux 11 langues
- PowerPoint: alignements explicites preserves, notes du presentateur,
  indice de police <a:cs> insert a sa place
- Excel: feuilles affichees de droite a gauche, feuilles graphiques ignorees
- PDF: faconnage bidi (arabic-reshaper + python-bidi), polices par ecriture
  (arabe/hebreu), TTF enregistree pour le PDF recompose
- 46 tests nouveaux (tests/test_translators/test_rtl_layout.py), 253 au total
2026-09-01 20:22:46 +02:00
b4e873ad2c feat(review,teams): review foundation — segments, side-by-side editor, rebuild, XLIFF, team workspaces
Some checks failed
Deploy to Production / Build and Deploy (push) Failing after 2m14s
Foundations:
- TranslationSegment model + migration f7e8d9c0b1a2 (segments, workspaces,
  workspace_members, glossaries.workspace_id)
- SegmentRecorder injected into all 4 translators: unique (source,
  translation) pairs captured per job and persisted (best-effort)
- set_segment_overrides: human-reviewed translations applied verbatim on
  rebuild — top priority over TM and provider, zero API calls

Review API (routes/review_routes.py):
- GET /translations/{id}/segments (owner or job token)
- PATCH /segments/{id} edit/approve — feeds the per-user TM so approved
  translations are reused in later jobs
- POST /translations/{id}/rebuild — rebuild document with reviewed text
- GET/POST /translations/{id}/xliff — XLIFF 1.2 export/import (edited
  segments export their reviewed text)

Review editor (frontend /dashboard/reviews/[jobId]):
- side-by-side source/translation table, inline edit, approve (single or
  all), rebuild & download (auth blob), XLIFF export/import, 13 locales
- 'Relire et corriger' link on the translation-complete screen

Team workspaces (routes/workspace_routes.py + /dashboard/teams):
- Workspace/WorkspaceMember models, roles owner/admin/member
- create (Business plan), list with seat usage, invite by email with
  seat-limit enforcement (Business=5, Enterprise unlimited), removal
- shared glossaries: workspace members can use a glossary shared to their
  workspace (access check extended)

Tests: 1184 passed / 0 failed (11 new: recorder, overrides, docx
capture->rebuild e2e, XLIFF structure/escaping, seats, workspace CRUD,
shared glossary access)
2026-08-29 19:04:32 +02:00
526c87348f feat(translation): quality pipeline overhaul + new features (audit 2026-08-29)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m20s
Translation quality & format preservation:
- Word: merge adjacent same-format runs into one unit (sentence-level
  coherence like inline-tag handling); translate comments/balloons;
  dedupe textbox collection (was translated twice); RTL no longer
  overrides center/justify alignment; CJK/Arabic font hints (eastAsia/cs)
- PPTX: chart translations now actually reach the output file
  (ChartPart.blob is read-only — rewrite chart XML in the saved ZIP);
  CJK typeface hints (a:ea)
- Excel: sheet renames no longer break references — rewrite cell
  formulas (3D/quoted), defined names, data validations, cond. formats
- PDF: bold/italic honored (hebo/heit/hebi); table cells never merge;
  unchanged blocks left untouched (typography preserved, fixes duplicate
  hyperlinks); attempted/changed stats + route gate now cover PDF;
  CJK font paths; scanned PDFs via Mistral OCR (detection + admin settings)

Features:
- formality param (formal/informal) + automatic regional-variant prompts
- output_mode=bilingual docx (source above translation)
- per-user translation memory on Redis (falls back to LRU), context-hashed
- QA report + 0-100 confidence score in job status; L0 on by default
- OpenAI-compatible providers: whole chunk in ONE numbered-JSON request
  (~15x fewer calls) with per-item fallback; base prompt always present
  (custom prompt no longer replaces translation instructions)

Infra & marketing alignment:
- plan-based engine gating + vision gating (closes paid-engine leak);
  /providers/available filtered per plan; 107 languages exposed
- zh-CN/zh-TW validation fixed; libmagic disabled on Windows (native crash)
- admin: Mistral OCR settings + engine status dashboard; httpx<0.28 pin
  (TestClient breakage); Prometheus test fixture fixed
- marketing docs aligned with code (PDF+OCR, retention, engines, pricing)
- security: .env.ionos/.env.production/provider_settings.json removed

Tests: 1173 passed / 0 failed (6 network tests deselected: free Google
endpoint temporarily blocked from this machine)
2026-08-29 18:38:09 +02:00
992f13d53c fix(translate): apply Excel chart text setters (were silently dropped)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m35s
Chart <a:t> elements (title, axis labels, series names) were collected and
sent to the LLM, but the apply loop never invoked their setters. Reason:
the apply loop iterated only text_elements[:sheet_name_offset], which
excluded the chart-text positions (sheet names were inserted in the
middle, pushing chart texts past sheet_name_offset). The LLM correctly
returned French for "Revenue by Product" / "Quantity Trend" / "Order #"
but the result was thrown away.

Fix: include the chart-text slice [sheet_name_offset + N .. total_texts]
in the apply loop. Setters for sheet names are still None, so they are
naturally skipped.

Add test_excel_chart_text_applied.py (2 end-to-end tests) using a fixed
provider that pre-translates every known chart text; the test asserts
all chart <a:t> values in the output .xlsx come from the FR table, not
the original English source.
2026-07-15 22:21:53 +02:00
07c4c12e6a fix(translate): chart labels, sheet-refs, sheet-name offset, paid-user Memento
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m19s
Multiple translation bugs in Word/Excel/PPTX that caused chart elements
to be left untranslated or charts to render as empty series.

Backend
-------
* providers (deepseek/openai/minimax): tighten system prompt so the LLM
  actually translates chart titles/axis labels/legend/category labels,
  month abbreviations, and clarifies what counts as a 'real' proper noun
  (people/place/company/product names) vs. technical labels. Old rule
  'keep proper nouns unchanged' was being read too broadly by the model
  and caused chart text to be skipped.
* excel_translator.py:
  - Sheet reference rewrite: when a sheet is renamed, the chart XML's
    c:f refs (e.g. 'Sales 2024'!$D$2:$D$61) are now rewritten to the
    new name with proper apostrophe escaping (Chiffre d'affaires ->
    'Chiffre d''affaires') and auto-quoting when the new name contains
    spaces or special chars. Without this, the chart points at a sheet
    that no longer exists and renders 0/empty series.
  - Sheet name offset bug: sheet_name_offset was computed after chart
    text was appended to text_elements, causing sheet names to receive
    chart text translations. Now captured BEFORE sheet names are added.
* New tests:
  - test_excel_chart_sheet_refs.py (10 unit tests, synthetic inputs)
  - test_chart_translation_prompt.py (3 contract tests on the prompt)

Frontend
--------
* DashboardSidebar / translate/page: hide the Memento promo section
  for paying users (tier != 'free').
* constants.ts: temporarily comment out the 'CLES API' nav item.
  Update constants.test.ts to match the new state.

All fixes are generic - no file-specific hardcoding, edge cases covered
(empty mapping, missing bang, apostrophe escaping, partial renames,
multi-series).
2026-07-15 22:04:40 +02:00
5ae1587428 feat(format): B1 — Word/Excel quick wins for format preservation
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m35s
Word fixes:
  W1 — Fix hyperlink double-collect: a run inside <w:hyperlink> was
       previously collected twice (once via paragraph.runs, once via
       the manual hyperlink iter). Now uses a dedup set of element
       ids to collect each run exactly once.

       NB: python-docx 1.x's paragraph.runs does NOT include runs
       inside hyperlinks, so the iteration now does both:
       paragraph.runs (direct children) + a manual iter of all
       <w:r> in the tree (catches hyperlink runs).

  W2 — Fix footnotes import: used document.part.package.part_related_by
       which doesn't exist in python-docx 1.x, so footnotes were never
       collected. Now uses document.part.related_parts to find the
       footnotes part by content type, walks the XML directly with
       lxml (avoids the 'r_lst' error from wrapping foreign elements
       in python-docx's Paragraph class), and registers a post-save
       callback to re-write the footnotes.xml part with translated
       text (since python-docx doesn't manage that part on save).
       Same fix applied to endnotes.

  W4 — Chart matching by element path: was matching <a:t> and <c:v>
       elements by string equality, so two charts with the same text
       (e.g. two 'Revenue' series) would only have the first one
       translated. Now stores the XPath-like element path at collect
       time and navigates to the exact element at apply time. Falls
       back to string matching for legacy entries without a path.

Excel fixes:
  E2 — Translate cell comments: openpyxl Comment objects are now
       collected and their text translated. The Comment object is
       replaced in place after translation.

  E3 — Translate cell hyperlink display labels: cell.hyperlink.display
       (or .target if no display) is collected and translated. The
       URL itself is never sent for translation, so it remains
       intact. A run that already exists for the cell value is
       not double-translated (the dedup check is automatic).

  E4 — Chart matching by element path: same fix as W4 but for
       Excel. Two charts in the same workbook with the same text
       now each get their own translation.

Tests:
  Added tests/test_translators/test_b1_format_fixes.py with 11 tests
  covering all the fixes. All 11 pass. Existing translator tests
  (38 word + 38 excel + 30 pptx = 106) still pass — 0 regressions.

  Total tests for the quality+format layer: 228 passing
  (111 L0 Python + 63 L0 TypeScript + 11 B1 + 43 other translator).

All fixes are surgical: existing translation flow is preserved.
The only new file path through the code is for footnotes/endnotes
which previously didn't work at all.
2026-07-14 16:28:17 +02:00
c1ea65f10f feat(translate): refonte du design de la page de traduction et du sélecteur de moteurs (Etapes 1-3)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m12s
2026-05-31 10:14:23 +02:00
c0f93501cc fix: use Google Cloud API key for classic mode + translation verification
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2s
Two critical fixes:

1. Provider "google" (default classic mode) now checks for a Google Cloud
   API key (GOOGLE_CLOUD_API_KEY in env or admin settings). If present,
   uses GoogleCloudTranslationProvider (official API). Previously it
   always fell through to deep_translator (free scraper) which gets
   blocked in production, silently returning untranslated text.

2. Added translation verification: each translator now tracks how many
   texts were attempted vs actually changed. If 0 texts were translated,
   the job is marked as FAILED with a clear error message instead of
   returning the original file as "completed".

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 12:09:26 +02:00
ce8e150a61 feat: homelab deployment - NPM + IONOS DNS + monitoring + NAS backup
- Restructured docker-compose for Nginx Proxy Manager (no custom nginx)
- Added domain wordly.art configuration
- Added Prometheus + Grafana monitoring stack with pre-configured dashboards
- Added PostgreSQL backup script to NAS (daily/weekly/monthly rotation)
- Added alert rules for backend, system, and Docker metrics
- Updated deployment guide for NPM + IONOS DNS homelab setup
- Added marketing plan document
- PDF translator and watermark support
- Enhanced middleware, routes, and translator modules

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-10 11:43:28 +02:00
Sepehr Ramezani
26bd096a06 feat: production deployment - full update with providers, admin, glossaries, pricing, tests
Major changes across backend, frontend, infrastructure:
- Provider system with model selection (Google, DeepL, OpenAI, Ollama, Google Cloud)
- Admin panel: user management, pricing, settings
- Glossary system with CSV import/export
- Subscription and tier quota management
- Security hardening (rate limiting, API key auth, path traversal fixes)
- Docker compose for dev, prod, and IONOS deployment
- Alembic migrations for new tables
- Frontend: dashboard, pricing page, landing page, i18n (en/fr)
- Test suite and verification scripts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-25 15:01:47 +02:00
Sepehr Ramezani
473b3e26c7 feat: revue de code, doc CODE_REVIEW, forfaits 2026, traduction LLM, providers avec modèle
Made-with: Cursor
2026-03-07 11:42:58 +01:00
8f9ca669cf Performance optimization: batch translation for 5-10x speed improvement
- GoogleTranslationProvider: Added batch translation with separator method
- DeepLTranslationProvider: Added translator caching and batch support
- LibreTranslationProvider: Added translator caching and batch support
- WordTranslator: Collect all texts -> batch translate -> apply pattern
- ExcelTranslator: Collect all texts -> batch translate -> apply pattern
- PowerPointTranslator: Collect all texts -> batch translate -> apply pattern
- Enhanced Ollama/OpenAI prompts with stricter translation-only rules
- Added rule: return original text if uncertain about translation
2025-11-30 20:41:20 +01:00
e48ea07e44 Add system prompt, glossary, presets for Ollama/WebLLM, image translation support 2025-11-30 16:45:41 +01:00
793d94c93e Initial commit: Document Translation API with Excel, Word, PowerPoint support 2025-11-30 10:48:58 +01:00