Les deux services traduisent desormais les documents par lots
numeros dans une seule requete, comme le provider OpenAI : vitesse
multipliee et moins de limites de debit. Repli automatique texte par
texte si la reponse du service est douteuse ou en erreur : aucune
traduction perdue, le document n'echoue jamais a cause d'un lot.
Tests: nouveau test_deepseek_provider.py + extension minimax.
Le defaut le plus sournois du pipeline etait silencieux : de l'arabe
livre pour une cible persane (meme ecriture, mauvaise langue) passe
inapercu et part chez le lecteur. Desormais chaque lot traduit est
verifie par le detecteur d'ecritures, et chaque segment fautif est
redemande une fois au moteur avec une consigne renforcee (nom de la
langue + lettres specifiques, ex. persan پ چ ژ گ). La seconde tentative
ne remplace la premiere que si elle passe le meme controle.
- services/quality/script_detector.py : extraction d'un controle
script_issue() reutilisable ; detect_arabic_variant signale des
ormais un long texte en ecriture arabe sans aucune lettre specifique
de la langue cible (arabe pur livre pour du persan/ourdou/pachto)
- translators/segments.py : retry_wrong_script() + construction de la
consigne renforcee, sans jamais faire echouer le travail
- Word, Excel, PDF : branchement apres la memoire de traduction et les
validations humaines ; le texte inchange (chiffres, noms propres) ne
declenche jamais de retentative
- 14 tests nouveaux (tests/test_translators/test_script_retry.py)
Pricing: full editorial redesign — serif card headers with accent pills
replace the colored font-black blocks, tone sweep across toggle/metrics/
features/CTAs, PLAN_COLORS removed; one design system app-wide.
Translate: decorative titles one step down (CTA hierarchy restored);
glossary and image-translation blocks hidden entirely for free users
(progressive disclosure — three controls for free).
Reviews: XLIFF hint line explains the exchange format; backend errors
routed through a friendly mapper (session/not-found/rate-limit/server).
Landing: fabricated hero UI cards (fake 'Context Engine' overlay)
removed — the photo no longer promises screens that don't exist.
Nav: single DashboardNavLinks component shared by sidebar and mobile
drawer (was duplicated markup).
API: GET /api/v1/translations (user job history, paginated; completed
jobs retained 24h) and POST /api/v1/translations/{id}/cancel —
cooperative cancellation with worker checkpoints before dispatch and
before finalisation, reserved quota released immediately. Translate
monitor now offers a real 'Cancel translation' next to 'Back to start';
recent-jobs list reads server history first, localStorage fallback.
DeepL purge (backend): provider module, registry registration, config
attrs/defaults, dispatch branch, admin settings schema + test branch,
legacy availability block, validation rules, plan provider lists,
error-code mappings, MCP enums, translator prompt mention, related
tests updated/removed. Fallback resolver skips unknown providers, so
stale chains containing 'deepl' degrade gracefully.
Verified: backend 110 tests passed; frontend build exit 0, vitest 9/9,
0 missing i18n keys, eslint 63 errors (vs 64 at HEAD).
Foundations:
- TranslationSegment model + migration f7e8d9c0b1a2 (segments, workspaces,
workspace_members, glossaries.workspace_id)
- SegmentRecorder injected into all 4 translators: unique (source,
translation) pairs captured per job and persisted (best-effort)
- set_segment_overrides: human-reviewed translations applied verbatim on
rebuild — top priority over TM and provider, zero API calls
Review API (routes/review_routes.py):
- GET /translations/{id}/segments (owner or job token)
- PATCH /segments/{id} edit/approve — feeds the per-user TM so approved
translations are reused in later jobs
- POST /translations/{id}/rebuild — rebuild document with reviewed text
- GET/POST /translations/{id}/xliff — XLIFF 1.2 export/import (edited
segments export their reviewed text)
Review editor (frontend /dashboard/reviews/[jobId]):
- side-by-side source/translation table, inline edit, approve (single or
all), rebuild & download (auth blob), XLIFF export/import, 13 locales
- 'Relire et corriger' link on the translation-complete screen
Team workspaces (routes/workspace_routes.py + /dashboard/teams):
- Workspace/WorkspaceMember models, roles owner/admin/member
- create (Business plan), list with seat usage, invite by email with
seat-limit enforcement (Business=5, Enterprise unlimited), removal
- shared glossaries: workspace members can use a glossary shared to their
workspace (access check extended)
Tests: 1184 passed / 0 failed (11 new: recorder, overrides, docx
capture->rebuild e2e, XLIFF structure/escaping, seats, workspace CRUD,
shared glossary access)
Multiple translation bugs in Word/Excel/PPTX that caused chart elements
to be left untranslated or charts to render as empty series.
Backend
-------
* providers (deepseek/openai/minimax): tighten system prompt so the LLM
actually translates chart titles/axis labels/legend/category labels,
month abbreviations, and clarifies what counts as a 'real' proper noun
(people/place/company/product names) vs. technical labels. Old rule
'keep proper nouns unchanged' was being read too broadly by the model
and caused chart text to be skipped.
* excel_translator.py:
- Sheet reference rewrite: when a sheet is renamed, the chart XML's
c:f refs (e.g. 'Sales 2024'!$D$2:$D$61) are now rewritten to the
new name with proper apostrophe escaping (Chiffre d'affaires ->
'Chiffre d''affaires') and auto-quoting when the new name contains
spaces or special chars. Without this, the chart points at a sheet
that no longer exists and renders 0/empty series.
- Sheet name offset bug: sheet_name_offset was computed after chart
text was appended to text_elements, causing sheet names to receive
chart text translations. Now captured BEFORE sheet names are added.
* New tests:
- test_excel_chart_sheet_refs.py (10 unit tests, synthetic inputs)
- test_chart_translation_prompt.py (3 contract tests on the prompt)
Frontend
--------
* DashboardSidebar / translate/page: hide the Memento promo section
for paying users (tier != 'free').
* constants.ts: temporarily comment out the 'CLES API' nav item.
Update constants.test.ts to match the new state.
All fixes are generic - no file-specific hardcoding, edge cases covered
(empty mapping, missing bang, apostrophe escaping, partial renames,
multi-series).
User said 'I want this to be automatic'. Three gaps were addressed.
Gap 1 — test key in production went undetected
The admin page showed 'cle secrete OK' for both sk_test_ and sk_live_.
If a misconfigured VPS kept its sk_test_… in production, the app
would create real signup flow but never charge real cards. Added
services.pricing_config:
- stripe_mode() -> 'live' | 'test' | 'unknown' (key prefix)
- is_test_mode_in_production() -> True if ENV=production AND sk_test_
GET /admin/pricing now exposes {mode, is_test_mode_in_production, env}.
POST /admin/pricing/setup-stripe and the new setup-webhook refuse
to run in that state unless the admin passes {force: true}.
Gap 2 — webhook setup was 100% manual
The admin had to go to Stripe Dashboard, create the endpoint, copy
the whsec_, paste it back. Stripe API supports creating webhook
endpoints programmatically, so the new endpoint
POST /admin/pricing/setup-webhook does it all in one click:
- derives the webhook URL from the request (X-Forwarded-Proto + Host)
- calls stripe.WebhookEndpoint.create() (or .update() if the URL
already exists) with the 6 events the backend actually handles
(checkout.session.completed, customer.subscription.*, invoice.*)
- persists the returned whsec_ to .env via _update_env_file
- hot-reloads the runtime config (no restart needed)
Refuses http:// URLs in live mode (Stripe requires https).
Gap 3 — obsolete script leaked a test secret
scripts/stripe_setup.py contained a hardcoded sk_test_… in source.
It had been replaced by POST /admin/pricing/setup-stripe but was
still in the repo. Deleted via git rm. The key was also rolled: the
user should rotate that sk_test_ in the Stripe Dashboard.
Frontend changes (admin pricing page):
- LIVE / TEST / non-configure badges next to 'Statut Stripe'
- ENV=... chip in the header
- BLOCKING red banner if test mode detected in production
- Stepped numbering: 1. Produits & prix / 2. Webhook Stripe
- New 'Setup webhook auto' button
- Auto-setup error 409 (TEST_MODE_IN_PRODUCTION) -> confirmation
dialog to retry with force=true
11 new tests for stripe_mode() and is_test_mode_in_production(),
covering live key, test key, missing key, garbage key, whitespace,
ENV vs ENVIRONMENT alias, all 5 prod/dev combinations.
Total: 471 tests pass (was 460), zero regression.
L1 quality layer — uses a cheap LLM via the OpenAI-compatible API to
validate translation quality. Designed to be the SECOND line of defense
after L0 (script detection, length, pattern).
Architecture:
- sampler.py — picks 5 representative chunks per job (longest first,
skips L0-failed indices, skips too-short or identical pairs)
- llm_judge.py — OpenAI-compatible client, binary verdict per chunk
(accurate / fluent / correct_language / no_leaks), JSON output,
hard timeout, defensive (never raises), cost estimation built in
- pipeline.py — defensive wrapper that integrates both, never breaks
a translation job, always logs a structured event
Integration:
- 5 feature flags in config.py (QUALITY_L1_ENABLED, _LOG_ONLY, etc.)
- QUALITY_L1_LOG_ONLY=true by default: log-only mode, verdict NEVER
blocks or retries a job
- Reuses the chunks extracted by L0 (no double work)
- Passes the set of L0-failed indices so L1 doesn't re-judge them
- Wrapped in try/except so a misconfigured L1 NEVER breaks a job
Default config: deepseek-chat via DeepSeek API
- Cost: ~0.0003 USD per job (5 chunks)
- Speed: typically 1-2s per call, hard ceiling at 8s
- Easy to swap: just set L1_JUDGE_BASE_URL and L1_JUDGE_MODEL
LLM judge is intentionally a SEPARATE model from the translator
(self-evaluation bias mitigation — Meta/Stanford papers 2024-2025).
Tests:
test_sampler.py — 9 tests covering the sampling strategy
test_llm_judge.py — 22 tests covering init, parsing, mocked API,
cost estimation, env factory
test_l1_pipeline.py — 6 tests covering the wrapper
Total new: 37 tests, all pass
Grand total quality+format: 264 tests passing (0 regression)
All 36 new tests + 111 L0 tests + 117 existing translator tests = 264
Phase 1 (observation) for 2 weeks. Then QUALITY_L1_LOG_ONLY=false
to enable auto-retry via the fallback chain.
L0 quality detection layer to catch translation failures BEFORE they
reach users. Pure Python/TypeScript, zero new dependencies, no API calls.
Backend (Python — services/quality/):
- Script detection: 145 langs mapped to 23 scripts (Latin, Cyrillic,
Greek, Arabic, Hebrew, CJK, Hangul, Kana, Devanagari, Bengali, etc.)
- Language confusion detection (e.g. Arabic text for French target)
- Arabic-script variant discrimination (Persian/Urdu/Pashto/Kurdish
confusion — e.g. Persian text returned when Arabic was requested)
- Length sanity check (with numeric/short-source exemptions)
- Prompt leak detection (Translation: / Voici la traduction: / 翻译:)
- Repetition hallucination detection (token + character level)
- File text extraction for .docx/.xlsx/.pptx/.pdf (no translator
changes needed)
- Defensive pipeline that never raises (L0 must NEVER break a job)
Frontend (TypeScript — wordly.art---traduction-de-documents/src/utils/):
- Exact 1:1 mirror of the Python module
- Zero dependencies, works in browser AND Node.js
- Native Unicode regex (\\p{L}/u) and codePoint iteration
- 63 tests using Node's built-in test runner
Integration:
- Feature-flagged: QUALITY_L0_ENABLED=false (default)
- Observation only: logs structured events, never modifies files
- try/except wrapped: impossible to break a translation job
- Lazy imports: only loaded when flag is on
- Zero impact on existing tests / behavior
Tests:
- 111 Python tests covering all paths (config, script, length, leak,
pipeline, file_extractor) — 100% pass
- 63 TypeScript tests (Node --test) — 100% pass
- 174/174 total tests for the L0 layer
Bug fixes in script mapping:
- yi (Yiddish) -> hebrew (was incorrectly mapped to arabic)
- dv (Maldivian) -> thaana (was incorrectly mapped to arabic)
- ja (Japanese) -> hiragana_katakana (distinguishes from Chinese CJK)
Phase 1 (backend) + Phase 2 (frontend) of Track A complete.
Next: Track B1 (Word/Excel format preservation quick wins).
Closes Track A phase 1+2 of the dev plan.
Frontend:
- Fix Framer Motion / motion-dom build error by pinning framer-motion to
11.18.2 (compatible with React 19 and Next.js 16).
- Add cross-env and build:local script to bypass standalone symlink errors
on Windows without Developer Mode.
- Allow NEXT_OUTPUT=default to disable standalone output for local builds.
- Refactor i18n: split 14,177-line src/lib/i18n.tsx into per-locale,
per-namespace JSON files under src/lib/i18n/messages/.
- Load English synchronously; other locales loaded on demand via dynamic
imports (reduces initial bundle, improves maintainability).
- Remove unused next-intl message files src/messages/en.json and fr.json.
Backend:
- Remove insecure legacy /api/v1/download/{filename} and /api/v1/cleanup/{filename}
endpoints. The job-based /api/v1/download/{job_id} already enforces ownership.
- Deduplicate texts in TranslationService.translate_batch before sending them
to the provider, reducing API calls for repeated strings.
- Pin httpx to <0.28 to fix TestClient incompatibility with starlette 0.35.1.
- Add pytest-cov and ruff dev dependencies/config.
DevOps:
- Remove hardcoded Grafana password from docker-compose.yml and
docker-compose.monitoring.yml; use GRAFANA_PASSWORD env var.
- Change default TRANSLATION_SERVICE from ollama to google in
docker-compose.yml (Ollama is an optional profile).
- Add GRAFANA_PASSWORD to .env.example.
- Add .coverage and frontend/pnpm-workspace.yaml to .gitignore.
Tests:
- Update API versioning tests for removed legacy endpoints.
- Add tests/test_translation_service.py for deduplication behavior.
Verified:
- pnpm run build:local passes.
- uv run pytest tests/test_providers/* tests/test_translation_service.py
tests/test_story_3_5_api_versioning.py tests/test_download_endpoint.py
tests/test_translators/test_excel_translator.py: provider/translator tests
pass; one pre-existing French error-message test still fails (message is
returned in English, unrelated to this change).
Two issues found in screenshot review:
1. Glossary preview always showed English 'target' field (foie → liver)
instead of the language-specific translation. Now looks up
term.translations[targetLang] first, falls back to term.target.
When target is Persian, preview shows 'foie → کبد' (Persian).
2. Previous migration b7c8d9e0f1a2 was already applied on server before
the rename was added. New migration c8d9e0f1a2b3 handles the rename
separately: glossaries with target_language='multi' get their name
changed from 'Anglais' to 'Multilingue'.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Full audit found 18 issues across backend, frontend, and data files.
Root cause: target_language had no single source of truth — 'en' was
hardcoded as default in 6+ places while templates are actually multilingual.
Fixes applied:
- routes/glossary_routes.py: list_glossaries() fallback 'en' → 'multi'
- routes/glossary_routes.py: import reads target_lang from index.json
(source of truth) instead of template file
- data/glossaries/*.json: all 8 template files target_lang 'en' → 'multi'
- services/glossary_service.py: get_glossary_terms() now returns
target_language field (was missing entirely)
- schemas/glossary_schemas.py: all defaults 'en' → 'multi'
(GlossaryCreate, GlossaryResponse, GlossaryListItem)
- useTranslationConfig.ts: only reset glossary on sourceLang change,
not targetLang (multilingual glossaries work with any target)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- Create Stripe products/prices (Starter/Pro/Business monthly+yearly)
- Fix CRITICAL bug: add subscription_ends_at + cancel_at_period_end columns to users table
- Alembic migration: f6a7b8c9d0e1_add_subscription_ends_at_cancel_at_period_end
- Fix: implement handle_payment_failed() to set subscription_status=PAST_DUE
- Fix: harmonize .env.production Stripe variable names to match pricing_config.py
- Fix: add missing FRONTEND_URL and STRIPE_PUBLISHABLE_KEY to .env.production
- Add all Stripe Price IDs (test mode) to .env.production
- Wire credit purchase buttons to /api/v1/auth/create-credits-checkout
- Dashboard sync post-checkout was already implemented (no change needed)
Stripe test keys: configured in .env.production
Webhook: must be configured on server via stripe CLI or Stripe Dashboard
Webhook URL: https://wordly.art/api/v1/auth/webhook/stripe
Backend:
- Add source_language column to glossaries table
- Add translations JSON column to glossary_terms table
- Alembic migration for schema changes
- format_glossary_for_prompt now language-aware: extracts correct
translation per target language, falls back to EN reference for
templates with only FR→EN data
- CRUD routes accept/return source_language and translations
- Pydantic schemas updated
Frontend:
- Types updated: GlossaryTerm now has translations: Record<string, string>
- Glossary/GlossaryListItem now have source_language
- Added SUPPORTED_LANGUAGES constant (13 languages)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New providers:
- DeepSeek: direct API with deepseek-chat model, very cost-effective
- Minimax: MiniMax-M1 model via OpenAI-compatible API, supports m2.7
Changes:
- Full provider implementations with retry, health check, batch support
- Provider config with env vars (DEEPSEEK_*, MINIMAX_*)
- Auto-registration in provider registry
- Updated fallback chain to include new providers
- Updated setup-env.sh wizard with options 6 (deepseek) and 7 (minimax)
- Updated manage-keys.sh with new menu entries and provider switching
- Updated docker-compose.yml with new env vars
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Fix MissingGreenlet: sync_engine now uses psycopg2 instead of asyncpg
- Fix bcrypt/passlib compat: pin bcrypt<4.1 in requirements
- Fix legacy password_hash NOT NULL: alter column to nullable in migration
- Add frontend password validation (uppercase + lowercase + digit)
- Add forgot-password and reset-password backend endpoints
- Add forgot-password and reset-password frontend pages
- Add email_service.py (SMTP via admin settings)
- Add reset_token/reset_token_expires columns to User model
- Migrate legacy JSON-only users to DB on password reset request
- Mount data/ volume in docker-compose.local.yml for persistence
- Add production deployment config (Dockerfile, nginx, deploy.sh)
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Major changes across backend, frontend, infrastructure:
- Provider system with model selection (Google, DeepL, OpenAI, Ollama, Google Cloud)
- Admin panel: user management, pricing, settings
- Glossary system with CSV import/export
- Subscription and tier quota management
- Security hardening (rate limiting, API key auth, path traversal fixes)
- Docker compose for dev, prod, and IONOS deployment
- Alembic migrations for new tables
- Frontend: dashboard, pricing page, landing page, i18n (en/fr)
- Test suite and verification scripts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add SQLAlchemy models for User, Translation, ApiKey, UsageLog, PaymentHistory
- Add database connection management with PostgreSQL/SQLite support
- Add repository layer for CRUD operations
- Add Alembic migration setup with initial migration
- Update auth_service to automatically use database when DATABASE_URL is set
- Update docker-compose.yml with PostgreSQL service and Redis (non-optional)
- Add database migration script (scripts/migrate_to_db.py)
- Update .env.example with database configuration
Changes:
- Removed hardcoded admin credentials (now requires env vars)
- Added Redis session storage with in-memory fallback
- Improved CORS configuration with warnings for development mode
- Added retry_with_backoff decorator for translation API calls
- Updated pricing: Starter=, Pro=, Business=
- Stripe price IDs now loaded from environment variables
- Added redis to requirements.txt
- Updated .env.example with all new configuration options
- Created COMPREHENSIVE_REVIEW_AND_PLAN.md with deployment roadmap
- Frontend: Updated pricing page, new UI components
