B3.6 — fix two visual bugs reported on the user's prod PDF:
1. Title 'Spécification technique : Office Translator v3.0' overflowed
its 2-line bbox and overlapped the 'Version du document...' block.
Root cause: MAX_VERTICAL_EXPANSION was 1.5x the original height,
way too small for a long French title. Bumped to 6.0x.
2. 'Avis important' blue background had white rectangular patches.
Root cause: redaction always used fill=(1,1,1) (opaque white),
which erased the colored drawing underneath the text.
Fix: detect when a block's bbox intersects a page drawing,
and use fill=None (transparent) for the redaction in that case.
The original drawing survives intact.
B3.7 — eliminate remaining block-vs-next-block overlap:
Computes each block's 'next_block_y' (the y0 of the nearest block
below it on the same page) and uses it as the ceiling for vertical
expansion. Previously the smart-fit logic used the page bottom as
the ceiling, which let long translated blocks flow into their
neighbour (e.g. 'Pour la dernière version...' overlapping
'8. Résolution des problèmes' in the TOC).
Also includes:
- 9 new tests (6 B3.6 + 3 B3.7) — total 446 tests pass, zero regression
- scripts/verify_b3_6_fix.py — visual+structural verification
- Updated sample_files/test_corpus/test_pdf_translated.pdf with the
clean B3.6+B3.7 output
ROOT CAUSE FIX: PyMuPDF silently raised AttributeError when fontname=None
was passed to insert_textbox. The try/except in _try_insert was swallowing
the error and returning None, causing every block to be skipped via the
graceful failure path. Setting fontname='helv' as the default unblocks
the entire PDF translation pipeline.
SMART-FIT: rewrite _write_translated_block with proper tier-fallback:
- Tier 0: original bbox at original size
- Tier 1: expanded horizontal
- Tier 2: expanded vertical (3x original height)
- Tier 3: shrink once (0.93x)
- Tier 4: shrink twice (0.87x cumulative)
- Tier 5: min size floor (90% for headings, 75% for body)
- Tier 6: graceful skip with visible placeholder
REDACTION: single redaction per block (was per sub-bbox, creating 100+
redaction rectangles per page). Now only 1 redaction per text block.
FEATURE FLAG: PDF_SMART_FIT_ENABLED (default true, observation-first).
METRICS: text_overflow -> format_elements_lost_total.
RESULT ON REAL PDF:
Before: fonts shrunk 22pt->5.6pt, hierarchy destroyed
After: fonts EXACT match: [8, 11, 12, 14, 16, 22] preserved
Frontend:
- Fix Framer Motion / motion-dom build error by pinning framer-motion to
11.18.2 (compatible with React 19 and Next.js 16).
- Add cross-env and build:local script to bypass standalone symlink errors
on Windows without Developer Mode.
- Allow NEXT_OUTPUT=default to disable standalone output for local builds.
- Refactor i18n: split 14,177-line src/lib/i18n.tsx into per-locale,
per-namespace JSON files under src/lib/i18n/messages/.
- Load English synchronously; other locales loaded on demand via dynamic
imports (reduces initial bundle, improves maintainability).
- Remove unused next-intl message files src/messages/en.json and fr.json.
Backend:
- Remove insecure legacy /api/v1/download/{filename} and /api/v1/cleanup/{filename}
endpoints. The job-based /api/v1/download/{job_id} already enforces ownership.
- Deduplicate texts in TranslationService.translate_batch before sending them
to the provider, reducing API calls for repeated strings.
- Pin httpx to <0.28 to fix TestClient incompatibility with starlette 0.35.1.
- Add pytest-cov and ruff dev dependencies/config.
DevOps:
- Remove hardcoded Grafana password from docker-compose.yml and
docker-compose.monitoring.yml; use GRAFANA_PASSWORD env var.
- Change default TRANSLATION_SERVICE from ollama to google in
docker-compose.yml (Ollama is an optional profile).
- Add GRAFANA_PASSWORD to .env.example.
- Add .coverage and frontend/pnpm-workspace.yaml to .gitignore.
Tests:
- Update API versioning tests for removed legacy endpoints.
- Add tests/test_translation_service.py for deduplication behavior.
Verified:
- pnpm run build:local passes.
- uv run pytest tests/test_providers/* tests/test_translation_service.py
tests/test_story_3_5_api_versioning.py tests/test_download_endpoint.py
tests/test_translators/test_excel_translator.py: provider/translator tests
pass; one pre-existing French error-message test still fails (message is
returned in English, unrelated to this change).
Nouveau script dedie a la migration 'multilingue uniquement' :
- Supprime tout glossaire dont le nom ne contient pas '-> Multilingue'
- Backup JSON integral des glossaires + termes avant suppression
- Mode --dry-run / confirmation interactive / --yes
- Heuristique par nom uniquement (target_language peut etre 'multi' partout
apres les migrations passees, donc non fiable)
Utilise apres le dedup (user_id, name) pour finir la migration.
Le groupement par template_id etait faux sur la prod :
- Les doublons historiques ont template_id=NULL (crees avant la migration)
- Deux glossaires 'Finance - FR->Anglais' et 'Finance - FR->Multilingue'
partagent le meme template_id mais DOIVENT etre conserves separement.
Changements :
- Groupement par (user_id, name) -> c'est ce que l'utilisateur voit dans l'UI
et la definition reelle d'un doublon.
- Les glossaires multilingues ('-> Multilingue') ont un nom distinct des
versions '-> Anglais' : ils ne sont jamais fusionnes (preserve par design).
- Fallback automatique si la colonne template_id est absente du schema
(dev DB) : warning + requete sans la colonne, aucun crash.
- Suppression du flag --allow-missing-template-id devenu inutile.
- Nettoyage des imports ORM inutiles (text brut uniquement, plus rapide).
- scripts/backup_duplicate_glossaries.py : exporte en JSON les doublons
(meme user_id + template_id) sans rien supprimer. Schema validation,
tri stable, mode degrade si colonne template_id absente.
- scripts/delete_duplicate_glossaries.py : lit un backup JSON et supprime
les doublons listes. Validation IDs, confirmation interactive,
commit par user, mode --dry-run / --yes.
- .gitea/workflows/cleanup-glossaries.yml : workflow_dispatch qui SSH
sur le serveur de prod et execute le script dans le conteneur backend
(postgres demarre, .env charge, env_file docker-compose).
- Create Stripe products/prices (Starter/Pro/Business monthly+yearly)
- Fix CRITICAL bug: add subscription_ends_at + cancel_at_period_end columns to users table
- Alembic migration: f6a7b8c9d0e1_add_subscription_ends_at_cancel_at_period_end
- Fix: implement handle_payment_failed() to set subscription_status=PAST_DUE
- Fix: harmonize .env.production Stripe variable names to match pricing_config.py
- Fix: add missing FRONTEND_URL and STRIPE_PUBLISHABLE_KEY to .env.production
- Add all Stripe Price IDs (test mode) to .env.production
- Wire credit purchase buttons to /api/v1/auth/create-credits-checkout
- Dashboard sync post-checkout was already implemented (no change needed)
Stripe test keys: configured in .env.production
Webhook: must be configured on server via stripe CLI or Stripe Dashboard
Webhook URL: https://wordly.art/api/v1/auth/webhook/stripe
- Enriched 8 glossary templates with 18,191 translations across 11 languages
using LLM batch generation + back-translation validation (99.98% confirmed)
- Rewrote GlossarySelector as inline section with template creation
- Fixed sidebar duplicate (single Glossaries link with proOnly flag)
- Added glossaryId reset when sourceLang changes
- Always show GlossarySelector (locked with Pro badge for free users)
- Added source_language flag on glossary cards
- Redirected /dashboard/context to /dashboard/glossaries
- Updated import endpoint to read translations from templates
- Added enrichment script (scripts/enrich_glossary_templates.py)
- Added 6 i18n keys across all 13 locales
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
setup-env.sh:
- Each provider choice now sets its *_ENABLEED=true flag explicitly
- All provider fields written to .env (model, base_url, api_key, enabled)
- Shows active provider status in summary
manage-keys.sh:
- Each provider menu sets *_ENABLEED=true when key is added
- Sets *_ENABLEED=false when key is removed
- Writes all required env vars (model, base_url) not just API key
- Shows provider status with enabled/disabled state
- Better organized menu
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New providers:
- DeepSeek: direct API with deepseek-chat model, very cost-effective
- Minimax: MiniMax-M1 model via OpenAI-compatible API, supports m2.7
Changes:
- Full provider implementations with retry, health check, batch support
- Provider config with env vars (DEEPSEEK_*, MINIMAX_*)
- Auto-registration in provider registry
- Updated fallback chain to include new providers
- Updated setup-env.sh wizard with options 6 (deepseek) and 7 (minimax)
- Updated manage-keys.sh with new menu entries and provider switching
- Updated docker-compose.yml with new env vars
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Interactive menu to add/update/clear API keys:
- Translation: OpenAI, DeepL, OpenRouter
- Payment: Stripe (secret, webhook, price IDs)
- Admin password change with bcrypt
- Switch translation provider
- Status overview with masked keys
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Use stty -echo instead of read -s for password input,
compatible with dash/sh shells found on Ubuntu/Debian servers.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Interactive bash script for server configuration:
- Generates all secrets automatically (JWT, admin token, DB password)
- Hashes admin password with bcrypt
- Configurable translation providers, Stripe, Grafana
- Validates inputs and confirms before writing
- Secures .env with chmod 600
Usage on server: bash scripts/setup-env.sh
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Restructured docker-compose for Nginx Proxy Manager (no custom nginx)
- Added domain wordly.art configuration
- Added Prometheus + Grafana monitoring stack with pre-configured dashboards
- Added PostgreSQL backup script to NAS (daily/weekly/monthly rotation)
- Added alert rules for backend, system, and Docker metrics
- Updated deployment guide for NPM + IONOS DNS homelab setup
- Added marketing plan document
- PDF translator and watermark support
- Enhanced middleware, routes, and translator modules
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Major changes across backend, frontend, infrastructure:
- Provider system with model selection (Google, DeepL, OpenAI, Ollama, Google Cloud)
- Admin panel: user management, pricing, settings
- Glossary system with CSV import/export
- Subscription and tier quota management
- Security hardening (rate limiting, API key auth, path traversal fixes)
- Docker compose for dev, prod, and IONOS deployment
- Alembic migrations for new tables
- Frontend: dashboard, pricing page, landing page, i18n (en/fr)
- Test suite and verification scripts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add SQLAlchemy models for User, Translation, ApiKey, UsageLog, PaymentHistory
- Add database connection management with PostgreSQL/SQLite support
- Add repository layer for CRUD operations
- Add Alembic migration setup with initial migration
- Update auth_service to automatically use database when DATABASE_URL is set
- Update docker-compose.yml with PostgreSQL service and Redis (non-optional)
- Add database migration script (scripts/migrate_to_db.py)
- Update .env.example with database configuration
