User said 'I want this to be automatic'. Three gaps were addressed.
Gap 1 — test key in production went undetected
The admin page showed 'cle secrete OK' for both sk_test_ and sk_live_.
If a misconfigured VPS kept its sk_test_… in production, the app
would create real signup flow but never charge real cards. Added
services.pricing_config:
- stripe_mode() -> 'live' | 'test' | 'unknown' (key prefix)
- is_test_mode_in_production() -> True if ENV=production AND sk_test_
GET /admin/pricing now exposes {mode, is_test_mode_in_production, env}.
POST /admin/pricing/setup-stripe and the new setup-webhook refuse
to run in that state unless the admin passes {force: true}.
Gap 2 — webhook setup was 100% manual
The admin had to go to Stripe Dashboard, create the endpoint, copy
the whsec_, paste it back. Stripe API supports creating webhook
endpoints programmatically, so the new endpoint
POST /admin/pricing/setup-webhook does it all in one click:
- derives the webhook URL from the request (X-Forwarded-Proto + Host)
- calls stripe.WebhookEndpoint.create() (or .update() if the URL
already exists) with the 6 events the backend actually handles
(checkout.session.completed, customer.subscription.*, invoice.*)
- persists the returned whsec_ to .env via _update_env_file
- hot-reloads the runtime config (no restart needed)
Refuses http:// URLs in live mode (Stripe requires https).
Gap 3 — obsolete script leaked a test secret
scripts/stripe_setup.py contained a hardcoded sk_test_… in source.
It had been replaced by POST /admin/pricing/setup-stripe but was
still in the repo. Deleted via git rm. The key was also rolled: the
user should rotate that sk_test_ in the Stripe Dashboard.
Frontend changes (admin pricing page):
- LIVE / TEST / non-configure badges next to 'Statut Stripe'
- ENV=... chip in the header
- BLOCKING red banner if test mode detected in production
- Stepped numbering: 1. Produits & prix / 2. Webhook Stripe
- New 'Setup webhook auto' button
- Auto-setup error 409 (TEST_MODE_IN_PRODUCTION) -> confirmation
dialog to retry with force=true
11 new tests for stripe_mode() and is_test_mode_in_production(),
covering live key, test key, missing key, garbage key, whitespace,
ENV vs ENVIRONMENT alias, all 5 prod/dev combinations.
Total: 471 tests pass (was 460), zero regression.
User asked whether B3.6+B3.7 are generic for ALL PDFs or just for the
test_pdf.pdf. Audit found 2 genericity bugs:
1. _populate_next_block_y was sorting blocks globally by y0. In a
multi-column PDF (journals, brochures, newspapers), the 'next
block' of a left-column block would point to the right-column
block at the same y, which is wrong. Fix: group blocks into
columns by x0 proximity (15pt tolerance), then sort each column
by y0. Each block's next_block_y is the y0 of its column-mate
directly below it, not just the next block in y-order globally.
2. max_expand_y could go negative if next_block_y was above the
current block (rare edge case in extracted blocks with weird
bbox ordering). A negative max_expand_y would create an invalid
fitz.Rect with y1 < y0, causing silent failures. Fix: clamp
max_expand_y to >= 0.
7 new tests added:
- test_two_columns_get_separate_next_block_y: 2-col layout,
left and right columns get independent next_block_y mappings
- test_centered_full_width_header_gets_own_column: full-width
header between 2 columns is its own column
- test_three_columns: 3-column newspaper layout
- test_single_block_page, test_empty_block_list: edge cases
- test_max_expand_y_clamped_to_zero: negative-expansion safety
- test_two_column_pdf_translation_end_to_end: e2e test on a
2-col journal PDF, 4 input blocks -> 4 output blocks preserved
at correct positions, no cross-column overlap
Visual verification:
scripts/verify_b3_8_multicolumn.py renders a 2-col journal PDF
before and after translation, confirms 4 left + 4 right blocks
preserved at exact positions.
Total tests: 453 (was 446), zero regression.
B3.6 — fix two visual bugs reported on the user's prod PDF:
1. Title 'Spécification technique : Office Translator v3.0' overflowed
its 2-line bbox and overlapped the 'Version du document...' block.
Root cause: MAX_VERTICAL_EXPANSION was 1.5x the original height,
way too small for a long French title. Bumped to 6.0x.
2. 'Avis important' blue background had white rectangular patches.
Root cause: redaction always used fill=(1,1,1) (opaque white),
which erased the colored drawing underneath the text.
Fix: detect when a block's bbox intersects a page drawing,
and use fill=None (transparent) for the redaction in that case.
The original drawing survives intact.
B3.7 — eliminate remaining block-vs-next-block overlap:
Computes each block's 'next_block_y' (the y0 of the nearest block
below it on the same page) and uses it as the ceiling for vertical
expansion. Previously the smart-fit logic used the page bottom as
the ceiling, which let long translated blocks flow into their
neighbour (e.g. 'Pour la dernière version...' overlapping
'8. Résolution des problèmes' in the TOC).
Also includes:
- 9 new tests (6 B3.6 + 3 B3.7) — total 446 tests pass, zero regression
- scripts/verify_b3_6_fix.py — visual+structural verification
- Updated sample_files/test_corpus/test_pdf_translated.pdf with the
clean B3.6+B3.7 output
ROOT CAUSE FIX: PyMuPDF silently raised AttributeError when fontname=None
was passed to insert_textbox. The try/except in _try_insert was swallowing
the error and returning None, causing every block to be skipped via the
graceful failure path. Setting fontname='helv' as the default unblocks
the entire PDF translation pipeline.
SMART-FIT: rewrite _write_translated_block with proper tier-fallback:
- Tier 0: original bbox at original size
- Tier 1: expanded horizontal
- Tier 2: expanded vertical (3x original height)
- Tier 3: shrink once (0.93x)
- Tier 4: shrink twice (0.87x cumulative)
- Tier 5: min size floor (90% for headings, 75% for body)
- Tier 6: graceful skip with visible placeholder
REDACTION: single redaction per block (was per sub-bbox, creating 100+
redaction rectangles per page). Now only 1 redaction per text block.
FEATURE FLAG: PDF_SMART_FIT_ENABLED (default true, observation-first).
METRICS: text_overflow -> format_elements_lost_total.
RESULT ON REAL PDF:
Before: fonts shrunk 22pt->5.6pt, hierarchy destroyed
After: fonts EXACT match: [8, 11, 12, 14, 16, 22] preserved
Frontend:
- Fix Framer Motion / motion-dom build error by pinning framer-motion to
11.18.2 (compatible with React 19 and Next.js 16).
- Add cross-env and build:local script to bypass standalone symlink errors
on Windows without Developer Mode.
- Allow NEXT_OUTPUT=default to disable standalone output for local builds.
- Refactor i18n: split 14,177-line src/lib/i18n.tsx into per-locale,
per-namespace JSON files under src/lib/i18n/messages/.
- Load English synchronously; other locales loaded on demand via dynamic
imports (reduces initial bundle, improves maintainability).
- Remove unused next-intl message files src/messages/en.json and fr.json.
Backend:
- Remove insecure legacy /api/v1/download/{filename} and /api/v1/cleanup/{filename}
endpoints. The job-based /api/v1/download/{job_id} already enforces ownership.
- Deduplicate texts in TranslationService.translate_batch before sending them
to the provider, reducing API calls for repeated strings.
- Pin httpx to <0.28 to fix TestClient incompatibility with starlette 0.35.1.
- Add pytest-cov and ruff dev dependencies/config.
DevOps:
- Remove hardcoded Grafana password from docker-compose.yml and
docker-compose.monitoring.yml; use GRAFANA_PASSWORD env var.
- Change default TRANSLATION_SERVICE from ollama to google in
docker-compose.yml (Ollama is an optional profile).
- Add GRAFANA_PASSWORD to .env.example.
- Add .coverage and frontend/pnpm-workspace.yaml to .gitignore.
Tests:
- Update API versioning tests for removed legacy endpoints.
- Add tests/test_translation_service.py for deduplication behavior.
Verified:
- pnpm run build:local passes.
- uv run pytest tests/test_providers/* tests/test_translation_service.py
tests/test_story_3_5_api_versioning.py tests/test_download_endpoint.py
tests/test_translators/test_excel_translator.py: provider/translator tests
pass; one pre-existing French error-message test still fails (message is
returned in English, unrelated to this change).
Nouveau script dedie a la migration 'multilingue uniquement' :
- Supprime tout glossaire dont le nom ne contient pas '-> Multilingue'
- Backup JSON integral des glossaires + termes avant suppression
- Mode --dry-run / confirmation interactive / --yes
- Heuristique par nom uniquement (target_language peut etre 'multi' partout
apres les migrations passees, donc non fiable)
Utilise apres le dedup (user_id, name) pour finir la migration.
Le groupement par template_id etait faux sur la prod :
- Les doublons historiques ont template_id=NULL (crees avant la migration)
- Deux glossaires 'Finance - FR->Anglais' et 'Finance - FR->Multilingue'
partagent le meme template_id mais DOIVENT etre conserves separement.
Changements :
- Groupement par (user_id, name) -> c'est ce que l'utilisateur voit dans l'UI
et la definition reelle d'un doublon.
- Les glossaires multilingues ('-> Multilingue') ont un nom distinct des
versions '-> Anglais' : ils ne sont jamais fusionnes (preserve par design).
- Fallback automatique si la colonne template_id est absente du schema
(dev DB) : warning + requete sans la colonne, aucun crash.
- Suppression du flag --allow-missing-template-id devenu inutile.
- Nettoyage des imports ORM inutiles (text brut uniquement, plus rapide).
- scripts/backup_duplicate_glossaries.py : exporte en JSON les doublons
(meme user_id + template_id) sans rien supprimer. Schema validation,
tri stable, mode degrade si colonne template_id absente.
- scripts/delete_duplicate_glossaries.py : lit un backup JSON et supprime
les doublons listes. Validation IDs, confirmation interactive,
commit par user, mode --dry-run / --yes.
- .gitea/workflows/cleanup-glossaries.yml : workflow_dispatch qui SSH
sur le serveur de prod et execute le script dans le conteneur backend
(postgres demarre, .env charge, env_file docker-compose).
- Create Stripe products/prices (Starter/Pro/Business monthly+yearly)
- Fix CRITICAL bug: add subscription_ends_at + cancel_at_period_end columns to users table
- Alembic migration: f6a7b8c9d0e1_add_subscription_ends_at_cancel_at_period_end
- Fix: implement handle_payment_failed() to set subscription_status=PAST_DUE
- Fix: harmonize .env.production Stripe variable names to match pricing_config.py
- Fix: add missing FRONTEND_URL and STRIPE_PUBLISHABLE_KEY to .env.production
- Add all Stripe Price IDs (test mode) to .env.production
- Wire credit purchase buttons to /api/v1/auth/create-credits-checkout
- Dashboard sync post-checkout was already implemented (no change needed)
Stripe test keys: configured in .env.production
Webhook: must be configured on server via stripe CLI or Stripe Dashboard
Webhook URL: https://wordly.art/api/v1/auth/webhook/stripe
- Enriched 8 glossary templates with 18,191 translations across 11 languages
using LLM batch generation + back-translation validation (99.98% confirmed)
- Rewrote GlossarySelector as inline section with template creation
- Fixed sidebar duplicate (single Glossaries link with proOnly flag)
- Added glossaryId reset when sourceLang changes
- Always show GlossarySelector (locked with Pro badge for free users)
- Added source_language flag on glossary cards
- Redirected /dashboard/context to /dashboard/glossaries
- Updated import endpoint to read translations from templates
- Added enrichment script (scripts/enrich_glossary_templates.py)
- Added 6 i18n keys across all 13 locales
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
setup-env.sh:
- Each provider choice now sets its *_ENABLEED=true flag explicitly
- All provider fields written to .env (model, base_url, api_key, enabled)
- Shows active provider status in summary
manage-keys.sh:
- Each provider menu sets *_ENABLEED=true when key is added
- Sets *_ENABLEED=false when key is removed
- Writes all required env vars (model, base_url) not just API key
- Shows provider status with enabled/disabled state
- Better organized menu
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
New providers:
- DeepSeek: direct API with deepseek-chat model, very cost-effective
- Minimax: MiniMax-M1 model via OpenAI-compatible API, supports m2.7
Changes:
- Full provider implementations with retry, health check, batch support
- Provider config with env vars (DEEPSEEK_*, MINIMAX_*)
- Auto-registration in provider registry
- Updated fallback chain to include new providers
- Updated setup-env.sh wizard with options 6 (deepseek) and 7 (minimax)
- Updated manage-keys.sh with new menu entries and provider switching
- Updated docker-compose.yml with new env vars
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Interactive menu to add/update/clear API keys:
- Translation: OpenAI, DeepL, OpenRouter
- Payment: Stripe (secret, webhook, price IDs)
- Admin password change with bcrypt
- Switch translation provider
- Status overview with masked keys
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Use stty -echo instead of read -s for password input,
compatible with dash/sh shells found on Ubuntu/Debian servers.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Interactive bash script for server configuration:
- Generates all secrets automatically (JWT, admin token, DB password)
- Hashes admin password with bcrypt
- Configurable translation providers, Stripe, Grafana
- Validates inputs and confirms before writing
- Secures .env with chmod 600
Usage on server: bash scripts/setup-env.sh
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
- Restructured docker-compose for Nginx Proxy Manager (no custom nginx)
- Added domain wordly.art configuration
- Added Prometheus + Grafana monitoring stack with pre-configured dashboards
- Added PostgreSQL backup script to NAS (daily/weekly/monthly rotation)
- Added alert rules for backend, system, and Docker metrics
- Updated deployment guide for NPM + IONOS DNS homelab setup
- Added marketing plan document
- PDF translator and watermark support
- Enhanced middleware, routes, and translator modules
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Major changes across backend, frontend, infrastructure:
- Provider system with model selection (Google, DeepL, OpenAI, Ollama, Google Cloud)
- Admin panel: user management, pricing, settings
- Glossary system with CSV import/export
- Subscription and tier quota management
- Security hardening (rate limiting, API key auth, path traversal fixes)
- Docker compose for dev, prod, and IONOS deployment
- Alembic migrations for new tables
- Frontend: dashboard, pricing page, landing page, i18n (en/fr)
- Test suite and verification scripts
Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add SQLAlchemy models for User, Translation, ApiKey, UsageLog, PaymentHistory
- Add database connection management with PostgreSQL/SQLite support
- Add repository layer for CRUD operations
- Add Alembic migration setup with initial migration
- Update auth_service to automatically use database when DATABASE_URL is set
- Update docker-compose.yml with PostgreSQL service and Redis (non-optional)
- Add database migration script (scripts/migrate_to_db.py)
- Update .env.example with database configuration
