Commit Graph

25 Commits

Author SHA1 Message Date
7398cae359 feat(glossaries): add script to translate non-multilingual database glossaries using Google Translate adapter
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m25s
2026-06-28 10:42:41 +02:00
fa637abff0 perf+security: fix build, secure downloads, dedupe translations, refactor i18n
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m49s
Frontend:
- Fix Framer Motion / motion-dom build error by pinning framer-motion to
  11.18.2 (compatible with React 19 and Next.js 16).
- Add cross-env and build:local script to bypass standalone symlink errors
  on Windows without Developer Mode.
- Allow NEXT_OUTPUT=default to disable standalone output for local builds.
- Refactor i18n: split 14,177-line src/lib/i18n.tsx into per-locale,
  per-namespace JSON files under src/lib/i18n/messages/.
- Load English synchronously; other locales loaded on demand via dynamic
  imports (reduces initial bundle, improves maintainability).
- Remove unused next-intl message files src/messages/en.json and fr.json.

Backend:
- Remove insecure legacy /api/v1/download/{filename} and /api/v1/cleanup/{filename}
  endpoints. The job-based /api/v1/download/{job_id} already enforces ownership.
- Deduplicate texts in TranslationService.translate_batch before sending them
  to the provider, reducing API calls for repeated strings.
- Pin httpx to <0.28 to fix TestClient incompatibility with starlette 0.35.1.
- Add pytest-cov and ruff dev dependencies/config.

DevOps:
- Remove hardcoded Grafana password from docker-compose.yml and
  docker-compose.monitoring.yml; use GRAFANA_PASSWORD env var.
- Change default TRANSLATION_SERVICE from ollama to google in
  docker-compose.yml (Ollama is an optional profile).
- Add GRAFANA_PASSWORD to .env.example.
- Add .coverage and frontend/pnpm-workspace.yaml to .gitignore.

Tests:
- Update API versioning tests for removed legacy endpoints.
- Add tests/test_translation_service.py for deduplication behavior.

Verified:
- pnpm run build:local passes.
- uv run pytest tests/test_providers/* tests/test_translation_service.py
  tests/test_story_3_5_api_versioning.py tests/test_download_endpoint.py
  tests/test_translators/test_excel_translator.py: provider/translator tests
  pass; one pre-existing French error-message test still fails (message is
  returned in English, unrelated to this change).
2026-06-14 16:44:18 +02:00
9bb02927c3 fix: redirect logs to stderr and reduce size thresholds in backup/verify scripts
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m42s
2026-06-07 11:16:45 +02:00
ddf6b8f6bc fix: ignore unbound variables when sourcing .env in all backup/DR scripts
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m35s
2026-06-07 11:14:04 +02:00
3f980ad537 feat: add NAS backup, verification, and DR scripts
Some checks failed
Deploy to Production / Build and Deploy (push) Has been cancelled
2026-06-07 11:12:01 +02:00
80b49ee354 Robustness: Implement multi-destination backups (LOCAL, NAS, SCP) and backup/restore of NPM configurations 2026-06-07 09:50:51 +02:00
c7299228cd Robustness: Add fallback path handling in disaster-recovery.sh for NAS offline cases
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m15s
2026-06-07 09:48:13 +02:00
670d3f4376 Documentation: Add French Disaster Recovery Playbook for server failovers 2026-06-07 09:39:26 +02:00
23373ab91a Robustness: Update backup-database.sh with robust python resolution and cross-platform path handling 2026-06-07 09:29:19 +02:00
e7b5ea9a61 Robustness: Add json-file log rotation limits to all docker services 2026-06-07 09:26:39 +02:00
Sepehr
1d883c3c69 feat(glossaries): add migrate_glossaries_to_multilingual.py
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 5m10s
Nouveau script dedie a la migration 'multilingue uniquement' :
- Supprime tout glossaire dont le nom ne contient pas '-> Multilingue'
- Backup JSON integral des glossaires + termes avant suppression
- Mode --dry-run / confirmation interactive / --yes
- Heuristique par nom uniquement (target_language peut etre 'multi' partout
  apres les migrations passees, donc non fiable)

Utilise apres le dedup (user_id, name) pour finir la migration.
2026-06-03 21:56:01 +02:00
Sepehr
8f55e3d9aa fix(glossaries): dedup by (user_id, name) instead of (user_id, template_id)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 3m1s
Le groupement par template_id etait faux sur la prod :
- Les doublons historiques ont template_id=NULL (crees avant la migration)
- Deux glossaires 'Finance - FR->Anglais' et 'Finance - FR->Multilingue'
  partagent le meme template_id mais DOIVENT etre conserves separement.

Changements :
- Groupement par (user_id, name) -> c'est ce que l'utilisateur voit dans l'UI
  et la definition reelle d'un doublon.
- Les glossaires multilingues ('-> Multilingue') ont un nom distinct des
  versions '-> Anglais' : ils ne sont jamais fusionnes (preserve par design).
- Fallback automatique si la colonne template_id est absente du schema
  (dev DB) : warning + requete sans la colonne, aucun crash.
- Suppression du flag --allow-missing-template-id devenu inutile.
- Nettoyage des imports ORM inutiles (text brut uniquement, plus rapide).
2026-06-03 21:48:53 +02:00
Sepehr
cd32a42b1a feat(glossaries): add backup + delete scripts and Gitea workflow for duplicate cleanup
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 6m31s
- scripts/backup_duplicate_glossaries.py : exporte en JSON les doublons
  (meme user_id + template_id) sans rien supprimer. Schema validation,
  tri stable, mode degrade si colonne template_id absente.
- scripts/delete_duplicate_glossaries.py : lit un backup JSON et supprime
  les doublons listes. Validation IDs, confirmation interactive,
  commit par user, mode --dry-run / --yes.
- .gitea/workflows/cleanup-glossaries.yml : workflow_dispatch qui SSH
  sur le serveur de prod et execute le script dans le conteneur backend
  (postgres demarre, .env charge, env_file docker-compose).
2026-06-03 21:21:11 +02:00
374c605027 feat: Stripe integration complete - products created, DB migration, payment_failed handler, credit buttons wired
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m5s
- Create Stripe products/prices (Starter/Pro/Business monthly+yearly)
- Fix CRITICAL bug: add subscription_ends_at + cancel_at_period_end columns to users table
- Alembic migration: f6a7b8c9d0e1_add_subscription_ends_at_cancel_at_period_end
- Fix: implement handle_payment_failed() to set subscription_status=PAST_DUE
- Fix: harmonize .env.production Stripe variable names to match pricing_config.py
- Fix: add missing FRONTEND_URL and STRIPE_PUBLISHABLE_KEY to .env.production
- Add all Stripe Price IDs (test mode) to .env.production
- Wire credit purchase buttons to /api/v1/auth/create-credits-checkout
- Dashboard sync post-checkout was already implemented (no change needed)

Stripe test keys: configured in .env.production
Webhook: must be configured on server via stripe CLI or Stripe Dashboard
Webhook URL: https://wordly.art/api/v1/auth/webhook/stripe
2026-05-31 21:40:31 +02:00
ca8abc560d feat: multilingual glossary templates + inline GlossarySelector rewrite
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 1m28s
- Enriched 8 glossary templates with 18,191 translations across 11 languages
  using LLM batch generation + back-translation validation (99.98% confirmed)
- Rewrote GlossarySelector as inline section with template creation
- Fixed sidebar duplicate (single Glossaries link with proOnly flag)
- Added glossaryId reset when sourceLang changes
- Always show GlossarySelector (locked with Pro badge for free users)
- Added source_language flag on glossary cards
- Redirected /dashboard/context to /dashboard/glossaries
- Updated import endpoint to read translations from templates
- Added enrichment script (scripts/enrich_glossary_templates.py)
- Added 6 i18n keys across all 13 locales

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-17 00:52:24 +02:00
98d82414bb feat: add Node.js setup wizard (replaces broken bash scripts)
Some checks failed
Deploy to Homelab / Deploy Wordly to 192.168.1.151 (push) Has been cancelled
Deploy to Homelab / Deploy Monitoring (if configured) (push) Has been cancelled
Node.js script that actually works:
- Generates JWT_SECRET_KEY, ADMIN_TOKEN_SECRET, POSTGRES_PASSWORD
- Hashes admin password with bcrypt via docker
- Writes ADMIN_PASSWORD (plaintext) + ADMIN_PASSWORD_HASH (bcrypt)
- Sets *_ENABLED=true for chosen provider, false for others
- Writes ALL provider fields (api_key, model, base_url)
- Optionally configures Stripe
- Optionally starts docker compose
- Verified: syntax OK, all env vars match backend expectations

Usage: node scripts/setup.js

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-10 13:29:28 +02:00
7e97bbd07b fix: scripts properly set ENABLED flags and all env vars for providers
Some checks failed
Deploy to Homelab / Deploy Wordly to 192.168.1.151 (push) Has been cancelled
Deploy to Homelab / Deploy Monitoring (if configured) (push) Has been cancelled
setup-env.sh:
- Each provider choice now sets its *_ENABLEED=true flag explicitly
- All provider fields written to .env (model, base_url, api_key, enabled)
- Shows active provider status in summary

manage-keys.sh:
- Each provider menu sets *_ENABLEED=true when key is added
- Sets *_ENABLEED=false when key is removed
- Writes all required env vars (model, base_url) not just API key
- Shows provider status with enabled/disabled state
- Better organized menu

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-10 12:52:06 +02:00
e6e1678b1d feat: add DeepSeek and Minimax (m2.7) translation providers
Some checks failed
Deploy to Homelab / Deploy Wordly to 192.168.1.151 (push) Has been cancelled
Deploy to Homelab / Deploy Monitoring (if configured) (push) Has been cancelled
New providers:
- DeepSeek: direct API with deepseek-chat model, very cost-effective
- Minimax: MiniMax-M1 model via OpenAI-compatible API, supports m2.7

Changes:
- Full provider implementations with retry, health check, batch support
- Provider config with env vars (DEEPSEEK_*, MINIMAX_*)
- Auto-registration in provider registry
- Updated fallback chain to include new providers
- Updated setup-env.sh wizard with options 6 (deepseek) and 7 (minimax)
- Updated manage-keys.sh with new menu entries and provider switching
- Updated docker-compose.yml with new env vars

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-10 12:30:36 +02:00
d6d19eaf0c feat: add API key management script
Some checks failed
Deploy to Homelab / Deploy Wordly to 192.168.1.151 (push) Has been cancelled
Deploy to Homelab / Deploy Monitoring (if configured) (push) Has been cancelled
Interactive menu to add/update/clear API keys:
- Translation: OpenAI, DeepL, OpenRouter
- Payment: Stripe (secret, webhook, price IDs)
- Admin password change with bcrypt
- Switch translation provider
- Status overview with masked keys

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-10 12:18:43 +02:00
3d01f53924 fix: setup wizard compatible sh/dash (read -s fallback)
Some checks failed
Deploy to Homelab / Deploy Wordly to 192.168.1.151 (push) Has been cancelled
Deploy to Homelab / Deploy Monitoring (if configured) (push) Has been cancelled
Use stty -echo instead of read -s for password input,
compatible with dash/sh shells found on Ubuntu/Debian servers.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-10 12:16:27 +02:00
a9fe6c1fc6 feat: add interactive .env setup wizard
Some checks failed
Deploy to Homelab / Deploy Wordly to 192.168.1.151 (push) Has been cancelled
Deploy to Homelab / Deploy Monitoring (if configured) (push) Has been cancelled
Interactive bash script for server configuration:
- Generates all secrets automatically (JWT, admin token, DB password)
- Hashes admin password with bcrypt
- Configurable translation providers, Stripe, Grafana
- Validates inputs and confirms before writing
- Secures .env with chmod 600

Usage on server: bash scripts/setup-env.sh

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-10 12:02:44 +02:00
ce8e150a61 feat: homelab deployment - NPM + IONOS DNS + monitoring + NAS backup
- Restructured docker-compose for Nginx Proxy Manager (no custom nginx)
- Added domain wordly.art configuration
- Added Prometheus + Grafana monitoring stack with pre-configured dashboards
- Added PostgreSQL backup script to NAS (daily/weekly/monthly rotation)
- Added alert rules for backend, system, and Docker metrics
- Updated deployment guide for NPM + IONOS DNS homelab setup
- Added marketing plan document
- PDF translator and watermark support
- Enhanced middleware, routes, and translator modules

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-05-10 11:43:28 +02:00
Sepehr Ramezani
26bd096a06 feat: production deployment - full update with providers, admin, glossaries, pricing, tests
Major changes across backend, frontend, infrastructure:
- Provider system with model selection (Google, DeepL, OpenAI, Ollama, Google Cloud)
- Admin panel: user management, pricing, settings
- Glossary system with CSV import/export
- Subscription and tier quota management
- Security hardening (rate limiting, API key auth, path traversal fixes)
- Docker compose for dev, prod, and IONOS deployment
- Alembic migrations for new tables
- Frontend: dashboard, pricing page, landing page, i18n (en/fr)
- Test suite and verification scripts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-25 15:01:47 +02:00
550f3516db feat: Add PostgreSQL database infrastructure
- Add SQLAlchemy models for User, Translation, ApiKey, UsageLog, PaymentHistory
- Add database connection management with PostgreSQL/SQLite support
- Add repository layer for CRUD operations
- Add Alembic migration setup with initial migration
- Update auth_service to automatically use database when DATABASE_URL is set
- Update docker-compose.yml with PostgreSQL service and Redis (non-optional)
- Add database migration script (scripts/migrate_to_db.py)
- Update .env.example with database configuration
2025-12-31 10:56:19 +01:00
29178a75a5 feat: Add complete production deployment infrastructure
- Docker configuration:
  - Multi-stage Dockerfiles for backend (Python 3.11) and frontend (Node 20)
  - Production docker-compose.yml with all services
  - Development docker-compose.dev.yml with hot-reload

- Nginx reverse proxy:
  - SSL/TLS termination with modern cipher suites
  - Rate limiting and security headers
  - Caching and compression
  - Load balancing ready

- Kubernetes manifests:
  - Deployment, Service, Ingress configurations
  - ConfigMap and Secrets
  - HPA for auto-scaling
  - PersistentVolumeClaims

- Deployment scripts:
  - deploy.sh: Automated deployment with health checks
  - backup.sh: Automated backup with retention
  - health-check.sh: Service health monitoring
  - setup-ssl.sh: Let's Encrypt SSL automation

- Monitoring:
  - Prometheus configuration
  - Grafana dashboards (optional)
  - Structured logging

- Documentation:
  - DEPLOYMENT_GUIDE.md: Complete deployment instructions
  - Environment templates (.env.production)

Ready for commercial deployment!
2025-11-30 20:56:15 +01:00