sepehr
4d466699fd
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m26s
L1 quality layer — uses a cheap LLM via the OpenAI-compatible API to
validate translation quality. Designed to be the SECOND line of defense
after L0 (script detection, length, pattern).
Architecture:
- sampler.py — picks 5 representative chunks per job (longest first,
skips L0-failed indices, skips too-short or identical pairs)
- llm_judge.py — OpenAI-compatible client, binary verdict per chunk
(accurate / fluent / correct_language / no_leaks), JSON output,
hard timeout, defensive (never raises), cost estimation built in
- pipeline.py — defensive wrapper that integrates both, never breaks
a translation job, always logs a structured event
Integration:
- 5 feature flags in config.py (QUALITY_L1_ENABLED, _LOG_ONLY, etc.)
- QUALITY_L1_LOG_ONLY=true by default: log-only mode, verdict NEVER
blocks or retries a job
- Reuses the chunks extracted by L0 (no double work)
- Passes the set of L0-failed indices so L1 doesn't re-judge them
- Wrapped in try/except so a misconfigured L1 NEVER breaks a job
Default config: deepseek-chat via DeepSeek API
- Cost: ~0.0003 USD per job (5 chunks)
- Speed: typically 1-2s per call, hard ceiling at 8s
- Easy to swap: just set L1_JUDGE_BASE_URL and L1_JUDGE_MODEL
LLM judge is intentionally a SEPARATE model from the translator
(self-evaluation bias mitigation — Meta/Stanford papers 2024-2025).
Tests:
test_sampler.py — 9 tests covering the sampling strategy
test_llm_judge.py — 22 tests covering init, parsing, mocked API,
cost estimation, env factory
test_l1_pipeline.py — 6 tests covering the wrapper
Total new: 37 tests, all pass
Grand total quality+format: 264 tests passing (0 regression)
All 36 new tests + 111 L0 tests + 117 existing translator tests = 264
Phase 1 (observation) for 2 weeks. Then QUALITY_L1_LOG_ONLY=false
to enable auto-retry via the fallback chain.
237 lines
9.0 KiB
Plaintext
237 lines
9.0 KiB
Plaintext
# Document Translation API - Environment Configuration
|
|
# Copy this file to .env and configure your settings
|
|
# ⚠️ NEVER commit .env to version control!
|
|
#
|
|
# LEGEND (Story 6.6 - NFR10):
|
|
# - Required: App fails at startup with a clear message listing missing variables if not set (see .env.example or README).
|
|
# - Optional: Default value used when absent (documented below or in code). No fail-fast.
|
|
# - No real secrets in this file: use placeholders only (e.g. your_secret_here, pk_..., sk_..., whsec_...).
|
|
#
|
|
# Production (ENV=production or docker compose prod):
|
|
# Required: JWT_SECRET_KEY, ADMIN_USERNAME, (ADMIN_PASSWORD or ADMIN_PASSWORD_HASH), ADMIN_TOKEN_SECRET,
|
|
# DATABASE_URL (or POSTGRES_*), REDIS_URL if RATE_LIMIT_ENABLED=true.
|
|
# Optional: Provider keys, Stripe, CORS_ORIGINS, LOG_LEVEL, etc. (defaults or feature disabled).
|
|
# Development: Some required vars may get a default or warning; REDIS_URL optional if rate limiting disabled.
|
|
# See sections below for "Required" vs "Optional" and "If absent" behavior.
|
|
|
|
# ============== Translation Services ==============
|
|
# Optional. Default: google. If absent: default used.
|
|
TRANSLATION_SERVICE=google
|
|
# Optional. Provider enable/disable flags. If absent: default true/false per line below.
|
|
GOOGLE_TRANSLATE_ENABLED=true
|
|
GOOGLE_CLOUD_ENABLED=false
|
|
DEEPL_ENABLED=false
|
|
OPENAI_ENABLED=false
|
|
OLLAMA_ENABLED=false
|
|
OPENROUTER_ENABLED=false
|
|
|
|
# Provider fallback chain (comma-separated, in priority order)
|
|
PROVIDER_FALLBACK_CHAIN=google,deepl,openai,ollama,openrouter
|
|
|
|
# Mode-specific fallback chains
|
|
# Classic mode: Google Translate -> DeepL
|
|
FALLBACK_CHAIN_CLASSIC=google,deepl
|
|
|
|
# LLM mode: Ollama (local) -> OpenAI (cloud)
|
|
FALLBACK_CHAIN_LLM=ollama,openai
|
|
|
|
# Google Translate Configuration (GRATUIT via deep_translator — accès web non officiel)
|
|
# Pas de clé API requise. Limites non garanties (usage raisonnable recommandé).
|
|
GOOGLE_TRANSLATE_TIMEOUT=30
|
|
GOOGLE_TRANSLATE_MAX_RETRIES=3
|
|
GOOGLE_TRANSLATE_RETRY_DELAY=1
|
|
|
|
# Google Cloud Translation API v2 — version officielle et facturable
|
|
# Activer : console.cloud.google.com → APIs & Services → Cloud Translation API
|
|
# 500 000 caractères/mois GRATUITS, puis ~$20/million de caractères
|
|
GOOGLE_CLOUD_ENABLED=false
|
|
GOOGLE_CLOUD_API_KEY=
|
|
GOOGLE_CLOUD_TIMEOUT=30
|
|
GOOGLE_CLOUD_MAX_RETRIES=3
|
|
GOOGLE_CLOUD_RETRY_DELAY=1
|
|
|
|
# DeepL Configuration. Optional. If absent: provider disabled.
|
|
# Get API key from: https://www.deepl.com/pro-api. Free tier keys end with :fx.
|
|
DEEPL_API_KEY=
|
|
DEEPL_TIMEOUT=30
|
|
DEEPL_MAX_RETRIES=3
|
|
DEEPL_RETRY_DELAY=1
|
|
|
|
# OpenAI Configuration. Optional. If absent: provider disabled.
|
|
# Get API key from: https://platform.openai.com/api-keys
|
|
OPENAI_API_KEY=
|
|
OPENAI_MODEL=gpt-4o-mini
|
|
OPENAI_TIMEOUT=60
|
|
OPENAI_MAX_RETRIES=3
|
|
OPENAI_RETRY_DELAY=1.0
|
|
OPENAI_HEALTH_CHECK_TIMEOUT=5
|
|
# OPENAI_BASE_URL=https://api.openai.com/v1 # Optional: for Azure OpenAI or proxies
|
|
|
|
# OpenRouter Configuration. Optional. If absent: provider disabled.
|
|
OPENROUTER_API_KEY=
|
|
OPENROUTER_MODEL=deepseek/deepseek-chat
|
|
|
|
# Ollama Configuration. Optional. If absent: default http://localhost:11434 (provider may be disabled).
|
|
OLLAMA_BASE_URL=http://localhost:11434
|
|
OLLAMA_MODEL=llama3
|
|
OLLAMA_VISION_MODEL=llava
|
|
OLLAMA_TIMEOUT=120
|
|
OLLAMA_MAX_RETRIES=2
|
|
OLLAMA_RETRY_DELAY=2
|
|
|
|
# ============== File Limits ==============
|
|
# Maximum file size in MB
|
|
MAX_FILE_SIZE_MB=50
|
|
|
|
# ============== Rate Limiting (SaaS) ==============
|
|
# Enable/disable rate limiting
|
|
RATE_LIMIT_ENABLED=true
|
|
|
|
# Request limits
|
|
RATE_LIMIT_PER_MINUTE=30
|
|
RATE_LIMIT_PER_HOUR=200
|
|
|
|
# Translation-specific limits
|
|
TRANSLATIONS_PER_MINUTE=10
|
|
TRANSLATIONS_PER_HOUR=50
|
|
MAX_CONCURRENT_TRANSLATIONS=5
|
|
|
|
# ============== Quality Layer (L0) ==============
|
|
# Track A1 of the dev plan — observability only, no behavior change.
|
|
# When enabled, the L0 layer logs the script detection result for each
|
|
# translation job but does NOT modify the file or the job status.
|
|
# Default: false (opt-in). Set to "true" to enable.
|
|
QUALITY_L0_ENABLED=false
|
|
QUALITY_L0_SAMPLE_SIZE=20
|
|
|
|
# ============== Quality Layer (L1) ==============
|
|
# Track A3 of the dev plan — API-based LLM judge.
|
|
# Sends 5 sampled chunks per job to a cheap LLM (deepseek-chat by default)
|
|
# and logs the verdict (pass/fail) but does NOT modify the file or job
|
|
# status. Log-only by default for the first 2 weeks of observation.
|
|
# After that, set QUALITY_L1_LOG_ONLY=false to enable auto-retry.
|
|
#
|
|
# Cost: ~$0.0003 per job with deepseek-chat. ~$0.001 with gpt-4o-mini.
|
|
QUALITY_L1_ENABLED=false
|
|
QUALITY_L1_LOG_ONLY=true
|
|
QUALITY_L1_SAMPLE_SIZE=5
|
|
QUALITY_L1_MIN_CHUNKS=10
|
|
QUALITY_L1_TIMEOUT_SEC=8.0
|
|
|
|
# L1 judge configuration (any OpenAI-compatible endpoint).
|
|
# DeepSeek is the default (cheapest, ~$0.14/M input tokens).
|
|
L1_JUDGE_API_KEY=
|
|
L1_JUDGE_BASE_URL=https://api.deepseek.com/v1
|
|
L1_JUDGE_MODEL=deepseek-chat
|
|
# L1_JUDGE_MODEL=gpt-4o-mini
|
|
# L1_JUDGE_BASE_URL=https://api.openai.com/v1
|
|
# L1_JUDGE_MODEL=google/gemini-2.5-flash-lite
|
|
# L1_JUDGE_BASE_URL=https://openrouter.ai/api/v1
|
|
|
|
# ============== Cleanup Service ==============
|
|
# Enable automatic file cleanup
|
|
CLEANUP_ENABLED=true
|
|
|
|
# Cleanup interval in minutes. Optional. Default: 5.
|
|
CLEANUP_INTERVAL_MINUTES=15
|
|
|
|
# File time-to-live in minutes
|
|
FILE_TTL_MINUTES=60
|
|
INPUT_FILE_TTL_MINUTES=30
|
|
OUTPUT_FILE_TTL_MINUTES=120
|
|
|
|
# Disk space warning thresholds (GB)
|
|
DISK_WARNING_THRESHOLD_GB=5.0
|
|
DISK_CRITICAL_THRESHOLD_GB=1.0
|
|
|
|
# ============== Security ==============
|
|
# Enable HSTS (only for HTTPS deployments)
|
|
ENABLE_HSTS=false
|
|
|
|
# CORS allowed origins (comma-separated)
|
|
# ⚠️ IMPORTANT: Set to your actual frontend domain(s) in production!
|
|
# Example: https://yourdomain.com,https://www.yourdomain.com
|
|
# Use "*" ONLY for local development
|
|
# Dev: include every origin your Next.js dev server uses (e.g. 3000 and 3001). Non-production also auto-merges common localhost ports.
|
|
CORS_ORIGINS=http://localhost:3000,http://localhost:3001
|
|
|
|
# Maximum request size in MB
|
|
MAX_REQUEST_SIZE_MB=100
|
|
|
|
# Request timeout in seconds
|
|
REQUEST_TIMEOUT_SECONDS=300
|
|
|
|
# ============== Database ==============
|
|
# Required in production: set either DATABASE_URL or POSTGRES_* (POSTGRES_HOST, POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB, optional POSTGRES_PORT). If absent in dev: SQLite used (SQLITE_PATH).
|
|
# Format: postgresql://user:password@host:port/database. Example (no real secret):
|
|
# DATABASE_URL=postgresql://user:your_password@localhost:5432/translate_db
|
|
# DATABASE_URL=
|
|
|
|
# Optional. Default: data/translate.db. Used when DATABASE_URL is not set.
|
|
SQLITE_PATH=data/translate.db
|
|
|
|
# Enable SQL query logging (for debugging)
|
|
DATABASE_ECHO=false
|
|
|
|
# Required when RATE_LIMIT_ENABLED=true (production). Optional in dev (in-memory fallback if supported).
|
|
# If absent and rate limiting enabled: app fails at startup with clear message (Story 6.6).
|
|
# Example (no real secret): redis://localhost:6379/0
|
|
# REDIS_URL=redis://localhost:6379/0
|
|
|
|
# ============== Admin Authentication ==============
|
|
# Required for admin endpoints. If absent: app fails at startup (fail-fast) in production; see README.
|
|
ADMIN_USERNAME=admin
|
|
|
|
# Required (one of): ADMIN_PASSWORD_HASH (recommended prod) or ADMIN_PASSWORD. If both empty: fail-fast in production.
|
|
# Generate hash: python -c "import hashlib; print(hashlib.sha256(b'your_password').hexdigest())"
|
|
ADMIN_PASSWORD_HASH=
|
|
|
|
# Plain password for dev only. Use placeholder or leave empty; never put real secret in .env.example.
|
|
ADMIN_PASSWORD=your_admin_password_here
|
|
|
|
# Required for admin session tokens. If absent: random key at startup (sessions invalidated on restart). Example: use secrets.token_hex(32)
|
|
ADMIN_TOKEN_SECRET=
|
|
|
|
# ============== User Authentication ==============
|
|
# Required. JWT signing key. If absent: app fails at startup in production. Generate: secrets.token_urlsafe(64)
|
|
JWT_SECRET_KEY=
|
|
|
|
# Frontend URL for redirects
|
|
FRONTEND_URL=http://localhost:3000
|
|
|
|
# ============== Stripe Payments ==============
|
|
# Optional. If absent: payment features disabled. Placeholders only (pk_..., sk_..., whsec_...).
|
|
STRIPE_PUBLISHABLE_KEY=pk_test_...
|
|
STRIPE_SECRET_KEY=sk_test_...
|
|
STRIPE_WEBHOOK_SECRET=whsec_...
|
|
|
|
# Stripe Price IDs (create products in Stripe Dashboard)
|
|
# https://dashboard.stripe.com/products
|
|
STRIPE_PRICE_STARTER_MONTHLY=price_xxx
|
|
STRIPE_PRICE_STARTER_YEARLY=price_xxx
|
|
STRIPE_PRICE_PRO_MONTHLY=price_xxx
|
|
STRIPE_PRICE_PRO_YEARLY=price_xxx
|
|
STRIPE_PRICE_BUSINESS_MONTHLY=price_xxx
|
|
STRIPE_PRICE_BUSINESS_YEARLY=price_xxx
|
|
|
|
# ============== Monitoring ==============
|
|
# Optional. Default: admin. Grafana admin username.
|
|
GRAFANA_USER=admin
|
|
# Required when Grafana is enabled. If absent: Grafana may fail to start. Generate with: secrets.token_urlsafe(32)
|
|
GRAFANA_PASSWORD=
|
|
|
|
# Optional. Default: INFO. Log format: json or console (default from ENV/LOG_FORMAT).
|
|
LOG_LEVEL=INFO
|
|
LOG_FORMAT=json
|
|
# Optional. Set to "production" for production; influences logging and some security defaults.
|
|
# ENV=development
|
|
|
|
# ============== Frontend (Next.js) ==============
|
|
# Optional. Default: http://localhost:8000 (dev), http://backend:8000 (Docker). No secrets here (NFR10).
|
|
# NEXT_PUBLIC_API_URL=http://localhost:8000
|
|
|
|
# Enable request logging
|
|
ENABLE_REQUEST_LOGGING=true
|
|
|
|
# Memory usage threshold (percentage)
|
|
MAX_MEMORY_PERCENT=80 |