perf+security: fix build, secure downloads, dedupe translations, refactor i18n
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m49s

Frontend:
- Fix Framer Motion / motion-dom build error by pinning framer-motion to
  11.18.2 (compatible with React 19 and Next.js 16).
- Add cross-env and build:local script to bypass standalone symlink errors
  on Windows without Developer Mode.
- Allow NEXT_OUTPUT=default to disable standalone output for local builds.
- Refactor i18n: split 14,177-line src/lib/i18n.tsx into per-locale,
  per-namespace JSON files under src/lib/i18n/messages/.
- Load English synchronously; other locales loaded on demand via dynamic
  imports (reduces initial bundle, improves maintainability).
- Remove unused next-intl message files src/messages/en.json and fr.json.

Backend:
- Remove insecure legacy /api/v1/download/{filename} and /api/v1/cleanup/{filename}
  endpoints. The job-based /api/v1/download/{job_id} already enforces ownership.
- Deduplicate texts in TranslationService.translate_batch before sending them
  to the provider, reducing API calls for repeated strings.
- Pin httpx to <0.28 to fix TestClient incompatibility with starlette 0.35.1.
- Add pytest-cov and ruff dev dependencies/config.

DevOps:
- Remove hardcoded Grafana password from docker-compose.yml and
  docker-compose.monitoring.yml; use GRAFANA_PASSWORD env var.
- Change default TRANSLATION_SERVICE from ollama to google in
  docker-compose.yml (Ollama is an optional profile).
- Add GRAFANA_PASSWORD to .env.example.
- Add .coverage and frontend/pnpm-workspace.yaml to .gitignore.

Tests:
- Update API versioning tests for removed legacy endpoints.
- Add tests/test_translation_service.py for deduplication behavior.

Verified:
- pnpm run build:local passes.
- uv run pytest tests/test_providers/* tests/test_translation_service.py
  tests/test_story_3_5_api_versioning.py tests/test_download_endpoint.py
  tests/test_translators/test_excel_translator.py: provider/translator tests
  pass; one pre-existing French error-message test still fails (message is
  returned in English, unrelated to this change).
This commit is contained in:
2026-06-14 16:44:18 +02:00
parent eda6821632
commit fa637abff0
365 changed files with 23910 additions and 14955 deletions

View File

@@ -1207,28 +1207,55 @@ class TranslationService:
"""
Translate multiple text strings efficiently using batch processing.
Deduplicates non-empty texts before sending them to the provider so that
repeated strings (common in documents with headers, formulas, tables)
are translated only once. This reduces API calls, cost and latency.
Args:
texts: List of texts to translate
target_language: Target language code
source_language: Source language code (default: 'auto')
Returns:
List of translated texts
List of translated texts in the same order as input
"""
if not texts:
return []
# Use provider's batch method if available
if hasattr(self.provider, "translate_batch"):
return self.provider.translate_batch(
texts, target_language, source_language
)
# Preserve original positions and filter out empty/whitespace-only texts
results: list[str] = [""] * len(texts)
unique_texts: list[str] = []
text_to_positions: dict[str, list[int]] = {}
# Fallback to individual translations
return [
self.translate_text(text, target_language, source_language)
for text in texts
]
for idx, text in enumerate(texts):
if not text or not text.strip():
results[idx] = text if text else ""
continue
if text not in text_to_positions:
text_to_positions[text] = []
unique_texts.append(text)
text_to_positions[text].append(idx)
if not unique_texts:
return results
# Translate unique texts only
if hasattr(self.provider, "translate_batch"):
translated_unique = self.provider.translate_batch(
unique_texts, target_language, source_language
)
else:
translated_unique = [
self.translate_text(text, target_language, source_language)
for text in unique_texts
]
# Map translated texts back to their original positions
for original, translated in zip(unique_texts, translated_unique):
for idx in text_to_positions[original]:
results[idx] = translated if translated else original
return results
# Global translation service instance