feat(translation): quality pipeline overhaul + new features (audit 2026-08-29)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m20s

Translation quality & format preservation:
- Word: merge adjacent same-format runs into one unit (sentence-level
  coherence like inline-tag handling); translate comments/balloons;
  dedupe textbox collection (was translated twice); RTL no longer
  overrides center/justify alignment; CJK/Arabic font hints (eastAsia/cs)
- PPTX: chart translations now actually reach the output file
  (ChartPart.blob is read-only — rewrite chart XML in the saved ZIP);
  CJK typeface hints (a:ea)
- Excel: sheet renames no longer break references — rewrite cell
  formulas (3D/quoted), defined names, data validations, cond. formats
- PDF: bold/italic honored (hebo/heit/hebi); table cells never merge;
  unchanged blocks left untouched (typography preserved, fixes duplicate
  hyperlinks); attempted/changed stats + route gate now cover PDF;
  CJK font paths; scanned PDFs via Mistral OCR (detection + admin settings)

Features:
- formality param (formal/informal) + automatic regional-variant prompts
- output_mode=bilingual docx (source above translation)
- per-user translation memory on Redis (falls back to LRU), context-hashed
- QA report + 0-100 confidence score in job status; L0 on by default
- OpenAI-compatible providers: whole chunk in ONE numbered-JSON request
  (~15x fewer calls) with per-item fallback; base prompt always present
  (custom prompt no longer replaces translation instructions)

Infra & marketing alignment:
- plan-based engine gating + vision gating (closes paid-engine leak);
  /providers/available filtered per plan; 107 languages exposed
- zh-CN/zh-TW validation fixed; libmagic disabled on Windows (native crash)
- admin: Mistral OCR settings + engine status dashboard; httpx<0.28 pin
  (TestClient breakage); Prometheus test fixture fixed
- marketing docs aligned with code (PDF+OCR, retention, engines, pricing)
- security: .env.ionos/.env.production/provider_settings.json removed

Tests: 1173 passed / 0 failed (6 network tests deselected: free Google
endpoint temporarily blocked from this machine)
This commit is contained in:
2026-08-29 18:38:09 +02:00
parent 992f13d53c
commit 526c87348f
87 changed files with 6996 additions and 1024 deletions

86
tests/test_plan_gating.py Normal file
View File

@@ -0,0 +1,86 @@
"""Plan-based engine gating and the expanded /languages endpoint."""
import pytest
from models.subscription import PlanType
from routes.translate_routes import (
_allowed_providers_for_plan,
_image_translation_allowed_for_plan,
_plan_from_user,
)
class _FakeUser:
def __init__(self, plan):
self.plan = plan
class TestAllowedProviders:
def test_free_gets_google_only(self):
assert _allowed_providers_for_plan(PlanType.FREE) == {"google"}
def test_starter_adds_deepl(self):
assert _allowed_providers_for_plan(PlanType.STARTER) == {"google", "deepl"}
def test_pro_adds_cloud_and_openrouter(self):
allowed = _allowed_providers_for_plan(PlanType.PRO)
assert {"google_cloud", "openrouter"} <= allowed
assert "openai" not in allowed
def test_business_adds_premium_and_xai(self):
allowed = _allowed_providers_for_plan(PlanType.BUSINESS)
assert {"openrouter_premium", "openai", "zai"} <= allowed
def test_enterprise_has_all(self):
enterprise = _allowed_providers_for_plan(PlanType.ENTERPRISE)
assert enterprise >= _allowed_providers_for_plan(PlanType.BUSINESS)
class TestImageTranslationGate:
@pytest.mark.parametrize("plan", [PlanType.FREE, PlanType.STARTER])
def test_refused_below_pro(self, plan):
assert _image_translation_allowed_for_plan(plan) is False
@pytest.mark.parametrize("plan", [PlanType.PRO, PlanType.BUSINESS, PlanType.ENTERPRISE])
def test_allowed_from_pro(self, plan):
assert _image_translation_allowed_for_plan(plan) is True
class TestPlanFromUser:
def test_anonymous_is_free(self):
assert _plan_from_user(None) is PlanType.FREE
def test_enum_plan_passthrough(self):
assert _plan_from_user(_FakeUser(PlanType.PRO)) is PlanType.PRO
def test_string_plan_accepted(self):
assert _plan_from_user(_FakeUser("pro")) is PlanType.PRO
def test_garbage_falls_back_to_free(self):
assert _plan_from_user(_FakeUser("nonsense")) is PlanType.FREE
class TestLanguagesEndpoint:
@pytest.mark.asyncio
async def test_exposes_at_least_60_languages(self):
from routes.legacy_routes import get_supported_languages
response = await get_supported_languages()
langs = response["supported_languages"]
assert response["count"] >= 60
assert len(langs) >= 60
@pytest.mark.asyncio
async def test_no_auto_and_canonical_chinese(self):
from routes.legacy_routes import get_supported_languages
langs = (await get_supported_languages())["supported_languages"]
assert "auto" not in langs
assert "zh-CN" in langs and "zh-TW" in langs
@pytest.mark.asyncio
async def test_every_language_has_a_name(self):
from routes.legacy_routes import get_supported_languages
langs = (await get_supported_languages())["supported_languages"]
assert all(name and name != code.upper() for code, name in langs.items())