feat(translation): quality pipeline overhaul + new features (audit 2026-08-29)
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m20s
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 2m20s
Translation quality & format preservation: - Word: merge adjacent same-format runs into one unit (sentence-level coherence like inline-tag handling); translate comments/balloons; dedupe textbox collection (was translated twice); RTL no longer overrides center/justify alignment; CJK/Arabic font hints (eastAsia/cs) - PPTX: chart translations now actually reach the output file (ChartPart.blob is read-only — rewrite chart XML in the saved ZIP); CJK typeface hints (a:ea) - Excel: sheet renames no longer break references — rewrite cell formulas (3D/quoted), defined names, data validations, cond. formats - PDF: bold/italic honored (hebo/heit/hebi); table cells never merge; unchanged blocks left untouched (typography preserved, fixes duplicate hyperlinks); attempted/changed stats + route gate now cover PDF; CJK font paths; scanned PDFs via Mistral OCR (detection + admin settings) Features: - formality param (formal/informal) + automatic regional-variant prompts - output_mode=bilingual docx (source above translation) - per-user translation memory on Redis (falls back to LRU), context-hashed - QA report + 0-100 confidence score in job status; L0 on by default - OpenAI-compatible providers: whole chunk in ONE numbered-JSON request (~15x fewer calls) with per-item fallback; base prompt always present (custom prompt no longer replaces translation instructions) Infra & marketing alignment: - plan-based engine gating + vision gating (closes paid-engine leak); /providers/available filtered per plan; 107 languages exposed - zh-CN/zh-TW validation fixed; libmagic disabled on Windows (native crash) - admin: Mistral OCR settings + engine status dashboard; httpx<0.28 pin (TestClient breakage); Prometheus test fixture fixed - marketing docs aligned with code (PDF+OCR, retention, engines, pricing) - security: .env.ionos/.env.production/provider_settings.json removed Tests: 1173 passed / 0 failed (6 network tests deselected: free Google endpoint temporarily blocked from this machine)
This commit is contained in:
@@ -179,3 +179,41 @@ class FileHandler:
|
||||
|
||||
# Global file handler instance
|
||||
file_handler = FileHandler()
|
||||
|
||||
|
||||
def validate_zip_safety(
|
||||
file_path: Path,
|
||||
max_compression_ratio: float = 100.0,
|
||||
max_total_uncompressed_mb: int = 1024,
|
||||
) -> None:
|
||||
"""Reject ZIP-based documents that expand dangerously (zip bombs).
|
||||
|
||||
Office files (.xlsx/.docx/.pptx) are ZIP archives: a small uploaded file
|
||||
can decompress to gigabytes in memory. Raises ValueError when the file
|
||||
is not a readable archive, expands beyond the allowed ratio, or when the
|
||||
total uncompressed size exceeds the cap.
|
||||
"""
|
||||
import zipfile
|
||||
|
||||
max_total_bytes = max_total_uncompressed_mb * 1024 * 1024
|
||||
try:
|
||||
with zipfile.ZipFile(file_path) as zf:
|
||||
total_uncompressed = 0
|
||||
for info in zf.infolist():
|
||||
if info.is_dir():
|
||||
continue
|
||||
total_uncompressed += info.file_size
|
||||
if (
|
||||
info.compress_size > 0
|
||||
and info.file_size / info.compress_size > max_compression_ratio
|
||||
):
|
||||
raise ValueError(
|
||||
f"Entry '{info.filename}' expands more than "
|
||||
f"{int(max_compression_ratio)}x its compressed size."
|
||||
)
|
||||
if total_uncompressed > max_total_bytes:
|
||||
raise ValueError(
|
||||
f"Archive expands beyond {max_total_uncompressed_mb} MB."
|
||||
)
|
||||
except zipfile.BadZipFile as e:
|
||||
raise ValueError(f"Not a valid ZIP-based document: {e}")
|
||||
|
||||
Reference in New Issue
Block a user