sepehr/office_translator
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat: Add admin dashboard with authentication - Admin login/logout with Bearer token authentication - Secure admin dashboard page in frontend - Real-time system monitoring (memory, disk, translations) - Rate limits and cleanup service monitoring - Protected admin endpoints - Updated README with full SaaS documentation

Browse Source
This commit is contained in:
Sepehr
2025-11-30 19:33:59 +01:00
parent 500502440c
commit 54d85f0b34
5 changed files with 845 additions and 86 deletions
Download Patch File Download Diff File Expand all files Collapse all files

271
README.md
View File

@@ -1,6 +1,6 @@
# 📄 Document Translation API
A powerful Python API for translating complex structured documents (Excel, Word, PowerPoint) while **strictly preserving** the original formatting, layout, and embedded media.
A powerful SaaS-ready Python API for translating complex structured documents (Excel, Word, PowerPoint) while **strictly preserving** the original formatting, layout, and embedded media.
## ✨ Features
@@ -12,6 +12,7 @@ A powerful Python API for translating complex structured documents (Excel, Word,
| **WebLLM** | Browser | Runs entirely in browser using WebGPU |
| **DeepL** | Cloud | High-quality translations (API key required) |
| **LibreTranslate** | Self-hosted | Open-source alternative |
| **OpenAI** | Cloud | GPT-4o/4o-mini with vision support |
### 📊 Excel Translation (.xlsx)
- ✅ Translates all cell content and sheet names
@@ -32,11 +33,19 @@ A powerful Python API for translating complex structured documents (Excel, Word,
- ✅ Image text extraction with text boxes added below images
- ✅ Keeps layering order and positions
### 🧠 LLM Features (Ollama/WebLLM)
### 🧠 LLM Features (Ollama/WebLLM/OpenAI)
- ✅ **Custom System Prompts**: Provide context for better translations
- ✅ **Technical Glossary**: Define term mappings (e.g., `batterie=coil`)
- ✅ **Presets**: HVAC, IT, Legal, Medical terminology
- ✅ **Vision Models**: Translate text within images (gemma3, qwen3-vl, llava)
- ✅ **Vision Models**: Translate text within images (gemma3, qwen3-vl, gpt-4o)
### 🏢 SaaS-Ready Features
- 🚦 **Rate Limiting**: Per-client IP with token bucket and sliding window algorithms
- 🔒 **Security Headers**: CSP, XSS protection, HSTS support
- 🧹 **Auto Cleanup**: Automatic file cleanup with TTL tracking
- 📊 **Monitoring**: Health checks, metrics, and system status
- 🔐 **Admin Dashboard**: Secure admin panel with authentication
- 📝 **Request Logging**: Structured logging with unique request IDs
## 🚀 Quick Start
@@ -60,9 +69,15 @@ python main.py
The API starts on `http://localhost:8000`
### Web Interface
### Frontend Setup
Open `http://localhost:8000/static/index.html` for the full-featured web interface.
```powershell
cd frontend
npm install
npm run dev
```
Frontend runs on `http://localhost:3000`
## 📚 API Documentation
@@ -71,7 +86,9 @@ Open `http://localhost:8000/static/index.html` for the full-featured web interfa
## 🔧 API Endpoints
### POST /translate
### Translation
#### POST /translate
Translate a document with full customization.
```bash
@@ -81,28 +98,63 @@ curl -X POST "http://localhost:8000/translate" \
-F "provider=ollama" \
-F "ollama_model=gemma3:12b" \
-F "translate_images=true" \
-F "system_prompt=You are translating HVAC documents. Use: batterie=coil, CTA=AHU"
-F "system_prompt=You are translating HVAC documents."
```
### Parameters
### Monitoring
| Parameter | Type | Default | Description |
|-----------|------|---------|-------------|
| `file` | File | required | Document to translate (.xlsx, .docx, .pptx) |
| `target_language` | string | required | Target language code (en, fr, es, fa, etc.) |
| `provider` | string | google | Translation provider (google, ollama, webllm, deepl, libre) |
| `ollama_model` | string | llama3.2 | Ollama model name |
| `translate_images` | bool | false | Extract and translate image text with vision |
| `system_prompt` | string | "" | Custom instructions and glossary for LLM |
#### GET /health
Comprehensive health check with system status.
### GET /ollama/models
List available Ollama models.
```json
{
"status": "healthy",
"translation_service": "google",
"memory": {"system_percent": 34.1, "system_available_gb": 61.7},
"disk": {"total_files": 0, "total_size_mb": 0},
"cleanup_service": {"is_running": true}
}
```
### POST /ollama/configure
Configure Ollama settings.
#### GET /metrics
System metrics and statistics.
### GET /health
Health check endpoint.
#### GET /rate-limit/status
Current rate limit status for the requesting client.
### Admin Endpoints (Authentication Required)
#### POST /admin/login
Login to admin dashboard.
```bash
curl -X POST "http://localhost:8000/admin/login" \
-F "username=admin" \
-F "password=your_password"
```
Response:
```json
{
"status": "success",
"token": "your_bearer_token",
"expires_in": 86400
}
```
#### GET /admin/dashboard
Get comprehensive dashboard data (requires Bearer token).
```bash
curl "http://localhost:8000/admin/dashboard" \
-H "Authorization: Bearer your_token"
```
#### POST /admin/cleanup/trigger
Manually trigger file cleanup.
#### GET /admin/files/tracked
List currently tracked files.
## 🌐 Supported Languages
@@ -120,22 +172,47 @@ Health check endpoint.
### Environment Variables (.env)
```env
# Translation Service
# ============== Translation Services ==============
TRANSLATION_SERVICE=google
DEEPL_API_KEY=your_deepl_api_key_here
# Ollama Configuration
OLLAMA_BASE_URL=http://localhost:11434
OLLAMA_MODEL=llama3.2
OLLAMA_MODEL=llama3
OLLAMA_VISION_MODEL=llava
# DeepL API Key (optional)
DEEPL_API_KEY=your_api_key_here
# File Limits
# ============== File Limits ==============
MAX_FILE_SIZE_MB=50
# Directories
UPLOAD_DIR=./uploads
OUTPUT_DIR=./outputs
# ============== Rate Limiting (SaaS) ==============
RATE_LIMIT_ENABLED=true
RATE_LIMIT_PER_MINUTE=30
RATE_LIMIT_PER_HOUR=200
TRANSLATIONS_PER_MINUTE=10
TRANSLATIONS_PER_HOUR=50
MAX_CONCURRENT_TRANSLATIONS=5
# ============== Cleanup Service ==============
CLEANUP_ENABLED=true
CLEANUP_INTERVAL_MINUTES=15
FILE_TTL_MINUTES=60
INPUT_FILE_TTL_MINUTES=30
OUTPUT_FILE_TTL_MINUTES=120
# ============== Security ==============
ENABLE_HSTS=false
CORS_ORIGINS=*
# ============== Admin Authentication ==============
ADMIN_USERNAME=admin
ADMIN_PASSWORD=changeme123 # Change in production!
# Or use a SHA256 hash:
# ADMIN_PASSWORD_HASH=your_sha256_hash
# ============== Monitoring ==============
LOG_LEVEL=INFO
ENABLE_REQUEST_LOGGING=true
MAX_MEMORY_PERCENT=80
```
### Ollama Setup
@@ -179,6 +256,73 @@ vanne 3 voies=3-way valve
- ⚖️ **Legal**: Legal documents
- 🏥 **Medical**: Healthcare terminology
## <20> Admin Dashboard
Access the admin dashboard at `/admin` in the frontend. Features:
- **System Status**: Health, uptime, and issues
- **Memory & Disk Monitoring**: Real-time usage stats
- **Translation Statistics**: Total translations, success rate
- **Rate Limit Management**: View active clients and limits
- **Cleanup Service**: Monitor and trigger manual cleanup
### Default Credentials
- **Username**: admin
- **Password**: changeme123
⚠️ **Change the default password in production!**
## 🏗️ Project Structure
```
Translate/
├── main.py # FastAPI application with SaaS features
├── config.py # Configuration with SaaS settings
├── requirements.txt # Dependencies
├── mcp_server.py # MCP server implementation
├── middleware/ # SaaS middleware
│ ├── __init__.py
│ ├── rate_limiting.py # Rate limiting with token bucket
│ ├── validation.py # Input validation
│ ├── security.py # Security headers & logging
│ └── cleanup.py # Auto cleanup service
├── services/
│ └── translation_service.py # Translation providers
├── translators/
│ ├── excel_translator.py # Excel with image support
│ ├── word_translator.py # Word with image support
│ └── pptx_translator.py # PowerPoint with image support
├── frontend/ # Next.js frontend
│ ├── src/
│ │ ├── app/
│ │ │ ├── page.tsx # Main translation page
│ │ │ ├── admin/ # Admin dashboard
│ │ │ └── settings/ # Settings pages
│ │ └── components/
│ └── package.json
├── static/
│ └── webllm.html # WebLLM standalone interface
├── uploads/ # Temporary uploads (auto-cleaned)
└── outputs/ # Translated files (auto-cleaned)
```
## 🛠️ Tech Stack
### Backend
- **FastAPI**: Modern async web framework
- **openpyxl**: Excel manipulation
- **python-docx**: Word documents
- **python-pptx**: PowerPoint presentations
- **deep-translator**: Google/DeepL/Libre translation
- **psutil**: System monitoring
- **python-magic**: File type validation
### Frontend
- **Next.js 15**: React framework
- **Tailwind CSS**: Styling
- **Lucide Icons**: Icon library
- **WebLLM**: Browser-based LLM
## 🔌 MCP Integration
This API can be used as an MCP (Model Context Protocol) server for AI assistants.
@@ -203,56 +347,27 @@ Add to your VS Code `settings.json` or `.vscode/mcp.json`:
}
```
### MCP Tools Available
## 🚀 Production Deployment
| Tool | Description |
|------|-------------|
| `translate_document` | Translate a document file |
| `list_ollama_models` | Get available Ollama models |
| `get_supported_languages` | List supported language codes |
| `configure_translation` | Set translation provider and options |
### Security Checklist
- [ ] Change `ADMIN_PASSWORD` or set `ADMIN_PASSWORD_HASH`
- [ ] Set `CORS_ORIGINS` to your frontend domain
- [ ] Enable `ENABLE_HSTS=true` if using HTTPS
- [ ] Configure rate limits appropriately
- [ ] Set up log rotation for `logs/` directory
- [ ] Use a reverse proxy (nginx/traefik) for HTTPS
## 🏗️ Project Structure
### Docker Deployment (Coming Soon)
```dockerfile
FROM python:3.11-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install -r requirements.txt
COPY . .
EXPOSE 8000
CMD ["uvicorn", "main:app", "--host", "0.0.0.0", "--port", "8000"]
```
Translate/
├── main.py # FastAPI application
├── config.py # Configuration
├── requirements.txt # Dependencies
├── mcp_server.py # MCP server implementation
├── services/
│ └── translation_service.py # Translation providers
├── translators/
│ ├── excel_translator.py # Excel with image support
│ ├── word_translator.py # Word with image support
│ └── pptx_translator.py # PowerPoint with image support
├── utils/
│ ├── file_handler.py # File operations
│ └── exceptions.py # Custom exceptions
├── static/
│ └── index.html # Web interface
├── uploads/ # Temporary uploads
└── outputs/ # Translated files
```
## 🧪 Testing
1. Start the API: `python main.py`
2. Open: http://localhost:8000/static/index.html
3. Configure Ollama model
4. Upload a document
5. Select target language and provider
6. Click "Translate Document"
## 🛠️ Tech Stack
- **FastAPI**: Modern async web framework
- **openpyxl**: Excel manipulation
- **python-docx**: Word documents
- **python-pptx**: PowerPoint presentations
- **deep-translator**: Google/DeepL/Libre translation
- **requests**: Ollama API communication
- **Uvicorn**: ASGI server
## 📝 License
@@ -264,4 +379,4 @@ Contributions welcome! Please submit a Pull Request.
---
**Built with ❤️ using Python, FastAPI, and Ollama**
**Built with ❤️ using Python, FastAPI, Next.js, and Ollama**