# Story 2.11: Progress Feedback Temps Reel

Status: done

## Story

As a **user**,
I want **to see real-time progress during translation**,
So that **I know the status of my translation**.

## Acceptance Criteria

1. **AC1: Progress Endpoint** - GET /api/v1/translations/{id} returns `{data: {id, status, progress_percent, current_step}, meta: {...}}`
2. **AC2: Real-time Latency** - Progress updates within 500ms of actual progress (NFR3)
3. **AC3: Status Values** - Status can be: "queued", "processing", "completed", "failed"
4. **AC4: LLM Progress Details** - For LLM mode, progress shows "translating slide X/Y" or "processing cell X/Y"
5. **AC5: Progress Percentage** - progress_percent field shows 0-100 based on completion
6. **AC6: Current Step** - current_step field describes what's being processed (e.g., "Validating file", "Translating sheet 1", "Generating output")
7. **AC7: Error State** - Failed translations return status "failed" with error_message in response
8. **AC8: Job Not Found** - Invalid job ID returns 404 with error "NOT_FOUND"

## Tasks / Subtasks

- [x] **Task 1: Extend Job Data Model** (AC: 1, 5, 6)
  - [x] 1.1 Add `progress_percent` (int 0-100) to job dict
  - [x] 1.2 Add `current_step` (str) to job dict
  - [x] 1.3 Add `total_items` and `processed_items` for granular tracking
  - [x] 1.4 Add `error_message` field for failed jobs

- [x] **Task 2: Create Progress Tracker Service** (AC: 2, 4)
  - [x] 2.1 Create `ProgressTracker` class with callback mechanism
  - [x] 2.2 Implement `update_progress(job_id, percent, step)` method
  - [x] 2.3 Support item-based progress: `update_item(job_id, current, total, item_name)`
  - [x] 2.4 Store progress in Redis or in-memory dict (consistent with existing pattern)

- [x] **Task 3: Update Translation Processors** (AC: 4)
  - [x] 3.1 Modify `excel_translator.py` to report progress per sheet/cell batch
  - [x] 3.2 Modify `word_translator.py` to report progress per paragraph/section
  - [x] 3.3 Modify `pptx_translator.py` to report progress per slide
  - [x] 3.4 Add `progress_callback` parameter to translator functions

- [x] **Task 4: Update Translation Job Processor** (AC: 2, 4)
  - [x] 4.1 Modify `_run_translation_job()` to use ProgressTracker
  - [x] 4.2 Set initial status "queued" before processing starts
  - [x] 4.3 Update to "processing" with progress during translation
  - [x] 4.4 Pass progress_callback to translators

- [x] **Task 5: Enhance Status Endpoint** (AC: 1, 3, 7, 8)
  - [x] 5.1 Update `GET /api/v1/translations/{job_id}` to include all new fields
  - [x] 5.2 Return proper 404 for non-existent jobs
  - [x] 5.3 Include error_message for failed status
  - [x] 5.4 Add response schema with all fields documented

- [x] **Task 6: Create Response Schemas** (AC: 1)
  - [x] 6.1 Create `TranslationStatusResponse` Pydantic model
  - [x] 6.2 Create `TranslationStatusData` with all progress fields
  - [x] 6.3 Create `TranslationStatusMeta` for metadata

- [x] **Task 7: Add Unit Tests** (AC: All)
  - [x] 7.1 Test progress updates are reflected correctly
  - [x] 7.2 Test status transitions (queued → processing → completed/failed)
  - [x] 7.3 Test 404 for non-existent job
  - [x] 7.4 Test error state includes error_message
  - [x] 7.5 Test LLM mode progress shows item details

- [x] **Task 8: Update OpenAPI Documentation** (AC: All)
  - [x] 8.1 Document all response fields
  - [x] 8.2 Add example responses for each status
  - [x] 8.3 Document error codes

## Dev Notes

### Previous Story Intelligence (Story 2.10)

**Existing Job Storage Pattern:**
```python
# In translate_routes.py
_translation_jobs: dict[str, dict] = {}

# Job structure (current):
_translation_jobs[job_id] = {
    "id": job_id,
    "status": "processing",
    "file_name": original_filename,
    "source_lang": source_lang,
    "target_lang": target_lang,
    "created_at": datetime.now(timezone.utc).isoformat(),
    "user_id": user_id,
    "input_path": str(input_path),
    "file_extension": file_extension,
    "provider": provider or mode,
    "webhook_url": webhook_url,
    "custom_prompt": custom_prompt,
    "glossary_id": glossary_id,
}
```

**Existing Status Endpoint (needs enhancement):**
```python
@router_v1.get("/translations/{job_id}")
async def get_translation_status(job_id: str, ...):
    job = _translation_jobs.get(job_id)
    if not job:
        return JSONResponse(status_code=404, ...)
    # Currently returns: id, status, file_name, source_lang, target_lang, created_at
    # NEEDS: progress_percent, current_step, error_message
```

**Background Processing Pattern:**
```python
asyncio.create_task(
    _run_translation_job(
        job_id=job_id,
        input_path=input_path,
        ...
    )
)
```

### Architecture Compliance

Per `_bmad-output/planning-artifacts/architecture.md`:

**Response Format (Success):**
```json
{
  "data": {
    "id": "tr_abc123",
    "status": "processing",
    "progress_percent": 45,
    "current_step": "Translating sheet 2/5",
    "file_name": "report.xlsx",
    "source_lang": "en",
    "target_lang": "fr",
    "created_at": "2024-01-15T10:30:00Z"
  },
  "meta": {
    "estimated_remaining_seconds": 8
  }
}
```

**Response Format (Completed):**
```json
{
  "data": {
    "id": "tr_abc123",
    "status": "completed",
    "progress_percent": 100,
    "current_step": "Translation complete",
    "file_name": "report.xlsx",
    "created_at": "2024-01-15T10:30:00Z",
    "completed_at": "2024-01-15T10:30:45Z"
  },
  "meta": {}
}
```

**Response Format (Failed):**
```json
{
  "data": {
    "id": "tr_abc123",
    "status": "failed",
    "progress_percent": 30,
    "current_step": "Error during translation",
    "error_message": "Provider unavailable: timeout after 30s",
    "file_name": "report.xlsx",
    "created_at": "2024-01-15T10:30:00Z",
    "failed_at": "2024-01-15T10:30:15Z"
  },
  "meta": {}
}
```

**Response Format (Not Found):**
```json
{
  "error": "NOT_FOUND",
  "message": "Job de traduction non trouve.",
  "details": {"job_id": "tr_nonexistent"}
}
```

### Naming Conventions (Per Architecture)

| Element | Convention | Example |
|---------|------------|---------|
| Variables | snake_case | `progress_percent`, `current_step` |
| JSON fields | snake_case | `progress_percent`, `current_step`, `error_message` |
| Classes | PascalCase | `ProgressTracker`, `TranslationStatusResponse` |
| Functions | snake_case | `update_progress()`, `get_translation_status()` |

### Implementation Approach

**ProgressTracker Class Design:**
```python
class ProgressTracker:
    """Track translation progress with callback support."""
    
    def __init__(self, job_id: str, storage: dict):
        self.job_id = job_id
        self.storage = storage  # Reference to _translation_jobs
    
    def update(self, percent: int, step: str) -> None:
        """Update progress percentage and current step."""
        job = self.storage.get(self.job_id)
        if job:
            job["progress_percent"] = min(100, max(0, percent))
            job["current_step"] = step
    
    def update_item(self, current: int, total: int, item_name: str) -> None:
        """Update progress based on item count (e.g., slides, sheets)."""
        percent = int((current / total) * 100) if total > 0 else 0
        step = f"{item_name} {current}/{total}"
        self.update(percent, step)
```

**Translator Integration Pattern:**
```python
# In translators/excel_translator.py
def translate_file(
    self, 
    input_path: Path, 
    output_path: Path, 
    target_lang: str, 
    source_lang: str = "auto",
    progress_callback: Optional[Callable[[int, str], None]] = None
) -> None:
    # ... validation ...
    
    total_sheets = len(workbook.sheetnames)
    for idx, sheet_name in enumerate(workbook.sheetnames):
        # Process sheet...
        if progress_callback:
            progress = int((idx + 1) / total_sheets * 100)
            progress_callback(progress, f"Translating sheet {idx + 1}/{total_sheets}")
```

### Project Structure Notes

**Files to Modify:**
- `routes/translate_routes.py` - Update job storage, status endpoint
- `translators/excel_translator.py` - Add progress callback
- `translators/word_translator.py` - Add progress callback
- `translators/pptx_translator.py` - Add progress callback

**Files to Create:**
- `services/progress_tracker.py` - ProgressTracker class (optional, can be inline)

### Git Intelligence - Recent Patterns

From recent commits:
- Translation cache implemented (5000 entry LRU cache)
- OpenRouter provider with DeepSeek support
- Parallel processing optimizations
- Redis sessions for production

### Dependencies on Previous Stories

| Story | Dependency |
|-------|------------|
| 2.10 | Job storage pattern (`_translation_jobs` dict) |
| 2.7 | ExcelProcessor structure for progress hooks |
| 2.8 | WordProcessor structure for progress hooks |
| 2.9 | PowerPointProcessor structure for progress hooks |

### Anti-Patterns to Avoid

1. **Don't use blocking calls** - Progress updates must be fast, non-blocking
2. **Don't exceed 500ms latency** - NFR3 requirement
3. **Don't log file content** - Only progress metadata (NFR11, NFR16)
4. **Don't break existing API contract** - Extend, don't modify existing response fields
5. **Don't create tight coupling** - Progress callback should be optional parameter

### Performance Considerations

- Progress updates should be O(1) - simple dict updates
- Avoid polling Redis on every progress update if using Redis
- Consider throttling updates (max 10 per second) for high-frequency operations
- Use in-memory dict for MVP (already established pattern in Story 2.10)

### Testing Strategy

```bash
# Unit tests
pytest tests/test_progress_tracking.py -v

# Integration tests
pytest tests/test_translate_endpoint.py -v -k "progress"

# With coverage
pytest tests/ --cov=routes/translate_routes --cov=translators -v
```

### References

- [Source: _bmad-output/planning-artifacts/epics.md#Story 2.11]
- [Source: _bmad-output/planning-artifacts/architecture.md#API Response Formats]
- [Source: _bmad-output/planning-artifacts/prd.md#FR47 Real-time progress]
- [Source: _bmad-output/planning-artifacts/prd.md#NFR3 Progress latency < 500ms]
- [Source: _bmad-output/implementation-artifacts/2-10-endpoint-post-api-v1-translate-core.md - Previous story]
- [Source: routes/translate_routes.py - Current job storage and status endpoint]
- [Source: services/translation_service.py - Translation batch processing pattern]

## Dev Agent Record

### Agent Model Used

glm-5

### Debug Log References

None

### Completion Notes List

1. **Task 1 Complete**: Extended job data model with progress_percent (0-100), current_step (str), total_items, processed_items, and error_message fields. Job status now starts as "queued" and transitions to "processing" when work begins.

2. **Task 2 Complete**: Created `services/progress_tracker.py` with ProgressTracker class. Includes `update()`, `update_item()`, `set_error()`, and `set_completed()` methods. Thread-safe with throttling (50ms min interval) to meet NFR3 <500ms latency requirement.

3. **Task 3 Complete**: All translators (excel, word, pptx) already have progress_callback support. Updated `_run_translation_job()` to create and pass progress callback to translators.

4. **Task 4 Complete**: Modified `_run_translation_job()` to use ProgressTracker. Sets initial status "queued", transitions to "processing", and passes progress_callback to translators. Uses `set_completed()` and `set_error()` for final states.

5. **Task 5 Complete**: Enhanced `GET /api/v1/translations/{job_id}` to include progress_percent, current_step, and error_message fields. Returns 404 for non-existent jobs with proper error format.

6. **Task 6 Complete**: Created Pydantic response schemas: TranslationStatusData, TranslationStatusMeta, TranslationStatusResponse with all progress fields documented.

7. **Task 7 Complete**: Created `tests/test_progress_tracking.py` with 18 comprehensive tests covering ProgressTracker class, job data model, status endpoint behavior, status transitions, and LLM progress details. All 18 tests pass.

8. **Task 8 Complete**: Added comprehensive OpenAPI documentation in the status endpoint docstring with example responses for each status (processing, completed, failed, not found).

### File List

**Created files:**
- `services/progress_tracker.py` - ProgressTracker class with update(), update_item(), set_error(), set_completed() methods
- `tests/test_progress_tracking.py` - 18 unit tests for progress tracking

**Modified files:**
- `routes/translate_routes.py` - Added progress fields to job creation, integrated ProgressTracker, enhanced status endpoint with new fields, added response schemas, job cleanup mechanism, estimated_remaining_seconds calculation
- `translators/excel_translator.py` - Standardized progress callback keys (current, total)
- `translators/word_translator.py` - Standardized progress callback keys, added granular progress during translation phase
- `translators/pptx_translator.py` - Standardized progress callback keys (current, total)

## Change Log

- 2026-02-21: Code Review Fixes - Fixed race condition in ProgressTracker throttling, added job cleanup mechanism (TTL 1h), implemented estimated_remaining_seconds calculation, standardized progress callback keys across all translators, added granular progress for Word translation, fixed type hints
- 2026-02-21: Implemented Story 2.11 - Real-time progress feedback with ProgressTracker service, extended job data model, enhanced status endpoint, comprehensive tests (18/18 passing)