Files
office_translator/_bmad-output/implementation-artifacts/1-1-setup-base-de-donnees-modele-user.md
Sepehr Ramezani 26bd096a06 feat: production deployment - full update with providers, admin, glossaries, pricing, tests
Major changes across backend, frontend, infrastructure:
- Provider system with model selection (Google, DeepL, OpenAI, Ollama, Google Cloud)
- Admin panel: user management, pricing, settings
- Glossary system with CSV import/export
- Subscription and tier quota management
- Security hardening (rate limiting, API key auth, path traversal fixes)
- Docker compose for dev, prod, and IONOS deployment
- Alembic migrations for new tables
- Frontend: dashboard, pricing page, landing page, i18n (en/fr)
- Test suite and verification scripts

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-04-25 15:01:47 +02:00

253 lines
11 KiB
Markdown

# Story 1.1: Setup Base de Données & Modèle User
Status: done
## Story
As a **Developer**,
I want **setup the database with Alembic (async) and refactor the User model with tier field**,
so that **the application can persist user data with simplified subscription management and async support**.
## Acceptance Criteria
1. **AC1: User Model Refactored** - `users` table has columns: id (UUID), email, hashed_password, tier (default "free"), daily_translation_count, created_at, updated_at
2. **AC2: Async SQLAlchemy** - Database connection uses async engine (SQLAlchemy 2.0 async)
3. **AC3: Alembic Async** - Alembic configured for async migrations
4. **AC4: Dual Database Support** - PostgreSQL works in production, SQLite in development
5. **AC5: Secrets from Environment** - All secrets loaded from environment variables (NFR10)
## Tasks / Subtasks
- [x] **Task 1: Refactor User Model** (AC: 1, 5)
- [x] 1.1 Add `tier` field (str: "free" | "pro") with default "free"
- [x] 1.2 Add `daily_translation_count` field (int, default 0)
- [x] 1.3 Rename `password_hash``hashed_password` for consistency with architecture
- [x] 1.4 Keep existing fields for backward compatibility (plan, stripe fields) but mark deprecated
- [x] 1.5 Add UUID column type (from String(36) to proper UUID) — DEFERRED: requires coordinated PK+FK migration across all tables (users, translations, api_keys, usage_logs, payment_history). Created as separate task in next sprint.
- [x] **Task 2: Convert to Async SQLAlchemy** (AC: 2, 4)
- [x] 2.1 Replace `create_engine` with `create_async_engine` in `database/connection.py`
- [x] 2.2 Replace `sessionmaker` with `AsyncSession` and `async_sessionmaker`
- [x] 2.3 Update `get_db()` to async generator with `AsyncSession`
- [x] 2.4 Keep SQLite support for development (with aiosqlite driver)
- [x] 2.5 Keep PostgreSQL support (with asyncpg driver)
- [x] **Task 3: Configure Alembic for Async** (AC: 3, 4)
- [x] 3.1 Update `alembic/env.py` for async engine support
- [x] 3.2 Add `asyncio.run()` wrapper for online migrations
- [x] 3.3 Ensure offline migrations still work
- [x] **Task 4: Create Migration** (AC: 1)
- [x] 4.1 Create migration `alembic/versions/002_add_tier_daily_count.py`
- [x] 4.2 Add `tier` column with default "free"
- [x] 4.3 Add `daily_translation_count` column with default 0
- [x] 4.4 Add `hashed_password` column (migration from password_hash)
- [x] 4.5 Data migration: set tier from existing plan field
- [x] **Task 5: Update Dependencies** (AC: 2, 4)
- [x] 5.1 Add `aiosqlite` for async SQLite support
- [x] 5.2 Add `asyncpg` for async PostgreSQL support
- [x] 5.3 Update `requirements.txt`
- [x] **Task 6: Verify Setup** (AC: 1-5)
- [x] 6.1 Test migration with SQLite: `alembic upgrade head`
- [x] 6.2 Test migration downgrade: `alembic downgrade -1`
- [x] 6.3 Verify User model can create/read with new fields
## Dev Notes
### 🚨 BROWNFIELD CONTEXT - CRITICAL
This is a **refactoring story**, NOT a fresh setup. The project already has:
| Existing File | Current State | Target State |
|---------------|---------------|--------------|
| `database/models.py` | Sync, `plan` enum (5 values) | Add `tier` (2 values), `daily_translation_count` |
| `database/connection.py` | Sync engine | Async engine |
| `alembic/env.py` | Sync migrations | Async migrations |
| `alembic/versions/001_initial.py` | Creates users table | Add migration 002 to modify |
### Technical Requirements from Architecture
**Database Stack:**
- PostgreSQL (prod) / SQLite (dev)
- SQLAlchemy 2.0 with async support
- Alembic migrations
**Auth Stack (already installed):**
- PyJWT 2.8.0
- passlib[bcrypt] 1.7.4
**New Dependencies Required:**
```
aiosqlite>=0.19.0 # Async SQLite driver
asyncpg>=0.29.0 # Async PostgreSQL driver
```
### User Model Changes
**Current Model** (`database/models.py:41-84`):
```python
class User(Base):
plan = Column(Enum(PlanType), default=PlanType.FREE) # 5 tiers
password_hash = Column(String(255)) # Wrong naming
docs_translated_this_month = Column(Integer, default=0) # Monthly
```
**Target Model (for this story):**
```python
class User(Base):
tier = Column(String(10), default="free") # Simple: "free" | "pro"
hashed_password = Column(String(255)) # Correct naming
daily_translation_count = Column(Integer, default=0) # Daily (reset at midnight)
```
**Migration Strategy:**
- Keep `plan` field temporarily for backward compatibility
- Add `tier` and `daily_translation_count` as new fields
- Future story will remove deprecated fields
### Async SQLAlchemy Pattern
**From** (`database/connection.py`):
```python
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
engine = create_engine(DATABASE_URL)
SessionLocal = sessionmaker(bind=engine)
```
**To:**
```python
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession, async_sessionmaker
# Convert URLs:
# postgresql://... → postgresql+asyncpg://...
# sqlite:///... → sqlite+aiosqlite:///...
engine = create_async_engine(ASYNC_DATABASE_URL)
AsyncSessionLocal = async_sessionmaker(bind=engine, class_=AsyncSession)
```
### Alembic Async Pattern
**Update** `alembic/env.py`:
```python
from sqlalchemy.ext.asyncio import create_async_engine
import asyncio
def run_migrations_online():
connectable = create_async_engine(DATABASE_URL)
async def run_async_migrations():
async with connectable.connect() as connection:
await connection.run_sync(do_migrations)
asyncio.run(run_async_migrations())
```
### Files to Modify
| File | Action | Notes |
|------|--------|-------|
| `database/models.py` | MODIFY | Add tier, daily_translation_count, hashed_password |
| `database/connection.py` | MODIFY | Convert to async |
| `alembic/env.py` | MODIFY | Add async support |
| `alembic/versions/002_*.py` | CREATE | New migration |
| `requirements.txt` | MODIFY | Add aiosqlite, asyncpg |
| `.env.example` | VERIFY | Has DATABASE_URL documented |
### Files NOT to Touch
- `main.py` - Will be updated in a later story
- `routes/auth_routes.py` - Will be updated in a later story
- `services/auth_service*.py` - Will be updated in a later story
- Existing migrations in `alembic/versions/` - Don't modify 001_initial.py
### Testing Commands
```bash
# Install new dependencies
pip install aiosqlite asyncpg
# Run migration
alembic upgrade head
# Verify migration
alembic current
# Rollback if needed
alembic downgrade -1
```
### Project Structure Notes
- Backend uses snake_case for files, variables, functions
- PascalCase for classes
- Follow existing patterns in `database/` folder
- Keep backward compatibility during transition
### References
- [Source: _bmad-output/planning-artifacts/architecture.md#Data Architecture]
- [Source: _bmad-output/planning-artifacts/architecture.md#Authentication & Security]
- [Source: _bmad-output/planning-artifacts/epics.md#Story 1.1]
- [Source: _bmad-output/planning-artifacts/prd.md#NFR6-NFR11 Security]
## Dev Agent Record
### Agent Model Used
GLM-5 (zai-coding-plan/glm-5)
### Debug Log References
- Installed aiosqlite, asyncpg, greenlet, pytest, pytest-asyncio dependencies
- All 11 tests pass successfully
- Migration upgrade/downgrade tested successfully
### Completion Notes List
1. **User Model Refactored**: Added `account_tier` (mapped via `tier` property), `daily_translation_count`, and `hashed_password` fields. Added backward-compatible `password_hash` property with deprecation warning.
2. **Async SQLAlchemy**: Converted `database/connection.py` to use `create_async_engine`, `AsyncSession`, and `async_sessionmaker`. Added URL conversion for postgresql→postgresql+asyncpg and sqlite→sqlite+aiosqlite.
3. **Alembic Async**: Updated `alembic/env.py` with async engine support using `asyncio.run()` wrapper and `run_sync()` pattern.
4. **Migration Created**: `002_add_tier_daily_count.py` adds new columns, copies data from `password_hash` to `hashed_password`, and migrates `plan` values to `tier` ("pro"/"business"/"enterprise" → "pro", others → "free").
5. **Dependencies Updated**: Added aiosqlite>=0.19.0, asyncpg>=0.29.0, greenlet>=3.0.0, pytest>=7.0.0, pytest-asyncio>=0.21.0.
6. **Tests Added**: Created `tests/conftest.py` with async fixtures and `tests/test_user_model.py` with 11 test cases covering all AC requirements.
7. **Code review (2026-02-21)**: H2: `database/repositories.py` and auth services now use `hashed_password`/`tier` in `create()`. H1+M3: Added `tests/test_database_utils.py` (AC4 URL conversion), `tests/test_alembic_async.py` (AC3/AC5 + Alembic upgrade/downgrade integration). M4: Migration `server_default=sa.text("0")` for Integer. M2: Comment in `database/connection.py` for sync engine. All 21 tests pass.
### File List
**Modified:**
- database/models.py - Added tier (direct column), daily_translation_count, hashed_password; CheckConstraint; datetime.now(timezone.utc); lazy="select"
- database/connection.py - Converted to async SQLAlchemy 2.0; fixed check_db_connection() with text(); uses shared convert_to_async_url
- database/__init__.py - Updated exports for async session; added Base export
- alembic/env.py - Added async migration support; uses shared convert_to_async_url
- requirements.txt - Added aiosqlite, asyncpg, greenlet, pytest, pytest-asyncio; removed psycopg2-binary (unused)
**Created:**
- database/utils.py - Shared convert_to_async_url() utility (DRY)
- alembic/versions/002_add_tier_daily_count.py - Migration for new user fields (op.execute, tier column, CHECK constraint)
- tests/__init__.py - Test package init
- tests/conftest.py - Pytest async fixtures (in-memory SQLite, asyncio_mode=auto)
- tests/test_user_model.py - User model tests (11 test cases)
- tests/test_database_utils.py - AC4: convert_to_async_url tests (code-review)
- tests/test_alembic_async.py - AC3/AC5 and Alembic upgrade/downgrade integration (code-review)
- pytest.ini - asyncio_mode = auto configuration
**Modified (code-review follow-up):**
- database/repositories.py - create() uses hashed_password and tier
- database/connection.py - Comment documenting sync engine (backward compat)
- alembic/versions/002_add_tier_daily_count.py - server_default=sa.text("0") for Integer
- services/auth_service.py - repo.create(..., hashed_password=, tier=)
- services/auth_service_db.py - repo.create(..., hashed_password=, tier=)
*Other files modified in repo (other stories): .env.example, main.py, models/subscription.py, routes/auth_routes.py, utils/__init__.py, utils/exceptions.py, etc.*
## Change Log
- 2026-02-20: Completed Story 1.1 - Setup database with async SQLAlchemy 2.0 and refactored User model with tier/daily_translation_count fields
- 2026-02-20: Code review fixes — C2: renamed account_tier→tier (direct column, ORM-queryable); C3: fixed check_db_connection() text(); H1: migration uses op.execute(); H2: extracted convert_to_async_url() to database/utils.py; H3: datetime.now(timezone.utc); H4: lazy="select"; M1: CHECK CONSTRAINT on tier; M2: removed psycopg2-binary; M3/M4: pytest.ini asyncio_mode=auto + in-memory test DB. Task 1.5 (UUID PK type) deferred — requires full FK cascade migration.
- 2026-02-21: Code review auto-fixes (option 1): H2 repositories/auth services → hashed_password/tier; H1+M3 tests for AC3/AC4/AC5 and Alembic integration; M4 migration server_default; M2 connection.py comment; story status → done.