Major changes across backend, frontend, infrastructure: - Provider system with model selection (Google, DeepL, OpenAI, Ollama, Google Cloud) - Admin panel: user management, pricing, settings - Glossary system with CSV import/export - Subscription and tier quota management - Security hardening (rate limiting, API key auth, path traversal fixes) - Docker compose for dev, prod, and IONOS deployment - Alembic migrations for new tables - Frontend: dashboard, pricing page, landing page, i18n (en/fr) - Test suite and verification scripts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
253 lines
11 KiB
Markdown
253 lines
11 KiB
Markdown
# Story 1.1: Setup Base de Données & Modèle User
|
|
|
|
Status: done
|
|
|
|
## Story
|
|
|
|
As a **Developer**,
|
|
I want **setup the database with Alembic (async) and refactor the User model with tier field**,
|
|
so that **the application can persist user data with simplified subscription management and async support**.
|
|
|
|
## Acceptance Criteria
|
|
|
|
1. **AC1: User Model Refactored** - `users` table has columns: id (UUID), email, hashed_password, tier (default "free"), daily_translation_count, created_at, updated_at
|
|
2. **AC2: Async SQLAlchemy** - Database connection uses async engine (SQLAlchemy 2.0 async)
|
|
3. **AC3: Alembic Async** - Alembic configured for async migrations
|
|
4. **AC4: Dual Database Support** - PostgreSQL works in production, SQLite in development
|
|
5. **AC5: Secrets from Environment** - All secrets loaded from environment variables (NFR10)
|
|
|
|
## Tasks / Subtasks
|
|
|
|
- [x] **Task 1: Refactor User Model** (AC: 1, 5)
|
|
- [x] 1.1 Add `tier` field (str: "free" | "pro") with default "free"
|
|
- [x] 1.2 Add `daily_translation_count` field (int, default 0)
|
|
- [x] 1.3 Rename `password_hash` → `hashed_password` for consistency with architecture
|
|
- [x] 1.4 Keep existing fields for backward compatibility (plan, stripe fields) but mark deprecated
|
|
- [x] 1.5 Add UUID column type (from String(36) to proper UUID) — DEFERRED: requires coordinated PK+FK migration across all tables (users, translations, api_keys, usage_logs, payment_history). Created as separate task in next sprint.
|
|
|
|
- [x] **Task 2: Convert to Async SQLAlchemy** (AC: 2, 4)
|
|
- [x] 2.1 Replace `create_engine` with `create_async_engine` in `database/connection.py`
|
|
- [x] 2.2 Replace `sessionmaker` with `AsyncSession` and `async_sessionmaker`
|
|
- [x] 2.3 Update `get_db()` to async generator with `AsyncSession`
|
|
- [x] 2.4 Keep SQLite support for development (with aiosqlite driver)
|
|
- [x] 2.5 Keep PostgreSQL support (with asyncpg driver)
|
|
|
|
- [x] **Task 3: Configure Alembic for Async** (AC: 3, 4)
|
|
- [x] 3.1 Update `alembic/env.py` for async engine support
|
|
- [x] 3.2 Add `asyncio.run()` wrapper for online migrations
|
|
- [x] 3.3 Ensure offline migrations still work
|
|
|
|
- [x] **Task 4: Create Migration** (AC: 1)
|
|
- [x] 4.1 Create migration `alembic/versions/002_add_tier_daily_count.py`
|
|
- [x] 4.2 Add `tier` column with default "free"
|
|
- [x] 4.3 Add `daily_translation_count` column with default 0
|
|
- [x] 4.4 Add `hashed_password` column (migration from password_hash)
|
|
- [x] 4.5 Data migration: set tier from existing plan field
|
|
|
|
- [x] **Task 5: Update Dependencies** (AC: 2, 4)
|
|
- [x] 5.1 Add `aiosqlite` for async SQLite support
|
|
- [x] 5.2 Add `asyncpg` for async PostgreSQL support
|
|
- [x] 5.3 Update `requirements.txt`
|
|
|
|
- [x] **Task 6: Verify Setup** (AC: 1-5)
|
|
- [x] 6.1 Test migration with SQLite: `alembic upgrade head`
|
|
- [x] 6.2 Test migration downgrade: `alembic downgrade -1`
|
|
- [x] 6.3 Verify User model can create/read with new fields
|
|
|
|
## Dev Notes
|
|
|
|
### 🚨 BROWNFIELD CONTEXT - CRITICAL
|
|
|
|
This is a **refactoring story**, NOT a fresh setup. The project already has:
|
|
|
|
| Existing File | Current State | Target State |
|
|
|---------------|---------------|--------------|
|
|
| `database/models.py` | Sync, `plan` enum (5 values) | Add `tier` (2 values), `daily_translation_count` |
|
|
| `database/connection.py` | Sync engine | Async engine |
|
|
| `alembic/env.py` | Sync migrations | Async migrations |
|
|
| `alembic/versions/001_initial.py` | Creates users table | Add migration 002 to modify |
|
|
|
|
### Technical Requirements from Architecture
|
|
|
|
**Database Stack:**
|
|
- PostgreSQL (prod) / SQLite (dev)
|
|
- SQLAlchemy 2.0 with async support
|
|
- Alembic migrations
|
|
|
|
**Auth Stack (already installed):**
|
|
- PyJWT 2.8.0
|
|
- passlib[bcrypt] 1.7.4
|
|
|
|
**New Dependencies Required:**
|
|
```
|
|
aiosqlite>=0.19.0 # Async SQLite driver
|
|
asyncpg>=0.29.0 # Async PostgreSQL driver
|
|
```
|
|
|
|
### User Model Changes
|
|
|
|
**Current Model** (`database/models.py:41-84`):
|
|
```python
|
|
class User(Base):
|
|
plan = Column(Enum(PlanType), default=PlanType.FREE) # 5 tiers
|
|
password_hash = Column(String(255)) # Wrong naming
|
|
docs_translated_this_month = Column(Integer, default=0) # Monthly
|
|
```
|
|
|
|
**Target Model (for this story):**
|
|
```python
|
|
class User(Base):
|
|
tier = Column(String(10), default="free") # Simple: "free" | "pro"
|
|
hashed_password = Column(String(255)) # Correct naming
|
|
daily_translation_count = Column(Integer, default=0) # Daily (reset at midnight)
|
|
```
|
|
|
|
**Migration Strategy:**
|
|
- Keep `plan` field temporarily for backward compatibility
|
|
- Add `tier` and `daily_translation_count` as new fields
|
|
- Future story will remove deprecated fields
|
|
|
|
### Async SQLAlchemy Pattern
|
|
|
|
**From** (`database/connection.py`):
|
|
```python
|
|
from sqlalchemy import create_engine
|
|
from sqlalchemy.orm import sessionmaker
|
|
|
|
engine = create_engine(DATABASE_URL)
|
|
SessionLocal = sessionmaker(bind=engine)
|
|
```
|
|
|
|
**To:**
|
|
```python
|
|
from sqlalchemy.ext.asyncio import create_async_engine, AsyncSession, async_sessionmaker
|
|
|
|
# Convert URLs:
|
|
# postgresql://... → postgresql+asyncpg://...
|
|
# sqlite:///... → sqlite+aiosqlite:///...
|
|
|
|
engine = create_async_engine(ASYNC_DATABASE_URL)
|
|
AsyncSessionLocal = async_sessionmaker(bind=engine, class_=AsyncSession)
|
|
```
|
|
|
|
### Alembic Async Pattern
|
|
|
|
**Update** `alembic/env.py`:
|
|
```python
|
|
from sqlalchemy.ext.asyncio import create_async_engine
|
|
import asyncio
|
|
|
|
def run_migrations_online():
|
|
connectable = create_async_engine(DATABASE_URL)
|
|
|
|
async def run_async_migrations():
|
|
async with connectable.connect() as connection:
|
|
await connection.run_sync(do_migrations)
|
|
|
|
asyncio.run(run_async_migrations())
|
|
```
|
|
|
|
### Files to Modify
|
|
|
|
| File | Action | Notes |
|
|
|------|--------|-------|
|
|
| `database/models.py` | MODIFY | Add tier, daily_translation_count, hashed_password |
|
|
| `database/connection.py` | MODIFY | Convert to async |
|
|
| `alembic/env.py` | MODIFY | Add async support |
|
|
| `alembic/versions/002_*.py` | CREATE | New migration |
|
|
| `requirements.txt` | MODIFY | Add aiosqlite, asyncpg |
|
|
| `.env.example` | VERIFY | Has DATABASE_URL documented |
|
|
|
|
### Files NOT to Touch
|
|
|
|
- `main.py` - Will be updated in a later story
|
|
- `routes/auth_routes.py` - Will be updated in a later story
|
|
- `services/auth_service*.py` - Will be updated in a later story
|
|
- Existing migrations in `alembic/versions/` - Don't modify 001_initial.py
|
|
|
|
### Testing Commands
|
|
|
|
```bash
|
|
# Install new dependencies
|
|
pip install aiosqlite asyncpg
|
|
|
|
# Run migration
|
|
alembic upgrade head
|
|
|
|
# Verify migration
|
|
alembic current
|
|
|
|
# Rollback if needed
|
|
alembic downgrade -1
|
|
```
|
|
|
|
### Project Structure Notes
|
|
|
|
- Backend uses snake_case for files, variables, functions
|
|
- PascalCase for classes
|
|
- Follow existing patterns in `database/` folder
|
|
- Keep backward compatibility during transition
|
|
|
|
### References
|
|
|
|
- [Source: _bmad-output/planning-artifacts/architecture.md#Data Architecture]
|
|
- [Source: _bmad-output/planning-artifacts/architecture.md#Authentication & Security]
|
|
- [Source: _bmad-output/planning-artifacts/epics.md#Story 1.1]
|
|
- [Source: _bmad-output/planning-artifacts/prd.md#NFR6-NFR11 Security]
|
|
|
|
## Dev Agent Record
|
|
|
|
### Agent Model Used
|
|
|
|
GLM-5 (zai-coding-plan/glm-5)
|
|
|
|
### Debug Log References
|
|
|
|
- Installed aiosqlite, asyncpg, greenlet, pytest, pytest-asyncio dependencies
|
|
- All 11 tests pass successfully
|
|
- Migration upgrade/downgrade tested successfully
|
|
|
|
### Completion Notes List
|
|
|
|
1. **User Model Refactored**: Added `account_tier` (mapped via `tier` property), `daily_translation_count`, and `hashed_password` fields. Added backward-compatible `password_hash` property with deprecation warning.
|
|
2. **Async SQLAlchemy**: Converted `database/connection.py` to use `create_async_engine`, `AsyncSession`, and `async_sessionmaker`. Added URL conversion for postgresql→postgresql+asyncpg and sqlite→sqlite+aiosqlite.
|
|
3. **Alembic Async**: Updated `alembic/env.py` with async engine support using `asyncio.run()` wrapper and `run_sync()` pattern.
|
|
4. **Migration Created**: `002_add_tier_daily_count.py` adds new columns, copies data from `password_hash` to `hashed_password`, and migrates `plan` values to `tier` ("pro"/"business"/"enterprise" → "pro", others → "free").
|
|
5. **Dependencies Updated**: Added aiosqlite>=0.19.0, asyncpg>=0.29.0, greenlet>=3.0.0, pytest>=7.0.0, pytest-asyncio>=0.21.0.
|
|
6. **Tests Added**: Created `tests/conftest.py` with async fixtures and `tests/test_user_model.py` with 11 test cases covering all AC requirements.
|
|
7. **Code review (2026-02-21)**: H2: `database/repositories.py` and auth services now use `hashed_password`/`tier` in `create()`. H1+M3: Added `tests/test_database_utils.py` (AC4 URL conversion), `tests/test_alembic_async.py` (AC3/AC5 + Alembic upgrade/downgrade integration). M4: Migration `server_default=sa.text("0")` for Integer. M2: Comment in `database/connection.py` for sync engine. All 21 tests pass.
|
|
|
|
### File List
|
|
|
|
**Modified:**
|
|
- database/models.py - Added tier (direct column), daily_translation_count, hashed_password; CheckConstraint; datetime.now(timezone.utc); lazy="select"
|
|
- database/connection.py - Converted to async SQLAlchemy 2.0; fixed check_db_connection() with text(); uses shared convert_to_async_url
|
|
- database/__init__.py - Updated exports for async session; added Base export
|
|
- alembic/env.py - Added async migration support; uses shared convert_to_async_url
|
|
- requirements.txt - Added aiosqlite, asyncpg, greenlet, pytest, pytest-asyncio; removed psycopg2-binary (unused)
|
|
|
|
**Created:**
|
|
- database/utils.py - Shared convert_to_async_url() utility (DRY)
|
|
- alembic/versions/002_add_tier_daily_count.py - Migration for new user fields (op.execute, tier column, CHECK constraint)
|
|
- tests/__init__.py - Test package init
|
|
- tests/conftest.py - Pytest async fixtures (in-memory SQLite, asyncio_mode=auto)
|
|
- tests/test_user_model.py - User model tests (11 test cases)
|
|
- tests/test_database_utils.py - AC4: convert_to_async_url tests (code-review)
|
|
- tests/test_alembic_async.py - AC3/AC5 and Alembic upgrade/downgrade integration (code-review)
|
|
- pytest.ini - asyncio_mode = auto configuration
|
|
|
|
**Modified (code-review follow-up):**
|
|
- database/repositories.py - create() uses hashed_password and tier
|
|
- database/connection.py - Comment documenting sync engine (backward compat)
|
|
- alembic/versions/002_add_tier_daily_count.py - server_default=sa.text("0") for Integer
|
|
- services/auth_service.py - repo.create(..., hashed_password=, tier=)
|
|
- services/auth_service_db.py - repo.create(..., hashed_password=, tier=)
|
|
|
|
*Other files modified in repo (other stories): .env.example, main.py, models/subscription.py, routes/auth_routes.py, utils/__init__.py, utils/exceptions.py, etc.*
|
|
|
|
## Change Log
|
|
|
|
- 2026-02-20: Completed Story 1.1 - Setup database with async SQLAlchemy 2.0 and refactored User model with tier/daily_translation_count fields
|
|
- 2026-02-20: Code review fixes — C2: renamed account_tier→tier (direct column, ORM-queryable); C3: fixed check_db_connection() text(); H1: migration uses op.execute(); H2: extracted convert_to_async_url() to database/utils.py; H3: datetime.now(timezone.utc); H4: lazy="select"; M1: CHECK CONSTRAINT on tier; M2: removed psycopg2-binary; M3/M4: pytest.ini asyncio_mode=auto + in-memory test DB. Task 1.5 (UUID PK type) deferred — requires full FK cascade migration.
|
|
- 2026-02-21: Code review auto-fixes (option 1): H2 repositories/auth services → hashed_password/tier; H1+M3 tests for AC3/AC4/AC5 and Alembic integration; M4 migration server_default; M2 connection.py comment; story status → done.
|