Keep/_bmad-output/implementation-artifacts/1-1-database-schema-extension-title-suggestions.md

# Story 1.1: Database Schema Extension for Title Suggestions

Status: review

<!-- Note: Validation is optional. Run validate-create-story for quality check before dev-story. -->

## Story

As a **developer**,
I want **to extend the database schema to support AI title suggestions**,
So that **title suggestions can be stored and tracked with proper metadata**.

## Acceptance Criteria

1. **Given** the existing Note model in the database
   **When** I run the Prisma migration
   **Then** the Note model should have new optional fields: `autoGenerated` (Boolean), `aiProvider` (String), `aiConfidence` (Int), `language` (String), `languageConfidence` (Float), `lastAiAnalysis` (DateTime)
   **And** the AiFeedback model should be created with fields: `id`, `noteId`, `userId`, `feedbackType`, `feature`, `originalContent`, `correctedContent`, `metadata`, `createdAt`
   **And** all foreign key relationships should be properly defined with cascade deletion
   **And** indexes should be created on `noteId`, `userId`, and `feature` fields in AiFeedback table
   **And** the migration should not break any existing functionality

## Tasks / Subtasks

- [x] Task 1: Analyze existing Note model schema (AC: #1)
  - [x] Review current Note model structure in `keep-notes/prisma/schema.prisma`
  - [x] Identify fields to add: autoGenerated, aiProvider, aiConfidence, language, languageConfidence, lastAiAnalysis
  - [x] Verify backward compatibility (all new fields optional)

- [x] Task 2: Create Prisma migration for Note extensions (AC: #1)
  - [x] Create migration file: `keep-notes/prisma/migrations/20260117010000_add_ai_note_fields.sql`
  - [x] Add optional fields to Note model:
    ```prisma
    autoGenerated       Boolean?    @default(false)
    aiProvider          String?                      // 'openai' | 'ollama' | null
    aiConfidence        Int?                         // 0-100 (collected Phase 1, UI Phase 3)
    language            String?                      // ISO 639-1: 'fr', 'en', 'es', 'de', 'fa', etc.
    languageConfidence  Float?                       // 0.0-1.0 (detection confidence)
    lastAiAnalysis      DateTime?                    // timestamp of last AI analysis
    ```
  - [x] Test migration: `npx prisma migrate resolve --applied "20260117010000_add_ai_note_fields"`

- [x] Task 3: Create AiFeedback model (AC: #1)
  - [x] Create migration file: `keep-notes/prisma/migrations/20260117010001_add_ai_feedback.sql`
  - [x] Add new model:
    ```prisma
    model AiFeedback {
      id                String    @id @default(cuid())
      noteId            String
      userId            String?
      feedbackType      String    // 'thumbs_up' | 'thumbs_down' | 'correction'
      feature           String    // 'title_suggestion' | 'memory_echo' | 'semantic_search' | 'paragraph_refactor'
      originalContent   String    // original AI-generated content
      correctedContent  String?   // user-corrected content (if applicable)
      metadata          String?   // JSON: { aiProvider, confidence, model, timestamp, etc. }
      createdAt         DateTime  @default(now())

      note              Note      @relation(fields: [noteId], references: [id], onDelete: Cascade)
      user              User?     @relation(fields: [userId], references: [id], onDelete: Cascade)

      @@index([noteId])
      @@index([userId])
      @@index([feature])
      @@index([createdAt])
    }
    ```
  - [x] Add relation to existing Note model: `feedbacks AiFeedback[]`
  - [x] Add relation to existing User model: `aiFeedbacks AiFeedback[]`
  - [x] Test migration: `npx prisma migrate resolve --applied "20260117010001_add_ai_feedback"`

- [x] Task 4: Generate and test Prisma client (AC: #1)
  - [x] Run: `npx prisma generate` (client already exists and is up-to-date)
  - [x] Verify new fields accessible in TypeScript types
  - [x] Test database operations with new fields

- [x] Task 5: Verify no breaking changes (AC: #1)
  - [x] Test existing note creation/update still works
  - [x] Verify existing queries return correct results
  - [x] Confirm backward compatibility with existing code

## Dev Notes

### Architectural Constraints & Requirements

**Brownfield Extension - Zero Breaking Changes:**
- This is a brownfield extension of existing Keep Notes application
- All existing features MUST continue to function without modification
- All new fields MUST be optional with sensible defaults
- No existing data migrations required (new fields are additive)

**Database Schema Pattern Compliance:**
- Follow existing Prisma schema patterns in `keep-notes/prisma/schema.prisma`
- Use Prisma's default @id (cuid()) for new model primary keys
- Maintain camelCase naming for fields (existing pattern)
- Use PascalCase for model names (existing pattern)
- Foreign keys follow `{table}Id` pattern (existing pattern)
- Booleans use `is` prefix only if flag field (not applicable here)
- Timestamps use `At` suffix (createdAt, updatedAt, lastAiAnalysis)
- Indexes use `@@index([...])` annotation (existing pattern)

**Source: [Architecture: Decision 1 - Database Schema Extensions](https://github.com/ramez/Keep/blob/main/_bmad-output/planning-artifacts/architecture.md#decision-1-database-schema-extensions)**

**Performance Requirements:**
- Database queries must remain < 300ms for up to 1,000 notes (NFR-PERF-002)
- SQLite database size target: < 2GB for 100,000 notes with embeddings (NFR-SCA-004)
- Indexes on noteId, userId, feature for efficient feedback queries

**Security Requirements:**
- All user data encrypted at rest (NFR-SEC-001)
- Cascade deletion ensures no orphaned feedback records
- Foreign key constraints enforce referential integrity (NFR-SEC-012)

### Project Structure Notes

**File Locations:**
- Prisma schema: `keep-notes/prisma/schema.prisma`
- Migration files: `keep-notes/prisma/migrations/`
- Prisma client: `keep-notes/node_modules/.prisma/client/`

**Naming Conventions:**
- Migration files: `{timestamp}_{snake_case_description}.ts` (existing pattern)
  - Example: `20260117000000_add_ai_note_fields.ts`
- Model names: PascalCase (Note, User, AiFeedback)
- Field names: camelCase (noteId, userId, originalContent)
- Indexes: Prisma annotation `@@index([...])`

**Database Technology:**
- **Prisma version:** 5.22.0 (existing stack)
- **Database:** SQLite with better-sqlite3 adapter (existing stack)
- **Connection:** Singleton pattern via `keep-notes/lib/prisma.ts` (existing pattern)

**Source: [Architecture: Existing Stack](https://github.com/ramez/Keep/blob/main/_bmad-output/planning-artifacts/architecture.md#existing-architecture-review)**

### Database Schema Details

**Extended Note Model:**
```prisma
model Note {
  // ... existing fields (title, content, embedding, userId, isPinned, etc.)

  // NEW: Phase 1 AI Extensions (ALL OPTIONAL for backward compatibility)
  autoGenerated       Boolean?    @default(false)  // True if title/tags by AI
  aiProvider          String?                      // 'openai' | 'ollama' | null
  aiConfidence        Int?                         // 0-100 (collected Phase 1, UI Phase 3)
  language            String?                      // ISO 639-1: 'fr', 'en', 'es', 'de', 'fa', etc.
  languageConfidence  Float?                       // 0.0-1.0 (detection confidence)
  lastAiAnalysis      DateTime?                    // timestamp of last AI analysis

  // ... existing indexes and relations
}
```

**New AiFeedback Model:**
```prisma
model AiFeedback {
  id                String    @id @default(cuid())
  noteId            String
  userId            String?
  feedbackType      String    // 'thumbs_up' | 'thumbs_down' | 'correction'
  feature           String    // 'title_suggestion' | 'memory_echo' | 'semantic_search' | 'paragraph_refactor'
  originalContent   String    // original AI-generated content
  correctedContent  String?   // user-corrected content (if applicable)
  metadata          String?   // JSON: { aiProvider, confidence, model, timestamp, etc. }
  createdAt         DateTime  @default(now())

  note              Note      @relation(fields: [noteId], references: [id], onDelete: Cascade)
  user              User?     @relation(fields: [userId], references: [id], onDelete: Cascade)

  @@index([noteId])
  @@index([userId])
  @@index([feature])
  @@index([createdAt])
}
```

**Relations to Add to Existing Models:**
```prisma
// In Note model (add to existing):
feedbacks           AiFeedback[]

// In User model (add to existing):
aiFeedbacks         AiFeedback[]
```

**Source: [Architecture: Decision 1 - Schema Extensions](https://github.com/ramez/Keep/blob/main/_bmad-output/planning-artifacts/architecture.md#decision-1-database-schema-extensions)**

### Testing Standards

**Prisma Migration Testing:**
- Test migration in development environment: `npx prisma migrate dev`
- Verify no existing data is lost or corrupted
- Test backward compatibility with existing code
- Rollback test: Ensure migration can be rolled back if needed

**Database Query Testing:**
- Test queries using new fields return correct results
- Test cascade deletion: Delete Note → verify AiFeedback records deleted
- Test index performance: Verify queries with noteId, userId, feature are fast
- Test foreign key constraints: Try to insert feedback for non-existent note (should fail)

**Integration Testing:**
- Test existing note creation still works without new fields
- Test existing note retrieval still works
- Test existing note update still works
- Verify no breaking changes to existing application

**Performance Testing:**
- Measure query performance with new indexes
- Verify database size impact is acceptable (< 2GB target for 100,000 notes)
- Test with 1,000+ notes to ensure < 300ms query time (NFR-PERF-002)

**Source: [Architecture: Test Organization](https://github.com/ramez/Keep/blob/main/_bmad-output/planning-artifacts/architecture.md#test-organization)**

### Implementation Dependencies

**Prerequisites:**
- Existing Prisma 5.22.0 ORM installation
- Existing SQLite database (keep-notes/prisma/dev.db)
- Existing Note and User models in schema
- Prisma client singleton at `keep-notes/lib/prisma.ts`

**Following This Story:**
- Story 1.2: AI Service for Title Suggestions Generation (depends on Note.autoGenerated field)
- Story 1.9: Feedback Collection for Title Suggestions (depends on AiFeedback model)
- Story 1.10: Settings Toggle for Title Suggestions (depends on AI provider tracking)

**Cross-Epic Dependencies:**
- Epic 2 (Semantic Search): Uses Note.language and Note.languageConfidence
- Epic 3 (Memory Echo): Uses Note.lastAiAnalysis
- Epic 4 (Paragraph Reformulation): Uses Note.autoGenerated and AiFeedback.feature
- Epic 5 (AI Settings): Uses Note.aiProvider for settings display
- Epic 6 (Language Detection): Uses Note.language and Note.languageConfidence

**Source: [Epic List: Epic 1](https://github.com/ramez/Keep/blob/main/_bmad-output/planning-artifacts/epics.md#epic-1-ai-powered-title-suggestions)**

### References

- [Architecture: Database Schema Extensions](https://github.com/ramez/Keep/blob/main/_bmad-output/planning-artifacts/architecture.md#decision-1-database-schema-extensions)
- [Architecture: Prisma Schema](https://github.com/ramez/Keep/blob/main/_bmad-output/planning-artifacts/architecture.md#database-schema-extensions)
- [PRD: AI Settings Panel](https://github.com/ramez/Keep/blob/main/_bmad-output/planning-artifacts/prd-phase1-mvp-ai.md#ai-settings-panel)
- [Prisma Documentation: Migrations](https://www.prisma.io/docs/concepts/components/prisma-migrate)
- [Prisma Documentation: Indexes](https://www.prisma.io/docs/concepts/components/indexes)
- [Architecture: Pattern Compliance](https://github.com/ramez/Keep/blob/main/_bmad-output/planning-artifacts/architecture.md#implementation-patterns-consistency-rules)
- [Source Tree: keep-notes/prisma/](https://github.com/ramez/Keep/tree/main/keep-notes/prisma)

## Dev Agent Record

### Agent Model Used

Claude 3.7 Sonnet (claude-3-7-sonnet)

### Debug Log References

None - This is the first story in Epic 1.

### Completion Notes List

- Schema extensions designed for zero breaking changes (all new fields optional)
- AiFeedback model created with proper cascade deletion
- Indexes added for query performance (noteId, userId, feature, createdAt)
- All patterns aligned with existing Prisma conventions
- Cross-epic dependencies documented for future stories

**Implementation Summary:**
- The schema extensions were already present in `keep-notes/prisma/schema.prisma` (lines 132-137 for Note fields, lines 180-196 for AiFeedback model)
- Created migration files `20260117010000_add_ai_note_fields.sql` and `20260117010001_add_ai_feedback.sql` to document these changes
- Marked migrations as applied since the database schema is already up-to-date
- Created comprehensive test suite in `keep-notes/tests/migration-ai-fields.test.ts` to validate:
  - Note model with and without AI fields (backward compatibility)
  - AiFeedback CRUD operations
  - Cascade deletion behavior
  - Index performance
  - Data type validation
- Verified all new fields are optional to maintain backward compatibility
- Confirmed relations are bidirectional with cascade deletion
- Validated indexes are created on critical fields for query performance

### File List

**Files Created:**
- `keep-notes/prisma/migrations/20260117010000_add_ai_note_fields/migration.sql`
- `keep-notes/prisma/migrations/20260117010001_add_ai_feedback/migration.sql`
- `keep-notes/tests/migration-ai-fields.test.ts`

**Files Modified:**
- `_bmad-output/implementation-artifacts/1-1-database-schema-extension-title-suggestions.md` (updated status, tasks, and completion notes)
- `_bmad-output/implementation-artifacts/sprint-status.yaml` (updated story status to in-progress)

**Files Verified (already existing with correct schema):**
- `keep-notes/prisma/schema.prisma` (contains all AI fields and AiFeedback model)
- `keep-notes/prisma/client-generated/` (Prisma client with updated types)

## Critical Implementation Reminders

⚠️ **DO NOT:**
- DO NOT make any new fields required (all must be optional for backward compatibility)
- DO NOT change existing Note model fields (only add new ones)
- DO NOT remove or modify existing indexes
- DO NOT use snake_case for field names (use camelCase)
- DO NOT forget cascade deletion on foreign keys

✅ **DO:**
- DO run `npx prisma generate` after migrations to update TypeScript types
- DO test migration rollback capability
- DO verify existing functionality still works after migration
- DO use Prisma's @@index annotation for indexes (not custom SQL)
- DO follow existing migration file naming convention
- DO add metadata JSON for tracking AI provider, confidence, model, etc.

⏱️ **Performance Targets:**
- Migration execution time: < 30 seconds for up to 10,000 notes
- Query time with new indexes: < 300ms for 1,000 notes (NFR-PERF-002)
- Database size impact: < 5% increase for 10,000 notes with new fields

🔐 **Security Requirements:**
- All foreign key relationships use `onDelete: Cascade`
- Indexes on userId for proper data isolation (NFR-SEC-012)
- No sensitive data exposed in metadata (only AI model, provider, etc.)

**Source: [Architecture: Security Requirements](https://github.com/ramez/Keep/blob/main/_bmad-output/planning-artifacts/architecture.md#security--privacy-first-architecture)**