Momento/docs/3-1-freemium-quota-tracking.md

# Story 3.1: Freemium "AI Discovery Pack" Quota Tracking

Status: ready-for-dev

---

## Story

As a business,
I want to track Freemium usage against a Redis quota limit,
So that I can limit my API cost exposure for free users.

**Given** a free user triggers an AI request
**When** the system intercepts the request
**Then** the quota is tracked and the UI updates
**And** (NFR-SC2) the Redis-backed check resolves in under 10ms.

---

## Epic Context

**Epic 3:** The SaaS Commercial Engine (Monetization & API Cost Protection)

**Epic Business Value:** The core backend logic allowing us to sell the product without bleeding API costs — freemium limits, router fallback, host-pays.

**Cross-Story Dependencies:**
- Story 3.1 (this story) establishes the quota tracking foundation
- Story 3.2 builds on this with the LLM Router and provider routing
- Story 3.3 adds smart-routing fallback when quota is low
- Story 3.4 (Host-Pays) extends quota tracking to collaborative sessions
- Story 3.5 (BYOK) bypasses quota for users with their own API keys
- Story 3.6 (Stripe) manages tier upgrades

**Technical Constraint:** (NFR-SC2) Redis-backed entitlement checks must complete in under 10ms.

---

## Acceptance Criteria

1. [AC1] When a free user (BASIC tier) makes an AI request, the system checks Redis for current usage count
2. [AC2] Each AI feature (semantic_search, auto_tag, auto_title) has its own Redis counter with format: `usage:{userId}:{feature}:{YYYY-MM}`
3. [AC3] Counter increments atomically via Redis INCRBY (not read-modify-write) to avoid race conditions
4. [AC4] If counter >= limit, return HTTP 402 with body `{ error: "QUOTA_EXCEEDED", feature, upgradeTier: "PRO", byokConfigured }`
5. [AC5] If counter < limit, allow request to proceed and increment counter asynchronously (fire-and-forget)
6. [AC6] Redis keys have 90-day TTL to auto-cleanup (covers grace period for monthly reconciliation)
7. [AC7] The Sidebar footer displays a usage gauge component showing Discovery Pack consumption in real-time

---

## Tasks / Subtasks

- [ ] Task 1: Add Prisma models for Subscription, UsageLog (AC: #1)
  - [ ] Subtask 1.1: Add `Subscription` model with tier/stripe fields
  - [ ] Subtask 1.2: Add `UsageLog` model for PostgreSQL sync target
  - [ ] Subtask 1.3: Create migration file
- [ ] Task 2: Create `lib/entitlements.ts` with Redis-backed `canUseFeature()` (AC: #2, #3, #4)
  - [ ] Subtask 2.1: Implement `getCurrentPeriodKey()` returning `YYYY-MM` format
  - [ ] Subtask 2.2: Implement Redis key format: `usage:{userId}:{feature}:{YYYY-MM}`
  - [ ] Subtask 2.3: Implement atomic `canUseFeature()` with < 10ms target (use Redis GET + pipeline INCRBY)
  - [ ] Subtask 2.4: Return `QuotaExceededError` when limit exceeded
- [ ] Task 3: Create `lib/usage-tracker.ts` with `trackFeatureUsage()` (AC: #5)
  - [ ] Subtask 3.1: Fire-and-forget Redis pipeline increment
  - [ ] Subtask 3.2: Include tokensUsed in metadata
- [ ] Task 4: Create `/api/usage/current` endpoint (AC: #6)
  - [ ] Subtask 4.1: Return remaining quota for all features for authenticated user
- [ ] Task 5: Create `<UsageMeter>` UI component for Sidebar footer (AC: #7)
  - [ ] Subtask 5.1: Show progress bar for Discovery Pack (semantic_search: 30 lifetime, auto_tag: 20, auto_title: 10)
  - [ ] Subtask 5.2: Real-time updates via React Query polling every 30s
  - [ ] Subtask 5.3: Show "Upgrade to Pro" paywall modal on 402 response
- [ ] Task 6: Create CRON sync worker `/api/cron/sync-usage` (AC: #6)
  - [ ] Subtask 6.1: Batch sync Redis counters → PostgreSQL UsageLog
  - [ ] Subtask 6.2: Handle monthly reset (new period = reset counters for that user/feature)

---

## Dev Notes

### Project Structure Notes

- **Code base:** `memento-note/` (Next.js app with App Router)
- **Redis client:** Currently `memento-note/lib/rate-limit.ts` uses in-memory Map (INSUFFICIENT). This story replaces it with actual Redis.
- **No existing `lib/entitlements.ts`** — creates from scratch
- **No existing `lib/usage-tracker.ts`** — creates from scratch
- **Sidebar:** `memento-note/components/sidebar.tsx` is the anchor for the UsageMeter UI
- **AI Factory:** `memento-note/lib/ai/factory.ts` has provider creation logic — quota checks must integrate BEFORE provider resolution

### Files to CREATE (NEW):
```
memento-note/lib/entitlements.ts       # Core quota check logic
memento-note/lib/usage-tracker.ts     # Track usage via Redis
memento-note/lib/redis.ts            # Redis client singleton
memento-note/app/api/usage/current/route.ts    # GET current quota
memento-note/app/api/cron/sync-usage/route.ts  # CRON sync Redis→PG
memento-note/components/usage-meter.tsx        # UI component
```

### Files to MODIFY (UPDATE):
```
memento-note/prisma/schema.prisma     # Add Subscription, UsageLog models
memento-note/components/sidebar.tsx  # Add UsageMeter to footer
memento-note/middleware.ts           # Add 402 handling for quota
memento-note/app/api/chat/route.ts   # Add canUseFeature() before AI call
```

### Testing Standards

- Unit tests for `canUseFeature()` with mocked Redis
- Unit tests for `trackFeatureUsage()` with Redis pipeline verification
- Integration tests for 402 response flow
- UI tests: verify UsageMeter renders correct progress

---

## Dev Agent Guardrails

### Technical Requirements

- **Redis client:** Use `@upstash/redis` for server-side queries (NOT client-side SDK). Self-hosted Redis via `docker-compose.yml` (see `saas-deployment-prep.md` Section I).
- **NFR-SC2 (< 10ms):** Use Redis GET + pipeline INCRBY — no read-modify-write patterns.
- **Atomic counters:** Always use `redis.pipeline().incrby()` not `GET → compute → SET`.
- **Async tracking:** `trackFeatureUsage()` must be fire-and-forget — do NOT await in the hot path.
- **Period key:** Use `new Date().toISOString().slice(0, 7)` for `YYYY-MM` format — UTC, not local.

### Architecture Compliance

- **Starter Pack limits (from saas-deployment-prep.md):**
  - BASIC: `semanticSearch: 30, autoTag: 20, autoTitle: 10` (lifetime, not monthly)
  - PRO: `semanticSearch: 100, autoTag: 200, autoTitle: 200, reformulate: 50, chat: 100` (monthly)
  - BUSINESS: `semanticSearch: 1000, autoTag: 1000, autoTitle: 1000, reformulate: 500, chat: 1000` (monthly)
- **Redis key format:** `usage:{userId}:{feature}:{YYYY-MM}` with 90-day TTL
- **CRON sync interval:** Every 5 minutes via Vercel Cron or node-cron
- **PostgreSQL sync:** Use UPSERT (INSERT ... ON CONFLICT UPDATE) via Prisma `$upsert()`

### Library / Framework Requirements

- **Redis:** `docker-compose.yml` with `redis:7-alpine` (self-hosted, no Upstash cost)
- **Prisma:** Already in use (`@prisma/client@5.22.0`)
- **React Query:** Already in use (`@tanstack/react-query@5.100.9`) for UsageMeter polling
- **AI SDK:** Already in use (`ai@6.0.23`) for provider calls

### File Structure Requirements

- Follow existing patterns in `memento-note/lib/` — TypeScript files, no default exports
- Use `import { redis } from '@/lib/redis'` singleton pattern (see existing `lib/prisma.ts`)
- API routes follow Next.js App Router: `app/api/[resource]/route.ts`

### Testing Requirements

- Use `vitest` (already configured)
- Mock Redis with `vi.mock('@upstash/redis')`
- No database tests — mock Prisma with `vi.mock('@prisma/client')`
- Target 80% coverage for `entitlements.ts`

---

## Previous Story Intelligence

N/A — This is the first story in Epic 3.

---

## Git Intelligence Summary

**Last 5 commits on modified paths:**

| Commit | Change |
|--------|--------|
| `195e845` | security: fix SQL injection in semantic search - use parameterized queries with bind params |
| `ff664f7` | fix: add missing await on reciprocalRankFusion call |
| `41596c2` | fix: openrouter provider fallback to CUSTOM_OPENAI_API_KEY |
| `cf2786d` | feat: migrate semantic search to pgvector + full-text search |
| `330c0c6` | feat: integrate Google Gemini, MiniMax, and GLM providers |

**Key insight:** Recent commits show security hardening on SQL queries — quota tracking must use parameterized queries for any SQL, and Redis must be used for the fast path.

---

## Latest Technical Information

**Redis Self-Hosted (from `saas-deployment-prep.md`):**
```yaml
# docker-compose.yml
redis:
  image: redis:7-alpine
  command: redis-server --requirepass ${REDIS_PASSWORD} --appendonly yes --maxmemory 256mb --maxmemory-policy allkeys-lru
  volumes:
    - redis_data:/data
  ports:
    - "127.0.0.1:6379:6379"
```

**@upstash/redis** package is NOT in package.json — use `ioredis` instead (already a dependency of socket.io):

```typescript
// lib/redis.ts
import Redis from 'ioredis';

const redis = new Redis({
  host: process.env.REDIS_HOST ?? 'localhost',
  port: parseInt(process.env.REDIS_PORT ?? '6379'),
  password: process.env.REDIS_PASSWORD,
  lazyConnect: true,
});

export { redis };
```

**Subscription tiers** defined in `saas-deployment-prep.md` Section B:
- BASIC: free with "AI Discovery Pack" (lifetime limits)
- PRO: €9.90/month (monthly limits)
- BUSINESS: €29.90/month (higher monthly limits)
- ENTERPRISE: €49.90 + €3.90/user/month

---

## Project Context Reference

- **PRD:** `docs/prd.md` — Product Requirements Document with full business context
- **Architecture:** `memento-note/docs/saas-deployment-prep.md` — V3 SaaS architecture including Redis quota system
- **BYOK/Billing:** `memento-note/docs/byok-billing-patch-v3.md` — Full technical spec for quota + BYOK + host-pays
- **UX Spec:** `docs/ux-design-specification.md` — Usage meter in sidebar footer (Section: Emplacement Quotas)
- **Epics:** `docs/epics.md` — Epic 3 with Story 3.1 requirements

---

## Story Completion Status

- Status: ready-for-dev
- Completion Note: Ultimate context engine analysis completed — comprehensive developer guide created
- Story ID: 3.1
- Story Key: 3-1-freemium-quota-tracking