Momento/docs/3-1-freemium-quota-tracking.md

Story 3.1: Freemium "AI Discovery Pack" Quota Tracking

Status: ready-for-dev

---

Story

As a business, I want to track Freemium usage against a Redis quota limit, So that I can limit my API cost exposure for free users.

Given a free user triggers an AI request When the system intercepts the request Then the quota is tracked and the UI updates And (NFR-SC2) the Redis-backed check resolves in under 10ms.

---

Epic Context

Epic 3: The SaaS Commercial Engine (Monetization & API Cost Protection)

Epic Business Value: The core backend logic allowing us to sell the product without bleeding API costs — freemium limits, router fallback, host-pays.

Cross-Story Dependencies:

• Story 3.1 (this story) establishes the quota tracking foundation
• Story 3.2 builds on this with the LLM Router and provider routing
• Story 3.3 adds smart-routing fallback when quota is low
• Story 3.4 (Host-Pays) extends quota tracking to collaborative sessions
• Story 3.5 (BYOK) bypasses quota for users with their own API keys
• Story 3.6 (Stripe) manages tier upgrades

Technical Constraint: (NFR-SC2) Redis-backed entitlement checks must complete in under 10ms.

---

Acceptance Criteria

1. [AC1] When a free user (BASIC tier) makes an AI request, the system checks Redis for current usage count
2. [AC2] Each AI feature (semantic_search, auto_tag, auto_title) has its own Redis counter with format: usage:{userId}:{feature}:{YYYY-MM}
3. [AC3] Counter increments atomically via Redis INCRBY (not read-modify-write) to avoid race conditions
4. [AC4] If counter >= limit, return HTTP 402 with body { error: "QUOTA_EXCEEDED", feature, upgradeTier: "PRO", byokConfigured }
5. [AC5] If counter < limit, allow request to proceed and increment counter asynchronously (fire-and-forget)
6. [AC6] Redis keys have 90-day TTL to auto-cleanup (covers grace period for monthly reconciliation)
7. [AC7] The Sidebar footer displays a usage gauge component showing Discovery Pack consumption in real-time

---

Tasks / Subtasks

• Task 1: Add Prisma models for Subscription, UsageLog (AC: #1)
  • Subtask 1.1: Add Subscription model with tier/stripe fields
  • Subtask 1.2: Add UsageLog model for PostgreSQL sync target
  • Subtask 1.3: Create migration file
• Task 2: Create lib/entitlements.ts with Redis-backed canUseFeature() (AC: #2, #3, #4)
  • Subtask 2.1: Implement getCurrentPeriodKey() returning YYYY-MM format
  • Subtask 2.2: Implement Redis key format: usage:{userId}:{feature}:{YYYY-MM}
  • Subtask 2.3: Implement atomic canUseFeature() with < 10ms target (use Redis GET + pipeline INCRBY)
  • Subtask 2.4: Return QuotaExceededError when limit exceeded
• Task 3: Create lib/usage-tracker.ts with trackFeatureUsage() (AC: #5)
  • Subtask 3.1: Fire-and-forget Redis pipeline increment
  • Subtask 3.2: Include tokensUsed in metadata
• Task 4: Create /api/usage/current endpoint (AC: #6)
  • Subtask 4.1: Return remaining quota for all features for authenticated user
• Task 5: Create <UsageMeter> UI component for Sidebar footer (AC: #7)
  • Subtask 5.1: Show progress bar for Discovery Pack (semantic_search: 30 lifetime, auto_tag: 20, auto_title: 10)
  • Subtask 5.2: Real-time updates via React Query polling every 30s
  • Subtask 5.3: Show "Upgrade to Pro" paywall modal on 402 response
• Task 6: Create CRON sync worker /api/cron/sync-usage (AC: #6)
  • Subtask 6.1: Batch sync Redis counters → PostgreSQL UsageLog
  • Subtask 6.2: Handle monthly reset (new period = reset counters for that user/feature)

---

Dev Notes

Project Structure Notes

• Code base: memento-note/ (Next.js app with App Router)
• Redis client: Currently memento-note/lib/rate-limit.ts uses in-memory Map (INSUFFICIENT). This story replaces it with actual Redis.
• No existing lib/entitlements.ts — creates from scratch
• No existing lib/usage-tracker.ts — creates from scratch
• Sidebar: memento-note/components/sidebar.tsx is the anchor for the UsageMeter UI
• AI Factory: memento-note/lib/ai/factory.ts has provider creation logic — quota checks must integrate BEFORE provider resolution

Files to CREATE (NEW):

memento-note/lib/entitlements.ts       # Core quota check logic
memento-note/lib/usage-tracker.ts     # Track usage via Redis
memento-note/lib/redis.ts            # Redis client singleton
memento-note/app/api/usage/current/route.ts    # GET current quota
memento-note/app/api/cron/sync-usage/route.ts  # CRON sync Redis→PG
memento-note/components/usage-meter.tsx        # UI component

Files to MODIFY (UPDATE):

memento-note/prisma/schema.prisma     # Add Subscription, UsageLog models
memento-note/components/sidebar.tsx  # Add UsageMeter to footer
memento-note/middleware.ts           # Add 402 handling for quota
memento-note/app/api/chat/route.ts   # Add canUseFeature() before AI call

Testing Standards

• Unit tests for canUseFeature() with mocked Redis
• Unit tests for trackFeatureUsage() with Redis pipeline verification
• Integration tests for 402 response flow
• UI tests: verify UsageMeter renders correct progress

---

Dev Agent Guardrails

Technical Requirements

• Redis client: Use @upstash/redis for server-side queries (NOT client-side SDK). Self-hosted Redis via docker-compose.yml (see saas-deployment-prep.md Section I).
• NFR-SC2 (< 10ms): Use Redis GET + pipeline INCRBY — no read-modify-write patterns.
• Atomic counters: Always use redis.pipeline().incrby() not GET → compute → SET.
• Async tracking: trackFeatureUsage() must be fire-and-forget — do NOT await in the hot path.
• Period key: Use new Date().toISOString().slice(0, 7) for YYYY-MM format — UTC, not local.

Architecture Compliance

• Starter Pack limits (from saas-deployment-prep.md):
  • BASIC: semanticSearch: 30, autoTag: 20, autoTitle: 10 (lifetime, not monthly)
  • PRO: semanticSearch: 100, autoTag: 200, autoTitle: 200, reformulate: 50, chat: 100 (monthly)
  • BUSINESS: semanticSearch: 1000, autoTag: 1000, autoTitle: 1000, reformulate: 500, chat: 1000 (monthly)
• Redis key format: usage:{userId}:{feature}:{YYYY-MM} with 90-day TTL
• CRON sync interval: Every 5 minutes via Vercel Cron or node-cron
• PostgreSQL sync: Use UPSERT (INSERT ... ON CONFLICT UPDATE) via Prisma $upsert()

Library / Framework Requirements

• Redis: docker-compose.yml with redis:7-alpine (self-hosted, no Upstash cost)
• Prisma: Already in use (@prisma/client@5.22.0)
• React Query: Already in use (@tanstack/react-query@5.100.9) for UsageMeter polling
• AI SDK: Already in use (ai@6.0.23) for provider calls

File Structure Requirements

• Follow existing patterns in memento-note/lib/ — TypeScript files, no default exports
• Use import { redis } from '@/lib/redis' singleton pattern (see existing lib/prisma.ts)
• API routes follow Next.js App Router: app/api/[resource]/route.ts

Testing Requirements

• Use vitest (already configured)
• Mock Redis with vi.mock('@upstash/redis')
• No database tests — mock Prisma with vi.mock('@prisma/client')
• Target 80% coverage for entitlements.ts

---

Previous Story Intelligence

N/A — This is the first story in Epic 3.

---

Git Intelligence Summary

Last 5 commits on modified paths:

Commit	Change
195e845	security: fix SQL injection in semantic search - use parameterized queries with bind params
ff664f7	fix: add missing await on reciprocalRankFusion call
41596c2	fix: openrouter provider fallback to CUSTOM_OPENAI_API_KEY
cf2786d	feat: migrate semantic search to pgvector + full-text search
330c0c6	feat: integrate Google Gemini, MiniMax, and GLM providers

Key insight: Recent commits show security hardening on SQL queries — quota tracking must use parameterized queries for any SQL, and Redis must be used for the fast path.

---

Latest Technical Information

Redis Self-Hosted (from saas-deployment-prep.md):

# docker-compose.yml
redis:
  image: redis:7-alpine
  command: redis-server --requirepass ${REDIS_PASSWORD} --appendonly yes --maxmemory 256mb --maxmemory-policy allkeys-lru
  volumes:
    - redis_data:/data
  ports:
    - "127.0.0.1:6379:6379"

@upstash/redis package is NOT in package.json — use ioredis instead (already a dependency of socket.io):

// lib/redis.ts
import Redis from 'ioredis';

const redis = new Redis({
  host: process.env.REDIS_HOST ?? 'localhost',
  port: parseInt(process.env.REDIS_PORT ?? '6379'),
  password: process.env.REDIS_PASSWORD,
  lazyConnect: true,
});

export { redis };

Subscription tiers defined in saas-deployment-prep.md Section B:

• BASIC: free with "AI Discovery Pack" (lifetime limits)
• PRO: €9.90/month (monthly limits)
• BUSINESS: €29.90/month (higher monthly limits)
• ENTERPRISE: €49.90 + €3.90/user/month

---

Project Context Reference

• PRD: docs/prd.md — Product Requirements Document with full business context
• Architecture: memento-note/docs/saas-deployment-prep.md — V3 SaaS architecture including Redis quota system
• BYOK/Billing: memento-note/docs/byok-billing-patch-v3.md — Full technical spec for quota + BYOK + host-pays
• UX Spec: docs/ux-design-specification.md — Usage meter in sidebar footer (Section: Emplacement Quotas)
• Epics: docs/epics.md — Epic 3 with Story 3.1 requirements

---

Story Completion Status

• Status: ready-for-dev
• Completion Note: Ultimate context engine analysis completed — comprehensive developer guide created
• Story ID: 3.1
• Story Key: 3-1-freemium-quota-tracking