Keep/docs/deployment-guide.md

# Deployment Guide - Memento Project

## Overview

Complete deployment guide for the Memento note-taking application using Docker and Docker Compose. Covers production deployment, environment configuration, and service orchestration.

**Architecture:** Multi-container deployment with keep-notes (Next.js web app) and mcp-server (Express MCP server)

---

## Prerequisites

### Required Software

| Tool | Version | Purpose |
|------|---------|---------|
| **Docker** | 20.10+ | Container runtime |
| **Docker Compose** | 2.0+ | Multi-container orchestration |
| **Git** | Latest | Clone repository |

### Verify Installation

```bash
docker --version
docker compose version
```

---

## Quick Start (Docker Compose)

### 1. Clone Repository

```bash
git clone <repository-url>
cd Keep
```

### 2. Configure Environment

```bash
# Copy environment template
cp .env.example .env

# Edit with your configuration
nano .env
```

### 3. Start Services

```bash
# Build and start all services
docker compose up -d

# View logs
docker compose logs -f

# Check service status
docker compose ps
```

### 4. Access Application

- **Web Application:** http://localhost:3000
- **Prisma Studio:** http://localhost:5555 (when running)

### 5. Stop Services

```bash
# Stop all services
docker compose down

# Stop and remove volumes (⚠️ deletes data)
docker compose down -v
```

---

## Docker Configuration Files

### Root Directory Structure

```
Keep/
├── docker-compose.yml           # Main orchestration
├── .env                         # Environment variables
├── keep-notes/
│   ├── Dockerfile              # Web app container
│   ├── .dockerignore           # Build exclusions
│   └── next.config.ts          # Next.js config
└── mcp-server/
    ├── Dockerfile              # MCP server container
    └── .dockerignore           # Build exclusions
```

---

## Docker Compose Configuration

### Complete docker-compose.yml

```yaml
version: '3.8'

services:
  # ============================================
  # keep-notes - Next.js Web Application
  # ============================================
  keep-notes:
    build:
      context: ./keep-notes
      dockerfile: Dockerfile
    container_name: memento-web
    ports:
      - "3000:3000"
    environment:
      - DATABASE_URL=file:/app/prisma/dev.db
      - NEXTAUTH_SECRET=${NEXTAUTH_SECRET}
      - NEXTAUTH_URL=${NEXTAUTH_URL:-http://localhost:3000}
      - NODE_ENV=production

      # Email Configuration (SMTP)
      - SMTP_HOST=${SMTP_HOST}
      - SMTP_PORT=${SMTP_PORT:-587}
      - SMTP_USER=${SMTP_USER}
      - SMTP_PASS=${SMTP_PASS}
      - SMTP_FROM=${SMTP_FROM:-noreply@memento.app}

      # AI Providers
      - OPENAI_API_KEY=${OPENAI_API_KEY}
      - OLLAMA_API_URL=${OLLAMA_API_URL:-http://ollama:11434}
    volumes:
      - db-data:/app/prisma
      - uploads-data:/app/public/uploads
    depends_on:
      - ollama
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:3000/api/health"]
      interval: 30s
      timeout: 10s
      retries: 3
      start_period: 40s
    networks:
      - memento-network

  # ============================================
  # mcp-server - MCP Protocol Server
  # ============================================
  mcp-server:
    build:
      context: ./mcp-server
      dockerfile: Dockerfile
    container_name: memento-mcp
    volumes:
      - db-data:/app/db
    depends_on:
      - keep-notes
    restart: unless-stopped
    networks:
      - memento-network

  # ============================================
  # Ollama - Local LLM Provider (Optional)
  # ============================================
  ollama:
    image: ollama/ollama:latest
    container_name: memento-ollama
    ports:
      - "11434:11434"
    volumes:
      - ollama-data:/root/.ollama
    restart: unless-stopped
    networks:
      - memento-network

  # ============================================
  # Prisma Studio - Database GUI (Optional)
  # ============================================
  prisma-studio:
    build:
      context: ./keep-notes
      dockerfile: Dockerfile
    container_name: memento-studio
    command: npx prisma studio
    ports:
      - "5555:5555"
    environment:
      - DATABASE_URL=file:/app/prisma/dev.db
    volumes:
      - db-data:/app/prisma
    depends_on:
      - keep-notes
    restart: unless-stopped
    networks:
      - memento-network
    profiles:
      - admin  # Only start with: docker compose --profile admin up

# ============================================
# Volumes - Data Persistence
# ============================================
volumes:
  db-data:
    driver: local
  uploads-data:
    driver: local
  ollama-data:
    driver: local

# ============================================
# Networks - Service Communication
# ============================================
networks:
  memento-network:
    driver: bridge
```

---

## Dockerfile: keep-notes

### Complete Dockerfile

```dockerfile
# ============================================
# Multi-stage build for Next.js 16
# ============================================

# Stage 1: Dependencies
FROM node:20-alpine AS deps
RUN apk add --no-cache libc6-compat
WORKDIR /app

# Install dependencies based on package manager
COPY package*.json ./
RUN npm ci

# Stage 2: Builder
FROM node:20-alpine AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .

# Generate Prisma client
RUN npx prisma generate

# Build Next.js application
ENV NEXT_TELEMETRY_DISABLED=1
RUN npm run build

# Stage 3: Production Runner
FROM node:20-alpine AS runner
WORKDIR /app

ENV NODE_ENV=production
ENV NEXT_TELEMETRY_DISABLED=1

# Create non-root user
RUN addgroup --system --gid 1001 nodejs
RUN adduser --system --uid 1001 nextjs

# Copy necessary files
COPY --from=builder /app/public ./public
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/prisma ./prisma
COPY --from=builder /app/node_modules/.prisma ./node_modules/.prisma
COPY --from=builder /app/node_modules/@prisma ./node_modules/@prisma

# Create directories with proper permissions
RUN mkdir -p /app/prisma /app/public/uploads/notes
RUN chown -R nextjs:nodejs /app

USER nextjs

EXPOSE 3000

ENV PORT=3000
ENV HOSTNAME="0.0.0.0"

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=40s --retries=3 \
  CMD curl -f http://localhost:3000/api/health || exit 1

CMD ["node", "server.js"]
```

### Update next.config.ts for Standalone Output

```typescript
// keep-notes/next.config.ts
import type { NextConfig } from 'next'

const nextConfig: NextConfig = {
  // ... existing config

  // Enable standalone output for Docker
  output: 'standalone',

  // Optimize for production
  reactStrictMode: true,
  swcMinify: true,

  // Image optimization
  images: {
    unoptimized: true, // Required for standalone
  },
}

export default nextConfig
```

### .dockerignore for keep-notes

```
node_modules
.next
.git
.gitignore
*.md
.env*.local
.vscode
idea
.DS_Store
*.log
coverage
test-results
playwright-report
```

---

## Dockerfile: mcp-server

### Complete Dockerfile

```dockerfile
# Base image
FROM node:20-alpine

# Install dependencies
WORKDIR /app

# Copy package files
COPY package*.json ./

# Install dependencies
RUN npm ci --only=production

# Copy application code
COPY . .

# Copy Prisma schema and client
COPY prisma ./prisma

# Create non-root user
RUN addgroup -g 1001 -S mcp && \
    adduser -u 1001 -S mcp -G mcp

# Create database directory
RUN mkdir -p /app/db && \
    chown -R mcp:mcp /app

USER mcp

WORKDIR /app

# Expose (not needed for stdio, but useful for SSE variant)
EXPOSE 3000

# Health check
HEALTHCHECK --interval=30s --timeout=10s --start-period=10s --retries=3 \
  CMD node -e "console.log('healthy')" || exit 1

# Start MCP server
CMD ["node", "index.js"]
```

### .dockerignore for mcp-server

```
node_modules
.git
.gitignore
*.md
.env
.env.*
*.log
.DS_Store
index-sse.js  # Only use main index.js
README-SSE.md
N8N-CONFIG.md
```

---

## Environment Configuration

### .env.example (Root Directory)

```bash
# ============================================
# Database Configuration
# ============================================
DATABASE_URL="file:/app/prisma/dev.db"

# ============================================
# NextAuth Configuration
# ============================================
# Generate with: openssl rand -base64 32
NEXTAUTH_SECRET="your-secret-key-here-min-32-chars"
NEXTAUTH_URL="http://localhost:3000"

# ============================================
# Email Configuration (SMTP)
# ============================================
# Required for password reset and reminders
SMTP_HOST="smtp.gmail.com"
SMTP_PORT="587"
SMTP_USER="your-email@gmail.com"
SMTP_PASS="your-app-password"
SMTP_FROM="noreply@memento.app"

# ============================================
# AI Provider Configuration
# ============================================

# OpenAI (Optional - for GPT models)
OPENAI_API_KEY="sk-..."

# Ollama (Optional - for local models)
OLLAMA_API_URL="http://ollama:11434"

# ============================================
# Application Settings
# ============================================
NODE_ENV="production"
PORT="3000"

# ============================================
# Docker-Specific Settings
# ============================================
# These are usually set in docker-compose.yml
# Keep for local development reference
```

### Generate NEXTAUTH_SECRET

```bash
# Linux/Mac
openssl rand -base64 32

# Windows (PowerShell)
-join ((48..57) + (65..90) + (97..122) | Get-Random -Count 32 | % {[char]$_})
```

---

## Production Deployment

### Option 1: Docker Compose (Recommended)

```bash
# 1. Clone repository
git clone <repository-url>
cd Keep

# 2. Configure environment
cp .env.example .env
nano .env  # Edit configuration

# 3. Start services
docker compose up -d

# 4. Initialize database (first time only)
docker compose exec keep-notes npx prisma migrate deploy

# 5. Create admin user (optional)
docker compose exec keep-notes node scripts/promote-admin.js your-email@example.com

# 6. Check status
docker compose ps
docker compose logs -f keep-notes
```

### Option 2: Individual Containers

#### Build keep-notes Image

```bash
cd keep-notes
docker build -t memento-web:latest .
docker run -d \
  --name memento-web \
  -p 3000:3000 \
  -v $(pwd)/prisma:/app/prisma \
  -e DATABASE_URL=file:/app/prisma/dev.db \
  -e NEXTAUTH_SECRET=your-secret \
  -e NEXTAUTH_URL=http://localhost:3000 \
  memento-web:latest
```

#### Build mcp-server Image

```bash
cd mcp-server
docker build -t memento-mcp:latest .
docker run -d \
  --name memento-mcp \
  -v /path/to/keep-notes/prisma:/app/db \
  memento-mcp:latest
```

---

## Database Management in Docker

### Run Migrations

```bash
# Apply pending migrations
docker compose exec keep-notes npx prisma migrate deploy

# Create new migration
docker compose exec keep-notes npx prisma migrate dev --name my_changes

# Reset database (⚠️ deletes all data)
docker compose exec keep-notes npx prisma migrate reset
```

### Seed Database (Optional)

```bash
# Create seed file
# keep-notes/prisma/seed.ts

# Run seed
docker compose exec keep-notes npx prisma db seed
```

### Access Database Directly

```bash
# Open SQLite CLI
docker compose exec keep-notes sqlite3 /app/prisma/dev.db

# Query notes
SELECT * FROM Note LIMIT 10;

# Exit
.quit
```

### Backup Database

```bash
# Backup database
docker compose exec keep-notes \
  sqlite3 /app/prisma/dev.db ".backup /app/prisma/backup.db"

# Copy backup to host
docker cp memento-web:/app/prisma/backup.db ./backup-$(date +%Y%m%d).db
```

### Restore Database

```bash
# Copy backup to container
docker cp ./backup.db memento-web:/app/prisma/restore.db

# Restore
docker compose exec keep-notes \
  sqlite3 /app/prisma/dev.db ".restore /app/prisma/restore.db"
```

---

## Service Management

### View Logs

```bash
# All services
docker compose logs -f

# Specific service
docker compose logs -f keep-notes
docker compose logs -f mcp-server

# Last 100 lines
docker compose logs --tail=100 keep-notes
```

### Restart Services

```bash
# Restart all
docker compose restart

# Restart specific service
docker compose restart keep-notes

# Rebuild and restart
docker compose up -d --build
```

### Update Services

```bash
# Pull latest code
git pull

# Rebuild and restart
docker compose up -d --build

# Or force rebuild without cache
docker compose build --no-cache
docker compose up -d
```

### Scale Services

```bash
# Scale keep-notes (requires load balancer)
docker compose up -d --scale keep-notes=3
```

---

## Reverse Proxy Configuration

### Nginx Configuration

```nginx
# /etc/nginx/sites-available/memento
server {
    listen 80;
    server_name your-domain.com;

    # Redirect to HTTPS
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl http2;
    server_name your-domain.com;

    # SSL certificates
    ssl_certificate /etc/letsencrypt/live/your-domain.com/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/your-domain.com/privkey.pem;

    # Next.js application
    location / {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection 'upgrade';
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_cache_bypass $http_upgrade;
    }

    # WebSocket support (if needed)
    location /ws {
        proxy_pass http://localhost:3000;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
    }
}
```

### Traefik Configuration (Alternative)

```yaml
# docker-compose.yml addition
services:
  traefik:
    image: traefik:v2.10
    command:
      - "--api.insecure=true"
      - "--providers.docker=true"
      - "--entrypoints.web.address=:80"
      - "--entrypoints.websecure.address=:443"
    ports:
      - "80:80"
      - "443:443"
      - "8080:8080"  # Dashboard
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"

  keep-notes:
    # ... existing config
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.keep-notes.rule=Host(`your-domain.com`)"
      - "traefik.http.routers.keep-notes.entrypoints=websecure"
      - "traefik.http.services.keep-notes.loadbalancer.server.port=3000"
```

---

## SSL/TLS Setup

### Let's Encrypt with Certbot

```bash
# Install Certbot
sudo apt install certbot python3-certbot-nginx

# Get certificate
sudo certbot --nginx -d your-domain.com

# Auto-renewal (configured automatically)
sudo certbot renew --dry-run
```

### Self-Signed Certificate (Development)

```bash
# Generate self-signed certificate
openssl req -x509 -nodes -days 365 -newkey rsa:2048 \
  -keyout /etc/ssl/private/memento-selfsigned.key \
  -out /etc/ssl/certs/memento-selfsigned.crt
```

---

## Monitoring & Logging

### Container Health

```bash
# Check health status
docker compose ps

# Inspect container
docker inspect memento-web

# Resource usage
docker stats
```

### Log Aggregation

```bash
# View logs from all services
docker compose logs > logs-$(date +%Y%m%d).txt

# Rotate logs
logrotate configure in /etc/logrotate.d/docker-compose
```

### Monitoring Tools

**Prometheus + Grafana** (Optional):
```yaml
# docker-compose.yml addition
services:
  prometheus:
    image: prom/prometheus
    volumes:
      - ./prometheus.yml:/etc/prometheus/prometheus.yml
    ports:
      - "9090:9090"

  grafana:
    image: grafana/grafana
    ports:
      - "3001:3000"
    volumes:
      - grafana-data:/var/lib/grafana
```

---

## Backup & Disaster Recovery

### Automated Backup Script

```bash
#!/bin/bash
# backup.sh - Daily backup script

BACKUP_DIR="/backups/memento"
DATE=$(date +%Y%m%d_%H%M%S)

# Create backup directory
mkdir -p "$BACKUP_DIR/$DATE"

# Backup database
docker compose exec keep-notes \
  sqlite3 /app/prisma/dev.db ".backup /app/prisma/backup.db"

# Copy to host
docker cp memento-web:/app/prisma/backup.db \
  "$BACKUP_DIR/$DATE/database.db"

# Backup uploads
docker cp memento-web:/app/public/uploads \
  "$BACKUP_DIR/$DATE/"

# Compress
tar -czf "$BACKUP_DIR/memento-$DATE.tar.gz" -C "$BACKUP_DIR" $DATE
rm -rf "$BACKUP_DIR/$DATE"

# Keep last 7 days
find "$BACKUP_DIR" -name "*.tar.gz" -mtime +7 -delete

echo "Backup completed: memento-$DATE.tar.gz"
```

### Schedule with Cron

```bash
# Edit crontab
crontab -e

# Add daily backup at 2 AM
0 2 * * * /path/to/backup.sh
```

### Restore from Backup

```bash
# Extract backup
tar -xzf memento-20240109_020000.tar.gz

# Restore database
docker cp memento-20240109_020000/database.db \
  memento-web:/app/prisma/restore.db

docker compose exec keep-notes \
  sqlite3 /app/prisma/dev.db ".restore /app/prisma/restore.db"

# Restore uploads
docker cp memento-20240109_020000/uploads/. \
  memento-web:/app/public/uploads/

# Restart services
docker compose restart
```

---

## Security Hardening

### 1. Docker Security

```yaml
# docker-compose.yml hardening
services:
  keep-notes:
    # ... existing config
    security_opt:
      - no-new-privileges:true
    read_only: true
    tmpfs:
      - /tmp
    cap_drop:
      - ALL
    cap_add:
      - NET_BIND_SERVICE
```

### 2. Network Security

```yaml
# Isolate services
networks:
  memento-network:
    driver: bridge
    internal: false  # Set to true to block external access
    ipam:
      config:
        - subnet: 172.20.0.0/16
```

### 3. Secrets Management

```bash
# Use Docker Secrets (Swarm mode)
echo "your-secret" | docker secret create nextauth_secret -

# In docker-compose.yml
services:
  keep-notes:
    secrets:
      - nextauth_secret
    environment:
      NEXTAUTH_SECRET_FILE: /run/secrets/nextauth_secret

secrets:
  nextauth_secret:
    external: true
```

### 4. Update Base Images

```bash
# Update to latest Alpine
docker pull node:20-alpine

# Rebuild
docker compose build --no-cache
```

---

## Troubleshooting

### Container Won't Start

```bash
# Check logs
docker compose logs keep-notes

# Inspect container
docker inspect memento-web

# Check resource usage
docker stats

# Restart Docker daemon
sudo systemctl restart docker
```

### Database Connection Issues

```bash
# Verify database file exists
docker compose exec keep-notes ls -la /app/prisma/dev.db

# Check permissions
docker compose exec keep-notes ls -la /app/prisma/

# Re-run migrations
docker compose exec keep-notes npx prisma migrate deploy
```

### Port Conflicts

```bash
# Check what's using port 3000
sudo lsof -i :3000

# Change port in docker-compose.yml
ports:
  - "3001:3000"  # Use 3001 instead
```

### Out of Memory

```bash
# Increase Docker memory limit
# Docker Desktop -> Settings -> Resources -> Memory: 4GB+

# Or add memory limit in docker-compose.yml
services:
  keep-notes:
    deploy:
      resources:
        limits:
          memory: 2G
```

### Permission Issues

```bash
# Fix file permissions
docker compose exec keep-notes chown -R nextjs:nodejs /app/prisma
docker compose exec keep-notes chmod 644 /app/prisma/dev.db
```

---

## Performance Optimization

### 1. Enable BuildKit

```bash
# Set environment variable
export DOCKER_BUILDKIT=1
export COMPOSE_DOCKER_CLI_BUILD=1
```

### 2. Multi-Stage Build

Already implemented in Dockerfiles to reduce image size.

### 3. Layer Caching

```dockerfile
# Cache node_modules
COPY package*.json ./
RUN npm ci
COPY . .
```

### 4. Resource Limits

```yaml
# docker-compose.yml
services:
  keep-notes:
    deploy:
      resources:
        limits:
          cpus: '1'
          memory: 2G
        reservations:
          cpus: '0.5'
          memory: 1G
```

### 5. Database Optimization

```bash
# Run VACUUM to optimize SQLite
docker compose exec keep-notes sqlite3 /app/prisma/dev.db "VACUUM;"

# Analyze query performance
docker compose exec keep-notes npx prisma studio
```

---

## CI/CD Integration

### GitHub Actions Example

```yaml
# .github/workflows/deploy.yml
name: Deploy to Production

on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKER_USERNAME }}
          password: ${{ secrets.DOCKER_PASSWORD }}

      - name: Build and push
        run: |
          docker build -t yourusername/memento-web:latest ./keep-notes
          docker push yourusername/memento-web:latest

      - name: Deploy to server
        uses: appleboy/ssh-action@master
        with:
          host: ${{ secrets.HOST }}
          username: ${{ secrets.USERNAME }}
          key: ${{ secrets.SSH_KEY }}
          script: |
            cd /app/memento
            docker compose pull
            docker compose up -d
```

---

## Development vs Production

### Development (Local)

```bash
# Use local Next.js dev server
npm run dev

# Hot reload enabled
# Source maps enabled
# Detailed error messages
```

### Production (Docker)

```bash
# Use optimized production build
docker compose up -d

# Minified code
# Optimized images
# Server-side rendering
# Health checks enabled
```

---

## Migration from Local to Docker

### 1. Export Existing Data

```bash
# From local development
sqlite3 keep-notes/prisma/dev.db ".dump" > backup.sql

# Or copy database file
cp keep-notes/prisma/dev.db ./dev-backup.db
```

### 2. Import to Docker

```bash
# Copy to container
docker cp ./dev-backup.db memento-web:/app/prisma/dev.db

# Verify
docker compose exec keep-notes sqlite3 /app/prisma/dev.db "SELECT COUNT(*) FROM Note;"
```

### 3. Update Environment

```bash
# Copy .env to .env.production
cp .env .env.production

# Update URLs for production
NEXTAUTH_URL="https://your-domain.com"
```

---

## Update Strategy

### Rolling Updates

```bash
# Zero-downtime deployment
docker compose up -d --no-deps --build keep-notes

# Or use multiple instances
docker compose up -d --scale keep-notes=2
```

### Rollback

```bash
# Revert to previous version
docker compose down
git checkout <previous-commit>
docker compose up -d --build
```

---

## Cost Estimation (Cloud Deployment)

### Self-Hosted Options

| Provider | Instance | Monthly Cost | Capacity |
|----------|----------|--------------|----------|
| DigitalOcean | Basic Droplet | $6-12 | 1-2 GB RAM, 1 vCPU |
| Linode | Nanode | $5 | 1 GB RAM, 1 vCPU |
| AWS EC2 | t3.micro | $8-15 | 1 GB RAM, 2 vCPU |
| Google Cloud | e2-micro | $6-12 | 1 GB RAM, 1 vCPU |
| Hetzner | CX11 | €4.20 | 4 GB RAM, 1 vCPU |

### Managed Services

| Service | Cost | Notes |
|---------|------|-------|
| Render | Free+$7/plan | Free tier available |
| Railway | $5-20/plan | Simple setup |
| Fly.io | $0-16/plan | Pay per use |
| Vercel | $0-20/plan | Next.js optimized |

---

## Summary

This deployment guide provides:
- ✅ Complete Docker setup for both services
- ✅ Docker Compose orchestration
- ✅ Production-ready configuration
- ✅ Database management in containers
- ✅ SSL/TLS setup
- ✅ Backup and restore procedures
- ✅ Security hardening
- ✅ Monitoring and troubleshooting
- ✅ CI/CD integration

**Quick Start:**
```bash
git clone <repo> && cd Keep
cp .env.example .env && nano .env
docker compose up -d
```

**Access:** http://localhost:3000

---

## Next Steps

After deployment:
1. Create admin user
2. Configure AI providers (OpenAI or Ollama)
3. Set up email for password reset
4. Configure backups
5. Enable SSL/TLS for production
6. Set up monitoring
7. Review security settings