fix: improve note interactions and markdown LaTeX support
## Bug Fixes ### Note Card Actions - Fix broken size change functionality (missing state declaration) - Implement React 19 useOptimistic for instant UI feedback - Add startTransition for non-blocking updates - Ensure smooth animations without page refresh - All note actions now work: pin, archive, color, size, checklist ### Markdown LaTeX Rendering - Add remark-math and rehype-katex plugins - Support inline equations with dollar sign syntax - Support block equations with double dollar sign syntax - Import KaTeX CSS for proper styling - Equations now render correctly instead of showing raw LaTeX ## Technical Details - Replace undefined currentNote references with optimistic state - Add optimistic updates before server actions for instant feedback - Use router.refresh() in transitions for smart cache invalidation - Install remark-math, rehype-katex, and katex packages ## Testing - Build passes successfully with no TypeScript errors - Dev server hot-reloads changes correctly
This commit is contained in:
682
docs/integration-architecture.md
Normal file
682
docs/integration-architecture.md
Normal file
@@ -0,0 +1,682 @@
|
||||
# Integration Architecture - Memento Project
|
||||
|
||||
## Overview
|
||||
|
||||
Document describing how the two parts of the Memento project (keep-notes web application and mcp-server) integrate and communicate with each other and external systems.
|
||||
|
||||
---
|
||||
|
||||
## System Architecture Diagram
|
||||
|
||||
```
|
||||
┌─────────────────────────────────────────────────────────────┐
|
||||
│ External Systems │
|
||||
├─────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌─────────┐ │
|
||||
│ │ User │ │ AI │ │ N8N │ │ Email │ │
|
||||
│ │ Browser │ │Assistant │ │ Workflow │ │ Service │ │
|
||||
│ └────┬─────┘ └────┬─────┘ └────┬─────┘ └────┬────┘ │
|
||||
│ │ │ │ │ │
|
||||
└───────┼─────────────┼─────────────┼──────────────┼─────────┘
|
||||
│ │ │ │
|
||||
↓ ↓ ↓ ↓
|
||||
┌───────────────────────────────────────────────────────────────┐
|
||||
│ Memento System │
|
||||
├───────────────────────────────────────────────────────────────┤
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ keep-notes (Next.js Web App) │ │
|
||||
│ │ │ │
|
||||
│ │ Frontend (React/Next.js) │ │
|
||||
│ │ ├─ User Interface (20+ components) │ │
|
||||
│ │ ├─ State Management (Context + Hooks) │ │
|
||||
│ │ └─ Client-Side Routing │ │
|
||||
│ │ │ │
|
||||
│ │ Backend (Next.js API Routes + Server Actions) │ │
|
||||
│ │ ├─ REST API Endpoints (12 routes) │ │
|
||||
│ │ ├─ Server Actions (mutations) │ │
|
||||
│ │ ├─ Authentication (NextAuth) │ │
|
||||
│ │ ├─ AI Integration (Vercel AI SDK) │ │
|
||||
│ │ └─ Cron Jobs (reminders) │ │
|
||||
│ │ │ │
|
||||
│ │ ┌──────────────────────────────────────────────┐ │ │
|
||||
│ │ │ Prisma ORM Layer │ │ │
|
||||
│ │ │ ┌──────────────────────────────────────┐ │ │ │
|
||||
│ │ │ │ SQLite Database (dev.db) │ │ │ │
|
||||
│ │ │ │ │ │ │ │
|
||||
│ │ │ │ - User │ │ │ │
|
||||
│ │ │ │ - Note │ │ │ │
|
||||
│ │ │ │ - Label │ │ │ │
|
||||
│ │ │ │ - Account, Session, etc. │ │ │ │
|
||||
│ │ │ │ │ │ │ │
|
||||
│ │ │ └──────────────────────────────────────┘ │ │ │
|
||||
│ │ └──────────────────────────────────────────────┘ │ │
|
||||
│ └────────────────────────────────────────────────────┘ │
|
||||
│ ↕ │
|
||||
│ Shared Data │
|
||||
│ │
|
||||
│ ┌─────────────────────────────────────────────────────┐ │
|
||||
│ │ mcp-server (Express MCP Server) │ │
|
||||
│ │ │ │
|
||||
│ │ MCP Protocol Layer │ │
|
||||
│ │ ├─ Tools (9 tools: create_note, search, etc.) │ │
|
||||
│ │ ├─ Request/Response Handling │ │
|
||||
│ │ └─ Error Handling │ │
|
||||
│ │ │ │
|
||||
│ │ ┌──────────────────────────────────────────────┐ │ │
|
||||
│ │ │ Prisma Client (Shared) │ │ │
|
||||
│ │ └──────────────┬───────────────────────────────┘ │ │
|
||||
│ │ │ Direct Access │ │
|
||||
│ │ ↓ │ │
|
||||
│ │ ┌──────────────────────────────────────────────┐ │ │
|
||||
│ │ │ ../keep-notes/prisma/dev.db (Same File) │◄───┼────┘
|
||||
│ │ └──────────────────────────────────────────────┘ │
|
||||
│ └────────────────────────────────────────────────────┘ │
|
||||
│ │
|
||||
└────────────────────────────────────────────────────────────┘
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Integration Points
|
||||
|
||||
### 1. Database Integration (Primary)
|
||||
|
||||
**Type:** Direct File Access
|
||||
**Protocol:** SQLite file sharing
|
||||
**Direction:** Both read/write
|
||||
|
||||
**keep-notes → Database:**
|
||||
```typescript
|
||||
// lib/prisma.ts
|
||||
import { PrismaClient } from '@prisma/client'
|
||||
const prisma = new PrismaClient()
|
||||
// Direct connection to prisma/dev.db
|
||||
```
|
||||
|
||||
**mcp-server → Database:**
|
||||
```javascript
|
||||
// index.js
|
||||
const prisma = new PrismaClient({
|
||||
datasources: {
|
||||
db: {
|
||||
url: `file:${join(__dirname, '../keep-notes/prisma/dev.db')}`
|
||||
}
|
||||
}
|
||||
});
|
||||
// Connects to SAME database file
|
||||
```
|
||||
|
||||
**Data Consistency:**
|
||||
- Both parts see same data immediately
|
||||
- No replication lag
|
||||
- Single source of truth
|
||||
- SQLite handles concurrent reads
|
||||
- Single writer limitation (acceptable for current scale)
|
||||
|
||||
**Implications:**
|
||||
- ✅ Real-time data sync
|
||||
- ✅ Simple architecture
|
||||
- ✅ No API needed between parts
|
||||
- ⚠️ Must be on same filesystem (or network mount)
|
||||
- ⚠️ SQLite concurrency limits
|
||||
- ⚠️ Single point of failure (database file)
|
||||
|
||||
---
|
||||
|
||||
### 2. HTTP Integration (Future/Optional)
|
||||
|
||||
**Current:** No HTTP communication between keep-notes and mcp-server
|
||||
|
||||
**Potential HTTP Integration:**
|
||||
```
|
||||
keep-notes → HTTP → mcp-server → Prisma → DB
|
||||
```
|
||||
|
||||
**Use Cases:**
|
||||
- Remote deployment (separate servers)
|
||||
- MCP server as API gateway
|
||||
- Webhook forwarding
|
||||
|
||||
**Current State:** Not implemented (database-mediated preferred)
|
||||
|
||||
---
|
||||
|
||||
### 3. User Authentication Integration
|
||||
|
||||
**keep-notes:**
|
||||
- NextAuth.js v5 for authentication
|
||||
- Session stored in `Session` table (DB)
|
||||
- Protected routes via middleware
|
||||
|
||||
**mcp-server:**
|
||||
- **NO authentication** currently
|
||||
- Direct database access
|
||||
- Trusts client environment
|
||||
|
||||
**Security Implications:**
|
||||
- ⚠️ MCP server has full DB access
|
||||
- ⚠️ No authentication layer
|
||||
- ✅ Acceptable for local/localhost
|
||||
- ❌ Not secure for public deployment
|
||||
|
||||
**Recommendation:** Add API key auth for MCP server in production
|
||||
|
||||
---
|
||||
|
||||
## Data Flow Patterns
|
||||
|
||||
### Pattern 1: Web App Data Flow
|
||||
|
||||
```
|
||||
User Browser
|
||||
↓ HTTP Request
|
||||
Next.js Route Handler
|
||||
↓
|
||||
Server Action or API Route
|
||||
↓
|
||||
Prisma Query (read/write)
|
||||
↓
|
||||
SQLite Database
|
||||
↓
|
||||
Prisma Response
|
||||
↓
|
||||
UI Update (React re-render)
|
||||
```
|
||||
|
||||
### Pattern 2: MCP Server Data Flow
|
||||
|
||||
```
|
||||
AI Assistant / N8N
|
||||
↓ MCP Protocol (stdio)
|
||||
MCP Server (index.js)
|
||||
↓ Tool Invocation
|
||||
Prisma Query (read/write)
|
||||
↓
|
||||
SQLite Database (shared file)
|
||||
↓
|
||||
Parse & Format Data
|
||||
↓ MCP Response (stdio)
|
||||
AI Assistant / N8N
|
||||
```
|
||||
|
||||
### Pattern 3: External Integrations
|
||||
|
||||
**Email Service:**
|
||||
```
|
||||
keep-notes → nodemailer → SMTP Server → User Email
|
||||
```
|
||||
|
||||
**AI Provider:**
|
||||
```
|
||||
keep-notes → Vercel AI SDK → OpenAI/Ollama API → AI Response
|
||||
```
|
||||
|
||||
**N8N Workflow:**
|
||||
```
|
||||
N8N → MCP Protocol → mcp-server → Prisma → DB
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Service Boundaries
|
||||
|
||||
### keep-notes Responsibilities
|
||||
|
||||
**User-Facing:**
|
||||
- Web UI (React components)
|
||||
- User authentication
|
||||
- Note management UI
|
||||
- Label management UI
|
||||
- Settings & admin panel
|
||||
|
||||
**Business Logic:**
|
||||
- Note CRUD operations
|
||||
- Label CRUD operations
|
||||
- Auto-tagging (AI)
|
||||
- Search functionality
|
||||
- Reminder scheduling
|
||||
|
||||
**Data Access:**
|
||||
- Prisma ORM
|
||||
- SQLite queries
|
||||
- File uploads
|
||||
- AI integrations
|
||||
|
||||
**External Communication:**
|
||||
- Email sending (nodemailer)
|
||||
- AI provider APIs (OpenAI, Ollama)
|
||||
- User authentication (NextAuth)
|
||||
|
||||
### mcp-server Responsibilities
|
||||
|
||||
**Protocol-Specific:**
|
||||
- MCP protocol implementation
|
||||
- Tool registration
|
||||
- Request/response handling
|
||||
- Error handling
|
||||
|
||||
**Data Access:**
|
||||
- Direct Prisma access
|
||||
- Note operations via MCP tools
|
||||
- Label queries
|
||||
- Search operations
|
||||
|
||||
**No Direct User Interface** - Protocol-only service
|
||||
|
||||
---
|
||||
|
||||
## Communication Protocols
|
||||
|
||||
### keep-nets → User
|
||||
|
||||
**Protocol:** HTTP/WebSocket
|
||||
**Format:** HTML/JSON
|
||||
**Authentication:** NextAuth session (cookies)
|
||||
|
||||
**Examples:**
|
||||
- Browser → `GET /` → Next.js SSR → HTML
|
||||
- Browser → `POST /api/notes` → JSON response
|
||||
- Browser → Server Action → Mutation → Revalidate
|
||||
|
||||
### keep-notes → External Services
|
||||
|
||||
**Email:**
|
||||
- Protocol: SMTP
|
||||
- Library: nodemailer
|
||||
- Purpose: Password reset, reminders
|
||||
|
||||
**AI Providers:**
|
||||
- Protocol: HTTP
|
||||
- Libraries: Vercel AI SDK
|
||||
- Providers: OpenAI API, Ollama (local)
|
||||
|
||||
### mcp-server → MCP Clients
|
||||
|
||||
**Protocol:** Model Context Protocol (MCP)
|
||||
**Transport:** Stdio (standard input/output)
|
||||
**Format:** JSON-RPC over stdio
|
||||
**Authentication:** None (currently)
|
||||
|
||||
**Clients:**
|
||||
- AI Assistants (ChatGPT, Claude)
|
||||
- N8N workflows
|
||||
- Custom automation scripts
|
||||
|
||||
---
|
||||
|
||||
## Deployment Topologies
|
||||
|
||||
### Current: Local Development
|
||||
|
||||
```
|
||||
Same Machine
|
||||
├── keep-notes (npm run dev)
|
||||
│ └── http://localhost:3000
|
||||
├── mcp-server (npm start)
|
||||
│ └── stdio://
|
||||
└── SQLite DB (shared file)
|
||||
└── keep-notes/prisma/dev.db
|
||||
```
|
||||
|
||||
### Production Option 1: Single Server
|
||||
|
||||
```
|
||||
Single Server (Docker/VM)
|
||||
├── keep-notes (Node.js process)
|
||||
├── mcp-server (Node.js process)
|
||||
└── SQLite DB (shared file)
|
||||
```
|
||||
|
||||
**Pros:** Simple, low latency
|
||||
**Cons:** Single point of failure
|
||||
|
||||
### Production Option 2: Microservices
|
||||
|
||||
```
|
||||
├── Web Server (keep-notes)
|
||||
│ └── Docker container
|
||||
├── MCP Server (mcp-server)
|
||||
│ └── Docker container
|
||||
└── Database Server (PostgreSQL)
|
||||
└── Connection pooling
|
||||
```
|
||||
|
||||
**Pros:** Scalable, resilient
|
||||
**Cons:** More complex, requires PostgreSQL migration
|
||||
|
||||
### Production Option 3: Cloud Native
|
||||
|
||||
```
|
||||
├── Vercel/Netlify (keep-notes frontend)
|
||||
├── Cloud Run/AWS Lambda (MCP server)
|
||||
└── Cloud SQL (PostgreSQL)
|
||||
```
|
||||
|
||||
**Pros:** Highly scalable, managed
|
||||
**Cons:** Vendor lock-in, higher cost
|
||||
|
||||
---
|
||||
|
||||
## API Contract Between Parts
|
||||
|
||||
### Currently: NO Direct HTTP API
|
||||
|
||||
**Integration Method:** Shared database
|
||||
**Communication:** Database-mediated
|
||||
**Synchronization:** Immediate (via shared DB)
|
||||
|
||||
**Future HTTP API (if needed):**
|
||||
|
||||
```
|
||||
keep-notes exposes internal API:
|
||||
GET /api/notes
|
||||
POST /api/notes
|
||||
etc.
|
||||
|
||||
mcp-server could call these:
|
||||
fetch('http://localhost:3000/api/notes')
|
||||
```
|
||||
|
||||
**Current Preference:** Database sharing (simpler, faster)
|
||||
|
||||
---
|
||||
|
||||
## Authentication Flow
|
||||
|
||||
### User Authentication (keep-notes)
|
||||
|
||||
```
|
||||
1. User enters credentials in LoginForm
|
||||
2. POST /api/auth/signin
|
||||
3. NextAuth validates credentials
|
||||
4. Creates session in DB
|
||||
5. Sets HTTP-only cookie
|
||||
6. Redirects to dashboard
|
||||
```
|
||||
|
||||
### MCP Authentication (mcp-server)
|
||||
|
||||
```
|
||||
1. MCP client connects via stdio
|
||||
2. No authentication required
|
||||
3. Full database access granted
|
||||
4. Executes tools directly
|
||||
```
|
||||
|
||||
**Security Note:** This is acceptable for local/trusted environments but should be enhanced for production deployment.
|
||||
|
||||
---
|
||||
|
||||
## Data Synchronization
|
||||
|
||||
### Consistency Model
|
||||
|
||||
**Type:** Strong consistency via shared database
|
||||
**Mechanism:** SQLite ACID properties
|
||||
**Lag:** Zero (immediate)
|
||||
|
||||
**Conflict Resolution:**
|
||||
- Last write wins (SQLite default)
|
||||
- Application-level locking for critical operations
|
||||
|
||||
**Caching:**
|
||||
- React Cache (server components)
|
||||
- No explicit cache invalidation needed (DB is source of truth)
|
||||
|
||||
---
|
||||
|
||||
## Error Handling Across Parts
|
||||
|
||||
### keep-notes Errors
|
||||
|
||||
**Database Errors:**
|
||||
```typescript
|
||||
try {
|
||||
const note = await prisma.note.create({ ... })
|
||||
} catch (error) {
|
||||
return NextResponse.json(
|
||||
{ success: false, error: 'Failed to create note' },
|
||||
{ status: 500 }
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
**Action Errors:**
|
||||
- Form validation errors (Zod)
|
||||
- User-facing error messages
|
||||
- Graceful degradation
|
||||
|
||||
### mcp-server Errors
|
||||
|
||||
**Tool Execution Errors:**
|
||||
```javascript
|
||||
try {
|
||||
const note = await prisma.note.create({ ... })
|
||||
} catch (error) {
|
||||
throw new McpError(
|
||||
ErrorCode.InternalError,
|
||||
`Tool execution failed: ${error.message}`
|
||||
)
|
||||
}
|
||||
```
|
||||
|
||||
**Error Types:**
|
||||
- `InvalidRequest`: Bad parameters
|
||||
- `InternalError`: DB/Server errors
|
||||
- `MethodNotFound`: Unknown tool
|
||||
|
||||
---
|
||||
|
||||
## Monitoring & Observability
|
||||
|
||||
### Current State: Minimal
|
||||
|
||||
**keep-notes:**
|
||||
- Console logging
|
||||
- Playwright test reports
|
||||
|
||||
**mcp-server:**
|
||||
- Stderr logging
|
||||
- No structured logging
|
||||
|
||||
### Recommended Enhancements
|
||||
|
||||
**1. Distributed Tracing**
|
||||
- Trace requests across both parts
|
||||
- Correlation IDs
|
||||
- Performance monitoring
|
||||
|
||||
**2. Centralized Logging**
|
||||
- Structured JSON logs
|
||||
- Log aggregation (ELK, Loki)
|
||||
- Error tracking (Sentry)
|
||||
|
||||
**3. Metrics**
|
||||
- Request/response times
|
||||
- Database query performance
|
||||
- Tool execution statistics
|
||||
|
||||
**4. Health Checks**
|
||||
- `/health` endpoint for keep-notes
|
||||
- Health check for mcp-server
|
||||
- Database connectivity checks
|
||||
|
||||
---
|
||||
|
||||
## Scaling Considerations
|
||||
|
||||
### Current Limitations (SQLite)
|
||||
|
||||
**Concurrency:**
|
||||
- Multiple readers OK
|
||||
- Single writer (limitation)
|
||||
- Lock contention possible under load
|
||||
|
||||
**Capacity:**
|
||||
- File-based storage
|
||||
- Manual backups required
|
||||
- No automatic failover
|
||||
|
||||
### Scaling Strategies
|
||||
|
||||
**Horizontal Scaling:**
|
||||
- **Web App:** Multiple instances behind load balancer
|
||||
- **MCP Server:** Multiple instances (round-robin)
|
||||
- **Database:** Migrate to PostgreSQL
|
||||
|
||||
**Vertical Scaling:**
|
||||
- More CPU cores
|
||||
- More RAM (for caching)
|
||||
- Faster SSD (for I/O)
|
||||
|
||||
**Database Migration:**
|
||||
```prisma
|
||||
// Change datasource in schema.prisma
|
||||
datasource db {
|
||||
provider = "postgresql"
|
||||
url = env("DATABASE_URL")
|
||||
}
|
||||
|
||||
// Update adapters
|
||||
// - @prisma/adapter-pg
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security Architecture
|
||||
|
||||
### Current Security Posture
|
||||
|
||||
**keep-notes:**
|
||||
- ✅ NextAuth authentication
|
||||
- ✅ Password hashing (bcrypt)
|
||||
- ✅ Session management
|
||||
- ✅ Protected routes
|
||||
- ✅ CSRF protection (NextAuth)
|
||||
- ⚠️ No rate limiting
|
||||
- ⚠️ No brute force protection
|
||||
|
||||
**mcp-server:**
|
||||
- ✅ Input validation (schemas)
|
||||
- ✅ SQL injection protection (Prisma)
|
||||
- ❌ No authentication
|
||||
- ❌ No authorization
|
||||
- ❌ No rate limiting
|
||||
|
||||
### Security Recommendations
|
||||
|
||||
**Immediate:**
|
||||
1. Add API key authentication to mcp-server
|
||||
2. Restrict mcp-server to localhost/VPN
|
||||
3. Add rate limiting to both services
|
||||
4. Implement brute force protection
|
||||
|
||||
**Future:**
|
||||
1. HTTPS/TLS for all communications
|
||||
2. Input sanitization
|
||||
3. Output encoding
|
||||
4. Security headers (CSP, HSTS, etc.)
|
||||
5. Regular security audits
|
||||
|
||||
---
|
||||
|
||||
## Integration Testing
|
||||
|
||||
### Test Scenarios
|
||||
|
||||
**1. Data Consistency**
|
||||
- Create note in keep-notes → Verify visible in mcp-server
|
||||
- Update via mcp-server → Verify visible in keep-notes
|
||||
|
||||
**2. Concurrent Access**
|
||||
- Simultaneous writes to same note
|
||||
- Race condition testing
|
||||
- Lock behavior verification
|
||||
|
||||
**3. Error Propagation**
|
||||
- Database connection failures
|
||||
- Invalid data handling
|
||||
- Graceful degradation
|
||||
|
||||
### Testing Tools
|
||||
|
||||
**Integration Tests:**
|
||||
- Playwright (E2E for keep-notes)
|
||||
- Custom MCP client tests
|
||||
- Database state verification
|
||||
|
||||
**Load Tests:**
|
||||
- Concurrent tool execution
|
||||
- Database query performance
|
||||
- Memory leak detection
|
||||
|
||||
---
|
||||
|
||||
## Deployment Integration
|
||||
|
||||
### Docker Compose (Recommended)
|
||||
|
||||
```yaml
|
||||
version: '3.8'
|
||||
services:
|
||||
keep-notes:
|
||||
build: ./keep-notes
|
||||
ports:
|
||||
- "3000:3000"
|
||||
volumes:
|
||||
- db-data:/app/prisma
|
||||
environment:
|
||||
- DATABASE_URL=file:/app/prisma/dev.db
|
||||
|
||||
mcp-server:
|
||||
build: ./mcp-server
|
||||
volumes:
|
||||
- db-data:/app/db
|
||||
depends_on:
|
||||
- keep-notes
|
||||
environment:
|
||||
- DATABASE_URL=file:/app/db/dev.db
|
||||
|
||||
volumes:
|
||||
db-data:
|
||||
```
|
||||
|
||||
**Benefits:**
|
||||
- Single command deployment
|
||||
- Shared volume for database
|
||||
- Environment configuration
|
||||
- Easy local development
|
||||
|
||||
---
|
||||
|
||||
## Summary
|
||||
|
||||
The Memento project uses a **database-mediated integration pattern**:
|
||||
|
||||
**Strengths:**
|
||||
- ✅ Simple architecture
|
||||
- ✅ Real-time data sync
|
||||
- ✅ No API complexity
|
||||
- ✅ Easy local development
|
||||
- ✅ Strong consistency
|
||||
|
||||
**Considerations:**
|
||||
- ⚠️ Single database file (scalability)
|
||||
- ⚠️ No authentication in MCP server
|
||||
- ⚠️ SQLite concurrency limits
|
||||
- ⚠️ Manual backup required
|
||||
|
||||
**Ideal For:**
|
||||
- Single-user or small team deployments
|
||||
- Local/self-hosted environments
|
||||
- Applications with modest write volume
|
||||
- Projects prioritizing simplicity
|
||||
|
||||
**Future Evolution:**
|
||||
- Add authentication layer
|
||||
- Migrate to PostgreSQL for scaling
|
||||
- Implement HTTP API between parts
|
||||
- Add caching layer (Redis)
|
||||
- Implement message queue for async ops
|
||||
Reference in New Issue
Block a user