feaeb075ce
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 4s
The original migration used a fragile add-copy-drop-rename pattern with _jsonb casts that silently failed, leaving the embedding column as text. Replace with a direct ALTER COLUMN TYPE ... USING embedding::vector(1536) that is fully idempotent and handles all partial states from previous failed attempts.
2.1 KiB
2.1 KiB
Search Broken — embedding column not converted to vector type
Date: 2026-05-12
Problem
The search fails with this error:
operator does not exist: text <=> vector
HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
Root cause
The NoteEmbedding.embedding column is still type text (old JSON string format), NOT vector(1536).
The Prisma migration marked itself as applied but the actual column type conversion was never executed.
The SQL query tries to use the <=> cosine distance operator on a text column, which fails.
Current state
- pgvector extension IS installed (CREATE EXTENSION worked)
- But the embedding column was NOT converted from text to vector(1536)
- There are 102 rows in NoteEmbedding with JSON string embeddings
- The migration SQL needs to: ALTER COLUMN embedding TYPE vector(1536) using proper casting
What needs to happen
- Check the actual column type: SELECT column_name, data_type, udt_name FROM information_schema.columns WHERE table_name = 'NoteEmbedding' AND column_name = 'embedding';
- The migration SQL must convert the column. The embedding values are stored as JSON strings like "[0.1, 0.2, ...]" — need to strip brackets, then cast to vector.
- The conversion SQL should be something like: ALTER TABLE "NoteEmbedding" ALTER COLUMN embedding TYPE vector(1536) USING embedding::vector(1536); OR if stored as JSON string: ALTER TABLE "NoteEmbedding" ALTER COLUMN embedding TYPE vector(1536) USING (replace(replace(embedding, '[', ''), ']', ''))::vector(1536);
- Also check if the tsvector column and trigger on Note table were created properly.
- The semantic-search.service.ts code uses $queryRawUnsafe with <=> operator — make sure the SQL is correct for pgvector.
Files to check/fix
- prisma/migrations/20260512120000_pgvector_and_fts_search/migration.sql — the actual migration SQL
- lib/ai/services/semantic-search.service.ts — the search service using vector queries
- lib/ai/services/embedding.service.ts — embedding service
- schema.prisma — NoteEmbedding model