feaeb075ce
All checks were successful
Deploy to Production / Build and Deploy (push) Successful in 4s
The original migration used a fragile add-copy-drop-rename pattern with _jsonb casts that silently failed, leaving the embedding column as text. Replace with a direct ALTER COLUMN TYPE ... USING embedding::vector(1536) that is fully idempotent and handles all partial states from previous failed attempts.
38 lines
2.1 KiB
Markdown
38 lines
2.1 KiB
Markdown
# Search Broken — embedding column not converted to vector type
|
|
|
|
## Date: 2026-05-12
|
|
|
|
## Problem
|
|
The search fails with this error:
|
|
```
|
|
operator does not exist: text <=> vector
|
|
HINT: No operator matches the given name and argument types. You might need to add explicit type casts.
|
|
```
|
|
|
|
## Root cause
|
|
The `NoteEmbedding.embedding` column is still type `text` (old JSON string format), NOT `vector(1536)`.
|
|
The Prisma migration marked itself as applied but the actual column type conversion was never executed.
|
|
The SQL query tries to use the `<=>` cosine distance operator on a text column, which fails.
|
|
|
|
## Current state
|
|
- pgvector extension IS installed (CREATE EXTENSION worked)
|
|
- But the embedding column was NOT converted from text to vector(1536)
|
|
- There are 102 rows in NoteEmbedding with JSON string embeddings
|
|
- The migration SQL needs to: ALTER COLUMN embedding TYPE vector(1536) using proper casting
|
|
|
|
## What needs to happen
|
|
1. Check the actual column type: SELECT column_name, data_type, udt_name FROM information_schema.columns WHERE table_name = 'NoteEmbedding' AND column_name = 'embedding';
|
|
2. The migration SQL must convert the column. The embedding values are stored as JSON strings like "[0.1, 0.2, ...]" — need to strip brackets, then cast to vector.
|
|
3. The conversion SQL should be something like:
|
|
ALTER TABLE "NoteEmbedding" ALTER COLUMN embedding TYPE vector(1536) USING embedding::vector(1536);
|
|
OR if stored as JSON string:
|
|
ALTER TABLE "NoteEmbedding" ALTER COLUMN embedding TYPE vector(1536) USING (replace(replace(embedding, '[', ''), ']', ''))::vector(1536);
|
|
4. Also check if the tsvector column and trigger on Note table were created properly.
|
|
5. The semantic-search.service.ts code uses $queryRawUnsafe with <=> operator — make sure the SQL is correct for pgvector.
|
|
|
|
## Files to check/fix
|
|
- prisma/migrations/20260512120000_pgvector_and_fts_search/migration.sql — the actual migration SQL
|
|
- lib/ai/services/semantic-search.service.ts — the search service using vector queries
|
|
- lib/ai/services/embedding.service.ts — embedding service
|
|
- schema.prisma — NoteEmbedding model
|