added pgvector implementation and updated doc

This commit is contained in:
2026-06-29 22:27:32 +02:00
parent 28080c7954
commit 47466a6c61
3 changed files with 117 additions and 27 deletions
+3
View File
@@ -48,6 +48,9 @@ dependencies {
implementation("org.springframework.boot:spring-boot-starter-webmvc") implementation("org.springframework.boot:spring-boot-starter-webmvc")
implementation("org.springframework.ai:spring-ai-starter-model-openai") implementation("org.springframework.ai:spring-ai-starter-model-openai")
implementation("org.postgresql:postgresql:$postgresqlVersion") implementation("org.postgresql:postgresql:$postgresqlVersion")
implementation("org.springframework.ai:spring-ai-starter-vector-store-pgvector")
compileOnly("org.projectlombok:lombok") compileOnly("org.projectlombok:lombok")
@@ -21,6 +21,12 @@ spring:
api-key: ${OPENAI_KEY} api-key: ${OPENAI_KEY}
chat: chat:
model: gpt-4o-mini model: gpt-4o-mini
vectorstore:
pgvector:
dimensions: 1536
distance-type: COSINE_DISTANCE
index-type: HNSW
initialize-schema: true
session: session:
store-type: jdbc store-type: jdbc
jdbc: jdbc:
@@ -165,43 +165,124 @@ services:
restart: unless-stopped restart: unless-stopped
``` ```
Note that I'm using my own locally hosted PostgreSQL instances for the main and test database. Just add databases via SQL or PgAdmin and install the pgvector extension to each database manually. There is an offical ready-made pgvector docker image but if you already host a PostGreSQL database you need to add the extension yourself. Note that I'm using my own locally hosted PostgreSQL instances for the main and test database. Just add databases via SQL or PgAdmin and install the pgvector extension to each database manually. There is an official ready-made pgvector docker image but if you already host a PostgreSQL database you need to add the extension yourself.
Check the name of your PostGreSQL container: ## Installing pgvector on a self-hosted PostgreSQL container
```
docker ps pgvector is a PostgreSQL extension that adds a `vector` data type. It does not create a new database — it adds a `vector_store` table to your existing database. The shared library (`vector.so`) must be installed on the PostgreSQL server itself.
**Do not install it manually inside a running container.** Manual installations do not survive container recreation (e.g. after a `docker compose up --force-recreate` or image update). Instead, bake it into a custom Docker image.
### Step 1: Create a custom Dockerfile for PostgreSQL
Create a `Dockerfile.postgres` next to your `docker-compose.yaml`:
```dockerfile
FROM postgres:18.4
RUN apt-get update && apt-get install -y --no-install-recommends \
ca-certificates git build-essential postgresql-server-dev-18 \
&& git clone --branch v0.8.3 https://github.com/pgvector/pgvector.git \
&& cd pgvector && make && make install \
&& cd .. && rm -rf pgvector \
&& apt-get purge -y ca-certificates git build-essential postgresql-server-dev-18 \
&& apt-get autoremove -y \
&& rm -rf /var/lib/apt/lists/*
``` ```
Enter your container via bash: - `FROM postgres:18.4` — this is still the standard postgres image, not the pgvector image
- `ca-certificates` — required for git to verify GitHub's SSL certificate during build
- `postgresql-server-dev-18` — provides the PostgreSQL header files needed to compile pgvector
``` ### Step 2: Update docker-compose.yaml to use the custom image
docker exec -it 876fb382969f bash
```
Before working on your database backup your databases:
```
su - postgres -c "pg_dumpall > /tmp/backup200526.sql"
#exit the container and copy the backup file to local file system
docker cp 876fb382969f:/tmp/backup200526.sql . ```yaml
services:
db:
container_name: postgres
labels:
- "com.centurylinklabs.watchtower.enable=false"
build:
context: .
dockerfile: Dockerfile.postgres
network: host
restart: always
environment:
POSTGRES_USER: ${DB_USER}
POSTGRES_PASSWORD: ${DB_PASSWORD}
POSTGRES_DB: ${DB_NAME}
networks:
- pg_network
volumes:
- /home/pi/docker/postgresql:/var/lib/postgresql
ports:
- "5432:5432"
networks:
pg_network:
external: true
``` ```
Install dependencies, build and install pgvector: ### Step 3: Build the image and recreate the container
apt-get update
apt-get install -y build-essential git postgresql-server-dev-all
```
git clone https://github.com/pgvector/pgvector.git
cd pgvector
make
make install
docker restart 876fb382969f
```
Enter PostGreSQL container and create pgvector extension for each databse:
```
docker exec -it <container-name> psql -h localhost -U <db-user> -d <db-name>
CREATE EXTENSION vector; Before recreating, back up your databases:
```bash
docker exec -it <container-name> bash
su - postgres -c "pg_dumpall > /tmp/backup.sql"
exit
docker cp <container-name>:/tmp/backup.sql .
``` ```
Build the image. Use `--network=host` to ensure the build container can reach GitHub:
```bash
docker build --network=host -t postgres-pgvector -f Dockerfile.postgres .
```
Recreate the container
```bash
docker compose up -d --force-recreate
```
### Step 4: Enable the extensions in each database
Run this once per database that needs pgvector:
```bash
docker exec -it <container-name> psql -U <db-user> -d <db-name>
```
```sql
CREATE EXTENSION IF NOT EXISTS vector;
CREATE EXTENSION IF NOT EXISTS "uuid-ossp";
```
`uuid-ossp` is also required — Spring AI's `vector_store` table uses `uuid_generate_v4()` for its primary key.
### Step 5: Add pgvector config to application.yaml
Spring AI's pgvector auto-configuration requires these properties:
```yaml
spring:
ai:
vectorstore:
pgvector:
dimensions: 1536 # must match your embedding model output size
distance-type: COSINE_DISTANCE
index-type: HNSW
initialize-schema: true # auto-creates the vector_store table on startup
```
`dimensions` depends on the embedding model:
- `text-embedding-ada-002` / `text-embedding-3-small` → 1536
- `text-embedding-3-large` → 3072
`initialize-schema: true` means Spring AI will create the `vector_store` table automatically on first startup — no manual SQL needed.
# Appendix: Additional config for developing in Code-Server # Appendix: Additional config for developing in Code-Server
When using the code-server container there are additional config steps to mind: When using the code-server container there are additional config steps to mind: