Editing Embeddings Vector Dbs (section)

== <span style="color: #FFFFFF;">Applying</span> ==
'''Generating and storing embeddings with Sentence Transformers + Chroma:'''

<syntaxhighlight lang="python">
from sentence_transformers import SentenceTransformer
import chromadb
import numpy as np

# Load embedding model
model = SentenceTransformer("BAAI/bge-m3")

# Sample documents
docs = [
    "Neural networks are the foundation of deep learning.",
    "The heart pumps blood through the circulatory system.",
    "Python is a popular programming language for data science.",
    "Transformers use self-attention mechanisms for NLP tasks.",
    "The mitochondria are the powerhouse of the cell.",
]

# Generate embeddings
embeddings = model.encode(docs, normalize_embeddings=True)
print(f"Embedding shape: {embeddings.shape}")  # (5, 1024)

# Store in vector database
client = chromadb.Client()
collection = client.create_collection("knowledge_base")

collection.add(
    documents=docs,
    embeddings=embeddings.tolist(),
    ids=[f"doc_{i}" for i in range(len(docs))]
)

# Semantic search
query = "How do attention mechanisms work?"
query_embedding = model.encode([query], normalize_embeddings=True).tolist()

results = collection.query(
    query_embeddings=query_embedding,
    n_results=2
)
print(results["documents"])
# [["Transformers use self-attention mechanisms for NLP tasks.",
#   "Neural networks are the foundation of deep learning."]]
</syntaxhighlight>

; Vector database selection guide
: '''Local/development''' → Chroma (in-memory, file-backed), FAISS (library)
: '''Self-hosted production''' → Qdrant (Rust, great performance), Weaviate (rich features), Milvus (scale)
: '''Managed cloud''' → Pinecone (simplest API), Weaviate Cloud, Zilliz Cloud
: '''Existing PostgreSQL stack''' → pgvector extension (good for <10M vectors)
: '''Multimodal (text + image)''' → Weaviate, Qdrant (both support multiple vector types)
</div>

<div style="background-color: #8B4500; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">