Vector embeddings explained for developers

This article explains how vector embeddings work and how to practically use them for semantic search, specifically for developers working with Python and OpenAI's API.

Requirements

- Python 3.10 or higher - `openai` Python package version 1.35.0 or higher - An OpenAI API key with access to the `text-embedding-3-small` model - `numpy` version 1.26.0 or higher for vector calculations

Install the dependencies:

pip install openai==1.35.0 numpy==1.26.0

Steps

1. Create an embedding

An embedding is a list of floating-point numbers that represents the semantic meaning of a text. OpenAI's `text-embedding-3-small` model converts text into a vector of 1536 dimensions.

from openai import OpenAI</p><p>client = OpenAI(api_key="your-api-key")</p><p>response = client.embeddings.create(
    model="text-embedding-3-small",
    input="The cat is sitting on the mat"
)</p><p>embedding = response.data[0].embedding
print(len(embedding))  Output: 1536

2. Store embeddings in a vector database

For production, use a dedicated vector database such as Pinecone, Weaviate, or pgvector. For local development, a NumPy array suffices.

import numpy as np</p><p>Store embeddings as a matrix
documents = [
    "The cat is sitting on the mat",
    "The dog is running through the park",
    "A bird is flying over the water"
]</p><p>embeddings = []
for doc in documents:
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=doc
    )
    embeddings.append(response.data[0].embedding)</p><p>embedding_matrix = np.array(embeddings)
np.save('document_embeddings.npy', embedding_matrix)

3. Calculate cosine similarity

Cosine similarity measures the angle between two vectors. A value of 1 means identical, 0 means no overlap. This is the standard way to determine semantic similarity.

def cosine_similarity(vec_a, vec_b):
    dot_product = np.dot(vec_a, vec_b)
    norm_a = np.linalg.norm(vec_a)
    norm_b = np.linalg.norm(vec_b)
    return dot_product / (norm_a * norm_b)</p><p>Test with two related sentences
query = "A cat is lying on the floor"
query_response = client.embeddings.create(
    model="text-embedding-3-small",
    input=query
)
query_embedding = query_response.data[0].embedding</p><p>similarities = [
    cosine_similarity(query_embedding, doc_emb)
    for doc_emb in embeddings
]</p><p>print(similarities)
Output: [0.82, 0.31, 0.28] — the first sentence is the most related

4. Build a semantic search function

Semantic search finds results based on meaning, not exact word matches. This works because embeddings place words with similar meanings close together in vector space.

def semantic_search(query, embeddings, documents, top_k=3):
    query_response = client.embeddings.create(
        model="text-embedding-3-small",
        input=query
    )
    query_embedding = query_response.data[0].embedding
    
    similarities = [
        cosine_similarity(query_embedding, doc_emb)
        for doc_emb in embeddings
    ]
    
    top_indices = np.argsort(similarities)[-top_k:][::-1]
    
    results = []
    for idx in top_indices:
        results.append({
            'document': documents[idx],
            'similarity': similarities[idx]
        })
    
    return results</p><p>results = semantic_search("An animal is lying asleep", embeddings, documents)
for r in results:
    print(f"{r['similarity']:.2f}: {r['document']}")
Output:
0.79: The cat is sitting on the mat
0.45: The dog is running through the park
0.33: A bird is flying over the water

5. Reduce dimensionality with PCA

1536 dimensions are a lot for visualization or storage in memory. Principal Component Analysis (PCA) reduces the dimensionality while preserving the relative distances.

from sklearn.decomposition import PCA</p><p>pca = PCA(n_components=2)
reduced_embeddings = pca.fit_transform(embedding_matrix)</p><p>print(reduced_embeddings.shape)  Output: (3, 2)

6. Implement caching for embeddings

Generating embeddings costs API calls and time. Cache them locally to avoid repeated calculations.

import json
import hashlib
from pathlib import Path</p><p>CACHE_FILE = Path("embedding_cache.json")</p><p>def get_cached_embedding(text):
    if CACHE_FILE.exists():
        cache = json.loads(CACHE_FILE.read_text())
    else:
        cache = {}
    
    text_hash = hashlib.sha256(text.encode()).hexdigest()
    
    if text_hash in cache:
        return cache[text_hash]
    
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=text
    )
    embedding = response.data[0].embedding
    
    cache[text_hash] = embedding
    CACHE_FILE.write_text(json.dumps(cache))
    
    return embedding</p><p>Use the cache
emb = get_cached_embedding("The cat is sitting on the mat")  API call
emb_cached = get_cached_embedding("The cat is sitting on the mat")  From cache

7. Batch processing for efficiency

Process multiple documents in one API call. This significantly reduces latency and costs.

texts = [
    "First document about artificial intelligence",
    "Second document about machine learning",
    "Third document about natural language processing"
]</p><p>batch_response = client.embeddings.create(
    model="text-embedding-3-small",
    input=texts
)</p><p>batch_embeddings = [item.embedding for item in batch_response.data]
print(len(batch_embeddings))  Output: 3

8. Implement a complete search pipeline

Combine all steps into a reusable pipeline that you can integrate directly into an application.

class EmbeddingSearch:
    def __init__(self, model="text-embedding-3-small"):
        self.client = OpenAI()
        self.model = model
        self.documents = []
        self.embeddings = np.array([])
    
    def add_documents(self, docs):
        self.documents.extend(docs)
        response = self.client.embeddings.create(
            model=self.model,
            input=docs
        )
        new_embeddings = np.array([item.embedding for item in response.data])
        
        if self.embeddings.size == 0:
            self.embeddings = new_embeddings
        else:
            self.embeddings = np.vstack([self.embeddings, new_embeddings])
    
    def search(self, query, top_k=5):
        response = self.client.embeddings.create(
            model=self.model,
            input=query
        )
        query_embedding = np.array(response.data[0].embedding)
        
        similarities = [
            cosine_similarity(query_embedding, doc_emb)
            for doc_emb in self.embeddings
        ]
        
        top_indices = np.argsort(similarities)[-top_k:][::-1]
        
        return [
            (self.documents[idx], similarities[idx])
            for idx in top_indices
        ]</p><p>Use the pipeline
search_engine = EmbeddingSearch()
search_engine.add_documents(documents)
results = search_engine.search("How does a cat work?", top_k=2)</p><p>for doc, score in results:
    print(f"{score:.3f} - {doc}")

Verification

Check that your embeddings work correctly by testing the dimensionality and semantic relationships:

python -c "
from openai import OpenAI
import numpy as np</p><p>client = OpenAI()
response = client.embeddings.create(
    model='text-embedding-3-small',
    input=['dog', 'cat', 'car']
)
embeddings = np.array([item.embedding for item in response.data])</p><p>Check dimensionality
assert embeddings.shape == (3, 1536), f'Error: expected (3, 1536), got {embeddings.shape}'</p><p>Check semantic relationships
def cos_sim(a, b):
    return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))</p><p>dog_cat = cos_sim(embeddings[0], embeddings[1])
dog_car = cos_sim(embeddings[0], embeddings[2])</p><p>print(f'Dimensionality: {embeddings.shape[1]}')
print(f'Dog vs cat similarity: {dog_cat:.3f}')
print(f'Dog vs car similarity: {dog_car:.3f}')
assert dog_cat > dog_car, 'Semantic relationship is incorrect'
print('All checks passed')
"

Frequently asked problems

1. API error: "model_not_found"

Your API key does not have access to the `text-embedding-3-small` model. Use an older model version or upgrade your account.

Solution: Replace `text-embedding-3-small` with `text-embedding-ada-002` and adjust the dimensionality from 1536 to 3072.

2. Low similarity scores for related texts

Embeddings work well for semantic similarity, but not for exact word matching. Two sentences with the same words but a different order can yield low scores.

Solution: Combine embeddings with a lexical search method such as BM25 for hybrid searches. Use `text-embedding-3-large` for better semantic precision.

3. Memory problems with many documents

Storing hundreds of thousands of embeddings of 1536 dimensions consumes a lot of RAM. One million documents uses about 6 GB.

Solution: Use a dedicated vector database such as Pinecone or Weaviate. These store embeddings on disk and use approximate nearest neighbor (ANN) algorithms for fast searches.

The shift from raw text to embeddings is a fundamental change: your code no longer works with words, but with positions in a continuous vector space — and that space ultimately determines what your search engine does and does not find.