Building a RAG system? Here’s what I learned about vectors and embeddings - the core concepts that make semantic search possible, explained as simply as possible.

What is a vector in machine learning?

In mathematics, a vector is an array of numbers that define a point in a dimensional space. In more practical terms, a vector is a list of numbers — like [1989, 22, 9, 180]. Each number indicates where the object is along a specified dimension. In machine learning, the use of vectors makes it possible to search for similar objects (text, image, audio, video…).

What is embedding?

Embedding is the process of creating vectors using deep learning – translating objects into a mathematical form (aka vector) according to the factors or traits each one may or may not have, and the categories they belong to. An “embedding” (aka vector) is the output of this process.

What is vector database?

A type of database designed to store and retrieve the vectors and their metadata. They also usually come with the capabilities for efficient and fast lookup of nearest-neighbors in the N-dimensional space.

What is vector indexing?

A vector index is a data structure that enables fast similarity search in high-dimensional spaces. Without indexing, the process of doing similarity search among 1000 chunks would be like: compute similarity between query vector and 1000 chunk vectors -> sort 1000results -> return top matches. This is an O(n) operation that becomes unbearably slow as dataset grows. With vector indexing, indexes pre-organize vectors so only a small subset will be checked. Common Vector Index Types

What is vector similarity?

Vector similarity is a measure of how closely related two data points are in a vector space. Common vector similarity metrics are Euclidean distance, cosine similarity, and dot product similarity.