Embeddings: Represent Similarity with Vectors
You want to find images that resemble one example without defining every possible category.
Embeddings: Represent Similarity with Vectors explained visually
You want to find images that resemble one example without defining every possible category. English captions are optional and start off. The complete transcript follows the written lesson.
What you will learn
You want to find images that resemble one example without defining every possible category.
Build a numerical representation
You want to find images that resemble one example without defining every possible category. An embedding represents each image as a vector of numbers. A model learns this representation so that certain relationships become easier to compare. The meaning of similarity comes from the training objective and data, not from the vector format alone.
Choose the similarity rule
Distances or similarities between embeddings can support retrieval, clustering, or matching. Euclidean distance measures separation, while cosine similarity compares direction. Normalization changes the relationship between these measures. A model trained to group object categories may ignore details that matter for individual identity, and a representation useful for photographs may behave poorly on a different image domain.
Evaluate the meaning of close
Evaluate retrieval with examples that reflect the intended use. Include look-alike objects, unusual viewpoints, and difficult negatives. A close vector match is evidence under the model's learned representation; it is not proof that two images show the same physical object. Record the embedding model version because updating it can change the entire search space. Stored vectors and newly generated vectors must remain compatible.
Worked example
Two feature vectors point in the same direction but have different lengths: one is one, two, and the other is two, four. Their cosine similarity is one, while their Euclidean distance is not zero. The chosen comparison determines what it means for the vectors to match.
Quick check: test your understanding
Try it yourself: Would a category embedding necessarily distinguish two nearly identical individual cars? Explain.
Check the reasoning
No. It may place both cars close together because they share a category and appearance, even when individual identity is the important distinction.
Can stored embeddings safely be mixed across unrelated model versions?
Not without checking compatibility. Different models may produce coordinates with different meanings.
Further reading
Dive into Deep Learning — Aston Zhang, Zachary C. Lipton, Mu Li, and Alexander J. Smola.
Deep Learning — Ian Goodfellow, Yoshua Bengio, and Aaron Courville.
Original ToolsInVision teaching examples. Editorial approach · Suggest a correction
Accessible referenceLesson 73 transcript230 words · Video transcript · Matches the published narration and captionsOpen transcript
How could a search tool find pictures similar to one example, even when we have not named every possible category? One useful approach is to turn each image into an embedding.
An embedding is a vector: an ordered list of numbers produced by a model. Those numbers describe the image in a learned feature space. We can compare the vectors instead of comparing every original pixel directly.
But what does similar mean? That depends on the training objective. A model trained to group animal categories may place two different dogs close together. A model trained to recognize individual animals needs to preserve a different kind of information.
The comparison method matters too. Euclidean distance measures separation. Cosine similarity compares direction. Take the vectors one, two and two, four. They point in the same direction, so their cosine similarity is one. Their lengths differ, so their Euclidean distance is not zero.
For an image search system, test more than easy matches. Include similar-looking objects, changed lighting, and unusual viewpoints. A nearby vector is a useful model prediction, not proof that two pictures show the same physical object.
Finally, keep track of the embedding model version. If you replace the model, the coordinates may change meaning. Old stored vectors and new query vectors need compatible representations. Good retrieval depends on both the learned features and the way you compare and maintain them.