What is visual search?
Visual search finds products from an image instead of a text query, by comparing visual features of the query image with those of items in a catalog.
In plain words: A shopper uploads or points at a photo, and the system returns products that look like the item in it. Behind that, a model detects the item, turns its crop into a numeric description of its appearance and compares that description with those of the product images in the catalog.
Also called: image search, search by photo, reverse image search
Published by Youzu · Editorial policy
- Published

- Round pedestal table. Exact match: the same table in the catalog.
- Rush-seat chair. Similar match: a chair of the same shape in another wood.
- Green ceramic jug. Broad match: other green jugs and vases to browse.
- Round pedestal table. Exact match: the same table in the catalog.
- Rush-seat chair. Similar match: a chair of the same shape in another wood.
- Green ceramic jug. Broad match: other green jugs and vases to browse.
Why it matters
Some products are easier to show than to name: a chair seen in a café, a pattern, a color. Visual search lets shoppers start from that picture. Its results are only as good as the catalog behind it, because products without clean images or attributes cannot be found or filtered.
How it works
Detect and crop
Find the item in the photo and crop it, or let the shopper adjust the frame.
Embed
Turn the crop into an embedding, a list of numbers that describes its appearance.
Retrieve and rank
Find the nearest catalog images and rank them, often with help from the category.
| Catalog item | Match |
|---|---|
| Velvet 3-seater, navy | Exact |
| Linen 3-seater | Similar |
| Cream 3-seater | Similar |
| Velvet loveseat, plum | Broad |
Examples
An exact match
A product photo of the terracotta armchair returns that armchair as an exact match, the plum accent chair as similar and the plum loveseat as a broad match.
An item picked out of a room
In a room photo, selecting the sofa returns 2 similar 3-seat sofas and 2 broad matches; the floor lamp returns nothing useful, because the example catalog has no lamps.
An image link
An image link of a navy sofa returns the navy velvet 3-seater as an exact match, followed by similar and broad matches.
Common mistakes
Testing on the vendor's demo. Demo queries and catalogs are chosen to look good. Evaluate on your own photos against your own catalog.
Hiding empty results. Filling the page with whatever is nearest when nothing close exists costs more trust than an honest empty result.
Related terms
- Attribute extraction
- Pulling structured values such as color, material or size out of text or images and storing them as fields on a product record.
- Product matching
- Deciding whether 2 records describe the same product by comparing identifiers, titles, attributes and images.
- Match level
- A match level states how close a visual search result must be to the query image: exact, similar or broad.
In Youzu
From the same catalog, shoppers can search with a photo, shop the products in a scene, see products in a room photo, and see products on new backgrounds or in 3D.
Sources
Visual Search at Pinterest, Yushi Jing, David Liu and others, arXiv. arxiv.org/abs/1505.07647. Accessed .
Vision API Product Search documentation, Google Cloud. docs.cloud.google.com/vision/product-search/docs. Accessed .
FAQ
Questions buyers ask
No. Image recognition labels what is in a picture, such as a sofa. Visual search goes further: it finds the specific products in a catalog that look most like the item in the picture. It usually uses recognition as a first step, to find and crop the item before comparing it with catalog images.
No. Visual search adds a way to start from a picture, while text search remains the main way most shoppers search. Text search stays in your existing search; Youzu adds photo search and cleaner product data. Many stores offer both from the same search bar, landing on the same product pages.
Product images. Visual search works from your catalog's own product images; products need images. Youzu Lens lets shoppers search your catalog with a photo: an upload, an image URL or an object detected in a photo; matches can be exact, similar or broad.
Last reviewed
Related
Keep reading
Product
Youzu Lens: visual search for e-commerce
Youzu Lens lets shoppers search your catalog with a photo, an image URL or an object detected in a photo; matches can be exact, similar or broad.
Guide
Visual search for e-commerce: how it works and how to evaluate it
How visual search turns a photo into ranked products, what drives quality (crops, embeddings, catalog data) and how to benchmark vendors on your images.
Glossary
What is attribute extraction?
Attribute extraction defined in plain words, with a worked example, common mistakes and links to the guides and Youzu products that apply it.