Skip to main content

Glossary

What is visual search?

Visual search finds products from an image instead of a text query, by comparing visual features of the query image with those of items in a catalog.

In plain words: A shopper uploads or points at a photo, and the system returns products that look like the item in it. Behind that, a model detects the item, turns its crop into a numeric description of its appearance and compares that description with those of the product images in the catalog.

Also called: image search, search by photo, reverse image search

Published by Youzu · Editorial policy

Published
Virtually staged: a kitchen dining corner with a round ash table, 2 rush-seat chairs and a green ceramic jug.
Virtually staged
Numbered on the photo:
  1. Round pedestal table. Exact match: the same table in the catalog.
  2. Rush-seat chair. Similar match: a chair of the same shape in another wood.
  3. Green ceramic jug. Broad match: other green jugs and vases to browse.

Why it matters

Some products are easier to show than to name: a chair seen in a café, a pattern, a color. Visual search lets shoppers start from that picture. Its results are only as good as the catalog behind it, because products without clean images or attributes cannot be found or filtered.

How it works

  1. Detect and crop

    Find the item in the photo and crop it, or let the shopper adjust the frame.

  2. Embed

    Turn the crop into an embedding, a list of numbers that describes its appearance.

  3. Retrieve and rank

    Find the nearest catalog images and rank them, often with help from the category.

Matches for an image link of a navy velvet sofa, from an 8-item example catalog
Catalog itemMatch
Velvet 3-seater, navyExact
Linen 3-seaterSimilar
Cream 3-seaterSimilar
Velvet loveseat, plumBroad

Examples

  • An exact match

    A product photo of the terracotta armchair returns that armchair as an exact match, the plum accent chair as similar and the plum loveseat as a broad match.

  • An item picked out of a room

    In a room photo, selecting the sofa returns 2 similar 3-seat sofas and 2 broad matches; the floor lamp returns nothing useful, because the example catalog has no lamps.

  • An image link

    An image link of a navy sofa returns the navy velvet 3-seater as an exact match, followed by similar and broad matches.

Common mistakes

  • Testing on the vendor's demo. Demo queries and catalogs are chosen to look good. Evaluate on your own photos against your own catalog.

  • Hiding empty results. Filling the page with whatever is nearest when nothing close exists costs more trust than an honest empty result.

Attribute extraction
Pulling structured values such as color, material or size out of text or images and storing them as fields on a product record.
Product matching
Deciding whether 2 records describe the same product by comparing identifiers, titles, attributes and images.
Match level
A match level states how close a visual search result must be to the query image: exact, similar or broad.

In Youzu

From the same catalog, shoppers can search with a photo, shop the products in a scene, see products in a room photo, and see products on new backgrounds or in 3D.

Youzu Lens

Sources

  1. Visual Search at Pinterest, Yushi Jing, David Liu and others, arXiv. arxiv.org/abs/1505.07647. Accessed .

  2. Vision API Product Search documentation, Google Cloud. docs.cloud.google.com/vision/product-search/docs. Accessed .

FAQ

Questions buyers ask

No. Image recognition labels what is in a picture, such as a sofa. Visual search goes further: it finds the specific products in a catalog that look most like the item in the picture. It usually uses recognition as a first step, to find and crop the item before comparing it with catalog images.

Product images. Visual search works from your catalog's own product images; products need images. Youzu Lens lets shoppers search your catalog with a photo: an upload, an image URL or an object detected in a photo; matches can be exact, similar or broad.

Last reviewed

Next step

Automate Commerce. Make shopping personal.

Bring your catalog, listing flow or shopping idea. Let's find the right place to start.

Book a demo

Tell us about your catalog and we will get back to you.