Back in 2017, Google Lens and Pinterest Lens were launched. Around the same time, ASOS Style Match application was brought to life. Differing in the offered functionality, those apps had a major common feature. They allowed users to snap or upload an image of an object and get the info about it or recommendations on similar objects.
And that proved beneficial. In just a year Pinterest Lens celebrated 600+ million searches a month. By 2019, people have addressed more than a billion queries to Google Lens. ASOS profited offering items from their product stock similar to users' queries.
All those apps played a bet on visual search. And it appeared to be something people have long anticipated. Bing, Amazon, eBay, and dozens of smaller companies joined the club bringing consumers' experience to a new level. So probably, it's high time to keep an eye on visual search.
In a wider meaning, visual search is a technology that allows looking for information using an image as a query. It also allows for returning visual search results to text-based queries. But in fact, there are three widely used visual search variations today.
Image meta search is an information retrieval system that allows searching for visual content like images, animations, etc. using text-based queries. It returns a set of image thumbnails as search results sorted by relevance. Judging by the name, it's quite clear that the system uses images metadata (file names, alt tags or descriptions associated with the image) indexed in its database to find a match to a query. This makes it the closest relative to the classic keyword-based search we all know. With the difference in the type of results.
Today, image meta search is the most widely adopted variation of visual search. There are, however, a couple of drawbacks. Sometimes it's quite difficult to formulate exactly what you look for. Especially if you search for some specific things. Which inevitably leads to getting wrong search results.
This is a search technique in which an image formulates a search query. It requires searchers to upload an example image or insert a link to it. To which a search engine answers with the same or similar images as search results. Thus searchers may clear up the image source or find similar images.
Reverse image search is powered by content-based image retrieval (CBIR). Content-based image retrieval implies retrieving images similar to a given example image from an image database. The search analyzes images content similarity. And the content here stands for colors, shapes, textures, and other information an image may contain.
What makes it possible, you ask. Oh, here comes a wider interdisciplinary scientific field called computer vision. Which is, in its turn, a part of the artificial intelligence field or the computer science field.
Oh, this is my favorite. This is search by image or a part of an image powered by image recognition technology. Being a part of already mentioned here computer vision, image recognition refers to the identification of different objects in images. They may be places, logos, people, buildings, and other objects. The technology roots in machine learning and neural networks. Machines recognize the size of an object in a picture, it's shape, color, etc. and return the information about such objects. Also, they show similar or alike items in search results.
Let's take a deeper look at the process. Once a user snapped, for example, a killer pair of fuchsia ankle boots (or uploaded an image of them found on Instagram) with "I want something like this" in mind. How would the machines do the magic? The process can be divided into three major steps:
Once uploaded, the image is being scanned by the visual search module's artificial neural network. The neural network will generate your image's main descriptors, such as the type of footwear, it's color, material, etc. They will be returned in a form of text tags.
With tags at hand, the system may look through its index to find matches. In our example, the index is the product catalog with all the items labeled with their descriptive tags.
The index, by the way, is also created with the help of the artificial neural network. If a retailer implements a visual search module into its system, the neural network within it processes all the products it the retailer's catalog to eliminate the metadata each item contains and generate descriptive tags instead. Thus the system may analyze the newly arrived tags against the existing tags of the same kind.
Finally, the system filters all the found matches based on their relevance to the input. So first will come the fuchsia ankle boots that look the most like to what you've been searching. The rest of the results will vary from the absolutely identical shoes to those just of the same color.
Below are some examples of different visual search solutions, like search engines and visual search applications:
Google Images is the biggest visual search service launched by Google in 2001. It allows for both textual-based and reverse image searches.
Bing Images is the visual search engine by Microsoft. It supports textual-based and reverse image searches. It also allows highlighting an object in the image and comes up with image results of what it believes to be similar objects.
Pinterest is one of the biggest and most popular visual search platforms. It gives inspiration and powers creativity for 250 million monthly users. Each textual query returns thousands of related pins. Moreover, each pin empowers you to search for visually similar results just by clicking a special button or highlighting objects.
Google Lens is a visual search app that empowers users to search for information literally through their camera lenses. It helps translate texts, read QR codes, find outfits or accessories you liked, explore nearby places, identify plants and animals, and many more.
Pinterest Lens — a great visual search powered app that allows users to look for ideas based on things they point their cameras at.
Bixby Vision — a visual search application by Samsung. It provides similar to Google Lens' functionality, giving users a new way of interaction with the world around them. From translating a sign to learning the calories in your lunch to even trying on makeup with AR.
Visual search technologies are changing users' search behavior and open new doors for companies to promote and sell their products. The ability to come up with what a user exactly wants (or very close to it) is what businesses strive to, don't they? In fact, the fashion industry was the first to recognize the potential benefits visual search technology can bring. Yet, it's a great solution for e-commerce in general. Here are some key points on how visual search may help both users and businesses:
For users, visual search is sometimes more valuable than text-based search. And way more time-saving. It frees users from trying to guess the right keyword that would describe the desired object and scramble through tons of not always relevant results. Just try to google something like "a red flower with lots of petals" — and good luck with finding the one you meant. Uploading a picture (or snapping it) is much easier and it tells a search engine exactly what to search for.
And this works not only with looking for things to buy, though it may look the most obvious. Identifying sights when traveling, getting to know unfamiliar plants, birds, and animals, finding home decor ideas — the areas of use may be various.
For businesses, visual search may become a win-win strategy. As more and more consumers would appreciate getting more shoppable content, according to this study, it would be quite unwise not to provide them with it.
Here are a couple of the most obvious options a business may benefit from:
All popular search engines allow for searching by image. And as interest in visual search is growing, they increase the image search share. For example, Google already tends to show more images in search results. So, here are some ways companies can leverage all kinds of visual search.
As I've already mentioned, image meta search is the most used type of visual search as of today. To increase your visual content visibility in search, the good old image SEO best practices still work very well. The aim of image SEO is and has been to give search engines all the possible information about the content of images. Consider these steps to make sure your visual content appears in search results.
Just create or open your project in WebSite Auditor and go to Site Structure>Site Audit.
Scroll down the results column to Images and select Empty alt text to see all the pages that contain images with empty alt-tags.
Create or open your project in WebSite Auditor and go to Content Analysis> Page Audit.
Choose or add the page to check and let the tool do the check for you.
Look through the Mobile Friendliness and Page Speed section of the results. It will give you insights about your page speed possible issue and recommend how to fix them.
Visual applications and search engines are able to identify and decipher different objects in images. However, you may help them do it. You need to ensure the images on your website are easily "readable". This means the objects are clearly visible, the backgrounds are light, and there's no clatter in them. All this makes image processing easier for machines, thus helping them define objects more accurately.
Being one of the most popular visual search platforms, Pinterest provides businesses with great advertising options that help businesses engage with users right at the moment they search for inspiration. The Catalogue feature allows retailers creating product catalogs and easily promote product pins through Shopping Ads. And this seems to be extremely beneficial, as 90% of weekly Pinners use Pinterest to make purchase decisions, according to this study.
Gartner predicted that by 2021 early adopter brands would increase revenue by 30% having redesigned their websites to support voice and visual search. While it's not likely that visual search will push the text-based search out completely, its share will definitely grow. What's more, visual search is becoming a significant part of the consumer's journey. Thus companies that want to be ahead of the competition should already take a closer look at visual search, examine it's possible business benefits, and make use of it.
Visual search seems like a great trend to me. I believe it'll evolve greatly in the upcoming years. What do you think? I'd appreciate your thoughts and arguments.
By: Volha Belakurskaya