'It's a game changer': Google, Pinterest, Snapchat and marketers are rushing to crack the latest advertising search trend

'It's a game changer': Google, Pinterest, Snapchat and marketers are rushing to crack the latest advertising search trend

home depot

Joe Raedle/Getty

Jose Ulloa Jr. works in a Home Depot store on May 17, 2016 in Miami, Florida

  • Visual search is emerging as the latest frontier that brands are looking to conquer in a bid to connect with consumers as well as drive sales.
  • Retailers like The Home Depot, fashion brands like Tommy Hilfiger, and auto brands like Toyota are active in the space, an increasing number have started testing out visual search features.
  • Platforms are also dabbling in the space, with Facebook, Amazon, Apple, Google, Pinterest and now, reportedly Snap rolling out visual search products.

You've just seen someone wearing the perfect jacket. What happens next?

You could string together a couple of descriptive words and search for it online, hoping to get back something similar. Or you could snap a photo on your phone and get all the information you want instantaneously.

Complimentary Tech Event
Transform talent with learning that works
Capability development is critical for businesses who want to push the envelope of innovation.Discover how business leaders are strategizing around building talent capabilities and empowering employee transformation.Know More

The latter is an example of what's called visual search, the latest new digital frontier that both major brands and tech giants are looking to conquer. While its parameters are still being defined, visual search generally refers to the use of digital photos to get information, rather than text or voice.

There are several different flavors emerging. One, like the example above, is a lot like traditional search. But instead of typing text into a search box, it typically involves looking for product information by snapping a picture with a smartphone. It's sort of the image version of using Shazam to get all the details of a song.


Another visual search tactic consists of using images or codes to drive people from the analog world to the web. And another version, popular on Pinterest, simply uses individual photos or pins to drive people to e-commerce sites.

Retailers like The Home Depot, fashion brands like Tommy Hilfiger and even auto brands like Toyota have started experimenting with these various tactics as a potential new way to connect with consumers - and hopefully drive sales.

Given the preponderance of smartphones, some ad industry insiders see massive potential in visual search.

"Visual search is a game changer," said Jeremy Sigel, global SVP of content and innovation at Essence. "It has the potential to rival 'traditional' search."

'Customer behavior is changing toward visual search'

While personal photos have existed for hundreds of years, they haven't provided much of a marketing opportunity until recently. But the rise of mobile, e-commerce and developments in the fields of machine learning and artificial intelligence has changed that.


Today, visual search technology can filter photos through image recognition software, culling through hoards of data to surface the details associated with the exact product that a photo captures.

This photo-centric data can be used by advertisers to better engage with customers and ideally drive them to purchase.

"Communication for anyone under 30 is camera-first, not text first," said James Douglas, SVP and head of media at the search agency Reprise. "There's tremendous opportunity for brands to tap into machine learning and artificial intelligence to figure out what people are communicating through pictures, and then build relational opportunities for themselves on top."

Brands are not letting the opportunity pass. According to Ted Mann, CEO of the visual recognition software company Slyce, at least 50 companies currently license its technology to power a camera search mode in their apps. And interest in the technology, both at Slyce and outside, is only increasing.

Screen Shot 2018 07 17 at 5.44.35 PM



Major brands are encouraging people to shop using their phones' cameras

Target, for instance, is using visual search to drive consumers toward its products. Instead of typing "leather couch" into a search bar via text and sifting through thousands of results, for example, its app simply lets you snap a picture of your inspiration using the visual search tool Pinterest Lens - and gives you a curated set of products similar to it in seconds.

Similarly, Toyota is employing the technology to get its customers to engage with its new 2019 Avalon. The auto brand is placing Pincodes - Pinterest's version of QR-like codes - in Shape magazine to drive interested consumers to a dedicated Pinboard for the car on Pinterest. Users can snap the code in the magazine using the Pinterest app, where they can further engage with the vehicle digitally and socially.

Visual search is also a huge priority for The Home Depot. The home retailer has not only been a Pinterest partner since 2014, but is currently undertaking the task of building a massive library of "Shop the Look" pins tagged with its products on the platform. This means that the company can link specific products within a pin directly to its site allowing users to easily purchase each item. These pins have been performing 4 times better than regular pins on average, according to Melanie Babcock, senior director of agile marketing at The Home Depot


Home Depot

Home Depot's image search

The company is making visual search a priority outside of Pinterest too. It has assembled its own in-house team dedicated to the technology, said Babcock, because sooner or later, customers will come to expect it.

"Humans are inherently visual, we know what we like when we see it," she said. "We believe that customer behavior is changing toward visual search, so it's a huge priority for us."


Text search still dominates. And don't forget about voice

To be sure, text or keyword-based targeting remains the backbone of search advertising. Marketers still mostly serve ads to users based on whether their queries include certain sets of words and phrases.

And Google remains the undisputed king of search, raking in more than $28 billion or nearly 74% of the net search ad revenue in the US in 2017, according to eMarketer.

But as spending on search advertising continues to grow at double-digit rates in the US, marketers are turning to additional targeting options, including voice and visual search. In fact, voice and image-based searches are expected to make up at least 50% of all searches by 2020, according to Kleiner Perkins Caufield & Byers partner Mary Meeker.

But visual search has some way to go before it truly bursts into the spotlight. More marketers are investing in voice search and developing their own Alexa skills and Google Home apps than visual search, which will continue to happen until consumer behavior shifts.

"Everyone's always been interested," said Layne Harris, head of innovation technology at 360i. "But there's always going to be hesitancy to invest in something where there isn't a proof point."


And optimizing for visual search is no easy feat either, warned Reprise's Douglas.

"A lot of effort needs to go into making sure that the images are clean and can be interpreted by partners, and into organizing the feeds," he said.

But the biggest tech companies are all making serious investments in visual search

As brands embrace the technology more, they also have an expanding roster of platforms and tech companies to rely on for help. Self-proclaimed visual search engine Pinterest may have the first-mover advantage with Pinterest Lens and a slew of other features, but others aren't far behind.

Facebook, Amazon, Apple and Google have all started either building their own visual search platforms or acquiring companies that analyze such photo content. Google, for example, has Lens, and plans to make your smartphone camera "like a visual browser for the world around you."

Snap is the latest platform to jump on the bandwagon, and is reportedly working on its own visual product search feature called "Eagle" on the Snapchat app, according to a Tech Crunch report last week. The move is the latest iteration of Snap's evolution as a camera company, proving that it is serious about its mission of the camera being a tool to help people navigate the world around them.


Snap's entry could finally take visual search mainstream, say experts, benefiting everyone in the space.

"One thing that Snap has done much more than other platforms is introducing behaviors that people haven't been familiar with before," said 360i's Harris. "They have been pioneers in pushing exploratory behaviors like vertical video, which has now become the standard."

Pinterest vs Snapchat

The fact that Snapchat's app opens into the camera is also the reason why it could ultimately deliver a better shop-by-image experience than Pinterest, said Alex Marsh, SVP and director of platform innovation at BBDO. Recode has reported on a possible partnership with Amazon, but the company could eventually bring in other retailers or innovate more on ad products.

"Snapchat has a good track record when it comes to rolling out new ad products for marketers," he said. "I would expect them to find success innovating a number of products around the camera search."

If Pinterest is threatened, it's not showing it. Visual search is not about just using image recognition to surface more similar photos, but also the relevant context. And Pinterest's algorithm is more nuanced because it is supplemented by a layer of human curation that its users bring with their boards and tags, according to Vikram Bhaskaran, head of market development at the company.


"We've created a whole new vertical in the search space, allowing brands to not only to use visual SEO but also target on the basis of it," he said. "Visual search is a commodity, but the ability to understand the nuances of the object with human curation is what gives us our edge."