5 Ways Computer Vision Is Transforming Online Shopping

In a Namogoo's Online Consumer Online survey, 88% of them said that product images are a key element for a great shopping experience, ahead of all other factors such as product reviews (78%), product descriptions (77%), and the checkout process (76%).

Retailers recognize this and acknowledge that images shouldn’t only be a form of content to be displayed. Rather, product images can and should play a dynamic role in delivering a retail experience that benefits shoppers and keeps them as loyal customers. To do this, ecommerce businesses have turned to computer vision (CV).

Computer vision is a form of artificial intelligence (AI) that enables software to derive meaning from digital images akin to human perception (and even surpassing it). Computer vision replicates the human visual system to perceive images (including videos) and carry out tasks such as recognition, distinction, and classification.

Computer vision has been applied to and continues to advance a wide variety of technologies, which include facial recognition, self-driving cars, diagnostic x-ray, and crop monitoring, among many others. Where human sight is necessary, computer vision can and will impact how tasks are done no matter the sector or industry.

In this blog, we focus on how computer vision is transforming online shopping. We specifically look at these five areas where computer vision is changing the way we buy things online:

• Visual product search
• Personalized product recommendations
• Virtual Try-On
• Inventory management through virtual warehouses
• Image moderation for marketplaces

1. Visual product search

Just a few years ago, online shoppers search for the product they’re looking for only through text-based queries. With the advent of visual search among popular ecommerce websites, shoppers can now upload a photo of any item they wish to buy. Computer vision is at the very heart of this capability.

Far from simply being a “nice-to-have” feature, today’s shoppers are acclimating to this functionality. A survey in the US and UK found that 62% of Gen Z and Millennial consumers want visual search capabilities in their online shopping experiences. While visual product search may still be a luxury for smaller online retailers, the more established ecommerce players have been demonstrating how to leverage computer vision to better engage their customers.

• eBay: This well-known marketplace and auction house was one of the first online retailers to introduce computer vision as early as 2017 through image-recognition on their mobile apps. This allowed shoppers to search for any physical item among their 1.1 Billion listings with pictures instead of text-based search.
  

Source: eBay, An Easier Way to Search eBay: Computer Vision with Find It On eBay and Image Search Is Now Live

• ASOS: The fashion retailer launched their visual search capabilities in their ecommerce platform to help their shoppers find clothes and accessories using their smartphone camera. They focused on allowing customers to find products from photos captured in fleeting moments — someone passing the street or a screengrab from social media. At the time of launch, ASOS had 85,000 products in its online store with 5,000 new items added every week.
• Amazon: In 2019, Amazon launched its visual search feature for their fashion products, which they branded as StyleSnap. When shopping, users simply upload an image of a look they’re looking for and then StyleSnap will present recommendations for items that best match the image. Amazon extended this capability to StyleSnap for Home whereby customers can search for furnishings and home products. Instead of scrolling through hundreds of options for a couch or a lamp, for example, the visual search capability of StyleSnap can provide the products that most closely resemble the image or screenshot users upload.

Source: Amazon StyleSnap

Visual search will only grow from here. As early as 2018, Pinterest reported that their users across its various tools (including Pinterest Lens, its camera search feature) have been doing more than 600 million visual searches every month. The business case for visual search couldn’t be clearer. As Amy Vener, retail vertical strategy lead at Pinterest, said: “Visual discovery allows the consumer to generate a more emotional connection, which translates into less price sensitivity.”

Source: Amazon Science, The science behind Amazon's new StyleSnap for Home feature

Early adopters that redesign their platforms to support visual (and voice) search could see their digital commerce revenue increase by 30%. Studies vary but IndustryARC report had projected that the global visual search market to reach US$ 15 Billion by 2023.

2. Personalized product recommendations

Customers desire highly personalized online shopping experiences. 71% of online shoppers say they feel frustrated when their online shopping experience is impersonal, according to Segment's report on state of personalization. Marketers have always recognized that more personalized product recommendations is a tested strategy to convert shoppers into buyers and buyers into repeat customers.

In addition to product search, computer vision has also been playing an increasingly important role in improving product recommendations, particularly in making them more targeted and personalized. In platforms wherein product recommendations are powered by computer vision, shoppers are shown products similar in visual attributes to items they search for. For fashion items, for instance, these attributes could be shape, pattern, color, sleeve type, neck type, and length.

Part of how computer vision enables personalized product recommendations (and visual product search as well) is the capability known as visual attribute extraction. This is how physical attributes are “seen” and taken in by the software as meaningful information, which it then uses to identify and define any given product.

For instance, eBay recently enabled their sellers to scan products, beginning with trading cards in 2021. The app then automatically uses the attributes from the scanned image to auto-populate new listings. In addition to the benefits in efficiency and ease-of-use, this ability allows for better product classification or categorization, which then paves the way for enhanced product search and recommendation.

The opportunities for online retailers in enhancing and retooling their product recommendation capabilities are evident. 31% of sales among ecommerce sites come from product recommendations. Further, the Segment's report indicates that about half (49%) of online shoppers have made a purchase they didn’t originally plan on buying after being shown a personalized product recommendation.

3. Virtual try-on (VTO)

Virtual Try-On (VTO) is a digital technology that allows customers to virtually sample or “try on” products — typically (but not exclusively) clothes, accessories, and personal effects — using augmented reality (AR), which overlay real-world and virtual objects or environments together.

For example, shoppers can see how a shirt might fit even without physically trying them on. A user could also virtually try on a pair of glasses. Fashion brands such as Burberry, Gucci, ASOS, Timberland, and Lacoste have introduced various forms of virtual try-ons powered by augmented reality.

Image source: VentureBeat

Even makeup brands have deployed virtual try-on whereby a shopper wanting to buy an eyeshadow or a foundation could use their smartphone to see which particular shade suits their skin tone. No matter the product, customers can access VTO in three (3) ways:

• Mobile: using an app or a web browser with their smartphone camera
• Desktop: using a web browser with their own photo or device camera
• In-store virtual fitting rooms: using digital mirrors and smart displays

Consider how Walmart has introduced virtual try-on to their digital platforms. With their “Choose My Model” feature, online shoppers can select a model that matches their appearance. Walmart customers get to choose from among 50 virtual models (with more on the way) who can best approximate their height (between 5’2″ and 6’0″), size (XS through XXXL), skin tone, body shape, and hair color.

Walmart’s VTO technology is further powered by their neural network and computer vision capabilities, which analyze images of garments and models in the Walmart catalog in order to create a wide variety of dressed images. These technologies help create the variations that can be generated from a single product such as options in size, color, and sleeve length, among other possible differences.

Those who’ve tried virtual try-on can attest that it makes the shopping experience more fun and engaging. It can bring people out of their comfort zone by allowing them to experiment with the latest fashion trends or new looks they might not have considered before. This is partly due to how virtual try-on makes shopping more efficient as shoppers can switch between items more seamlessly.

Virtual try-on also brings benefits for retailers. It can drive revenue due to greater personalization, improved customer experience, location independence, higher customer engagement, and increased competitiveness. VTO can also reduce costs owing to fewer returns, minimized losses, and prevention of wasted time and resources.

For businesses looking to assess whether virtual try-on is truly a game-changer or mere hype, we at Netguru unpacked this in a comprehensive report. If you’re looking to build virtual try-on capabilities whether for retail or any other sector, Netguru has the expertise and experience to deliver for your business.

4. Inventory management via virtual warehouses

Online shoppers today expect fast delivery — next day or same-day delivery whenever possible. With this pressure, retailers have had to rely on a more decentralized supply chain whereby inventory can be housed anywhere ranging from a traditional warehouse, a distribution center, a temporary facility, even to the stockroom of a store.

This is where virtual warehouses come into play. Virtual warehousing tracks inventory wherever they’re physically stocked to enable fast and efficient distribution and last-mile delivery when needed. When an order is placed, virtual warehouse solutions help retailers identify the most optimal way (typically the fastest way) to fulfill the order.

However, the smallest mistake can trip up inventory management. A mislabeled product, even with the tiniest detail, can cause headaches for logistics and supply chain professionals. Virtual warehouses coupled with computer vision capabilities primarily address these types of potential problems.

Computer vision solutions regularly scan inventory in a virtual warehouse. Upon encountering product attribution mistakes (e.g. color, weight, volume, size, expiration date), the system will flag these as potential errors. The relevant employees will then be notified to physically inspect the inventory for verification. If there is, indeed, an error, employees can take the appropriate action, such as correcting the label or pulling out the product. Furthermore, because of machine learning, the computer vision capabilities of any given virtual warehouse will only get better in spotting errors moving forward.

Computer vision in virtual warehousing can be part of a larger monitoring system integrated with a variety of systems and sensors built for physical warehouses and logistics facilities. In this context, computer vision software can be streamlined with cameras and intelligent video analytics, not only for warehousing but also for in-store operations. These types of AI-driven solutions process footage in real-time to optimize space (from shelves to warehouses) and manage the entire inventory more efficiently.

5. Automated image moderation for marketplaces

Retailers that sell their own inventory have a high degree of control over the product images they have on their websites. This isn’t the case for online marketplaces that have hundreds (even thousands) of merchants or sellers on their digital platforms. Think of Amazon, eBay, Rakuten, and even social media platforms such as Pinterest and Facebook.

Before, these companies relied on a lot of human moderators who approve or disapprove these product images that merchants upload on their sites. In recent years, image moderation solutions powered by computer vision and other AI capabilities have automated much of this work.

Image moderation technologies assess product images whether they comply with the respective policies and guidelines of any given ecommerce firm. For example, as part of its technical interventions for the pandemic, eBay deployed computer vision to flag listings for masks and hand sanitizers with unreasonably inflated prices. This also had the ability to detect and mark fad products offering false health claims.

The typical use case of image moderation tools is to block product images that may contain adult content or promote violence. Most online marketplaces also have strict rules against explicit medical content such as disease symptoms, open wounds, or before-and-after photos. Some platforms also have policies against copyrighted photos, memes, political content, and other images unrelated to the product listing.

Instead of manually reviewing each image that merchants and sellers upload, content moderation tools flag inappropriate images automatically. There can still be a few “false positives” (images mistakenly tagged as noncompliant or inappropriate), which a supporting team of manual reviewers can easily correct.

This technology ultimately redounds to the benefit of shoppers as it protects them from inappropriate or unwarranted visual content. At the same time, ecommerce marketplaces get to keep the integrity of the retail experience they want for their shoppers, not to mention achieve cost savings and operational efficiency.

Build with computer vision

Computer vision and the retail sector go hand in hand. With consumers expecting omnichannel experiences, the application of computer vision into online retail is already being extended and integrated in real-world shopping use cases such as automatic checkouts, queue monitoring, foot traffic analysis, and theft detection.

This emerging technology has quickly moved from a research or experimental-driven field to a commercially driven tool. Computer vision continues to show that it benefits both shoppers and retailers. For customers, it delivers effective product discovery and a seamless shopping experience. For retailers, the benefits are obvious: customer engagement, cost savings, operational efficiency, and clicks and conversions, ultimately.