Trax – Reinventing Retail with Computer Vision

After Edward Snowden leaked that the National Security Agency (NSA) used backdoor channels to gain access to smartphones, webcams, and traffic cameras, people have become paranoid that the government is watching through our devices. The rise of sticky notes and cover slides attached to webcams is a testament to that fact. (Never mind the hackers trying to see what you’re up to.) And to all of our great dismay, the tin foil hat conspiracy theorists were actually on to something. If the NSA is willing to spend billions of dollars to gather all that image and video data, it must mean it’s worth something.

But even the United States government can’t hire enough college interns to stare at the petabytes of images and video produced by all the nation’s electronics. So what does a data-hungry intelligence agency do? It has to rely on artificial intelligence, and specifically computer vision, to transform all of those private moments from people who didn’t know anyone was watching into actionable insights.

What is Computer Vision?

Computer vision is a subdiscipline of artificial intelligence that involves pulling insights from digital images or video. The best example of how powerful computer vision can be is the development of sensors for autonomous vehicles, which can identify, track, and make decisions around complex road environments. While we’re not yet living in a world of self-driving cars, computer vision has many other great uses beyond the futuristic experience of getting piping-hot pizza delivered to our front door by a driverless vehicle in 30 minutes or less.

Wood Nots - Computer vision versus human vision
Computer vision versus human vision. Credit: Wood Nots

In the example of a human being, computer vision is the complex part of your brain that can instantly register attractive people in your peripheral view. It’s the evolutionary logic behind how we make sense of the objects in our world, connecting the data we receive from our eyeballs with the synapses in our brain that compel us to overcome our fear of talking to hot strangers. Today, machines are still relatively stupid, so any additional data from the outside world can only help them make better decisions.

Computer vision is a game changer that gives artificial intelligence algorithms the ability to see and interact with physical reality. McKinsey reported that Google (GOOG) completed 24 mergers and acquisitions since 2010, with eight of those in computer vision. Since venture capitalists began pouring money into artificial intelligence startups, computer vision has been a big chunk of those investments.

Computer Vision - McKinsey & Co.
Computer vision makes up a nice slice of the AI pie. Credit: McKinsey

We’ve talked about startups in the computer vision game here and here, even traveling to Moscow to learn about how computer vision is being used to fight crime in Russia. Some of China’s most funded AI startups are solving problems using computer vision and achieving some incredible feats. For example, a Chinese police car is now able to identify a criminal on the sidewalk while traveling at 60 miles per hour.

While computer vision is already finding great uses in national intelligence and high-tech sectors, one underrated place where computer vision and artificial intelligence are set to boost performance is the humble corner store.

Computer Vision for Retail

Click for company website

Founded in 2010, Trax is a Singapore-based startup that recently took in a massive Series E in April 2021 for $640 million. In total, Trax has raised a whopping $1 billion in disclosed funding from notable investors including SoftBank (SFTBY), Sony (SONY), and BlackRock (BLK). The company is using computer vision to help retail markets and grocery stores keep track of their wares in real time and ensure that out-of-stock items are repurchased on time, while expired are pulled off from the shelves before unsuspecting consumers take a swig of root beer manufactured in the 90s. The company holds 23 patents on its technology and can analyze images from phones, in-store cameras, and grocery store robots.

Trax - Computer Vision
Trax robot taking images of the grocery shelves. Credit: Trax

The company has over 150 customers on the docket, including food and beverage giants Coca-Cola (KO), Nestlé (NSRGY), Diageo (DEO), Anheuser-Busch InBev (BUD), and one of our dividend champions, Procter & Gamble (PG).

In 2019, Trax announced a partnership with Google Cloud Platform to deliver its Retail Watch image recognition product to more customers across the globe. (The company has 292 customers across more than 90 countries.) With a billion-dollar war chest, they’ve been on the hunt to acquire smaller companies that can bolster their core technology, gobbling up six companies to date, including its most recent acquisition of retail insights provider Survey.com in 2020.

Use Cases for Computer Vision in Retail

The Trax platform allows retail and grocery store employees to take images of shelves with an app on their phones. The images are sliced and diced using the Trax computer vision algorithms to transform pixels into real data that can be understood by the software. That data gets shoved into a cloud-based data storage system and fed into another series of algorithms that stitches the images together into a giant panoramic collage that represents the entire full-store shelves. The individual items on the shelves are transformed into stock-keeping units (SKUs), which are the alphanumerical code associated with those scannable, unintelligible bar codes found on the bottom of a food or retail product. But here’s the magic.

Those SKUs actually mean something to the ancient system of retail inventory management, where these numbers are not even standardized between brands and companies (a can of baby formula and a bottle of rum can have the same exact SKU). That means Trax provides a real-time sense of how many retail items are actually on the shelf and when to order more.

Human versus Computer Vision - openFrameworks
What a human sees versus what a computer sees. Credit: openFrameworks

Using a combined accuracy and verification system, even if new items are put on the shelf, the Trax platform can learn what needs to be purchased over time. The system works in tandem with human domain experts who validate the decision-making process of the computer algorithm and trains the machine learning process. And the system can detect tiny differences in food products, so no more ordering 60 pallets of the slightly wrong milk brand that customers never buy. Stored and analyzed data on SKUs also provide retailers with ongoing insights into the performance of each item, and so they can determine if more product needs to be purchased or taken off the shelf.

SKU - Veeqo
SKU and barcode associated with it. Credit: Veeqo

The Trax platform compares items based on six primary metrics – price, share of shelf, presence, positioning, campaign, and planogram. Price is self-explanatory. Share of shelf represents how much of a specific product takes up the shelf compared to other categorically similar products. Presence refers to whether or not the item is in stock and positioning deals with the issue regarding at what level the item is to the eye, as retailers love to put fast-moving products at eye level. For example, when’s the last time you crouched down to the bottom shelf to grab a bottle of off-brand barbecue sauce? Campaign deals with if the product is on sale, which always makes customers jump for joy (unless we’re talking about expired hot dogs for half off). Planogram refers to whether or not that teenage grocery stocker actually put the item on the right shelf or was busy taking Instagram selfies on the job.

Trax - In-Store Breakdown in Real-Time
Trax real-time data analytics platform tracks product categories. Credit: Trax

These metrics are shoved together by the big brains at Trax to create a holistic view of the shelf health for the retailer. That means, in theory, better sales for the retail industry, where retailers regularly over sell their items and can’t restock them fast enough. With the growth of curbside pickup and online ordering as a result of big ‘Rona, the pressure’s on for retailer inventories to be up to date at all times.


While the cameras at the grocery store may or may not be spying on shoppers, they’re certainly peeking into their purchasing habits. Retailers have been using SKUs to gather insight into consumer behavior for a long time, but Trax takes the concept of inventory management to the next level.

By gathering inventory data in real time, Trax provides a serious upgrade for an industry that still uses physical paperwork to keep track of store reports, which are probably already outdated by the time the manager gets around to perusing them for insights. Most importantly, all this information can be used throughout the supply chain to provide distributors and manufacturers with a holistic view of the fast-moving consumer goods (FMCG) lifecycle.


Leave a Reply

Your email address will not be published.