How to Detect Manufacturing Defects Using AI

May 19. 2019. 7 mins read

While most consumers don’t often see the process of manufacturing their products, they are most certainly aware of when there is a defect in the product they receive. This kind of experience can range from disappointing to downright disastrous – neither of which is ideal for the company producing the product. Unfortunately, the reality of modern manufacturing is that unexpected defects are more frequent than most realize, and most companies that produce physical products don’t know about critical defects until it is already too late: the product is in their customers’ hands.

A sour patch of Amazon reviews alone can tank the life of a product that ships with a small but severe issue. Incidentally, many companies have actually turned to Amazon reviews to find issues that went under the radar in their manufacturing facilities. While this is an interesting trend in its own right, most companies don’t want things to come to this point. There needs to be a more reliable way to prepare for unexpected yet major issues in a product – be it a discrepancy in sourced parts, a design flaw, or something we can’t even imagine. From there, how does a company keep that data close at hand and continually iterate on known and unknown problems? That is where Instrumental comes in.

How Instrumental Helps Find Defects

Click for company websiteFounded in 2015, Silicon Valley’s Instrumental Inc. has raised $10.3 million in funding to develop real-time defect detection of both known and unanticipated issues on manufacturing lines. We’ve looked at lots of startups using computer vision for various applications, but what sets Instrumental apart goes beyond cameras. Instrumental aggregates all of the image data into a cloud database, where it can be analyzed by tens or even hundreds of machine learning algorithms to identify defects or changes that engineers care about. In addition, they analyze those results to provide risk assessment of the line process and program as a whole. While it’s reasonable to expect that by the time a product gets to the production stage, all of the kinks are worked out and new defects would not occur, Instrumental has shown that 2% or more of units on stable mobile phone lines can have defects that go undetected by the existing, non-Instrumental test stations.

How can a machine learning algorithm detect a problem when it doesn’t know what a problem looks like? Well, just like humans do, the algorithms learn over the course of about thirty units what normal units look like. (It’s okay if there is some normal variation in those units, they don’t have to be perfect.) Next, Instrumental works with their customer engineers to identify the key regions of interest to examine – they can be big or small. Instrumental’s algorithms focus on each of these locations and identify anything that looks different from the normal state. They get better over time, but the coolest feature might be that the same algorithm can find multiple types of defects, such as in the example below: dark regions, dark vias, and pin damage.

In conventional vision systems, each one of these defects would have had to be anticipated and pre-programmed in by an expensive consultant before it could be intercepted by the system. With Instrumental, no anticipation or pre-programming is required, and one algorithm can do the job of many. No failure examples are needed to set up a live algorithm that can intercept defects on the line.

That’s how Instrumental’s technology works. It quickly learns how things should look, operate, and is able to autonomously detect and catch potential issues along the production line in real-time, saving rework and preventing quality escapes that result in returns. This proactive approach relieves many pain-points for human engineers so they spend more time on value-added activities, something that’s particularly valuable early on in the manufacturing process.

The Process of Manufacturing

Instead of a checklist of five items that a human inspector might have, Instrumental’s algorithms can check 25 items or more, all at once, allowing human engineers to isolate a subset of products that failed a test or might have a discrepancy. That’s cool, but in order to understand the real value being added here, we need to understand a bit more about a typical manufacturing process, which is for the most part industry agnostic.

Since most of us don’t work in manufacturing, we may think of a manufacturing process as consisting of numerous steps with sophisticated things happening throughout. 98% of electronics assembly lines, like the ones that built your smartphone, tablets, and other devices, are primarily manual assembly lines – people sit shoulder-to-shoulder doing various assembly, testing, and inspection tasks. That’s referred to as “mass production,” and what you’re probably unaware of is how companies go from “prototype” to “mass production.”

From prototype to mass production
Source: Ben Einstein

Above you can see the common stages that a manufacturing firm will move through as they go from a functioning prototype to mass producing a product. It’s the process of designing and testing the actual production process itself. After you have your engineering prototype, the first step of this process is referred to as the Engineering Validation Test (EVT) which Instrumental describes as follows:

The EVT build is the first time you combine looks-like and works-like into one form factor, with production intent materials and manufacturing processes.

This consists of a production run with 100 to 1000 units that are fully functioning and testable, and that use the same parts you plan on using when you finally reach “mass production.” You may decide to test multiple designs, with the goal of eventually settling on a single design and then fixing as many issues as possible. Everyone who works in quality assurance knows how important it is to find defects at this stage, not later. Just imagine not catching a defect and then finding out about it when customers begin calling customer service or leaving one-star reviews on Amazon. Think about how tough it will be then to trace back the problem in your production process.

One problem you face during the EVT phase is the high number of failures in the production process. Depending on the complexity of what you’re building, up to 40% of the units that come off the line may fail for a variety of functional or performance reasons and need to be analyzed. Most companies are highly globalized, so how exactly do you explain a problem you’re having with a defective unit in your Chinese factory to an engineer in Geneva? You could FedEx the part overnight, but what if the engineer wants to see the parts that were produced immediately before the defective part? One solution is to take pictures of every single part as it passes through every single stage of the production line – like Instrumental does.

Instrumental’s solution works really well. With just one click, engineers can dive in deeper to see the full context provided by a 20-megapixel image. Engineers report that they’re twice as efficient during their time in the factory and find an average of twenty issues they claim they would not have found until production. That’s particularly notable because at the EVT phase, you’re still a long way away from mass production. The EVT phase is only complete when you can produce “one production-worthy configuration that meets all of the product requirements for functionality, performance, and reliability.” That’s according to a great blog piece by Instrumental Inc. that describes the remaining phases you need to go through to finally arrive at mass production.

Success Stories

Many machine learning applications sound cool on paper until you try them in the real world. Since this technology was built by a couple of ex-Apple engineers, it’s already being put to good use in improving quality assurance for various use cases across all types of manufacturing. A recent example is Motorola, which has rolled Instrumental out across all of their smartphone development lines. Motorola has found that Instrumental technology has enabled them to identify defects faster, save significant amounts of money in development by avoiding extra experiments, and ramp into production much more quickly.

Another use case on the Instrumental Inc. website is from August Home, which uses Instrumental’s technology to help identify issues in complex smart door locks. While you might think shipping someone a scratched door lock is bad for business, what about when one of your customers arrives home from a night out and realizes their smart door won’t open because of a dark yield defect?

Above you can see a “dark yield defect” which is a problem that’s missed by functional tests on the assembly line and causes units to fail only after making it to customers. In the above picture, you can see a connector that is only partially connected (framed in red outline.) Below that, you can see another connector (framed in green outline) which is how the connector ought to appear. Both units pass a functional test on the manufacturing line, but one unit fails after some drunk bloke shakes it one too many times. This is just one of many examples where Instrumental’s technology can identify problems that standard test functions can’t.


In the world of artificial intelligence, much emphasis is being placed lately on the notion of “general artificial intelligence” which is the ability for a machine to learn without much training. Generalized manufacturing technologies are one of the great dividends of the Industry 4.0 emphasis, and Instrumental is a good example. Instrumental’s technology has the ability to add value very early on in the manufacturing process by catching defects so they don’t make their way into the hands of customers or waste valuable time before mass production. It also helps shorten the early stages of the manufacturing process which sit right on the critical path, meaning you can ship product sooner and try to keep all the promises your sales and marketing team has been making to your customers. It’s easy to see how such a technology could also prove useful to “on demand manufacturing” business models where you’re constantly producing limited-run quantities for customers. There are a huge number of use cases for machine learning in manufacturing, and Instrumental Inc. appears to be just getting started.


Leave a Reply

Your email address will not be published.