Engineers use AI/ML to improve test

December 10, 2024 by Martin Rowe

Collected at: https://www.eeworldonline.com/engineers-use-ai-ml-to-improve-test/

Engineers can use AI/ML to reduce test development time, analyze test data from manufacturing, automate visual inspection and calibration, and identify counterfeit components. Here’s how some do it.

Testing takes too much time; just ask any engineering or production manager. Shortening test-development time can reduce time to market. Shortening production test time lowers manufacturing costs. Analyzing test data can predict weaknesses before they become reliability problems. Increased automation can further cut costs.

The test industry has developed many tools to help engineers write test code, analyze test data, and reduce the tests needed to sufficiently test dice, packaged ICs, modules, boards, and systems. AI/ML has become another tool.

Test engineers have begun implementing AI/ML in writing code, minimizing test sequences, analyzing test data, and automating functions traditionally performed manually. EE World investigated how engineers are incorporating traditional and generative AI.

Engineering productivity enhancements

Engineers can use generative AI even when starting to develop automated test systems. According to Jayson Wilkinson at Testeract, you should think of AI as an intern. It’s not going to replace test engineers, but it can improve productivity, starting with developing test plans. For example, Wilkinson said test engineers can use AI to recommend test equipment for an automated test station. Feed an AI engine with a DUT’s measurement specifications, add datasheets on test equipment, and let it provide instrument options. Doing so can save engineers time by not having to pour over datasheets. An article from NI says “Generative AI can translate requirements into tests, search existing code for reuse, and identify devices or instruments suited to the test system requirements.”

While generative AI can scan online manufacturer datasheets and suggest which equipment to use, what about proprietary data? For example, data on a device that’s not included in a datasheet, or at least not while its test system is under development prior to product launch. In that case, engineers must train AI with internal documents to learn the test system’s inner workings.

At Semicon West 2024, ATE manufacturer Teradyne announced an AI tool that contains knowledge about the company’s test systems based on internal documentation. Engineers can ask the tool about how the company’s proprietary software works. It’s a ChatGPT for the company’s systems. Teradyne’s Trent Weaver expects a release to customers later this year or early in 2025.

In addition to selecting test instruments, AI can generate code for automated test systems. Wilkinson noted, “We train models that we use internally to help our engineers get answers to questions while they work to generate hardware-abstraction-layer code. We want test-system development to go from weeks and months to hours and days.” Currently, Testeract uses AI to generate code in C or Python, as explained in this webinar.

“Writing test programs is complicated,” noted Keith Shaub of Advantest. “The systems use hundreds, if not thousands, of software libraries. AI/ML can assist a test engineer in writing a higher-quality test program more efficiently. Given that some companies have hundreds of test engineers, that’s millions of dollars a year in productivity gains and cost savings.”

ATE systems need test fixtures and load boards to bring signals to and from test instrumentation, excite a DUT, and take measurements. AI tools developed at ATE companies can help test engineers where ChatGPT can’t. A proprietary AI/ML can access the company’s internal test system and load-board documentation. Schaub noted that tools for mapping a device’s pins to the tester have been around for years. Today, engineers can use AI/ML to learn about test instrumentation, load-board design, and DUT.

Advantest engineers developed an AI/ML tool that helps test engineers develop load boards using images. They used an Arduino board to simulate the tester for a gyro/accelerometer because it has similar features needed for testing. Schaub explained how they fed pictures to the large language model (LLM) and asked it to create the load board for this DUT by matching the tester to the chip. Figure 1 shows the gyro/accelerometer on the right and the Arduino board on the left. Generative AI selected five connections needed to connect and test the DUT, one of which was D2, which connected to the interrupt pin on the DUT.

LLM Semiconductor Test Engineer assistant — *Figure 1. AI chose pin D2 from the tester to connect to the DUT’s interrupt pin. Image: Advantest*

AI chose D2 because it knew that D2 was the optimal choice. That required deep knowledge of the tester and the DUT, information not publicly available. Test engineers routinely make such decisions; now AI/ML can help. In this case, engineers fed sufficient information into the AI/ML to make the right decision, though it needed information about the Arduino, the DUT, and the tester itself.

You can’t do that with ChatGPT. Teradyne’s Trent Weaver noted that asking ChatGPT about ATE or semiconductor test can be dangerous because it can “hallucinate,” producing made-up answers if it isn’t trained with sufficient proprietary information.

AI/ML in manufacturing data analysis

Engineers have begun using AI/ML to analyze test data, both for engineering and for manufacturing purposes. Goals include shortening test time without compromising test coverage and identifying weaknesses that can lead to field failures.

All electronic devices must operate over a range of conditions such as power-supply voltage, minimum and maximum signal amplitudes, input frequency, and temperature. On production lines, tests can occur on thousands of DUTs across multiple test stations. Engineering evaluations, while performed on fewer devices, can still generate tons of data because the devices are often subjected to many more tests than those in production.

Testing for every possible condition is impractical in manufacturing. Thus, manufacturing test is a tradeoff between time and coverage. Analyzing with AI/ML assists engineers in optimizing that balance. Test engineers have used statistical analysis tools for years, so how does AI/ML change the game?

In semiconductor manufacturing, engineers can obtain data from devices while still on the wafer, after dicing, and after packaging. The goal, particularly with digital ICs, is to minimize test patterns. “Design cycles keep shrinking, even though the design sizes are growing,” said Sri Ganta of Synopsys. “It’s a problem that the industry is trying to address.” Optimizing test coverage is an iterative process where engineers change test parameters and look at the results. “You use many engineering resources, but the process still takes a long time. We see AI helping to optimize this process. AI can tweak these different parameters on the fly, and then you can decide which best optimizes the design.” Synopsys simulation software lets the AI engine tweak parameters and make dynamic decisions based on the results for each parameter. This process moves the use of test data earlier in the design and simulation process.

Production generates massive amounts of test data, enough for AI/ML to decide on process variables. Here, factors such as packaging can play a role in overall quality. According to Ganta, engineers can use data to narrow down failure analysis, perhaps identifying failures in a semiconductor’s failing nets or in a particular metal, metal pattern, or metal layer. With that information, engineers can decide if the problem is a process or design-related issue.

manufacturing data analytics — *Figure 2. Analytics software with AI helps engineers identify problems in electronics manufacturing. Image: Virinco*

“AI can reduce the time-consuming ‘hunt and peck’ process to find correlations in test data by telling you where to look,” added Chris Nelson of Virinco. AL/ML enhanced data-analysis software can generate a “process heatmap” that searches for patterns in manufacturing data — which might include part number, test station, operator, and test fixture — combining them with process and measurement data to look for patterns that can reveal problems such as bent pins, bad cables, and so on. Figure 2 shows some of the data-analysis results.

AI in visual inspection

In addition to using AI/ML for test development and for analyzing production data, engineers also use it for visual inspection. Engineers have used pattern-recognition software for many years, but AI can do things traditional inspection software can’t.

Averna’s Matt Thompson has begun using AI/ML to visually inspect medical devices and printed circuits. “Our primary use of AI is in automated visual inspection. We have found that while we prefer to use an algorithm rather than AI to identify a defect, there are instances where an algorithm is not effective.”

Thompson cited an example of inspecting a solder joint (Figure 3). For example, some solder joints might contain a glob, but that’s acceptable if the solder doesn’t cause a short-circuit failure. Human inspectors know that, but algorithms that compare many solder joints to a high-quality solder joint might reject one with excess solder even though the connection will properly perform in the field.

Circuit board and semiconductor — *Figure 3. Although the pins on this IC have some excess solder, it’s not enough to cause a short-circuit failure. AI needs training to identify when these conditions occur. Image: Adobe Stock*

“You can have an unsupervised AI model that can help with anomaly detection. You train it on all good manufacturing data, even those that have some excess solder. Anything outside of that data, it’s capable of flagging, and you can figure out from there if it’s a true defect,” said Thompson.

Thompson noted that companies are not yet comfortable trusting AI/ML to visually inspect products used in medical devices because of the risk of passing a questionable product — with sufficient training, that could change.

In another example, suppose your DUT is a meter that requires a physical alteration, such as an adjustment or trim, to bring it into specification. That might involve removing some material. Thompson is investigating the use of AI/ML to look at test instrument displays and then figuring out where and how much material to remove. It’s currently a manual operation that he hopes to automate. “We’re looking at automating that, and we want to figure out a mathematical algorithm to do it. We also want to entertain the use of AI in that application and see if we can find a way to look at one test result, modify the UUT to bring it into specification, and then keep it flowing down the production line.”

Engineers at Fluke Calibration have done something similar. The company has trained AI to read the digits of a handheld multimeter’s display using a webcam or laptop camera. Most handheld DMMs lack a data port, so there’s no way to connect them to a computer. By reading the display’s digits, the AI can digitize them and send the result to software that controls a calibrator, thus completing the loop. As Jeff Gust explains in the video, training AI to read a meter’s digits took months of work:

Identify counterfeit parts

Using AI/ML for manufacturing inspection and for reading digital displays adds to its use cases for electronics test. Another application covers counterfeit part detection. Here, engineers can use AI to analyze electrical measurements and images for visual inspection. “A review on machine learning based counterfeit integrated circuit detection” Highlights how ML and neural networks help to uncover recycled parts sold as new. The paper describes how engineers used a ring oscillator to create signals at various frequencies. It also covers techniques for comparing simulated results to measured results on genuine and questionable semiconductors. The paper discusses parameters such as path delay, negative-bias temperature, phase margin, bandwidth, harmonics, and more. It also notes that these measurements can reveal aging, which is apparent in recycled parts that can have shorter times to failure that new parts. With enough data on simulated and genuine parts, AI can learn to look for deviations indicating fake parts.

The paper also discusses how AI/ML can enhance image processing to identify recycled parts (Figure 4). In one example, AI detected scratches on an IC package following image processing that removed the device’s identifying part number.

AI finds scratches in counterfeit part — *AI can help identify counterfeit or used parts sold as new by finding scratches on IC packages. Image: IOP Publishing.*

If you subscribe to the notion that “data is the new currency,” you must go to great lengths to protect that data. Remember, data needed to test electronic devices include proprietary information on the inner workings of semiconductors, boards, systems, and test stations. Analytical software often runs in the cloud, but proprietary data originates on the manufacturing floor. That means data must move from its source to the cloud for analysis. Thus, you need to make sure the data and the networks that carry it are secure. You may need help from your IT department.

For more information

Engineering productivity enhancements

AI/ML in manufacturing data analysis

AI in visual inspection

Identify counterfeit parts

Leave a Reply Cancel reply