Revolutionizing AMS Verification with Metamorphic Testing

Metamorphic Testing in AMS: A Game-Changer for Verification

In the ever-evolving world of Analog and Mixed-Signal (AMS) systems, where traditional testing methodologies often struggle to find elusive bugs, a novel testing technique is gaining traction: Metamorphic Testing (MT). This innovative approach, which has made waves in the software industry, is now proving its worth in the hardware verification world, specifically in verifying Phase-Locked Loops (PLLs).

What is Metamorphic Testing and Why Does It Matter?

Metamorphic testing is a powerful testing strategy used when creating a meaningful oracle (or reference model) is difficult or even impossible. Unlike traditional testing methods, where we compare outputs with expected results, Metamorphic Testing focuses on testing a system by running different simulations and ensuring certain relationships or invariants hold true across these runs.

For example, if you have a design that implements a mathematical function like sin(x), a metamorphic relation (MR) could state that sin(x) = sin(180 - x) for all x. This approach helps uncover hidden issues without requiring a direct comparison to an oracle or golden reference, which can be especially challenging in AMS systems where defining oracles inside circuits can be a complex task.

Metamorphic Testing in Action: The Case of PLLs

One remarkable application of Metamorphic Testing in the hardware world is its use in verifying Phase-Locked Loops (PLLs) in AMS systems. PLLs, which are essential in many communication and control systems, combine both analog and digital components, making them particularly tricky to test effectively.

In a groundbreaking paper presented at the 2021 DATE conference by researchers from the University of Bremen and Johannes Kepler University, they applied Metamorphic Testing to verify a production PLL. The researchers identified several metamorphic relations (MRs) for PLLs, such as:

• The PLL remains locked if the input frequency stays within the lock range, and the lock detector signal stays "on."

• The feedback loop frequency inside the PLL should scale proportionally with the input clock frequency.

By applying these MRs to a commercial PLL simulation coded in SystemC and running it through COSIDE simulations, they uncovered a rare but significant bug. The bug occurred when a small change in the input clock frequency from 1 MHz to 1.01 MHz caused the PLL to lock to the wrong frequency due to a dead-zone effect in the Phase Frequency Detector (PFD).

This discovery, though subtle, is a perfect example of how Metamorphic Testing can expose issues that traditional methods might miss, especially in complex AMS systems where debugging can be notoriously difficult.

The Power of Metamorphic Testing: Innovation in AMS Verification

What makes Metamorphic Testing truly innovative is its ability to work without the need for a pre-existing reference model. By focusing on relationships between different test runs, it uncovers corner-case bugs that might otherwise be overlooked.

In the case of the PLL verification, the researchers were able to define invariants that hold across simulations. By running the PLL through multiple simulations with slightly varying conditions, they were able to detect discrepancies, proving that Metamorphic Testing is an invaluable tool for ensuring the reliability and accuracy of AMS systems.

The bug discovered in the PLL serves as a reminder of the complexities involved in AMS verification and the need for more innovative verification methodologies. Metamorphic Testing has the potential to not only improve the efficiency of the verification process but also to increase the reliability of designs, especially when testing is particularly challenging due to the inherent complexities of analog systems.

The Takeaway: Innovating with Metamorphic Testing

In summary, Metamorphic Testing is rapidly becoming a game-changer in the world of Analog/Mixed-Signal verification. Its application to PLLs is just the beginning. As the industry continues to push the boundaries of what's possible in AI-driven design and complex hardware systems, Metamorphic Testing will undoubtedly play a crucial role in uncovering hard-to-find bugs and ensuring the robustness of next-generation hardware.

By adopting this innovative approach, engineers can significantly improve their Design Verification (DV) suites and enhance the reliability of their systems. Whether you're working with analog, digital, or mixed-signal systems, Metamorphic Testing can help ensure that your designs stand up to the toughest challenges.

Source - Semiwiki