In a 128-entry memory-controller FIFO, a generated test can hit cp_fill: bins hwm_cross = {[120:127]} at 122 entries and still miss the design intent entirely. The intended scenario is upstream back-pressure when the downstream consumer is throttling. The test reaches 120, satisfies the bin, and never applies throttling.
A Covered Bin Is Not a Covered Scenario
A coverage bin defines an observation, not its purpose. The range [120:127] says that the fill level must enter a value interval. It does not say whether the FIFO arrived there during a clean transfer, an AXI stall, an error recovery sequence, or active downstream throttling.
An agent that reads the bin and generates constrained-random stimulus to hit it is complying with the formal input it received. The verification plan annotation, RTL comment, and engineer's mental model may describe a failure mode that is absent from the bin syntax. Coverage databases such as UCDB or UCIS record the hit, but they do not restore the missing intent.
That boundary matters. We are not saying generated tests are wrong when they satisfy the bin. We are saying that metric satisfaction and scenario satisfaction are different verification claims.
Where Intent Commonly Disappears
Interaction coverage is one example. A cross requiring {req == 1, ack == 0, data_valid == 1} can be hit in one clock cycle by forcing the three observable states together. The intended test may require req to assert, arbitration to consume several cycles, and data_valid to rise during the wait. Both executions close the cross. Only one exercises the handshake state machine.
Threshold bins have the same weakness. A watermark can be reached from an idle FIFO without protocol pressure, even when the coverage group was written to check producer stalling under a throttled consumer. The value is covered; the back-pressure behavior is not.
Boundary parameters also need context. Reaching a limit after reset, a retry, or error recovery can traverse different RTL paths from reaching it during clean initialization. A test that enters the range from a quiet starting state may leave the relevant corner case untouched.
What We Changed in ChipAgents
During an early-access run with a fabless ASIC team, a 32-bit compute block reached 96 percent functional coverage with tests generated through ChipAgents. The team marked the block complete after a two-week sprint. Integration testing two weeks later exposed a back-pressure stall. We had generated a test for that failure mode, but the test only closed the watermark bin. It did not keep downstream throttling active while the watermark was crossed.
Reviewing the coverage group made the cause plain. Nothing in the bin definition mentioned throttling. The engineer who authored the group knew the intended scenario, but that knowledge had not been represented in the formal coverage expression.
That experience led us to make review a workflow checkpoint rather than treating generated SystemVerilog tests as ready-to-run output. ChipAgents can generate tests, track coverage, and explain failures across Cadence Xcelium, Synopsys VCS, and Mentor QuestaSim. The engineer still confirms whether a closed bin represents the behavior the block requires.
The Checkpoint in Practice
This is not a line-by-line audit of every generated test. The reviewer examines tests that close previously uncovered groups and asks whether the path used to reach each bin matches the group's intended scenario.
First, did the test reach the bin through a valid protocol sequence, or through a shortcut that merely produced the required signal combination? This matters for handshake crosses, arbitration, and state-machine transitions.
Next, are the values representative of the failure mode? A boundary bin should be exercised at the boundary with the relevant preceding conditions, not reached through an easy interior path that happens to fall inside the declared range.
Finally, for protocol-oriented groups, did the test follow the intended flow, or did it reproduce the same observable states through a non-protocol path? A coverage report cannot answer that distinction by itself.
In our workflow, an engineer familiar with the block can inspect 20 to 30 generated tests in 15 to 20 minutes. The review is limited to tests that closed gaps. Tests duplicating bins already exercised by hand-written tests generally need no additional inspection.
Why the Review Pays for Itself
Suppose 40 generated tests address 30 coverage gaps. A 20-minute review is a small cost compared with writing those tests manually, especially when the review is restricted to the gap-closing set.
It also moves discovery earlier. If a test closes a metric but misses intent, the team can correct its stimulus, coverage annotation, or scenario description while the generation context is still available. Finding the same omission during tape-out sign-off, after the block was declared complete from a coverage number, is more expensive.
There is a useful counterpoint: adding review to a fast generation loop can become ceremony if every test receives identical scrutiny. That is why the checkpoint should be risk-based. Focus on new bins, threshold crossings, protocol crosses, reset and recovery paths, and tests whose stimulus looks unusually simple for the behavior claimed.
We are not arguing that human engineers should replace automated generation. We are arguing for a narrow human decision at the point where a formal metric is being interpreted as evidence of design behavior. A test can be syntactically valid, reproducible in Xcelium, VCS, or QuestaSim, and present in the UCIS database while still missing the scenario that motivated the coverage group. The practical implication is to review the path that closed a new bin, not merely the percentage shown in the report.