Seven Rules for Pass/Fail Criteria That Hold Up Under Review
"Adequate performance" is not a pass/fail criterion. It's a wish.
I still see phrases like that in test packages that make it to TRR. Reviewers still sometimes let them through — usually because the review meeting is packed and the criterion table is at the back of the document. Then post-test review turns into an argument about what "adequate" meant, and the data becomes indefensible.
Here are the seven rules I apply to every pass/fail criterion before it goes into a TRR package. If a criterion doesn't meet all seven, it gets rewritten or dropped.
1. Numeric Threshold with a Unit
"3000 psi" is a criterion. "Sufficient pressure" is not. If you cannot express the threshold as a number with a unit, it isn't testable — it's a design goal, and it belongs in the requirements document, not the test package.
Rewrite the requirement to give it a number, or accept that this parameter isn't going to be verified by test. Either is fine. Ambiguity in the criterion is not.
2. Explicit Tolerance
"3000 ± 100 psi" — not "approximately 3000."
Tolerance is where post-test arguments live. If you hit 2950 psi against a spec of "3000," you have three possible outcomes: pass, fail, or a debate. The debate is what happens when tolerance isn't written down, and reviewers won't let you slide.
Tolerance can come from the requirement, from a design margin analysis, or from the instrumentation accuracy budget. Cite the source.
3. Defined Sample Window
"Peak within 200 ms of command" — not "shortly after command."
The window tells the DAQ engineer where to look and what to compute. Undefined windows mean whoever reduces the data chooses, and their choice may not match your intent. If the criterion is "peak pressure," is that the peak in the first cycle, or the max across the whole run? Different windows, different answers.
Write it down. Even if the answer is "for the duration of the run," write that.
4. Named Reduction Method
Peak? Mean? RMS? Filtered — and if filtered, what cutoff frequency and filter type?
Two engineers will reduce the same trace differently if you don't specify. Then the pass/fail result depends on who ran the reduction, not on the article. That's the failure mode you're trying to avoid.
A concrete example: strain gauge output on a load frame. Peak? Peak-to-peak? Range? Standard deviation? All defensible; only one is what you meant. Pick it.
5. Specified Measurement Channel
If you can't name the sensor part number, its calibration status, and the DAQ channel it feeds, the criterion is not testable — you're describing an intent, not a test.
Every criterion needs to trace to a physical measurement. Every physical measurement needs to trace to a calibrated instrument. If either link is missing, the criterion goes back to whoever wrote it until it's complete.
6. Linked to a Requirement ID
No orphan criteria. Every pass/fail line ties to a requirement in the RVM.
If a criterion doesn't link to a requirement, one of two things is true: you forgot the requirement (add it), or the criterion doesn't need to exist (delete it). Either way, resolve it before TRR.
Orphan criteria are how test scope creeps. Every added criterion means more instrumentation, more DAQ channels, more reduction work, more review time. If it's not tied to a requirement, ask why it's there.
7. Compound Criteria Split
"Pressure AND flow within limits" is two criteria, not one. Split them. Evaluate them independently.
Otherwise you get a fail where you don't know which condition triggered it, and the failure investigation stalls until someone re-reduces the data. Or worse, both fail and you report it as one line, when it's actually two independent issues that need separate root cause work.
Why This Matters
Every one of these gets argued about at post-test review if it wasn't nailed down at TRR.
"We hit 2950 psi — close enough, right?" — No, without tolerance, you don't have a criterion, you have an opinion.
"The peak was at 250 ms, not 200 — is that pass?" — Depends on what you wrote. If you didn't write a window, you're negotiating in the review, not defending it.
"Which reduction did you use?" — If the answer is "we chose one that made it pass," reviewers stop trusting the entire test report.
The discipline is upstream. Get the criterion right at TRR and post-test review becomes a formality. Get it wrong, and every arguable point becomes a fight — often with reviewers who are looking for reasons to send the package back.
Better to spend an extra hour at TRR writing tight criteria than a week defending loose ones later.
The Ground TRR Checklist I built includes a section on pass/fail criterion review. Free — 101 items across ten sections.
Get the Ground TRR Checklist → https://solriseengineering.gumroad.com/l/ground-trr-checklist