| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
A failed configuration method used to be reported as an invented test result. Such a result is counted in the run statistics, and its history id — md5(testCaseId + parameters) — is one no execution ever produces, so no retry could ever replace it: a green rerun still showed the error. The failure now becomes a run-level GlobalError instead, keeping the broken fixture and its evidence in the scope. The tests it prevented keep their own results: TestNG skips them with the real parameters of every data provider row, so their identity matches a green rerun. The one exception is a parameterised method whose values were never produced — its data provider failed, or a failed dependency or fixture kept the provider from running. TestNG ends such a method once with no parameters, and a result built from that end has an identity nothing can replace. It is reported as a run-level error too. Whether a method is parameterised is read from its own declaration, because ITestNGMethod#isDataDriven reads only the method-level annotation and misses a provider inherited from a class-level @test. A data provider that skips on purpose keeps its skipped result: TestNG wraps the SkipException, so the intent is found through the cause chain. After configuration failures are worded neutrally, since an alwaysRun after method also runs for a test a failed before fixture already skipped.
…s global errors A container that failed or aborted — typically a class whose @BeforeAll threw or failed an assumption — used to be reported as an invented test result. Such a result is counted in the run statistics, and its history id is one no execution ever produces, so no retry could ever replace it. The platform sends no events for the tests inside such a container, so nothing can be backfilled either; the failure now becomes a run-level GlobalError carrying the original exception. A skipped subtree still backfills a skipped result for every test the plan can name, with the identity it has in a real run. Test templates, test factories and parameterized classes are containers whose invocations only exist once they run: a skipped one has nothing to report, and a result for the container itself would carry a history id no invocation ever produces. Such containers are collected and reported once as a run-level error instead. With no non-test identifier reaching stopTest any more, the AS_ID=-1 marker that hid the invented results from the report goes too.
… errors A @BeforeClass, @afterclass or @ClassRule that threw, or a @BeforeClass assumption, used to vanish from the report entirely: JUnit 4 reports it on the description of the class itself, with no events for the tests it prevented, and the listener's update of a test it never started was a silent no-op. The run looked green with fewer tests than expected. Such a failure is now a run-level GlobalError naming the class, the count of tests inside it and the cause. Nothing is backfilled: JUnit 4 does not name the tests a class-level failure prevented, and the message claims nothing about whether they ran, since a before and an after failure arrive on the same description. An invalid test class, or a Parameterized class whose parameters method threw, is reported by JUnit 4 as a synthetic initializationError test. That used to become a broken test result whose identity matches none of the cases a green run produces, so no retry could replace it; it is now a run-level error too.
A failed setupSpec used to be reported as an invented test result, and every feature it prevented was backfilled as skipped — including data-driven features, backfilled once with no parameters. The fixture result is not a test case, and the parameterless feature result carries a history id that no iteration ever produces: both are counted in the run statistics under identities a green rerun cannot replace. The failure now becomes a run-level GlobalError, with the broken fixture kept in the spec scope. Plain features are still backfilled as skipped with the identity they have in a real run. Data-driven features are not: their iterations, and the values that tell them apart, only exist once the feature runs, so they are named in the run-level error instead. A failed cleanupSpec used to be visible only as a broken fixture inside the spec scope; it is now a run-level error too, its features keeping the results they already produced.
| Back | FazBrowse Home | New Git URL |
Context
Report failures outside individual tests as run-level errors across TestNG, JUnit Platform, JUnit 4, Spock 2, Cucumber 7, and JBehave 5. This covers configuration methods, data providers, failed containers, class and specification fixtures, run hooks, and story lifecycle steps.
These failures are no longer lost or represented as synthetic test cases that distort statistics and remain in history after successful retries. Real tests retain stable identities, blocked tests are reported as skipped when their identities are known, and unknown parameterized or template invocations are not invented.
fixes #643
fixes #912
fixes #1128
fixes #1155
Checklist