Skip to main content
The Anomalies tab shows what changed in a test run compared to the project’s recent runs. It surfaces tests that newly failed, turned flaky, slowed down, or recovered, so you catch regressions that a plain pass/fail view misses. Find it as the Anomalies tab on any test run’s details page. Anomalies tab header showing severity meter with named band, run summary, clickable section chips, the score breakdown, and the New failures and Slower than usual sections

​
What it compares against

Each run is compared against the project’s recent baseline: its last few runs, up to 10. Detection needs a baseline to work, so it starts once the project has 5 prior runs. Before that, the tab shows how many runs are still needed.

​
Anomaly types

The tab groups changes into 5 sections. New failures counts only tests that have run before. A brand-new test failing on its first appearance is a new test, not a regression, so it is left out. Each section shows the share of the run it accounts for, and expands to list its tests. Sections start expanded.

​
What each anomaly shows

Every section lists the affected tests. Click a test to open its details for this run.

​
Run header

The header opens with a one-line summary of the run, such as 5 tests changed or Nothing changed, followed by how many tests ran and how many were already failing before this run. A vs. N runs before pill shows how many prior runs it was compared against. When a run has more than one kind of change, the header also shows a clickable chip per section, such as 2 new failures or 1 slower than usual. Click a chip to jump to that section.

​
Severity score

The header shows a severity score from 0 to 100 for the run, on a meter with a named band: Clean, Minor, Elevated, High, or Critical. New failures, slowdowns, and shared error clusters raise the score; fixes lower it. A higher score means more, or worse, changes in this run. Below the meter, What made up this score breaks the number down into the signed points each kind of change contributed, so you can see what drove it, for example +30 from new failures and -5 from fixes.

​
Shared error clusters

When 3 or more tests fail with the same error, they form a cluster. Clusters raise the severity score, since a group of tests failing for one reason is a single root cause to fix rather than several separate failures. At the bottom of the tab, All failures in this run, by type breaks failures down by category, with badges like timeout: 8 or assertion: 3.

​
States you may see

Anomaly detection is per run. It reads run data, so a signed-in user with access to the project can view it, in the app or through a personal access token. CI pipeline keys cannot read it.

Run Summary

Review failed, flaky, and skipped tests by cause

Errors

Group failures by error message within a run

Flaky Tests

Detect and fix flaky tests

Analytics

Track reliability and performance trends over time