Create a release
- Open Releases in the project sidebar and click New release.
- Enter a Release name and select a Release branch. The branch can’t change later.
- Choose which runs to include.
- Click Create release.
Collect new runs automatically adds new runs as they finish. It’s on by default, except for Pick runs.
Version, Target date, Deploying to, Release goal, and Run tags are optional. Run tags limit the release to runs carrying any one of the tags.
Read the verdict
Each test counts its newest result across the release’s runs. A test that fails and then passes on the same commit counts as flaky.
The score never reads higher than the verdict allows: at most 49 when Blocked, and at most 89 when Needs attention.
Fix before release lists failing tests, critical ones first, with the reason, failed step, and when each started failing. The Spec files, All tests, and Runs tabs show the detail behind the verdict.
Choose critical tests
A failing critical test blocks the release. A test is critical when any of these apply:
Open Critical tests on the report to add or remove tests. Changes apply to every release in the project. A test from Test Controls can only be changed there. Adding back a test you removed returns it with its original source, such as Folder name.
To turn off AI picks, go to Project Settings > AI Settings and switch off Release Critical Tests. They stop counting within about 15 minutes.
Remove a run from the report
If a run was cut short by mistake, open the Runs tab and use Remove from this release. The report moves to the run before it. Use Count this run again to bring it back.Close a release
Click Close release, add a note if you want one, and confirm. The report stops changing and new runs no longer join. An owner or admin can Reopen it.Share the result
- Release brief: an AI summary of the report and what to fix next, below Fix before release. Available on Pro and above. Test Run Insights in AI Settings turns it on or off. When the report changes, the earlier brief stays, marked
Updating for the latest report…, until the new one is written. - Export PDF: a copy of the report to share.
Who can use it
Troubleshooting
The verdict says Not enough data
The verdict says Not enough data
The release needs at least one run where tests executed. You see this verdict when no run has results yet, when the newest run didn’t finish, or when every test in the newest run was skipped. Add a run that finishes, or remove the run that was cut short.
Only N of M tests ran
Only N of M tests ran
The release’s runs used filters such as
--grep or --project, or a shard is missing, so part of the suite never ran. The verdict reads Not enough data until you add a full run to the release. If a critical test is already failing, it reads Blocked and the warning stays. This check needs @testdino/playwright 2.11.2 or later.We can't confirm the whole suite ran
We can't confirm the whole suite ran
The release’s runs used filters or are missing a shard, and the branch has no full run to compare them with. Run the full suite on the branch to clear this.
A run I expected isn't in the release
A run I expected isn't in the release
Only finished runs join. Check that the run is on the release’s branch, carries one of its run tags, and that Collect new runs automatically is on.
Related
Test Controls
Mark tests critical for your builds
Test Runs
See the runs behind a release