Skip to main content
A flaky test produces different results across runs without code changes. It passes on one execution and fails on the next, or passes only after a retry.

​
Quick Reference

​
How Detection Works

Flaky test detection activates automatically when retries are enabled in Playwright. No additional configuration required.
playwright.config.ts
TestDino detects flaky tests in two ways: Within a single run. A test that fails initially but passes on retry is marked flaky. The retry count appears in the test details. Test marked as flaky after passing on retry Across multiple runs. Tests with inconsistent outcomes on the same code are flagged. TestDino tracks pass/fail patterns and calculates a stability percentage. Stability percentage showing inconsistent test results across runs
NoteBoth detection methods indicate that the test result depends on something other than your code.

​
Flaky Test Categories

TestDino classifies flaky tests by root cause:

​
Common causes

  • Fixed waits instead of waiting for the page to be ready
  • Missing await causes steps to run out of order
  • Weak selectors that match more than one element
  • Tests share data and affect each other
  • Parallel runs collide on the same user or record
  • Slow or unstable network or third-party APIs
  • CI setup differs from local environment

​
Where to Find Flaky Tests

​
Dashboard

Open the Dashboard. The Most Flaky Tests panel lists tests with the highest flaky rates in the selected period. Each entry shows the test name, spec file, flaky rate percentage, and a link to the latest run. Click any test to open its most recent execution. Most Flaky Tests panel showing test names with flaky percentages

​
Analytics Summary

Open Analytics → Summary. The Flakiness & Test Issues chart shows the flaky rate trend over time and a list of flaky tests with spec file and execution date. A rising trend indicates increasing instability in your test suite. Flakiness trend chart with percentage over time and list of affected tests

​
Test Run Summary

Open any test run. The Summary tab shows flaky test counts grouped by category: Timing Related, Environment Dependent, Network Dependent, Assertion Intermittent, and Other Flaky. Test run summary showing flaky test counts by category Click a category to filter the detailed analysis table.

​
Test Case History

Open a specific test case and go to the History tab. The stability percentage shows how often the test passes: Stability = (Passed Runs / Total Runs) x 100 A test with 100% stability has never failed or been flaky. Any value below 100% indicates inconsistent behavior. The Last Flaky tile links to the most recent run where the test was marked flaky. Test case history showing stability percentage and last flaky run

​
Test Explorer

Open Test Explorer from the sidebar. The Flaky Rate column shows the percentage of executions with flaky results for each spec file or test case. Sort by flaky rate to find the most unstable tests across the project.

​
Flakiness trend

Add the Flakiness trend widget to an analytics dashboard to track flaky-run rate over time. Set the widget’s environment filter to compare stability across staging, production, and other environments.
NoteHigh flaky rates in specific environments suggest environment-dependent issues like resource constraints or service availability.

​
CI Check Behavior

GitHub CI Checks handle flaky tests in two modes: See GitHub CI Checks for configuration details.

​
Export Flaky Test Data

Use the TestDino MCP server to query flaky tests programmatically:
TipThe MCP server returns test names, flaky rates, and run IDs for further analysis.
See TestDino MCP for more details.

Test Explorer

Analyze flaky rates across spec files and test cases

GitHub CI Checks

Configure flaky handling in CI

TestDino MCP

Query flaky data with AI

Analytics

Project-wide test analytics