CI Optimization
TestDino offers a way to reduce CI time and cost by re-running only the tests that failed in the previous run.
This guide provides a method for optimizing Playwright test execution in your GitHub Actions CI pipeline.
Tip: Since @playwright/test@1.50+
has improved support for --last-failed
on CI, we recommend upgrading your package version accordingly.
How It Works
-
Run the full suite.
npx playwright test
executes all tests. -
Cache results.
npx tdpw cache
reads Playwright’s report and stores pass/fail metadata in TestDino (branch/commit aware, shard aware). -
Rerun failures.
npx tdpw last-failed
returns the exact tests that failed last time. Pass that list to Playwright to run only those tests.
Benefits over a local --last-failed
cache: survives new runners, works across shards and jobs, and can be triggered from a separate workflow.
Requirements
-
Playwright installed and working in CI.
-
TestDino CLI (
tdpw
) available vianpx
. -
TestDino API token stored as a GitHub secret (e.g.,
TESTDINO_TOKEN
). -
Use the TestDino GitHub App for automated uploads/comments (optional but recommended).
Sample GitHub Actions Workflow
1. Full run & cache (playwright-tests.yml
)
Runs on push; always caches results immediately after tests.
This workflow is the primary testing process. Its job is to run the entire test suite and record the results to establish a baseline.
name: Playwright Tests
on:
push:
workflow_dispatch:
jobs:
test:
runs-on: ubuntu-latest
container: mcr.microsoft.com/playwright:latest
env:
TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: '22.x' }
- name: Install deps
run: |
npm ci
npx playwright install --with-deps
- name: Run all tests
run: npx playwright test
- name: Cache test metadata (always)
if: always()
run: npx tdpw cache --token="${{ secrets.TESTDINO_TOKEN }}"
place tdpw cache
immediately after npx playwright test
, guarded by if: always()
.
2. Rerun failed only (rerun-failed-tests.yml
)
Triggered manually; fetches failures, then runs only those tests.
This workflow is the targeted retry process. Its only function is to execute the tests that failed during the main run.
name: failed-only-reruns
on:
workflow_dispatch:
jobs:
rerun:
runs-on: ubuntu-latest
container: mcr.microsoft.com/playwright:latest
env:
TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: '22.x' }
- name: Install deps
run: |
npm ci
npx playwright install --with-deps
- name: Get last failed tests with tdpw
id: last_failed
run: |
npx tdpw last-failed --token="${{ secrets.TESTDINO_TOKEN }}" > last_failed.txt
echo "args=$(cat last_failed.txt)" >> $GITHUB_OUTPUT
- name: Run only failed tests
run: npx playwright test ${{ steps.last_failed.outputs.args }}
- name: Cache rerun metadata (always)
if: always()
run: npx tdpw cache --token="${{ secrets.TESTDINO_TOKEN }}"
Call tdpw last-failed
before Playwright, then pass its output to npx playwright test
.
Example
Imagine a pull request triggers a full CI run with 50 tests, which takes 10 minutes to complete. 47 tests pass, but 3 fail in the login.spec.ts, cart.spec.ts, and checkout.spec.ts files due to a temporary network issue.
Instead of re-running all 50 tests, a developer can trigger a targeted rerun.
This second run will execute only those 3 failed tests, completing in under a minute and quickly confirming that the issue is resolved.
General rules (any workflow)
- After tests:
npx tdpw cache
(saves what happened). - Before rerun:
npx tdpw last-failed
(gets what failed). - Tokens: Provide
--token
or setTESTDINO_TOKEN
in env/secrets. - Branch/commit overrides (optional):
npx tdpw last-failed --branch="main" --commit="$GITHUB_SHA" --token="$TESTDINO_TOKEN"
- Artifacts (optional): continue uploading Playwright reports for debugging.
The Two-Part Process
The workflow operates in two distinct stages to ensure efficiency.
-
Run & Save: The
playwright-tests.yml
workflow runs first on every code push. It executes all tests to establish a complete quality baseline and usestdpw cache
to save any failures to TestDino. -
Fetch & Rerun: If the main run has failures, a user can manually trigger the
rerun-failed-tests.yml
workflow. It asks TestDino what failed previously and then runs only those tests for a quick re-check.
This setup provides a fast, reliable “run everything → cache → rerun failures” loop for GitHub Actions.