CI Optimization

CI Optimization

TestDino offers a way to reduce CI time and cost by re-running only the tests that failed in the previous run.

This guide provides a method for optimizing Playwright test execution in your GitHub Actions CI pipeline.

Tip: Since @playwright/test@1.50+ has improved support for --last-failed on CI, we recommend upgrading your package version accordingly.

How It Works

  1. Run the full suite. npx playwright test executes all tests.

  2. Cache results. npx tdpw cache reads Playwright’s report and stores pass/fail metadata in TestDino (branch/commit aware, shard aware).

  3. Rerun failures. npx tdpw last-failed returns the exact tests that failed last time. Pass that list to Playwright to run only those tests.

Benefits over a local --last-failed cache: survives new runners, works across shards and jobs, and can be triggered from a separate workflow.

Requirements

  • Playwright installed and working in CI.

  • TestDino CLI (tdpw) available via npx.

  • TestDino API token stored as a GitHub secret (e.g., TESTDINO_TOKEN).

  • Use the TestDino GitHub App for automated uploads/comments (optional but recommended).

Sample GitHub Actions Workflow

1. Full run & cache (playwright-tests.yml)

Runs on push; always caches results immediately after tests.

This workflow is the primary testing process. Its job is to run the entire test suite and record the results to establish a baseline.

.github/workflows/test.yml
name: Playwright Tests  
on:  
  push:  
  workflow_dispatch:
 
jobs:  
  test:  
    runs-on: ubuntu-latest  
    container: mcr.microsoft.com/playwright:latest  
    env:  
      TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}  
    steps:  
      - uses: actions/checkout@v4  
      - uses: actions/setup-node@v4  
        with: { node-version: '22.x' }
 
      - name: Install deps  
        run: |  
          npm ci  
          npx playwright install --with-deps
 
      - name: Run all tests  
        run: npx playwright test
 
      - name: Cache test metadata (always)  
        if: always()  
        run: npx tdpw cache --token="${{ secrets.TESTDINO_TOKEN }}"

place tdpw cache immediately after npx playwright test, guarded by if: always().

2. Rerun failed only (rerun-failed-tests.yml)

Triggered manually; fetches failures, then runs only those tests.

This workflow is the targeted retry process. Its only function is to execute the tests that failed during the main run.

.github/workflows/test.yml
name: failed-only-reruns  
on:  
  workflow_dispatch:
 
jobs:  
  rerun:  
    runs-on: ubuntu-latest  
    container: mcr.microsoft.com/playwright:latest  
    env:  
      TESTDINO_TOKEN: ${{ secrets.TESTDINO_TOKEN }}  
    steps:  
      - uses: actions/checkout@v4  
      - uses: actions/setup-node@v4  
        with: { node-version: '22.x' }
 
      - name: Install deps  
        run: |  
          npm ci  
          npx playwright install --with-deps
 
      - name: Get last failed tests with tdpw  
        id: last_failed  
        run: |  
          npx tdpw last-failed --token="${{ secrets.TESTDINO_TOKEN }}" > last_failed.txt  
          echo "args=$(cat last_failed.txt)" >> $GITHUB_OUTPUT
 
      - name: Run only failed tests  
        run: npx playwright test ${{ steps.last_failed.outputs.args }}
 
      - name: Cache rerun metadata (always)  
        if: always()  
        run: npx tdpw cache --token="${{ secrets.TESTDINO_TOKEN }}"

Call tdpw last-failed before Playwright, then pass its output to npx playwright test.

Example

Imagine a pull request triggers a full CI run with 50 tests, which takes 10 minutes to complete. 47 tests pass, but 3 fail in the login.spec.ts, cart.spec.ts, and checkout.spec.ts files due to a temporary network issue.

Instead of re-running all 50 tests, a developer can trigger a targeted rerun.

This second run will execute only those 3 failed tests, completing in under a minute and quickly confirming that the issue is resolved.

General rules (any workflow)

  • After tests: npx tdpw cache (saves what happened).
  • Before rerun: npx tdpw last-failed (gets what failed).
  • Tokens: Provide --token or set TESTDINO_TOKEN in env/secrets.
  • Branch/commit overrides (optional):
    npx tdpw last-failed --branch="main" --commit="$GITHUB_SHA" --token="$TESTDINO_TOKEN"
  • Artifacts (optional): continue uploading Playwright reports for debugging.

The Two-Part Process

The workflow operates in two distinct stages to ensure efficiency.

  1. Run & Save: The playwright-tests.yml workflow runs first on every code push. It executes all tests to establish a complete quality baseline and uses tdpw cache to save any failures to TestDino.

  2. Fetch & Rerun: If the main run has failures, a user can manually trigger the rerun-failed-tests.yml workflow. It asks TestDino what failed previously and then runs only those tests for a quick re-check.

This setup provides a fast, reliable “run everything → cache → rerun failuresloop for GitHub Actions.