Playwright Harness CI Setup

Harness CI runs your pipeline from a YAML definition, stored in .harness/ in your repository or inline in the Harness UI. This guide covers a sharded setup that streams results to TestDino during the run.

Prerequisites

A TestDino account with a project created
A TestDino API key (Generate API Keys)
A Harness CI project with a Git connector to your repository
A Playwright test project (or clone the TestDino Example Repository to get started)
@testdino/playwright installed and configured in playwright.config.ts|js|mjs:

npm install @testdino/playwright

playwright.config.ts

import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['@testdino/playwright', {
      token: process.env.TESTDINO_TOKEN,
    }],
  ],
});

Set Up Your API Key

Store your TestDino API key as a Harness secret so the pipeline reads it by reference instead of holding the value in YAML.

In Harness, open Project Settings (or Organization Settings to share it across projects)
Go to Secrets
Click New Secret, then Text
Set the Secret Name to TESTDINO_TOKEN
Paste your TestDino API key into the value
Save the secret

Reference it in the pipeline with <+secrets.getValue("TESTDINO_TOKEN")>. Harness masks secret values in build logs.

Basic Pipeline Config

For a simple setup without sharding, run your tests in one Run step with TESTDINO_TOKEN set. Results stream to TestDino during the run, so no separate upload step is needed. Replace <ORG_ID>, <PROJECT_ID>, <GIT_CONNECTOR_ID>, and <YOUR_REPO> with your Harness values. This example targets Harness Cloud runners; self-hosted (Docker or Kubernetes) runners use the same step with a different platform and runtime.

.harness/playwright.yaml: Basic pipeline

.harness/playwright.yaml

pipeline:
  name: playwright-testdino
  identifier: playwright_testdino
  projectIdentifier: <PROJECT_ID>
  orgIdentifier: <ORG_ID>
  properties:
    ci:
      codebase:
        connectorRef: <GIT_CONNECTOR_ID>
        repoName: <YOUR_REPO>
        build: <+input>
  stages:
    - stage:
        name: test
        identifier: test
        type: CI
        spec:
          cloneCodebase: true
          platform:
            os: Linux
            arch: Amd64
          runtime:
            type: Cloud
            spec: {}
          execution:
            steps:
              - step:
                  type: Run
                  name: playwright tests
                  identifier: playwright_tests
                  spec:
                    image: mcr.microsoft.com/playwright:v1.61.1-jammy
                    shell: Sh
                    envVariables:
                      TESTDINO_TOKEN: <+secrets.getValue("TESTDINO_TOKEN")>
                    command: |
                      npm ci
                      npx playwright install
                      npx playwright test

TipTESTDINO_TOKEN is resolved from the Harness secret you added earlier. Results stream live even when tests fail, so failures land on the dashboard without a separate upload step.

Sharded Test Runs

For large test suites, run shards in parallel. Each shard streams its own results, and TestDino groups them into one run when they share a ci-run-id.

How it works

A Run step installs dependencies once, then a second Run step uses strategy.parallelism to fan out into shards
Each shard runs Playwright with --shard=<n>/<total> from its iteration index
Each shard streams its results to TestDino during the run
Every shard sets the same TESTDINO_CI_RUN_ID so TestDino merges them into a single run
No merge or upload step runs after the shards finish

Full sharded config

Replace <ORG_ID>, <PROJECT_ID>, <GIT_CONNECTOR_ID>, and <YOUR_REPO> with your Harness values. This example targets Harness Cloud runners; self-hosted (Docker or Kubernetes) runners use the same steps with a different platform and runtime.

.harness/playwright.yaml: Sharded pipeline

.harness/playwright.yaml

pipeline:
  name: playwright-testdino
  identifier: playwright_testdino
  projectIdentifier: <PROJECT_ID>
  orgIdentifier: <ORG_ID>
  properties:
    ci:
      codebase:
        connectorRef: <GIT_CONNECTOR_ID>
        repoName: <YOUR_REPO>
        build: <+input>
  stages:
    - stage:
        name: test
        identifier: test
        type: CI
        spec:
          cloneCodebase: true
          platform:
            os: Linux
            arch: Amd64
          runtime:
            type: Cloud
            spec: {}
          execution:
            steps:
              # Install dependencies once. Steps in a CI stage share the workspace,
              # so a single install serves every shard.
              - step:
                  type: Run
                  name: install deps
                  identifier: install_deps
                  spec:
                    image: mcr.microsoft.com/playwright:v1.61.1-jammy
                    shell: Sh
                    command: |
                      npm ci
              - step:
                  type: Run
                  name: playwright shard
                  identifier: playwright_shard
                  strategy:
                    parallelism: 4
                  spec:
                    image: mcr.microsoft.com/playwright:v1.61.1-jammy
                    shell: Sh
                    envVariables:
                      TESTDINO_TOKEN: <+secrets.getValue("TESTDINO_TOKEN")>
                      # Same run id across all shards groups them into one run in TestDino.
                      TESTDINO_CI_RUN_ID: <+pipeline.sequenceId>
                    command: |
                      # strategy.iteration is 0-based; Playwright shards are 1-based.
                      export SHARD=$((<+strategy.iteration> + 1))
                      export TOTAL=<+strategy.iterations>
                      npx playwright install
                      npx playwright test --shard=$SHARD/$TOTAL

Key details

Detail	Notes
`strategy.parallelism: 4`	Harness runs 4 copies of the step as parallel shards.
`<+strategy.iteration>`	0-based index of the current shard. Add 1 for Playwright’s 1-based `--shard`.
`<+strategy.iterations>`	Total shard count, passed as the denominator of `--shard`.
`<+pipeline.sequenceId>`	Build number for this pipeline. Used as `TESTDINO_CI_RUN_ID` to group shards.
`TESTDINO_CI_RUN_ID`	Read by `@testdino/playwright` to merge all shards into one run.
`TESTDINO_TOKEN`	Resolved from the Harness secret and masked in logs.

Set the ciRunId reporter option in playwright.config to process.env.TESTDINO_CI_RUN_ID if you prefer to pass it through the config instead of relying on the environment variable.

Pipeline execution

After the pipeline starts, Harness shows the parallel shards under the playwright shard step. Each shard streams its results, and TestDino merges them into one run as they complete.

Results in TestDino

The test run appears in your TestDino dashboard with full failure details, flaky detection, and trend data as tests complete.

Troubleshooting

Results not appearing on the dashboard

Confirm @testdino/playwright is in your playwright.config reporter array and TESTDINO_TOKEN resolves in the step’s envVariables. Results stream during the run, so no separate upload step is required.

Sharded runs show up as separate runs

Confirm every shard sets TESTDINO_CI_RUN_ID to <+pipeline.sequenceId>. A per-shard value creates one run per shard.

Shards overwrite the same node_modules

On runners that share one workspace across steps, run npm ci once in a separate step before the sharded step, as shown above. Concurrent installs in the sharded step race on the shared node_modules.

CI Optimization

Reduce CI time with smart reruns

Environment Mapping

Route test results to Dev, Staging, or Production by branch

Integrations

Connect Slack, Jira, Linear, Asana, and more

TestDino MCP

Access test results and fix issues with AI agents

Getting Started

Installation & Setup

AI & MCP

Guides

Project

Test Management

Organization

Integrations

Data Privacy

Resources

Prerequisites

Set Up Your API Key

Basic Pipeline Config

Sharded Test Runs

How it works

Full sharded config

Key details

Pipeline execution

Results in TestDino

Troubleshooting

CI Optimization

Environment Mapping

Integrations

TestDino MCP

​Prerequisites

​Set Up Your API Key

​Basic Pipeline Config

​Sharded Test Runs

​How it works

​Full sharded config

​Key details

​Pipeline execution

​Results in TestDino

​Troubleshooting

​Related

CI Optimization

Environment Mapping

Integrations

TestDino MCP

Prerequisites

Set Up Your API Key

Basic Pipeline Config

Sharded Test Runs

How it works

Full sharded config

Key details

Pipeline execution

Results in TestDino

Troubleshooting

Related