> ## Documentation Index
> Fetch the complete documentation index at: https://docs.testdino.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Playwright Buildkite Setup

> Stream Playwright results from Buildkite to TestDino live, with sharded parallel jobs grouped into one run.

Buildkite auto-detects a `.buildkite/pipeline.yml` in your repository. This guide covers a basic run and a parallel sharded setup that streams results to TestDino during the run.

## Prerequisites

* A <a href="https://app.testdino.com" target="_blank" rel="noopener noreferrer">TestDino account <Icon icon="arrow-up-right-from-square" size={12} /></a> with a project created
* A TestDino API key ([Generate API Keys](/guides/generate-api-keys))
* A Buildkite pipeline connected to your repository, with an agent in a cluster
* A Playwright test project (or clone the <a href="https://github.com/testdino-hq/TestDino-Example" target="_blank" rel="noopener noreferrer">TestDino Example Repository <Icon icon="arrow-up-right-from-square" size={12} /></a> to get started)
* `@testdino/playwright` installed and configured in `playwright.config.ts|js|mjs`:

```bash theme={null}
npm install @testdino/playwright
```

```typescript playwright.config.ts theme={null}
import { defineConfig } from '@playwright/test';

export default defineConfig({
  reporter: [
    ['@testdino/playwright', {
      token: process.env.TESTDINO_TOKEN,
    }],
  ],
});
```

## Set Up Your API Key

Store your TestDino API key as a Buildkite cluster secret so it reaches the job without being written into `.buildkite/pipeline.yml`.

1. In Buildkite, go to **Agents**
2. Open **Clusters** and select the cluster your pipeline runs on
3. Open **Secrets**, then click **New Secret**
4. Set the **Key** to `TESTDINO_TOKEN`
5. Paste your TestDino API key into the value
6. Save the secret

The `secrets` plugin reads the secret at runtime and exposes it as an environment variable. Buildkite redacts secret values from build logs automatically.

## Basic Pipeline Config

For a simple setup without sharding, run your tests with `TESTDINO_TOKEN` injected. Results stream to TestDino during the run, so no separate upload step is needed.

<Accordion title=".buildkite/pipeline.yml: Basic pipeline">
  ```yaml .buildkite/pipeline.yml theme={null}
  steps:
    - label: ":playwright: Run tests with TestDino"
      plugins:
        - secrets#v2.4.0:
            variables:
              TESTDINO_TOKEN: "TESTDINO_TOKEN"
        - docker#v5.11.0:
            image: "mcr.microsoft.com/playwright:v1.61.1-jammy"
            propagate-environment: true
            environment:
              - "TESTDINO_TOKEN"
      command:
        - npm ci
        - npx playwright install --with-deps
        - npx playwright test
  ```
</Accordion>

<Tip>
  **Tip**

  The `secrets` plugin injects `TESTDINO_TOKEN` and the `docker` plugin forwards it into the container with `propagate-environment`. Results stream live even when tests fail, so failures land on the dashboard without a separate upload step.
</Tip>

## Sharded Test Runs

For large test suites, run shards in parallel. Each shard streams its own results, and TestDino groups them into one run when they share a `ci-run-id`.

### How it works

1. `parallelism: 4` fans the step out into 4 Buildkite jobs
2. Each shard runs Playwright with `--shard=<n>/<total>` from its parallel-job index
3. Each shard streams its results to TestDino during the run
4. Every shard sets the same `TESTDINO_CI_RUN_ID` so TestDino merges them into a single run
5. No merge or upload step runs after the shards finish

### Full sharded config

<Accordion title=".buildkite/pipeline.yml: Sharded pipeline">
  ```yaml .buildkite/pipeline.yml theme={null}
  steps:
    - label: ":playwright: shard %n"
      parallelism: 4
      plugins:
        - secrets#v2.4.0:
            variables:
              TESTDINO_TOKEN: "TESTDINO_TOKEN"
        - docker#v5.11.0:
            image: "mcr.microsoft.com/playwright:v1.61.1-jammy"
            propagate-environment: true
            environment:
              - "TESTDINO_TOKEN"
      env:
        # Same run id across all shards groups them into one run in TestDino.
        # $$ defers interpolation to job runtime (Buildkite interpolates the YAML at upload time).
        TESTDINO_CI_RUN_ID: "$$BUILDKITE_BUILD_NUMBER"
      command:
        # BUILDKITE_PARALLEL_JOB is 0-based; Playwright shards are 1-based.
        - export SHARD=$$(($$BUILDKITE_PARALLEL_JOB + 1))
        - export TOTAL=$$BUILDKITE_PARALLEL_JOB_COUNT
        - npm ci
        - npx playwright install --with-deps
        - npx playwright test --shard=$$SHARD/$$TOTAL
  ```
</Accordion>

### Key details

| Detail                         | Notes                                                                                                                           |
| :----------------------------- | :------------------------------------------------------------------------------------------------------------------------------ |
| `parallelism: 4`               | Buildkite runs 4 copies of the step as separate parallel jobs.                                                                  |
| `BUILDKITE_PARALLEL_JOB`       | 0-based index of the current job. Add 1 for Playwright's 1-based `--shard`.                                                     |
| `BUILDKITE_PARALLEL_JOB_COUNT` | Total shard count, passed as the denominator of `--shard`.                                                                      |
| `TESTDINO_CI_RUN_ID`           | Read by `@testdino/playwright` to group all shards into one run.                                                                |
| `$$` prefix                    | Buildkite interpolates the YAML at upload time. `$$` defers runtime variables so they resolve when the job runs, not at upload. |
| `TESTDINO_TOKEN`               | Injected by the `secrets` plugin and redacted from logs.                                                                        |

Set the `ciRunId` reporter option in `playwright.config` to `process.env.TESTDINO_CI_RUN_ID` if you prefer to pass it through the config instead of relying on the environment variable.

### Pipeline execution

After the build starts, Buildkite shows each shard as a separate job under the step. Each shard streams its results, and TestDino merges them into one run as they complete.

### Results in TestDino

The test run appears in your TestDino dashboard with full failure details, flaky detection, and trend data as tests complete.

## Troubleshooting

<AccordionGroup>
  <Accordion title="Results not appearing on the dashboard">
    Confirm `@testdino/playwright` is in your `playwright.config` `reporter` array and `TESTDINO_TOKEN` is injected. The `docker` plugin must list `TESTDINO_TOKEN` under `environment` so the value reaches the container. Results stream during the run, so no separate upload step is required.
  </Accordion>

  <Accordion title="Sharded runs show up as separate runs">
    Confirm every shard sets the same `TESTDINO_CI_RUN_ID`. If the id resolves to an empty value, check the `$$` prefix: without it Buildkite interpolates `BUILDKITE_BUILD_NUMBER` at upload time, before the value exists.
  </Accordion>

  <Accordion title="TESTDINO_TOKEN not available in the job">
    Open **Agents → Clusters → your cluster → Secrets** and confirm a secret with the exact key `TESTDINO_TOKEN` exists. The `secrets` plugin `variables` map is `ENV_VAR: "SecretKey"`, and secret keys are case-sensitive.
  </Accordion>
</AccordionGroup>

## Related

<CardGroup cols={2}>
  <Card title="CI Optimization" icon="gauge-high" href="/guides/playwright-ci-optimization">
    Reduce CI time with smart reruns
  </Card>

  <Card title="Environment Mapping" icon="code-branch" href="/guides/environment-mapping">
    Route test results to Dev, Staging, or Production by branch
  </Card>

  <Card title="Integrations" icon="puzzle-piece" href="/integrations/overview">
    Connect Slack, Jira, Linear, Asana, and more
  </Card>

  <Card title="TestDino MCP" icon="plug" href="/mcp/overview">
    Access test results and fix issues with AI agents
  </Card>
</CardGroup>