TestDino CircleCI Orb Setup

Set up CircleCI to upload Playwright test results to TestDino using the TestDino Orb , a prebuilt package that wraps the TestDino CLI so you don’t need to install it separately.

RecommendationFor more configuration options and the latest features, use the TestDino CLI approach instead. The Orb wraps the CLI but may not include all new updates immediately.

Pick an upload mode

The orb supports 4 upload modes. Pick one based on what you want in the dashboard.

Mode	Uploads	Use it when
Basic Upload	The JSON results file	You only need pass/fail data and trends
Upload with HTML Report	JSON plus the Playwright HTML report	You want the HTML report attached to the run
Upload Full Bundle	JSON, images, and videos	You need screenshots and videos for failure debugging
Upload with Custom Paths	Whatever paths you name	Your reports are not in the default locations

Prerequisites

A TestDino account with a project created
A TestDino API key (Generate API Keys)
TestDino Example Repository for sample tests and ready-to-use CI configs
A CircleCI account with access to your repository
Playwright configured with JSON and HTML reporters in playwright.config.js:

playwright.config.js

// ...existing config

reporter: [
  ['html', { outputDir: './playwright-report' }],  // Optional
  ['json', { outputFile: './playwright-report/report.json' }],  // ✅ Required
]

Set Up Your API Key

Open your project in CircleCI
Go to Project Settings → Environment Variables
Click Add Environment Variable
Set the name to TESTDINO_TOKEN
Paste your TestDino API key as the value
Save the variable

WarningNever commit your API key directly in config files. Always use environment variables.

Basic Upload

.circleci/config.yml: Basic upload

.circleci/config.yml

version: '2.1'
orbs:
  testdino: testdino/testdino@1.0.0
jobs:
  run-tests:
    docker:
      - image: mcr.microsoft.com/playwright:v1.40.0-focal
    steps:
      - checkout
      - run:
          command: npm ci
          name: Install dependencies
      - run:
          command: npx playwright test
          name: Run Playwright tests
      - testdino/upload-report:
          report_directory: ./playwright-report
          token: TESTDINO_TOKEN
workflows:
  test-and-upload:
    jobs:
      - run-tests

Upload with HTML Report

.circleci/config.yml: Upload with HTML report

.circleci/config.yml

version: '2.1'
orbs:
  testdino: testdino/testdino@1.0.0
jobs:
  upload-with-html:
    docker:
      - image: mcr.microsoft.com/playwright:v1.40.0-focal
    steps:
      - checkout
      - run:
          command: npm ci
          name: Install dependencies
      - run:
          command: npx playwright test
          name: Run Playwright tests
      - testdino/upload-report:
          report_directory: ./playwright-report
          token: TESTDINO_TOKEN
          upload_html: true
          verbose: true
      - store_artifacts:
          destination: playwright-report
          path: ./playwright-report
workflows:
  test-with-testdino:
    jobs:
      - upload-with-html:
          context: testdino-credentials

Upload Full Bundle

Include all artifacts (JSON, images, videos) in one upload.

.circleci/config.yml: Upload full bundle

.circleci/config.yml

version: '2.1'
orbs:
  testdino: testdino/testdino@1.0.0
jobs:
  upload-full-bundle:
    docker:
      - image: mcr.microsoft.com/playwright:v1.40.0-focal
    steps:
      - checkout
      - run:
          command: npm ci
          name: Install dependencies
      - run:
          command: npx playwright test
          name: Run Playwright tests
      - testdino/upload-report:
          report_directory: ./playwright-report
          token: TESTDINO_TOKEN
          upload_full_json: true
          verbose: true
workflows:
  test-with-testdino:
    jobs:
      - upload-full-bundle:
          context: testdino-credentials

Upload with Custom Paths

Specify custom report paths and include images and videos.

.circleci/config.yml: Upload with custom paths

.circleci/config.yml

version: '2.1'
orbs:
  testdino: testdino/testdino@1.0.0
jobs:
  upload-with-custom-paths:
    docker:
      - image: mcr.microsoft.com/playwright:v1.40.0-focal
    steps:
      - checkout
      - run:
          command: npm ci
          name: Install dependencies
      - run:
          command: npx playwright test
          name: Run Playwright tests
      - testdino/upload-report:
          report_directory: ./test-results
          json_report: ./test-results/results.json
          html_report: ./test-results/index.html
          token: TESTDINO_TOKEN
          upload_images: true
          upload_videos: true
workflows:
  test-with-testdino:
    jobs:
      - upload-with-custom-paths:
          context: testdino-credentials

Orb Parameters

Parameter	Required	Description
`report_directory`	Yes	Path to the Playwright report directory
`token`	Yes	CircleCI environment variable name holding the TestDino API key
`upload_html`	No	Upload HTML report (`true` / `false`)
`upload_full_json`	No	Upload full JSON bundle with all artifacts
`upload_images`	No	Upload screenshot attachments
`upload_videos`	No	Upload video recordings
`json_report`	No	Custom path to JSON report file
`html_report`	No	Custom path to HTML report file
`verbose`	No	Enable verbose logging output

TipUse CircleCI contexts (e.g., testdino-credentials) to manage secrets across multiple jobs without repeating environment variable setup.

Sharded Test Runs

For larger test suites, CircleCI parallelism splits tests across multiple containers. Each container uploads its own report, and TestDino groups them into one run when they share a ci-run-id.

How it works

CircleCI runs Playwright across 4 containers using parallelism: 4
Each container writes its results to its own report directory
The orb uploads each report with the same ci_run_id so TestDino merges them into a single run
No merge job runs after the containers finish

Full sharded config

.circleci/config.yml: Sharded config

.circleci/config.yml

version: '2.1'
orbs:
  testdino: testdino/testdino@1.0.0
jobs:
  run-tests:
    docker:
      - image: mcr.microsoft.com/playwright:v1.40.0-focal
    parallelism: 4
    steps:
      - checkout
      - run:
          command: npm ci
          name: Install dependencies
      - run:
          command: |
            npx playwright test \
              --shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL
          name: Run shard
      - testdino/upload-report:
          report_directory: ./playwright-report
          token: TESTDINO_TOKEN
          ci_run_id: $CIRCLE_WORKFLOW_ID
workflows:
  test-and-upload:
    jobs:
      - run-tests

Key details

Config Block	What It Does
`parallelism: 4`	Runs 4 shard containers in parallel
`--shard=$((CIRCLE_NODE_INDEX + 1))/$CIRCLE_NODE_TOTAL`	CircleCI uses 0-based indexing, Playwright uses 1-based
`ci_run_id: $CIRCLE_WORKFLOW_ID`	Groups all shards into one run on the dashboard
`token: TESTDINO_TOKEN`	Names the environment variable holding the TestDino API key

Pipeline execution

After the workflow runs, CircleCI shows all shard containers in the pipeline view. Each container uploads its report, and TestDino merges them into one run. CircleCI pipeline view showing 4 parallel Playwright shard containers, each uploading Playwright results to TestDino

CircleCI pipeline view showing 4 parallel Playwright shard containers, each uploading Playwright results to TestDino

Results in TestDino

The test run appears in your TestDino dashboard with full failure details, flaky detection, and trend data once the shards finish uploading. TestDino Test Runs dashboard showing results uploaded from CircleCI with pass/fail counts and AI Insights

TestDino Test Runs dashboard showing results uploaded from CircleCI with pass/fail counts and AI Insights

Troubleshooting

Orb not found or version error

Ensure the orb is imported correctly: testdino: testdino/testdino@1.0.0. Check the CircleCI Orb Registry for the latest version.

Sharded runs show up as separate runs

Pass the same ci_run_id: $CIRCLE_WORKFLOW_ID to every shard container. Different values create one run per shard.

TESTDINO_TOKEN not found

Confirm the environment variable is set in CircleCI → Project Settings → Environment Variables. If using contexts, verify the job references the correct context name.

CI Optimization

Reduce CI time with smart reruns

Environment Mapping

Route test results to Dev, Staging, or Production by branch

Integrations

Connect Slack, Jira, Linear, Asana, and more

TestDino MCP

Access test results and fix issues with AI agents

Getting Started

Installation & Setup

AI & MCP

Guides

Project

Test Management

Organization

Integrations

Data Privacy

Resources

Pick an upload mode

Prerequisites

Set Up Your API Key

Basic Upload

Upload with HTML Report

Upload Full Bundle

Upload with Custom Paths

Orb Parameters

Sharded Test Runs

How it works

Full sharded config

Key details

Pipeline execution

Results in TestDino

Troubleshooting

CI Optimization

Environment Mapping

Integrations

TestDino MCP

​Pick an upload mode

​Prerequisites

​Set Up Your API Key

​Basic Upload

​Upload with HTML Report

​Upload Full Bundle

​Upload with Custom Paths

​Orb Parameters

​Sharded Test Runs

​How it works

​Full sharded config

​Key details

​Pipeline execution

​Results in TestDino

​Troubleshooting

​Related

CI Optimization

Environment Mapping

Integrations

TestDino MCP

Pick an upload mode

Prerequisites

Set Up Your API Key

Basic Upload

Upload with HTML Report

Upload Full Bundle

Upload with Custom Paths

Orb Parameters

Sharded Test Runs

How it works

Full sharded config

Key details

Pipeline execution

Results in TestDino

Troubleshooting

Related