Troubleshoot TestDino Playwright Errors

This page collects the errors TestDino users hit most often, grouped by where they happen: uploading results, CI and pull requests, signing in, the API, integrations, and plan limits. Find your symptom in the table, then open its fix below.

Quick Reference

Symptom	Area	Fix
`Token is required but not provided`	Upload	Token errors
`Invalid API key or unauthorized access`	Upload	Token errors
`report.json not found`	Upload	Missing report.json
`report.json` deleted mid-run	Upload	Reporter order
Upload succeeds but run is absent	Upload	Runs not appearing
Upload step never ran after a failure	CI	Upload skipped in CI
Sharded run shows partial results	CI	Sharded reports not merging
No check or comment on the pull request	CI	GitHub checks and comments
CI check fails with a high pass rate	CI	GitHub checks and comments
Run has no environment label	CI	Environment label missing
Cannot sign in with SSO	Access	SSO sign-in failures
Password login refused	Access	SSO sign-in failures
Cannot invite, or change a role	Access	Permission errors
`401 AUTH_INVALID_TOKEN`	API	API token and scope errors
`429 RATE_LIMIT_EXCEEDED`	API	API rate limits
Slack alerts not arriving	Integrations	Slack alerts missing
MCP server or tool errors	Integrations	MCP errors
`Execution limit reached`	Plan	Plan and seat limits

Upload and report errors

The CLI uploads the report.json that Playwright writes after a run. Most upload failures are a token, a path, or a reporter-config problem.

Token errors

The CLI authenticates with the TESTDINO_TOKEN environment variable. Project tokens start with tdp_ and come from Project Settings → API Keys.Confirm the token is set, then pass it explicitly and watch the verbose output:

echo $TESTDINO_TOKEN
npx tdpw upload ./playwright-report --token="$TESTDINO_TOKEN" --verbose

Token is required but not provided means the variable is unset or empty. Export it, or pass --token directly.Invalid API key or unauthorized access means the token is wrong, expired, revoked, or scoped to a different project. The CLI returns:

{ "success": false, "error": { "code": "AUTH_ERROR", "message": "Invalid API key or unauthorized access" } }

Generate a fresh key and confirm it belongs to the project you are uploading to. In CI, token names are case-sensitive and must be stored as secrets, never hardcoded. A secret saved as TestDino_Token will not resolve as TESTDINO_TOKEN. Create and scope keys from API Keys.

Missing report.json

The CLI reads the report.json produced by Playwright’s JSON reporter. If it is missing, the upload has nothing to send. Two things must be true:

The JSON reporter is configured and writes the file.
The path you pass to upload is the directory that holds it.

playwright.config.ts

reporter: [
  ['html', { outputDir: './playwright-report' }],
  ['json', { outputFile: './playwright-report/report.json' }],
]

npx tdpw upload ./playwright-report --token="$TESTDINO_TOKEN"

A path mismatch produces report.json not found even when the file exists in another folder. Confirm the upload path matches the reporter’s outputFile directory. The Python flow is equivalent: pytest --playwright-json=test-results/report.json --html=test-results/index.html --self-contained-html, then testdino upload ./test-results.

Reporter order

Playwright’s HTML reporter clears its output directory on every run. When it is listed after the JSON reporter and points at the same directory, it deletes the report.json from that run. This is the most common first-upload failure, and it looks identical to a missing report.List the HTML reporter before the JSON reporter:

playwright.config.ts

reporter: [
  ['html', { outputDir: './playwright-report' }],   // first: clears the dir
  ['json', { outputFile: './playwright-report/report.json' }],  // then writes
]

The Python flow writes JSON and HTML to separate files, so reporter order is not a factor there.

Runs not appearing

The upload reports success but the run is missing from the dashboard. Work through these in order:

Wrong project. A valid token scoped to another project uploads the run there. Confirm the token belongs to the project you are viewing.
Empty report. Confirm the report folder holds a non-empty report.json.
Verbose check. Re-run with --verbose to see the exact upload target and any files the CLI skipped.

npx tdpw upload ./playwright-report --token="$TESTDINO_TOKEN" --verbose

The FAQ walks through this case in more detail under Why are my uploaded runs not appearing?.

CI and pull request errors

These appear once uploads run inside a pipeline: the upload step is skipped, shards do not merge, or the result never links to a pull request.

Upload skipped in CI

When tests fail, most CI providers stop the job before the upload step runs, so failing runs, the ones you most want to see, never reach TestDino. Mark the upload to run regardless of the test result. The keyword differs by provider:

Provider	Make upload always run
GitHub Actions	`if: always()` on the step
GitLab CI	`artifacts: when: always` on the job
Azure DevOps	`condition: always()` on the step
AWS CodeBuild	run the upload in the `post_build` phase
Bitbucket	`\|\| true` after the test command
Jenkins	upload in the `post.always` block
CircleCI	`when: always` on the step

The token must also be available to the step. In CI, secret names are case-sensitive, and some providers scope secrets to protected branches only.

Sharded reports not merging

A sharded run produces one blob report per shard. Uploaded as-is, the dashboard shows a single shard’s results. Merge the blobs into one report first, then upload the merged report:

npx playwright merge-reports --config=playwright.config.ts ./all-blob-reports
npx tdpw upload ./playwright-report --token="$TESTDINO_TOKEN" --upload-html --upload-traces

Each shard must persist its blob report as a CI artifact, and the merge job must collect every shard before merging. A merge that finds fewer blobs than shards produces a partial report. The persistence step (artifact, stash, or workspace) is provider-specific, so follow the sharding section in your CI provider guide.

GitHub checks and comments

A pull request shows no TestDino check or comment even though the run uploaded. Both depend on TestDino linking the run to a commit and the connected repository. Common causes:

GitHub Checks is off. Enable it in the CI Checks tab. See GitHub status checks.
Comments toggle is off. Enable comments in Settings → Integrations → GitHub.
No commit SHA. Run the upload inside CI so git context is captured automatically.
Repository mismatch. The TestDino project must be connected to the same repo the PR lives in.
App not installed. The GitHub App must be installed with access to the repository.

A check can also fail while the pass rate looks high when a mandatory tag test fails: that overrides the pass rate. The FAQ covers this under Why might a TestDino CI Check fail even if my pass rate looks high?. For repo connection and comment settings, see the GitHub integration guide.

Environment label missing

A run appears without an environment label and drops out of environment-filtered views. The branch matched no mapping pattern. The usual cause is a regex mistake:

Unanchored: dev matches any branch that contains “dev”. Anchor it: ^dev/.
Too broad: .* or .+ matches every branch and overrides your specific rules.
Invalid regex: an unclosed bracket or a bad escape never matches.

Test patterns on regex101.com before saving, and add a catch-all rule for branches outside your naming convention. See Environment mapping for the full pattern reference.

These block people from signing in or acting in the organization. Most are SSO configuration or role permissions.

SSO sign-in failures

SSO errors trace to a mismatch between your identity provider and the values saved in TestDino. The most common ones:

Error	Cause	Fix
`redirect_uri must be a Login redirect URI`	Callback URL does not match the organization key	Set the IdP redirect URI to exactly `…/sso/<your-org-key>/callback` on `api.testdino.com`, not `app`, with no trailing slash
`Policy evaluation failed`	No access policy, or the user is not assigned	Add an access policy and rule on the default auth server, then assign the user
`idpCert is not in PEM format`	Certificate not pasted as a full PEM block	Paste the whole certificate, including the `BEGIN` and `END` lines
Password login refused	SSO enforcement is on for that email domain	Sign in with SSO, or remove the domain from enforcement
No Single Sign-On tab	Not an admin, or the org is not SSO-entitled	Get admin access, or contact support to enable SSO

For the full setup and the SCIM provisioning errors, see Single Sign-On.

Permission errors

Permissions are organization-level, so a blocked action usually means your role is too low, not a bug. Common cases:

Cannot invite a member. Only Owners and Administrators can invite. The org may also be at its member limit for the plan.
Cannot change a role. You can only assign roles at or below your own. Administrators cannot grant the Owner role or change their own role.
Cannot delete a run or test case. Members cannot delete; Viewers are read-only.
Cannot manage an integration. Project-level integrations (GitHub, GitLab, Slack, monday, Azure DevOps) need Owner or Administrator.
Cannot manage billing. Only Owners and Billing Managers can change a subscription.

See the permission matrix and each case in Users, Roles & Permissions.

API errors

These come from the Public API and the MCP server, which both use a project token. Authentication, scope, and rate limits are the usual causes.

API token and scope errors

Status	Code	Cause	Fix
401	`AUTH_INVALID_TOKEN`	Token missing, malformed, expired, or revoked	Send `Authorization: Bearer tdp_...` with one space. Re-issue the token if it was rotated
403	`FORBIDDEN`	Valid token, wrong project	Use the project token that matches the `projectId` in the URL. Tokens are scoped to one project
404	`NOT_FOUND`	Resource missing, or not visible to the token	Verify the ID and that the token can access that project. Confirm a run ID exists with `list_testruns`

The full error table is in API conventions, and the auth setup is in the API quickstart.

API rate limits

A 429 RATE_LIMIT_EXCEEDED means you crossed a per-token window:

{ "success": false, "error": { "code": "RATE_LIMIT_EXCEEDED", "message": "Too many requests" } }

The limits are 100 requests per minute per token, 200 per minute per IP before auth, and 1 per minute per token for PDF generation. Read RateLimit-Reset from the response headers and wait that many seconds, or back off exponentially. See Rate limits for the header reference.

Integration errors

Slack and MCP each have their own connection and configuration failures. These accordions cover the top symptoms and link to the full guides.

Slack alerts missing

No TestDino messages arrive in Slack. Check the connection and channel setup:

App not connected, or no default channel. Verify the connection in Settings → Integrations → Slack and set a default channel. Use the Test button to send a sample.
Private channel not listed. Add the TestDino app to the channel first (channel name → Integrations → Add an App), then click Refresh in Slack settings.
Messages in the wrong channel. Check the environment-to-channel mapping. Unmapped environments fall back to the default channel.
Annotation alerts silent. The test needs a testdino:notify-slack annotation with a mapped target, and annotation alerts fire only on failure.

Each case is detailed in Slack alerts.

MCP errors

The MCP server has its own setup, authentication, and network failures, with a dedicated page. The categories:

Installation: npm ERR! 404 Not Found, command not found: testdino-mcp.
Editor integration: the server does not appear in Claude Code, Cursor, or Claude Desktop.
Authentication: health returns “Invalid token” or “No projects found”.
Network: ECONNREFUSED, timeouts, or a 429 from the server.

Each category and its fix is in MCP troubleshooting.

Plan and seat limits

When the organization reaches a plan limit, tests still run but TestDino stops accepting new data. Two limits apply. Execution limit reached is the monthly test-run quota. Tests run locally, but uploads pause until you upgrade the plan or the quota resets on the billing date. You can rebalance the quota across projects from Settings → Test Limits, where each project shows used versus allocated executions. With Auto-Borrow on, a project that exhausts its allocation borrows from the unallocated pool instead of blocking; with it off, streaming stops at the limit. user limit reached is the seat count for net-new members. A new SSO sign-in or SCIM create is blocked at the limit, but existing members signing in again are never blocked. Free a seat or upgrade the plan. See Test limits for quota allocation and auto-borrow settings.

Other setup gaps

Commit, branch, or author blank. The CLI reads git metadata from the repository, so run uploads inside an initialized git repo.
Run tags rejected. Use up to 5 tags with letters, numbers, hyphens, underscores, and dots only.
CLI errors on an old runtime. The Node.js CLI needs Node.js 18+ and @playwright/test 1.52+. The Python CLI needs Python 3.9+.

Get help

Still stuck? Collect this before contacting support, so the team can reproduce it without a back-and-forth:

The Run ID or dashboard URL of the affected run.
The CLI version: tdpw --version or testdino --version.
The framework and runtime: Playwright version, Node.js or Python version.
The full error and stack trace.
The CI logs with tokens and credentials redacted.
The --verbose output from the failing command.

Reach the team at support@testdino.com or on Discord. See Support for hours and response times.

Node.js Reporter

Upload flags, reporter config, and CLI commands for the Node.js CLI.

Python Reporter

Upload flags and pytest config for the Python CLI.

Generate API Keys

Create and scope the token the CLI and API use.

CI Setup Overview

Per-provider pipeline guides with sharding and artifact handling.

​Quick Reference

​Upload and report errors

​CI and pull request errors

​Sign-in and access errors

​API errors

​Integration errors

​Plan and seat limits

​Other setup gaps

​Get help

​Related