Before you start, you need:
- The Owner, Admin, or Member role. Every plan includes Run tests.
- GitHub connected to the project. If it is not, the dialog asks you to connect it first. See GitHub App.
- A workflow with the
testdino_selectioninput, as shown below. @testdino/playwright2.11.1 or later and Playwright 1.56 or later on the branch you run. Running whole spec files from New on GitHub needs@testdino/playwright2.11.6 or later.
Set up your workflow
Run tests uses the sametestdino_selection workflow input as Run automation in Test Management. A workflow already set up for Run automation needs no change.
Otherwise, add the input to the GitHub Actions workflow that runs your Playwright tests, and pass it to npx tdpw test. Follow Set up your workflow for the lines to add and the rules for the test command. Add the workflow to every branch you want to run from TestDino.
When the Run tests button appears
Run tests sits in the Test Runs page header, next to the Sync icon. It appears for the Owner, Admin, and Member roles when the project has test runs in the last 90 days and 1 of these is true:
The button is hidden for the Viewer and Billing Manager roles, and when runs come only from another CI provider and GitHub is not connected.
Start a run
1
Open Run tests
On the Test Runs page, click Run tests. The dialog opens on Pick tests with every test TestDino has seen, and no filter applied.
2
Pick tests
Tick tests, or tick a folder, spec file, or describe block to pick every test under it. Click a folder, spec file, or describe block row to expand or collapse it; click a test row to tick it. Switch between the tree view (folder, spec file, describe block, test) and the list view with the 2 buttons next to the search box. All shows what the search and filters leave, Selected shows your picks. Select all picks every test shown; Clear removes your picks.
The list shows only tests from the repository connected to the project, because that is where the run starts. If the project also has test runs from other repositories, a note above the list names them.Tests not reported in the last 30 days sit in a Not run in 30+ days group at the bottom.A run takes up to 10,000 tests and 500 new spec files. Click Next.
3
Choose a branch, browsers, and workflow
- Branch: starts as the branch you filtered on in step 1, otherwise the repository’s default branch. Search the list or type a name.
- Browsers: leave it on All browsers to run each test in the browsers it recently ran in, or pick 1 or more browsers. The list shows how many of your picks have run in each browser.
- Workflow: the workflows on that branch that are set up for TestDino.
N workflows can't run selected tests. Show whylists the others and what to change. - If a test isn’t on
<branch>: Skip missing tests (the default) or Fail the job. See Tests missing on the branch.
42 of 47 tests will run, and a line for each reason a pick is skipped. Click it to list the tests that will run, and click Show tests on a reason to list the tests it skips. See Tests skipped before the run.4
Run
Click the Run button, such as
Run 42 tests. The dialog shows Run started. Click View in GitHub to open the workflow run, or Done to close the dialog.Discard your test picks?. Click Keep editing to return to your picks.
Run spec files that are new on GitHub
New on GitHub lists spec files on the branch that TestDino has never seen, such as a test file added since the last test run. Picking one runs every test in it. Files with no test in them, such as a fully commented-out file, are left out. The group appears below the tests in the All view, without a Status, Tag, or From a run filter. While TestDino checks the branch, the list showsChecking GitHub for new spec files…. It appears only when the latest test run used @testdino/playwright 2.11.6 or later.
See the results
The results arrive as a new test run in Test Runs when the GitHub run finishes. It shows who started it, such asManual · <name>. To retry only the tests that failed, open the new test run and click Re-run. See Re-run failed tests.
Tests skipped before the run
Tests to run lists each reason a pick won’t run, before CI starts. Each line starts with what won’t run, such as2 tests or 1 new file:
When no pick can run, the card shows
None of these tests will run and the Run tests button stays disabled.
Troubleshooting
No workflow on <branch> can run selected tests
No workflow on <branch> can run selected tests
No active workflow on the chosen branch has the
testdino_selection input. The dialog shows Set up a workflow with the lines to add. Add them on that branch, then come back and try again. Show why explains the other workflows.GitHub has no branch named <branch>.
GitHub has no branch named <branch>.
The branch is not on GitHub, so nothing can run and Run tests stays disabled. Pick an existing branch, or push the branch first.
New on GitHub doesn't appear
New on GitHub doesn't appear
The group shows only when the latest test run used
@testdino/playwright 2.11.6 or later, in the All view without a Status, Tag, or From a run filter. Upgrade the CLI and run your tests once, or clear the filter. A branch whose spec files TestDino has all seen shows no group.Selection <id> includes whole test files, which need a newer @testdino/playwright
Selection <id> includes whole test files, which need a newer @testdino/playwright
The workflow ran a selection with a file from New on GitHub on
@testdino/playwright older than 2.11.6, so nothing ran. Upgrade it on that branch, then start the run again.A test is skipped as not run on <browser>
A test is skipped as not run on <browser>
The test hasn’t run recently in the browser you chose. Clear the browser choice so Browsers reads All browsers, and the test runs where it last ran.
Some tests are missing from the list
Some tests are missing from the list
The list shows only tests from the repository connected to the project. Tests reported from another repository are not listed, and the note above the list names those repositories. To run them, send that repository’s test runs to a project connected to it, and run them from that project.
Related
Run automated tests
Run the automated tests linked to test cases in Test Management.
Re-run failed tests
Run only the failed or flaky tests from a finished test run.
Node.js CLI
Every
npx tdpw test flag, including --selection.GitHub App
Connect GitHub to a TestDino project.