What you’ll learn
- How to install and connect the TestDino GitHub App
- How to configure PR comments and commit summaries
- How to set up CI checks with quality gates
- How the Pull Requests dashboard syncs with GitHub
What the GitHub Integration Does
| Feature | What It Does | Where It Appears |
|---|---|---|
| PR Comments | Posts AI-generated summary with pass/fail counts, failure analysis grouped by file, and a link to the full report | GitHub pull request conversation tab |
| Commit Comments | Posts the same AI summary attached to the specific commit | GitHub commit detail page |
| CI Checks | Posts pass/fail status based on your quality gate settings (pass rate, mandatory tags, flaky handling) | GitHub PR status checks section |
| Pull Request Dashboard | Syncs PR metadata (author, reviewers, status, branches) and displays test run KPIs, pass rate trends, and AI failure insights per PR | TestDino → Pull Requests |
| Timeline | Syncs commits, test runs, and code review events into a chronological feed per PR | TestDino → PR Timeline |
| Files Changed | Syncs file diffs, additions, deletions, and code-level comments with resolved/unresolved status | TestDino → PR Files Changed |
Prerequisites
- A TestDino account with a project created (Getting Started)
- An API key generated for your project (Generate API Keys)
- The Playwright CLI configured and uploading results (CLI Overview)
- GitHub organization or account admin access (required to install the GitHub App)
Connect TestDino with GitHub
Connection is a two-step process: install the GitHub App (grants repo access in GitHub), then select which repository to connect in TestDino.1. Install the TestDino GitHub App
You can start from either location:- From GitHub: Open the TestDino | Playwright Reporter marketplace listing and click Install & Authorize
- From TestDino: Go to Settings → Integrations → GitHub and click Connect. This opens the same GitHub installation flow.
- All repositories: grants access to every current and future repo in the organization
- Only select repositories: grants access to specific repos you pick from the list
| Scope | Access | Used For |
|---|---|---|
| Actions | Read | Detecting Playwright CI runs and attaching results to commits |
| Checks | Read & Write | Posting pass/fail CI status checks on PRs and commits |
| Metadata | Read | Reading repo, branch, and organization info |
| Code, Issues, PRs, Workflows | Read & Write | Syncing PR metadata, file diffs, code review comments, and posting test summaries |
NoteThis step only grants the app access in GitHub. It does not connect a repo to a TestDino project yet. You can change repo access anytime from GitHub → Settings → Applications → TestDino.
2. Connect a repository to your TestDino project
After you approve the app in GitHub, the browser redirects back to TestDino. You land on the repository selection screen inside your TestDino project. Pick the repository you want to link to this project. Only repos you granted access to in the previous step appear in the list.
After selecting a repo, the GitHub card in Settings → Integrations shows the connected repository name and organization.
TestDino starts syncing PR metadata and attaching test runs to commits from this point forward.
After a successful connection, the integration unlocks PR comments, commit summaries, CI checks with quality gates, and a Pull Requests dashboard inside TestDino. Let’s configure each of these below.
Configure PR Comments
When a Playwright run finishes, TestDino can post an AI-generated test summary directly to the pull request or commit in GitHub. To control where and when these comments appear, you configure them through GitHub Settings in TestDino.
Open Settings → Integrations → GitHub and click the ⚙️ icon on the GitHub card. This opens the GitHub Settings panel with two tabs: Comments and CI Checks.
Default comment toggles
At the top of the Comments tab, two toggles control the global defaults:- Pull Request Comments: posts a test summary to every PR on a mapped branch
- Commit Comments: posts a test summary to every commit on a mapped branch
Environment overrides
Below the defaults, the Environment Overrides table lists each environment you have configured in your project (e.g., PROD, DEV, MAIN, QA, STAGE). Each row shows:| Column | What It Controls |
|---|---|
| Environment | The environment name (configured in Environment Mapping) |
| Branch Patterns | Which branches map to this environment (e.g., main, dev, staging) |
| PR | Toggle PR comments on or off for this specific environment |
| Commits | Toggle commit comments on or off for this specific environment |
Configure CI Checks
CI checks post a pass or fail status to GitHub based on your quality gate rules. If a required check fails, GitHub blocks the merge.
| Setting | Default | What It Controls |
|---|---|---|
| Pass Rate | 90% | Minimum percentage of tests that must pass |
| Mandatory Tags | None | Tags (e.g., @critical) where any failure fails the entire check |
| Flaky Handling | Neutral | Whether flaky tests count as failures (Strict) or are excluded (Neutral) |
| Environment Overrides | None | Per-environment rules that override the defaults |
InfoFor the full CI checks setup including environment overrides, making checks required in GitHub rulesets, troubleshooting, and common scenarios, see GitHub Status Checks.
Pull Requests in TestDino
Once connected, TestDino syncs your GitHub pull requests into a dedicated Pull Requests view. Each PR shows:| Tab | What It Shows |
|---|---|
| Overview | PR status, test run KPIs (pass rate, duration, files changed), latest run with AI failure insights, and test results trend graph |
| Timeline | Chronological feed of commits, test runs, and code review events synced from GitHub |
| Files Changed | File diffs with additions, deletions, and code-level comments with resolved/unresolved status |
Troubleshooting
PR comments not appearing
PR comments not appearing
- Verify the global comment toggles are enabled in Settings → Integrations → GitHub → ⚙️ → Comments
- Confirm the branch that triggered the run is mapped to an environment (if using overrides)
- Check that the CLI is uploading with the correct project API key
CI check not showing on PR
CI check not showing on PR
- Ensure Enable GitHub Checks is toggled on in the CI Checks tab
- Verify the commit SHA is present in the test run metadata (the CLI sends this automatically)
- Confirm the repository connected in TestDino matches the repo where the PR was opened
Pull Requests not syncing in TestDino
Pull Requests not syncing in TestDino
- Check that the GitHub App is still installed on the organization (GitHub → Settings → Applications)
- Verify the correct repository is connected in Settings → Integrations → GitHub
- Ensure tests have run on the PR branch at least once after connecting
Repository not appearing in selection list
Repository not appearing in selection list
- The GitHub App may not have access to that repo. Update repo access from GitHub → Settings → Applications → TestDino
- After changing access, return to TestDino and refresh the repository list
GitHub Status Checks
Configure quality gates, mandatory tags, and environment overrides
Environment Mapping
Map branches to environments for targeted reporting
Pull Requests Dashboard
View all PRs with test run results, trends, and AI insights
Test Health Badges
Add test status badges to your GitHub README