Slack Playwright Test Alerts

​

How does it work?

• Sends run summaries (status, counts, duration, environment/branch, author, commit).
• Routes notifications by environment to specific channels; unmatched events fall back to a default channel.
• Sends annotation-based alerts when tests with testdino:notify-slack fail, routed to specific channels or users.
• Sends tag-based notifications when a run contains tests carrying configured Playwright tags, with per-rule environments, triggers, and filters.
• Supports test posts for verification and quick reconfiguration.

​

Installation & Setup

​

1. Connect Slack

In Project → Integrations → Slack App, click Connect to Slack and complete the OAuth flow. After connecting, the Slack App card shows the connected status with your workspace name.

​

2. Configure channel routing

After connecting, click the ⚙️ icon on the Slack App card to open the Slack Notification Configuration dialog. The Test Run Alerts tab has two sections: Default Channel: Set a fallback channel that receives all test run alerts. Any run that doesn’t match an environment-specific channel is routed here. Environment Channels: Assign specific Slack channels to your project environments (e.g., PROD → #td-prod, STAGE → #td-stage). Runs on mapped environments route to their designated channel instead of the default.

​

3. Save and test

Click Save Configuration to apply your settings. Use the Test button next to any channel to send a sample message and verify it lands in the correct channel.

​

Configuration Scenarios

The integration supports both default and environment-based channel configurations, giving you control over how alerts are delivered.

​

1. Default Channel Only

All test run alerts, regardless of the branch or environment, are sent to a single default channel. This is useful for centralizing all notifications.

​

2. Default + Environment-Specific Channels

Alerts for mapped environments (e.g., PROD Alerts) are sent to their designated channel (e.g., #prod-alerts). All alerts from unmapped branches or environments automatically fall back to the default channel (e.g., #daily-updates).

​

Why this helps

• Reduce notification noise by routing alerts to environment-specific channels so teams see only relevant updates.
• Improve incident response by sending critical failure alerts directly to the responsible team’s channel.
• Enable faster triage with real-time summaries that link directly to detailed test evidence.

​

Annotation-Based Alerts

Beyond run-level alerts, the Slack App can notify specific channels or users when individual tests fail. This is driven by the testdino:notify-slack annotation in your Playwright test code. For example, if a test has testdino:notify-slack set to @ashish, and that test fails, TestDino sends a Slack message directly to Ashish. This is different from test run alerts, which fire on every run completion regardless of which tests failed.

​

How to set it up

1. Add a testdino:notify-slack annotation to your test with a channel (#e2e-alerts) or user (@ashish) as the target.
2. In the Slack Notification Configuration dialog, switch to the Annotation Alerts tab.
3. Map each annotation target to a Slack channel or user from your workspace.
4. Save the configuration.

See the Annotations guide for full setup instructions, code examples, and all supported annotation types.

​

Tag-Based Notifications

Tag-based notifications route Slack alerts by Playwright tag. A rule watches one or more tags, such as @smoke or @critical, and posts a Slack message when a completed run contains matching tests. Each rule controls when it fires (Trigger On), which tests the message lists (Tests), and which environments it watches, so @critical failures from Production go to an incident channel while @smoke results from Staging go to a QA channel.

​

How to set it up

1. Tag your Playwright tests. Tags must include the leading @.
   checkout.spec.ts

   import { test } from '@playwright/test';
   
   test('completes checkout with valid card', { tag: ['@smoke', '@payment'] }, async ({ page }) => {
     // ...
   });
   
2. In the Slack Notification Configuration dialog, switch to the Tags Notification tab.
3. Use the left Rules pane to manage your notification rules:
   • Click Add to create a new rule.
   • Select an existing rule to view or edit its configuration.
   • Each rule can watch different tags, environments, filters, and destinations.
4. Configure the selected rule in the right details pane:
   • Tags: one or more tags to watch. A test matches if it carries any of them.
   • Scope → Environment: choose where this rule applies.
   • Scope → Trigger On: choose when the rule fires.
   • Filters → Tests: choose which type of tests to include in the message: all tests, failed, flaky, or failed/flaky.
   • Destinations → Slack Channel / User: choose one or more Slack destinations.
5. Save the configuration. Use the Test button on a rule to send a sample message to its destinations.

​

Rule configuration

The Tags Notification tab has two panes. The left Rules pane lists every rule for the project; select one to inspect it, or click Add to create a new one. The right pane configures the selected rule with these controls: Environments come from the Branch Environment Mapping feature, which maps Git branches to environments such as Production, Staging, and Dev. Scope a rule to an environment when the same tag should notify different people depending on where it runs:

• @critical + Production → #incident-response.
• @smoke + Staging → #qa-review.
• @flaky + Any environment → #test-stability.

​

Potential Combinations

Trigger On and Tests combine into 8 behaviors. The table reads for a rule watching @smoke: Trigger On decides which runs post, Tests decides which tagged tests the message lists. Tag rules fire on run completion only when the run contains tests carrying the rule’s tags and the run matches the rule’s environment scope. A run with no matching tagged tests sends nothing.

​

Format & Limits

Tag format. When adding a tag in a rule:

• It must start with @ (e.g. @smoke).
• There must be no space right after the @ (@ smoke is rejected).
• It cannot contain a backslash (\).
• Any other character is allowed, matching Playwright (e.g. @#hdi, @[]hie).

Tags must match your Playwright test tags exactly, including the leading @. Limits per rule:

• Up to 20 tags.
• Up to 10 channels/users.
• Each message lists the first 10 matching tests; the total matched count still shows in the header.

​

How it’s Different from Slack Webhook

The Slack App provides three types of alerts:

• Test Run Alerts send environment-aware run summaries when any run completes.
• Annotation-Based Alerts notify specific channels or users when individual annotated tests fail.
• Tag-Based Notifications route alerts by Playwright tag, with per-rule environments, triggers, and filters.

The Slack Webhook sends all notifications to a single channel. It does not support environment routing, annotation-based alerts, or tag-based notifications.

​

Troubleshooting