Before you start
- A PagerDuty account with the Events API v2 integration (any paid tier, plus the free trial).
- The PagerDuty service these alerts belong to. A service represents one thing that can fail, so pick or create the matching one.
1. Add the integration in PagerDuty
1
Open the service
Go to Services → Service Directory and open the service that should receive Tallwatch alerts.
2
Add an Events API V2 integration
On the Integrations tab, click Add an integration, search for Events API V2, and select it.
3
Copy the integration key
PagerDuty shows a 32-character Integration Key. Copy it.
PagerDuty’s docs sometimes call this the “Routing Key.” It’s the same value. Tallwatch’s form labels it Integration key.
2. Add the channel in Tallwatch
1
Open the channels page
Go to Settings → Alerts → Channels and click Add channel.
2
Pick PagerDuty
Choose the PagerDuty kind and name it for the service, like
PagerDuty: billing prod.3
Paste the key and test
Put the 32-character key in Integration key, click Save, then Send test alert. A triggered incident should open in PagerDuty within seconds.
How events map
Tallwatch translates incident lifecycle events into PagerDuty event actions:
The
dedup_key is tallwatch:incident:<id>, so the resolve ties back to the incident the trigger opened rather than spawning a duplicate.
What the alert carries
A triggered PagerDuty incident with:- A summary of
<monitor name> <event>, for exampleAPI healthcheck incident.opened - Severity
criticalon open sourceset to the monitor’s URL (or its name, for monitors without one)componentset to the monitor type andgroupto your workspace slugcustom_detailswith the full event, monitor, org, incident, and last-check data, so on-call can see the failing regions and error class without leaving PagerDuty
Acknowledge and resolve
Resolving in Tallwatch sends aresolve to PagerDuty. Acknowledging in Tallwatch does not: in v1, the only actions sent to PagerDuty are trigger and resolve. Acknowledge the page in PagerDuty itself. Two-way sync of acknowledgements is planned for v1.1.
Troubleshooting
The test dispatched, but no incident in PagerDuty
The test dispatched, but no incident in PagerDuty
Check the dispatch history. A 4xx usually means the integration key was revoked or pasted wrong, so regenerate the integration and update Tallwatch.If PagerDuty returned 202 but no incident shows, the service may be suppressing a re-trigger because the same incident resolved recently. Wait the dedup window out, or trigger from a different monitor.
The dispatch shows `failed`
The dispatch shows `failed`
Open the row for the error. Common ones:
- A
pagerduty_errorresponse, often a disabled integration. Re-enable it on the service’s Integrations tab. - A 400, which means PagerDuty rejected the request body. Open an issue with the error text.
The PagerDuty incident isn't auto-resolving
The PagerDuty incident isn't auto-resolving
Auto-resolve depends on the
resolve carrying the same dedup_key as the original trigger. If the incident was resolved manually in PagerDuty first, PagerDuty treats the key as stale and a later trigger opens a fresh incident instead.Can I send to multiple PagerDuty services?
Can I send to multiple PagerDuty services?
Yes. Add one Tallwatch channel per service, each with its own integration key, and bind them to different monitors or policies.