ArgoCD Notifications Deep Dive - Triggers, Templates, and Subscriptions

by Bala

How the ArgoCD Notifications engine's trigger/template model works, and configuring per-application subscriptions to Slack, email, and webhooks

Introduction

The argocd-monitoring guide on this site covers wiring up a couple of notification channels quickly. This one covers the actual model underneath: triggers (conditions evaluated against Application state), templates (the message content), and subscriptions (which triggers send to which destination for which Application) - understanding the three lets you build genuinely custom notification logic instead of copy-pasting the built-in catalog.

The Three Pieces

apiVersion: v1
kind: ConfigMap
metadata:
  name: argocd-notifications-cm
  namespace: argocd
data:
  # 1. Service: where a notification can be sent
  service.slack: |
    token: $slack-token

  # 2. Template: what the notification says
  template.app-sync-succeeded: |
    message: |
      Application {{.app.metadata.name}} is now running new version.
      Sync status: {{.app.status.sync.status}}
    slack:
      attachments: |
        [{
          "title": "{{.app.metadata.name}}",
          "color": "#18be52"
        }]

  # 3. Trigger: when to fire
  trigger.on-sync-succeeded: |
    - when: app.status.operationState.phase in ['Succeeded']
      send: [app-sync-succeeded]
apiVersion: v1
kind: Secret
metadata:
  name: argocd-notifications-secret
  namespace: argocd
stringData:
  slack-token: xoxb-your-actual-bot-token

Credentials always go in argocd-notifications-secret, referenced from argocd-notifications-cm with a $ prefix ($slack-token) - the ConfigMap itself is safe to check into Git as part of the GitOps-managed ArgoCD install; the Secret is not.

Subscribing an Application

A trigger existing in the ConfigMap doesn’t send anything by itself - an Application (or AppProject, for a subscription shared across every app in the project) has to subscribe to it via annotation:

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: web-app
  annotations:
    notifications.argoproj.io/subscribe.on-sync-succeeded.slack: platform-alerts
    notifications.argoproj.io/subscribe.on-sync-failed.slack: platform-alerts
    notifications.argoproj.io/subscribe.on-sync-failed.email: oncall@example.com
spec:
  # ...

The annotation format is notifications.argoproj.io/subscribe.<trigger-name>.<service-name>: <destination> - on-sync-succeeded fires to the platform-alerts Slack channel, on-sync-failed fires to both Slack and a specific email address. This is deliberately self-service: a team owning an Application can add their own subscription annotations without touching the shared argocd-notifications-cm, as long as the trigger and service they reference already exist there.

The Built-In Catalog

Installing the notifications catalog (kubectl apply against the catalog’s install manifest, or as part of the ArgoCD Helm chart with notifications.argocdUrl set) pre-populates a standard set of triggers you don’t have to write yourself: on-deployed, on-health-degraded, on-sync-failed, on-sync-running, on-sync-status-unknown, on-sync-succeeded - covering the common health/sync-state transitions most teams actually want alerted on, with matching Slack/Teams/email templates already written.

Writing a Custom Trigger

data:
  trigger.on-deployed-stale: |
    - when: app.status.sync.status == 'Synced' and app.status.operationState.finishedAt != nil and time.Now().Sub(time.Parse(app.status.operationState.finishedAt)) > time.Duration(24 * time.Hour)
      send: [app-stale-warning]

  template.app-stale-warning: |
    message: |
      {{.app.metadata.name}} hasn't synced in over 24 hours.

Trigger when conditions are expr expressions evaluated against the full Application object (available as app) - anything in app.status/app.spec is fair game, letting you build conditions the built-in catalog doesn’t cover, like this one flagging an app that’s synced but hasn’t actually run a sync operation recently.

Multiple Channels From One Trigger

data:
  service.webhook.internal-audit-log: |
    url: https://audit.internal.example.com/webhook
    headers:
      - name: Content-Type
        value: application/json
metadata:
  annotations:
    notifications.argoproj.io/subscribe.on-sync-succeeded.webhook: internal-audit-log

Any number of service.<type>.<name> blocks can exist side by side - Slack, email, Microsoft Teams, PagerDuty, and generic webhooks are all supported service types, and a single trigger can be subscribed to several simultaneously (as the two on-sync-failed subscriptions above show), so a sync failure can page on-call and post to Slack and hit an internal audit webhook from the same trigger firing once.

Best Practices

  1. Subscribe at the AppProject level for org-wide alerting, and let individual Application annotations add team-specific channels on top - avoids every team having to remember to wire up the baseline alerts themselves.
  2. Keep the Secret and ConfigMap changes in separate PRs/reviews where practical - the ConfigMap (templates, triggers) is low-risk to review casually; the Secret (bot tokens, webhook URLs) deserves tighter scrutiny.
  3. Test a new trigger’s when expression against a real Application before rolling it out broadly - a subtly wrong expr condition either never fires or fires on every reconcile, and both failure modes are easy to miss until someone notices the silence or the spam.
  4. Use the built-in catalog as a starting point, not a ceiling - it covers the common cases, but the trigger/template model is fully generic, so don’t avoid a custom trigger just because it’s not in the catalog.

Conclusion

The notifications engine’s real design is triggers (state conditions) decoupled from templates (message content) decoupled from subscriptions (per-app opt-in) - once that split clicks, adding a new alert is usually a new trigger and a subscription annotation, not a redesign.

Additional Resources