Notifications and Audit
Complete guide to notification channels, personal routing, escalation policies, audit logging, and compliance evidence.
GitOpsHQ provides a comprehensive notification system for alerting teams about important events and a full audit trail for compliance and incident investigation. Together, these features ensure that the right people are informed at the right time and that every action is traceable.
Key Concept
Notifications answer "what just happened?" in real-time. Audit logs answer "what happened, when, and who did it?" after the fact. Both are essential for operational excellence and regulatory compliance.
Notification Channels
GitOpsHQ supports multiple notification channels to meet different team workflows:
| Channel | Tier | Description | Configuration |
|---|---|---|---|
| In-App | Free | Notifications displayed in the GitOpsHQ UI bell icon | Automatic — always available |
| Free | Email notifications via SMTP | Configured at org level | |
| HTTP Webhook | Free | Custom HTTP endpoint for integration | URL + optional secret |
| Slack | Pro | Slack workspace integration via Bot or Incoming Webhook | OAuth or webhook URL |
| Discord | Pro | Discord channel notifications | Webhook URL |
| Microsoft Teams | Pro | Teams channel cards | Incoming webhook connector |
| PagerDuty | Pro | PagerDuty incident creation | Integration key |
| OpsGenie | Pro | OpsGenie alert creation | API key |
Webhook Delivery
HTTP webhook notifications are delivered with a single HTTP attempt. There is no automatic retry logic. If the target endpoint is unavailable, the notification is lost for that delivery attempt.
Notification Event Categories
Events are organized into categories so you can subscribe to the signals that matter for your role:
Deployment Events
| Event | Description | Typical Audience |
|---|---|---|
release.create | A new release was created | DevOps, SRE |
release.approve | A release was approved for deployment | DevOps, Manager |
release.deploy | A release was successfully deployed | DevOps, SRE, Stakeholders |
release.fail | A release deployment failed | SRE, On-Call |
Promotion Events
| Event | Description | Typical Audience |
|---|---|---|
promotion.request | A promotion was requested between environments | DevOps |
promotion.approve | A promotion was approved | DevOps, Manager |
promotion.execute | A promotion was completed | DevOps, SRE |
Rollback Events
| Event | Description | Typical Audience |
|---|---|---|
rollback.initiate | A rollback was started | SRE, On-Call |
rollback.complete | A rollback finished successfully | SRE, DevOps |
Governance Events
| Event | Description | Typical Audience |
|---|---|---|
policy.violation.detect | A policy violation was found | Security, DevOps |
policy.violation.resolve | A policy violation was resolved | Security |
approval.request | An action requires approval | Approvers |
Cluster Events
| Event | Description | Typical Audience |
|---|---|---|
agent.connect | A cluster agent came online | SRE |
agent.disconnect | A cluster agent went offline | SRE, On-Call |
drift.detect | Cluster state diverged from Git | SRE, DevOps |
Security Events
| Event | Description | Typical Audience |
|---|---|---|
break_glass.opened | A break-glass session was initiated | Security, Manager |
apikey.create | A new API key was generated | Security |
member.role.change | A member's role was modified | Security, Manager |
Personal Notification Preferences
Each user configures their own notification routing. This allows different team members to receive different signals through different channels.
Open Notification Settings
Navigate to your Profile → Notification Preferences to configure your personal routing rules.
Select Event Categories
Choose which event categories you want to receive. You can subscribe to entire categories or individual events.
Assign Channels
For each selected event, choose which notification channel(s) should deliver the message. You can route different events to different channels.
Set Scope
Optionally scope your preferences to specific projects. For example, only receive production deployment notifications for the api-platform project.
Enable Digest Mode
For high-volume events, enable digest mode to receive a batched summary instead of individual notifications. Digests are sent at configurable intervals (e.g., every 15 minutes or hourly).
Role-Based Recommendations
| Role | Recommended Events | Recommended Channels |
|---|---|---|
| DevOps Engineer | Deployment, promotion, rollback events | Slack + In-App |
| SRE / On-Call | Cluster health, deployment failures, rollbacks | Slack + Email + PagerDuty (via webhook) |
| Security Engineer | Break-glass, API key, role changes, policy violations | Email + Slack |
| Engineering Manager | Approval requests, production deployments, policy violations | Email + Teams |
| Developer | Release created, promotion executed (own projects) | In-App + Slack |
Escalation Policies (Pro/Enterprise)
Escalation policies ensure that critical notifications are not missed by defining automatic escalation rules.
Pro/Enterprise Feature
Escalation policies are available on Pro and Enterprise plans. They are essential for organizations with on-call rotations and SLA requirements.
How Escalation Works
Escalation Configuration
| Field | Description | Example |
|---|---|---|
| Policy Name | Descriptive name | Production Deployment Failure |
| Trigger Events | Which events activate this policy | release.failed, agent.disconnected |
| Escalation Tiers | Ordered list of notification targets | Tier 1: SRE on-call, Tier 2: SRE lead, Tier 3: VP Engineering |
| Timeout | Time before escalating to next tier | 10 minutes |
| Repeat | Whether to re-notify if still unacknowledged | Yes, every 30 minutes |
Alert Fatigue Reduction
Escalation policies include built-in protections against alert fatigue:
- Aggregation: Multiple similar events within a time window are grouped into a single notification
- Deduplication: Identical events are suppressed after the first notification
- Cooldown: After an event is acknowledged, similar events within a cooldown period are suppressed
- Business hours: Different escalation rules for business hours vs. off-hours
Audit Log
The audit log at /audit provides a complete, immutable record of all organizational activity.
Delivery and Organizational Audit
Tracks all operational events in the organization:
| Event Type | Examples |
|---|---|
| Deployment | Release created, approved, deployed, failed, rolled back |
| Configuration | Project settings changed, environment updated, variable modified |
| Delivery | Delivery generated, preview executed, bindings changed |
| Cluster | Agent registered, sync triggered, manifests applied |
| Git | Commits created, files edited, delivery output written |
| Promotion | Promotion requested, approved, executed |
| Webhook | Webhook created, updated, delivery attempted |
IAM Audit
Tracks all identity and access management events:
| Event Type | Examples |
|---|---|
| Member Management | User invited, role changed, removed |
| Service Accounts | Created, updated, key rotated, deleted |
| API Keys | Created, scoped, revoked |
| Break-Glass | Session started, action performed, session ended |
| Role Changes | Role assigned, permission granted, scope modified |
| Authentication | Login, logout, MFA changes |
Audit Entry Structure
Every audit entry contains:
The AuditLog model contains these fields:
| Field | Description | Example |
|---|---|---|
| id | Unique audit entry identifier | uuid |
| projectId | Project associated with this event | project-uuid |
| tenantId | Tenant associated with this event (if applicable) | tenant-uuid |
| userId | User who performed the action | user-uuid |
| actorType | Category of the actor | user, service_account, system |
| action | What was done | release.deploy, image.update |
| details | Action-specific metadata (JSON) | {"environment": "production", "previousVersion": "2.0.5"} |
| gitCommitSha | Associated Git commit SHA (if applicable) | abc123def456 |
| createdAt | Precise time of the event (UTC) | 2026-03-27T10:30:15.123Z |
Filtering and Search
The audit explorer provides powerful filtering:
| Filter | Description |
|---|---|
| Date Range | Start and end timestamps |
| Event Type | Filter by action category |
| Actor | Filter by specific user or service account |
| Actor Type | Filter by user, service_account, or system |
| Resource | Filter by resource type and ID |
| Project | Scope to a specific project |
| Status | Filter by success, failure, or blocked |
Audit for Compliance
Compliance-Ready
GitOpsHQ's audit system is designed to meet SOC 2, ISO 27001, and similar compliance frameworks that require comprehensive change tracking and access logging.
Immutability
Audit records are immutable — they cannot be modified, deleted, or tampered with. This provides a trustworthy evidence chain for compliance auditors.
Traceability
Every change in the system is traceable to an actor:
- User actions: Linked to the authenticated user identity
- Service account actions: Linked to the service account and its creator
- System actions: Labeled as
systemwith the originating subsystem identified - Break-glass actions: Prominently flagged with the break-glass session ID and justification
API Access
Audit logs are available via GET /api/v2/audit-logs, which returns paginated JSON. There is no CSV or JSON file export endpoint. For external compliance reporting, consume the paginated API and aggregate externally.
Correlating Events
Related events can be correlated by filtering on shared fields such as projectId, tenantId, and gitCommitSha. For example, a production deployment creates a chain of events:
Events in this chain share the same projectId and gitCommitSha, making it possible to trace the full lifecycle during incident investigation.
Troubleshooting
| Error | Meaning | Resolution |
|---|---|---|
400 missing organization context | Request missing org context | Ensure you're authenticated in an active organization |
400 project not found in active organization | Invalid project scope in preference | Verify the project belongs to your current organization |
404 notification policy not found | Stale policy reference | Refresh the policy list; the policy may have been deleted |
500 failed to list | Backend fetch error | Retry the request; if persistent, check system status |
Best Practices
- Configure notifications for production events — Production deployments and failures should always notify the responsible team
- Use escalation policies for critical alerts — Never rely on a single person receiving an alert
- Review audit logs during post-mortems — The audit trail provides the timeline needed for incident analysis
- Consume the audit API regularly — Use
GET /api/v2/audit-logsto pull logs into external systems for compliance retention - Set up digest mode for noisy environments — Avoid alert fatigue by batching low-priority notifications
- Scope preferences to projects — Only receive notifications for projects you actively work on
- Correlate events using shared fields — Filter by
projectId,tenantId, orgitCommitShato trace the full context of any event - Monitor break-glass events — These should be rare and always warrant review