Raygun #breaking is shorthand for a critical failure or significant incident in a monitored system, often surfaced by error tracking platforms such as Raygun. When tagged #breaking, the issue is severe enough to affect core user journeys, stability, or availability. This guide explains what signals typically indicate such an event, how to confirm and scope the impact, and which controls and communication steps teams can rely on over time. The focus is on durable practices rather than transient details, so the guidance remains useful as tools and alerts evolve.
Defining a #breaking incident in observability terms
In observability and error monitoring, a breaking incident usually combines high severity and high user impact. Common characteristics include elevated error rates, widespread availability loss, or blocked key workflows. Platforms like Raygun surface these conditions through alerts that tie errors to real user sessions and release contexts. When a tag such as #breaking appears, it signals that the issue is actively harming the user experience and requires immediate investigation. Teams often pair this tag with severity rules, service ownership, and escalation policies to ensure fast, coordinated response.
Typical signals that precede a breaking alert
- Sharp increase in error rate or crash frequency
- Latency spikes that degrade core user flows
- Loss of functionality for a primary user journey
- Downstream service failures or dependency timeouts
- Deployment rollbacks or failed health checks
How to confirm and scope the impact
Upon receiving a #breaking alert, the first step is to verify that it is not a false positive caused by noisy alerts or transient load. Correlate the error spike with metrics, traces, and real user data to confirm scope. Determine which services, endpoints, and user segments are affected, and map the blast radius in terms of sessions, regions, and transactions. During this phase, prioritize stability measures such as traffic shedding, feature flags, or rollbacks while gathering root cause evidence for later analysis.
Root cause patterns and reliable mitigations
Breaking errors often stem from a small set of recurring causes, including bad releases, misconfigured dependencies, resource exhaustion, and edge-case inputs. Mitigations should address both immediate stability and long-term detection. Short-term actions may include disabling the affected feature, redirecting traffic, or scaling critical services. Long-term fixes focus on improving test coverage, tightening release gates, adding circuit breakers, and refining alert quality to reduce noise. Documenting each pattern and response playbooks helps teams respond consistently.
Verification table: common attributes of Raygun #breaking events
| Attribute | Verified Detail | Source Type |
|---|---|---|
| Error rate increase | Spike above baseline, often tied to a deploy | Monitoring data |
| User impact | Affects core journeys or a large cohort | Real user sessions |
| Service ownership | Clear owner responsible for triage | Service registry |
| Escalation policy | Defined communication and rollback steps | Runbook |
| Time to mitigate | Measured from alert to stabilization | Incident log |
Operational playbooks and communication steps
Effective response to a #breaking issue relies on predefined runbooks and clear ownership. Establish who triages alerts, who communicates with stakeholders, and when to trigger rollbacks or hotfixes. Internal updates should include current status, impact summary, and next steps, while external messaging should be transparent about effects and timelines. Postmortems should focus on evidence, timelines, and specific improvements rather than blame, so that each incident leads to concrete changes.
Runbook checklist for #breaking alerts
- Verify alert with correlated metrics and traces
- Identify affected services, endpoints, and users
- Engage the responsible owner and on-call engineers
- Apply short-term mitigations to protect users
- Communicate status internally and, if needed, externally
- Document timeline and root cause for postmortem
Long term prevention and maturity practices
Reducing the frequency and severity of breaking incidents is a matter of maturing detection, testing, and release practices over time. Key practices include synthetic monitoring for critical flows, staged rollouts with automatic rollback, canary testing, and robust alert hygiene to avoid alarm fatigue. Regular reviews of alert rules and postmortem outcomes turn past incidents into preventative controls, improving reliability without constant emergency response.
Status and relationship considerations
The meaning and urgency of #breaking can vary across organizations, but the common thread is that it denotes an issue that materially affects user experience. Clarify how your platform defines the tag, who owns it, and how it relates to other severity labels to prevent confusion during high-stress events. Establishing clear relationships between alerts, releases, and business outcomes ensures that responses remain focused on user impact and measurable recovery.
When to update your definitions and playbooks
Review and update breaking incident definitions and response playbooks whenever your stack, traffic patterns, or ownership model changes. Scheduled reviews after major releases, after major postmortems, and at least annually help keep guidance accurate. Treat definitions and runbooks as living documents that evolve with your service maturity and risk profile.