CloudWatch Alarms

Configure and manage CloudWatch alarms including metric, composite, and anomaly detection types with evaluation mechanics and recommended defaults.

Alarm types
Missing data treatment
Evaluation mechanics
Composite alarms
Anomaly detection
Recommended defaults
Common mistakes
CDK patterns

Alarm types

Metric Alarm

Watches a single metric or metric math expression.

States: OK, ALARM, INSUFFICIENT_DATA
Actions: SNS, EC2 (stop/terminate/reboot/recover), Auto Scaling, Lambda, SSM OpsItems, SSM Incident Manager, CloudWatch Investigations
M-of-N evaluation: DatapointsToAlarm (M) out of EvaluationPeriods (N)
Rate limit: PutMetricAlarm = 3 TPS (adjustable)

put-metric-alarm is an UPSERT, keyed on --alarm-name. Reusing the name of an existing alarm silently overwrites its whole configuration — threshold, evaluation periods, and its actions, including the SNS topics it notifies. There is no error and no previous version to restore, so an alarm someone depends on can stop paging with nothing to indicate it changed. Before writing to a name you did not just create, check whether it exists (describe-alarms --alarm-names <name>); if it does, show the customer the diff and confirm rather than overwriting on inference.

Composite Alarm

Combines states of other alarms with Boolean logic.

Rule operators: AND, OR, NOT, AT_LEAST(M, STATE, (alarms...))
AT_LEAST supports percentages: AT_LEAST(50%, ALARM, (a1, a2, a3))
Actions: SNS, Lambda, SSM — cannot perform EC2 or Auto Scaling actions
Limits: max 100 underlying alarms per composite, 150 composites per underlying, 500 rule elements
Composite and all underlying alarms must be in the same account and Region
Action suppression: ActionsSuppressor alarm can suppress composite alarm actions during known events (deployments, maintenance)

PromQL Alarm (OpenTelemetry metrics)

Monitors OTel metrics using PromQL instant queries with duration-based pending/recovery periods. Use for metrics sent via OTLP (150 labels, 30-day retention).

Missing data treatment

Four options — the most misunderstood CloudWatch feature.

Value	Behavior	Use when
`missing` (DEFAULT)	All missing → INSUFFICIENT_DATA	EC2 stop/terminate/reboot actions
`notBreaching`	Missing = within threshold	Error-count metrics (absence = no errors)
`breaching`	Missing = violating threshold	Heartbeat/health-check metrics
`ignore`	Maintain current state	DynamoDB metrics (service overrides default to `ignore`)

Note: The CloudWatch console defaults DynamoDB alarms to ignore instead of the usual missing. The API stores whatever you specify.

Premature alarm transitions

With treatMissingData=missing, the pattern M, M, B, M, M can trigger ALARM even with only 1 breaching datapoint. CloudWatch goes to ALARM when the oldest available breaching datapoint is at least as old as datapointsToAlarm and all more recent points are breaching or missing.

Fix: For non-sparse metrics, explicitly set notBreaching or breaching — don't rely on the default.

Evaluation mechanics

Three core settings

Period — seconds per data point aggregation (valid: 10, 20, 30, or any multiple of 60)
Evaluation Periods (N) — number of most recent periods to evaluate
Datapoints to Alarm (M) — how many of N must breach

Evaluation frequency

Period ≥ 1 min → evaluated every minute
Period = 10s/20s/30s → evaluated every 10 seconds
If EvaluationPeriods × Period > 1 day → evaluated once per hour

Evaluation Range

CloudWatch fetches more data points than the configured Evaluation Periods — the actual lookback window is wider than expected.

Example: Alarm with 1-day period, 1 evaluation period, treatMissingData=breaching:

You expect it to fire after 1 day of no data
CloudWatch actually looks back ~3 days before firing
Dead man switch alarms fire later than expected due to hourly evaluation

Evaluation period quotas

Period ≥ 1 hour → max evaluation window: 7 days
Period < 1 hour → max evaluation window: 1 day

Composite alarms

When to use

Reduce alert fatigue: only page when BOTH high CPU AND high error rate
Service-level health: aggregate per-resource alarms into one service alarm
Suppress during deployments: use ActionsSuppressor to mute during known events

Rule expression syntax

ALARM("error-rate-alarm") AND ALARM("latency-alarm")
ALARM("error-rate-alarm") OR ALARM("throttle-alarm")
NOT ALARM("maintenance-window")
AT_LEAST(2, ALARM, (a1, a2, a3))
AT_LEAST(50%, ALARM, (a1, a2, a3, a4))

Limitations

Cannot perform EC2 actions (stop, terminate, reboot, recover)
Cannot perform Auto Scaling actions
Composite and all underlying alarms must be in the same account and Region (underlying alarms must be same account + Region; monitoring accounts via OAM can watch source account metrics)
Cross-account observability monitoring account CAN watch source account alarms

Anomaly detection

Uses ANOMALY_DETECTION_BAND function as threshold
Band width = anomaly detection threshold value (configurable; higher value = thicker band of expected values)
Trains on up to 2 weeks of metric data (works with less, accuracy improves over time)
Cost: Higher than a regular alarm — see CloudWatch pricing for current anomaly detection alarm rates
Rate limit: 1,000 ANOMALY_DETECTION_BAND usages in GetMetricData per second
Use when: baselines are unknown, workloads are seasonal/variable

Recommended defaults

Parameter	Common mistake	Recommendation
`evaluationPeriods`	1	3–5
`datapointsToAlarm`	1	2–3 (M-of-N)
`treatMissingData`	`missing`	Explicitly choose based on metric type
`period`	300s (5 min)	60s (1 min) for faster detection
Error rate threshold	1%	5% (then tune down with data)
Latency threshold	1s	P99 of baseline + 2× (data-driven)

WARNING: Never use Average for duration/latency alarms. Average hides tail latency — use p99 or p90. A function averaging 100ms but with p99 at 5s has a serious problem that Average won't catch.

Common mistakes

M=N=1 with 1-minute periods — Too sensitive. The most recent datapoint may not have full information. Use "1 out of 2" or "1 out of 3" minimum.
Relying on default missing treatment — Explicitly configure for your metric type. Error metrics should use notBreaching. Health checks should use breaching.
Not understanding Evaluation Range — Alarms look back further than configured. Dead man switches with multi-day periods are evaluated once per hour, causing significant delay.
Metric math alarms for EC2 actions — Alarms based on metric math expressions cannot perform EC2 actions (stop, terminate, reboot, recover). Use a simple metric alarm instead.
High-resolution alarms without need — 10-second evaluation costs more. Each metric in a math expression is billed separately.
Using Average statistic for duration/latency alarms — Average hides tail latency. A function averaging 100ms with p99 at 5s has a serious problem Average won't catch. Always use p99 or p90 via --extended-statistic p99.
Ignoring DynamoDB's default override — DynamoDB alarms default to ignore for missing data, not the global missing.
Alarms on INSUFFICIENT_DATA state — Alarms invoke actions only on state changes, except Auto Scaling actions which continue invoking while in the new state.

CDK patterns

Error rate alarm (production pattern)

Note: Alarm on error rate (percentage via math expression), not raw error count. Raw counts trigger on a single error even during 10,000 successful invocations.

For CLI:

aws cloudwatch put-metric-alarm --alarm-name MyFunc-ErrorRate \
  --metrics '[
    {"Id":"errors","MetricStat":{"Metric":{"Namespace":"AWS/Lambda","MetricName":"Errors","Dimensions":[{"Name":"FunctionName","Value":"MyFunc"}]},"Period":60,"Stat":"Sum"},"ReturnData":false},
    {"Id":"invocations","MetricStat":{"Metric":{"Namespace":"AWS/Lambda","MetricName":"Invocations","Dimensions":[{"Name":"FunctionName","Value":"MyFunc"}]},"Period":60,"Stat":"Sum"},"ReturnData":false},
    {"Id":"error_rate","Expression":"IF(invocations > 0, errors * 100 / invocations, 0)","Label":"Error Rate %"}
  ]' \
  --threshold 5 --comparison-operator GreaterThanThreshold \
  --evaluation-periods 3 --datapoints-to-alarm 2 \
  --treat-missing-data notBreaching

For CDK:

import { Alarm, ComparisonOperator, MathExpression, TreatMissingData } from 'aws-cdk-lib/aws-cloudwatch';
import { Duration } from 'aws-cdk-lib';

const errorRateAlarm = new Alarm(this, 'ErrorRateAlarm', {
  metric: new MathExpression({
    expression: 'IF(invocations > 0, errors * 100 / invocations, 0)',
    usingMetrics: {
      errors: fn.metricErrors({ period: Duration.minutes(1) }),
      invocations: fn.metricInvocations({ period: Duration.minutes(1) }),
    },
  }),
  threshold: 5,
  evaluationPeriods: 3,
  datapointsToAlarm: 2,
  comparisonOperator: ComparisonOperator.GREATER_THAN_THRESHOLD,
  treatMissingData: TreatMissingData.NOT_BREACHING,
});

Duration/latency alarm (use p99, never Average)

const durationAlarm = new Alarm(this, 'DurationP99Alarm', {
  metric: fn.metricDuration({ statistic: 'p99', period: Duration.minutes(1) }),
  threshold: 3000, // 3 seconds
  evaluationPeriods: 3,
  datapointsToAlarm: 2,
  comparisonOperator: ComparisonOperator.GREATER_THAN_THRESHOLD,
  treatMissingData: TreatMissingData.NOT_BREACHING,
});