All skills
aws avatar

/aws-observability

@bb272a8

Builds, configures, debugs, and optimizes AWS observability — operator-symptom questions and detecting Omni vs classic CloudWatch. CloudWatch: Log Insights, alarms, Dynamic Instrumentation, and Application Signals — instrumenting/onboarding a service to Application Signals with ADOT on EC2/ECS/EKS/Lambda: auto-instrumentation, monitored service, reporting telemetry, ServiceEvents, CI/CD metadata, Terraform/manifest. Also fleet health views. CloudWatch Omni on an existing Space: SQL over logs and traces, PromQL over metrics, Omni dashboards, Omni alerts, context graph for root cause, programmatic/IaC access (API/SDK/CLI/CloudFormation) and driving Omni from a coding agent or skills, and evaluating AI agent quality from traces — on-demand and continuous online scoring of live agent traffic, readback, and custom trace evaluators. For first-time Omni setup — creating a Space, granting access, ingestion, or ADOT instrumentation — use setting-up-cloudwatch-observability. Not for app logging or threat detection.

Use this Skill: https://skilld.dev/gh/aws/agent-toolkit-for-aws/aws-observability

This session only. Nothing lands on disk.

referencescloudwatchappsignals-guideseks-nodejs.md

≈1.4k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Enable AWS Application Signals for Node.js Applications on Amazon EKS

This guide shows how to modify existing CDK and Terraform infrastructure code to enable AWS Application Signals for Node.js applications running on Amazon EKS.

Prerequisites

  • Application Signals enabled in your AWS account (see Enable Application Signals in your account)
  • Existing EKS cluster deployed using CDK or Terraform code
  • Node.js application containerized and pushed to ECR
  • AWS CLI configured with appropriate permissions

Critical Requirements

Do NOT:

  • Run deployment commands automatically (cdk deploy, terraform apply, etc.)
  • Remove existing application startup logic
  • Skip the user approval step before deployment

CDK Implementation

1. Install CloudWatch Observability Add-on

import * as eks from 'aws-cdk-lib/aws-eks';
import * as iam from 'aws-cdk-lib/aws-iam';

const cloudwatchRole = new iam.Role(this, 'CloudWatchAgentAddOnRole', {
  assumedBy: new iam.OpenIdConnectPrincipal(cluster.openIdConnectProvider),
  managedPolicies: [
    iam.ManagedPolicy.fromAwsManagedPolicyName('CloudWatchAgentServerPolicy')
  ],
});

new eks.CfnAddon(this, 'CloudWatchAddon', {
  addonName: 'amazon-cloudwatch-observability',
  clusterName: cluster.clusterName,
  serviceAccountRoleArn: cloudwatchRole.roleArn
});

2. Add Node.js Instrumentation Annotation

template: {
  metadata: {
    labels: { app: config.appName },
    annotations: {
      'instrumentation.opentelemetry.io/inject-nodejs': 'true'
    }
  },
}

Terraform Implementation

1. Add CloudWatch Agent IAM Permissions

Neither path in this guide confines CloudWatchAgentServerPolicy to the agent. Attaching to node_role grants it to every pod scheduled on those nodes, since any pod can reach the node's instance credentials unless IMDS access is blocked. The CDK path above uses an IRSA-style role instead of the node role, but as written its trust policy has no :sub condition — new iam.OpenIdConnectPrincipal(cluster.openIdConnectProvider) with no conditions lets any pod holding a projected service-account token call AssumeRoleWithWebIdentity on it. To actually scope it, add a :sub condition naming the addon's service accounts (the addon installs two — cloudwatch-agent and cloudwatch-agent-cluster-scraper) plus :aud sts.amazonaws.com. Tell the customer which of the three states the change leaves them in.

resource "aws_iam_role_policy_attachment" "cloudwatch_agent_policy" {
  policy_arn = "arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy"
  role       = aws_iam_role.node_role.name
}

Add to node group's depends_on:

resource "aws_eks_node_group" "app_nodes" {
  depends_on = [
    aws_iam_role_policy_attachment.node_policy,
    aws_iam_role_policy_attachment.cloudwatch_agent_policy
  ]
}

2. Install CloudWatch Observability Add-on

resource "aws_eks_addon" "cloudwatch_observability" {
  cluster_name = aws_eks_cluster.app_cluster.name
  addon_name   = "amazon-cloudwatch-observability"

  depends_on = [
    aws_eks_node_group.app_nodes
  ]
}

3. Add Node.js Instrumentation Annotation

template {
  metadata {
    labels = {
      app = var.app_name
    }
    annotations = {
      "instrumentation.opentelemetry.io/inject-nodejs" = "true"
    }
  }
}

Important Notes

  • The Node.js instrumentation annotation will cause pods to restart automatically
  • For Node.js applications with ESM module format, see special configuration requirements in the AWS documentation
  • It may take a few minutes for data to appear in the Application Signals console after deployment

Completion

Before reciting the summary below (guidance for you, not for the user): say which role the policy actually landed on, because the two paths differ in blast radius. The CDK path puts it on an IRSA-style role; the Terraform path attaches it to the node role, which extends it to every pod scheduled on those nodes. Also state that the CDK role's trust policy has no :sub condition unless one was added, so as written neither path confines the policy to the agent. The summary bullet has a your-node-role slot — fill it in with the role this change actually used before reciting.

Tell the user:

"I've completed the Application Signals enablement for your Node.js application. Here's what I modified:

Files Changed:

  • IAM role: Added CloudWatchAgentServerPolicy to your-node-role
  • CloudWatch Observability EKS add-on: Added to the EKS Cluster
  • Kubernetes Deployment: Instrumentation annotation added with inject-nodejs set to true

Next Steps:

  1. Ensure that Application Signals is enabled in AWS account.
  2. Review the changes using git diff
  3. Deploy your infrastructure
  4. After deployment, wait 5-10 minutes for telemetry data to start flowing

Verification:

  • Open AWS CloudWatch Console → Application Signals → Services

Troubleshooting Refer to the CloudWatch APM troubleshooting guide.

Let me know if you'd like me to make any adjustments before you deploy!"

Source: SKILL.md on GitHub

1 warning6d3 checks · Risk SAFE
  • Gen Agent Trust Hub6d

    This skill provides comprehensive capabilities for AWS observability and debugging, including agent evaluation and dynamic instrumentation. It includes some security considerations, such as the processing of untrusted telemetry data and the use of external scripts for service onboarding, which are handled with a focus on user confirmation and best practices.

  • Socket6d

    2 alerts: gptAnomaly

  • Snyk6d

    Risk: LOW · No issues

Signed by skilld at bb272a8. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated last week
metadata
{
  "version": "6"
}

README badge

README badge for aws/agent-toolkit-for-aws/aws-observability