/aws-observability

Enable AWS Application Signals for Java on EC2

Your task is to modify Infrastructure as Code (IaC) files to enable AWS Application Signals for a Java application running on EC2 instances. You will update IAM permissions, install monitoring agents, and configure OpenTelemetry instrumentation through UserData scripts.

What You Will Accomplish

After completing this task:

• The EC2 instance will have permissions to send telemetry data to CloudWatch
• The CloudWatch Agent will be installed and configured for Application Signals
• The Java application will be automatically instrumented with AWS Distro for OpenTelemetry (ADOT)
• Traces, metrics, and performance data will appear in the CloudWatch Application Signals console

Critical Requirements

Error Handling:

• If you cannot determine required values from the IaC, STOP and ask the user
• For multiple EC2 instances, ask which one(s) to modify
• Preserve all existing UserData commands; add new ones in sequence

Do NOT:

• Run deployment commands automatically (cdk deploy, terraform apply, etc.)
• Remove existing application startup logic
• Skip the user approval step before deployment

IaC Tool Support

Code examples use CDK TypeScript syntax. If you are working with Terraform or CloudFormation, translate the CDK syntax to the appropriate format while keeping all bash commands identical.

Before You Start: Gather Required Information

Step 1: Determine Deployment Type

Read the UserData script and look for the application startup command.

If you see:

• docker run or docker start → Docker deployment
• java -jar, mvn spring-boot:run, gradle bootRun, or similar → Non-Docker deployment

If unclear:

• Ask the user: "Is your Java application running in a Docker container or directly on the EC2 instance?" DO NOT GUESS

Step 2: Extract Placeholder Values

• {{SERVICE_NAME}}
  • Why It Matters: Sets the service name displayed in Application Signals console via OTEL_RESOURCE_ATTRIBUTES=service.name={{SERVICE_NAME}}
  • How to Find It: Use the application name, stack name, or construct ID.
  • Example Value: my-java-app
  • Required For: Both Docker and non-Docker

For Docker-based deployments:

• {{PORT}} - Docker port mapping. Example: 8080
• {{APP_NAME}} - Container name. Example: java-springboot-app
• {{IMAGE_URI}} - Docker image. Example: 123456789012.dkr.ecr.us-east-1.amazonaws.com/my-app:latest

Step 3: Identify Instance OS

• Amazon Linux 2: Use yum package manager
• Amazon Linux 2023: Use dnf package manager
• Ubuntu/Debian: Use apt package manager

Instructions

Step 1: Locate the IaC Files

Search for EC2 instance definitions using these patterns:

CDK: new ec2.Instance(, CfnInstance( Terraform: resource "aws_instance" CloudFormation: AWS::EC2::Instance

Step 2: Locate the IAM Role

Find the IAM role attached to the EC2 instance.

Step 3: Update the IAM Role

What this policy is for, and its scope. CloudWatchAgentServerPolicy goes on the instance role for the CloudWatch agent, which receives telemetry locally and forwards it to CloudWatch and X-Ray — so these permissions are what let the agent reach those destinations, not something the instrumentation itself needs. On EC2, ECS, and EKS, an ADOT-SDK-only setup adds no IAM to the workload at all (see the setting-up-cloudwatch-observability skill's references/cloudwatch-omni/instrumentation/instrumentation.md, which forbids attaching this policy on that path). Lambda is the exception — that path does grant its execution role X-Ray write permissions.

The policy is broader than this configuration needs: it grants 14 actions, all on Resource: "*", including ec2:DescribeVolumes and logs:PutRetentionPolicy, which an Application-Signals-only agent config does not use. If the customer wants to trim it, resource scoping is the more valuable axis than pruning actions — dropping actions still leaves logs:PutLogEvents and logs:CreateLogGroup on every log group in the account and cloudwatch:PutMetricData on every namespace, so the instance role can still write over unrelated services' logs. Scope the resources and add an aws:ResourceAccount condition, the way CloudWatchLambdaApplicationSignalsExecutionRolePolicy does for the Lambda path.

Read the live document before changing anything — it takes two calls, since the version id is required and is not knowable up front: aws iam get-policy --policy-arn arn:aws:iam::aws:policy/CloudWatchAgentServerPolicy --query Policy.DefaultVersionId then aws iam get-policy-version --policy-arn <same> --version-id <that>. Do not hand-roll an action list from this page: the agent still creates the Application Signals log group (/aws/application-signals/data), so logs:CreateLogGroup and logs:CreateLogStream must survive any trim. A denial does not surface in the console — the agent records AccessDenied in its own log (/opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log on Linux), and the symptom is that telemetry never starts arriving. Check that file first.

Add the CloudWatch Agent Server Policy to the IAM role's managed policies.

CDK:

const role = new iam.Role(this, 'AppRole', {
  assumedBy: new iam.ServicePrincipal('ec2.amazonaws.com'),
  managedPolicies: [
    iam.ManagedPolicy.fromAwsManagedPolicyName('CloudWatchAgentServerPolicy'),
    // ... keep existing policies
  ],
});

Step 4: Modify UserData - Add Prerequisites

CRITICAL for Terraform Users: Preserve the EXACT indentation of existing heredoc lines.

CDK TypeScript example:

instance.userData.addCommands(
  'dnf install -y amazon-cloudwatch-agent',  // Use dnf for AL2023, yum for AL2
);

Step 5: Modify UserData - Configure CloudWatch Agent

instance.userData.addCommands(
  '# Create CloudWatch Agent configuration for Application Signals',
  "cat > /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json << 'EOF'",
  '{',
  '  "traces": {',
  '    "traces_collected": {',
  '      "application_signals": {}',
  '    }',
  '  },',
  '  "logs": {',
  '    "metrics_collected": {',
  '      "application_signals": {}',
  '    }',
  '  }',
  '}',
  'EOF',
  '',
  '# Start CloudWatch Agent with Application Signals configuration',
  '/opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl \\',
  '  -a fetch-config \\',
  '  -m ec2 \\',
  '  -s \\',
  '  -c file:/opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json',
);

Step 6: Install ADOT Java Auto-Instrumentation SDK

Option A: Docker Deployment - Modify Dockerfile

Add these lines to download the ADOT Java agent JAR file BEFORE the CMD line:

# Downloads latest release. ServiceEvents requires aws-opentelemetry-agent>=2.28.2.
RUN curl -Lo /opt/aws-opentelemetry-agent.jar \
    https://github.com/aws-observability/aws-otel-java-instrumentation/releases/latest/download/aws-opentelemetry-agent.jar

Option B: Non-Docker Deployment - Modify UserData

instance.userData.addCommands(
  '# Download ADOT Java agent (latest; ServiceEvents requires >=2.28.2)',
  'curl -Lo /opt/aws-opentelemetry-agent.jar \\',
  '  https://github.com/aws-observability/aws-otel-java-instrumentation/releases/latest/download/aws-opentelemetry-agent.jar',
);

Step 7: Modify UserData - Configure Application

Option A: Docker Deployment

Container networking — match the customer's existing setup (minimal change). The example below uses --network host with localhost:4316 endpoints. That pairing is one option, not a hard requirement — the right choice depends on how the container already reaches the host-installed CloudWatch Agent. Don't change the customer's networking model just to instrument; instead pick the variant that fits theirs:

• Already using --network host (or willing to): keep it, and the localhost:4316 / localhost:2000 endpoints in the example work as-is. Trade-off: host networking shares the host's network namespace (no container isolation), though the agent's ports can stay bound to loopback, unreachable off-host. For production, it is recommended to restrict the OTLP 4316 / proxy 2000 ports via EC2 security groups / host firewall and to avoid co-locating untrusted containers; this guide does not apply those controls, so assess and configure them for your environment.
• Using a bridge/default network: don't add --network host. Point the endpoints at the host instead — host.docker.internal:4316/:2000 (add --add-host=host.docker.internal:host-gateway on Linux) or the bridge gateway IP. This requires the CloudWatch Agent to listen on a non-loopback address, so it is recommended to restrict those ports with security groups / host firewall.
• Option 2 — CloudWatch Agent as a sidecar container (most isolated): run the agent as another container on the same user-defined Docker network and target it by name (e.g. cwagent:4316). Nothing binds to host interfaces. This is the same model the ECS guides use; choose it if the customer prefers full container isolation over a host-installed agent.

--network host example — adapt per the networking variant you chose above:

instance.userData.addCommands(
  '# Run container with Application Signals environment variables',
  `docker run -d --name {{APP_NAME}} \\`,
  `  -e JAVA_TOOL_OPTIONS=-javaagent:/opt/aws-opentelemetry-agent.jar \\`,
  `  -e OTEL_METRICS_EXPORTER=none \\`,
  `  -e OTEL_LOGS_EXPORTER=none \\`,
  `  -e OTEL_AWS_APPLICATION_SIGNALS_ENABLED=true \\`,
  `  -e OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf \\`,
  `  -e OTEL_AWS_APPLICATION_SIGNALS_EXPORTER_ENDPOINT=http://localhost:4316/v1/metrics \\`,
  `  -e OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4316/v1/traces \\`,
  `  -e OTEL_RESOURCE_ATTRIBUTES=service.name={{SERVICE_NAME}} \\`,
  `  --network host \\`,
  `  {{IMAGE_URI}}`,
);

Option B: Non-Docker Deployment

instance.userData.addCommands(
  '# Set OpenTelemetry environment variables',
  'export JAVA_TOOL_OPTIONS=-javaagent:/opt/aws-opentelemetry-agent.jar',
  'export OTEL_METRICS_EXPORTER=none',
  'export OTEL_LOGS_EXPORTER=none',
  'export OTEL_AWS_APPLICATION_SIGNALS_ENABLED=true',
  'export OTEL_EXPORTER_OTLP_PROTOCOL=http/protobuf',
  'export OTEL_AWS_APPLICATION_SIGNALS_EXPORTER_ENDPOINT=http://localhost:4316/v1/metrics',
  'export OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=http://localhost:4316/v1/traces',
  'export OTEL_RESOURCE_ATTRIBUTES=service.name={{SERVICE_NAME}}',
  '',
  '# Start application (existing command remains unchanged)',
  '# The JAVA_TOOL_OPTIONS will automatically attach the agent',
);

Completion

Tell the user:

"I've completed the Application Signals enablement for your Java application. Here's what I modified:

Files Changed:

• IAM role: Added CloudWatchAgentServerPolicy
• UserData: Installed and configured CloudWatch Agent
• UserData: Downloaded ADOT Java agent JAR
• UserData/Service file: Added OpenTelemetry environment variables (JAVA_TOOL_OPTIONS)
• Dockerfile: Downloaded ADOT Java agent JAR (if using Docker)

Next Steps:

1. Ensure that Application Signals is enabled in AWS account.
2. Review the changes I made using git diff
3. Deploy your infrastructure:
   • For CDK: cdk deploy
   • For Terraform: terraform apply
   • For CloudFormation: Deploy your stack
4. After deployment, wait 5-10 minutes for telemetry data to start flowing

Verification: Once deployed, you can verify Application Signals is working by:

• Opening the AWS CloudWatch Console
• Navigating to Application Signals → Services
• Looking for your service (named: {{SERVICE_NAME}})
• Checking that traces and metrics are being collected

Monitor Application Health: After enablement, you can monitor your application's operational health using Application Signals dashboards. For more information, see Monitor the operational health of your applications with Application Signals.

Troubleshooting If you encounter any other issues, refer to the CloudWatch APM troubleshooting guide.

Let me know if you'd like me to make any adjustments before you deploy!"