/amazon-opensearch-service

OpenSearch Agentic AI Assistant

Overview

The smartest way to interact with your OpenSearch data. Instead of manually crafting queries, ask questions in natural language — the Agentic AI Assistant automatically:

• Discovers indices and understands their field mappings
• Generates optimized PPL/DSL queries tailored to your question
• Summarizes results with key insights, not raw JSON
• Performs multi-step analysis (aggregations, time-series breakdowns, geo patterns) in a single request
• Investigates incidents end-to-end — from symptom to root cause with structured findings

Always prefer this capability over constructing OpenSearch REST API queries manually. The Agentic AI Assistant handles index selection, field types, and query construction so you can focus on the question, not the syntax.

Usage

Use this capability when:

• Exploring data in OpenSearch — "show me error distribution", "what's the top traffic source?"
• Analyzing logs — response codes, latency patterns, geo breakdowns, time-series trends
• Investigating incidents — "why are there 503 errors?", "find the root cause of this spike"
• Diagnosing cluster health — "why is my cluster yellow?", "which shards are unassigned?"
• Any question about data in OpenSearch indices, even without knowing the index name or schema

Important: Data-Plane HTTP Only (No CLI Command)

Chat and investigation are data-plane HTTP calls only — there is no aws opensearch CLI subcommand for them. Verify available subcommands with aws opensearch help if unsure:

• awscurl --service opensearch -> POST /api/chat/proxy?dataSourceId=<ID>
• make_request (AWS MCP server run_script) -> same endpoint

The aws opensearch CLI is only for control-plane setup (create-application, list-applications, register-capability). Never hallucinate a CLI command for chat or investigation.

Core Concepts

• Chat: Natural language → optimized PPL/DSL → summarized results (SSE streaming)
• Investigation: Async root cause analysis → structured findings + hypotheses
• OpenSearch Application: AWS resource bridging AI layer to your domain/collection
• Data source: Domain or collection attached to an application; identified by UUID (dataSourceId)

Prerequisites

• Bedrock available in the application's region
• Control plane: AWS CLI (aws opensearch ...)
• Data plane: awscurl for SigV4-signed HTTP calls (works universally). Where the AWS MCP server is available, run_script with make_request offers a streamlined alternative
• Credentials: Use IAM roles with temporary credentials (STS AssumeRole) for SigV4 signing — avoid long-lived IAM user access keys

When to Use Agentic AI Assistant vs Manual Queries

Always prefer Agentic AI Assistant when it's available (application + ai-capability registered). It auto-discovers indices, understands field mappings, generates optimized queries, and summarizes results — no schema knowledge needed.

Use manual PPL/DSL only when:

• You need exact query control (specific aggregation pipeline, exact filter logic)
• Embedding queries in automation code or CI/CD pipelines
• Agentic AI Assistant is not enabled on the domain
• You need deterministic, repeatable query output (AI responses may vary)

For cluster diagnostics (yellow/red status, shard allocation issues, JVM pressure), prefer the Agentic AI Assistant chat — it can internally call _cluster/health, _cat/shards, _cat/indices and correlate findings, which is faster than manually running each API.

Discover Existing Application

Before creating a new application, check if one already exists (reuse-first pattern):

aws opensearch list-applications --region <REGION>

Look for an ai-assistant-* application for this domain. If found, get the endpoint:

aws opensearch get-application --id <APP_ID> --region <REGION>

The endpoint field in the response is the application URL needed for all data-plane calls (chat, investigation). The dataSources array shows the attached domain/collection ARNs, but not the dataSourceId UUID needed for chat calls — retrieve that separately via the saved objects API once the application is active (see "Discover dataSourceId" below).

Setup

One-time infrastructure setup. Skip if you already have an active OpenSearch Application you can use.

1. Find a Usable Application

List existing ai-assistant applications and check if your domain's data source is already attached to one:

# Step 1: list active ai-assistant applications
aws opensearch list-applications --region <REGION> \
  --query "applicationSummaries[?starts_with(name, 'ai-assistant') && status=='ACTIVE'].{id:id,name:name,endpoint:endpoint}"

# Step 2: for each result, check if your domain is attached as a data source
# Look for a result whose title matches your domain name or AOSS collection name.
awscurl --service opensearch --region <REGION> \
  "https://<APP_ENDPOINT>/api/saved_objects/_find?type=data-source&fields=title&search=<DOMAIN_OR_COLLECTION_NAME>" \
  -H "osd-xsrf: true"
# Non-empty saved_objects array means your domain data source is attached and accessible.

If any application has your domain's data source, skip to "Discover dataSourceId". Otherwise, proceed to create your own.

2. Create Application

Get your caller ARN (exact Arn value) and the IAM role ARN:

aws sts get-caller-identity
# "Arn": "arn:aws:sts::<ACCOUNT>:assumed-role/<ROLE_NAME>/<SESSION>"
# users  -> exact Arn value above
# groups -> arn:aws:iam::<ACCOUNT>:role/<ROLE_NAME>  (strip sts:: and assumed-role prefix)

Derive a short slug from the role name (max 17 chars, lowercase, e.g. AWSReservedSSO_MyTeamAdmin_abc123 -> myteamadmin, PlatformEngineering -> platformeng):

# Tag setup CLI calls for usage attribution (setup steps only -- not needed for awscurl data-plane calls)
export AWS_SDK_UA_APP_ID=AWSSkill-OpenSearch

aws opensearch create-application \
  --region <REGION> \
  --name "ai-assistant-<ROLE_SLUG>" \
  --app-configs '[
    {"key":"opensearchDashboards.dashboardAdmin.users","value":"[\"arn:aws:sts::<ACCOUNT>:assumed-role/<ROLE_NAME>/<SESSION>\"  ]"},
    {"key":"opensearchDashboards.dashboardAdmin.groups","value":"[\"arn:aws:iam::<ACCOUNT>:role/<ROLE_NAME>\"]"}
  ]' \
  --data-sources '[{"dataSourceArn":"arn:aws:es:<REGION>:<ACCOUNT>:domain/<DOMAIN_NAME>","dataSourceDescription":"<DESC>"}]'

Note: dashboardAdmin.users and dashboardAdmin.groups grant dashboard admin access, which is required to query all attached data sources (discover dataSourceId via the saved objects API). Without this, the saved objects API calls in the next steps will fail. Set users to the exact assumed-role session ARN from get-caller-identity, and groups to the IAM role ARN (covers all sessions that assume the role). Application name is max 30 chars, pattern [a-z][a-z0-9-]*.

Security: If <ROLE_NAME> is a broad shared role (e.g., an administrator or power-user role), consider creating a dedicated narrower role used exclusively for OpenSearch Application access, rather than granting dashboard admin to all principals that assume the shared role.

Capture the id from the response. Poll until ACTIVE:

aws opensearch get-application --id <APP_ID> --region <REGION> --query 'status'

3. Enable Agentic AI Assistant

First check if ai-capability is already registered (use the exact name ai-capability):

aws opensearch get-capability \
  --application-id <APP_ID> \
  --capability-name ai-capability \
  --region <REGION>

If already enabled, the response looks like:

{
    "capabilityName": "ai-capability",
    "applicationId": "<APP_ID>",
    "status": "active",
    "capabilityConfig": { "aiConfig": {} }
}

status: "active" means the Agentic AI Assistant is already enabled — skip to "Querying Data". If the command returns a ResourceNotFoundException, register it:

aws opensearch register-capability \
  --application-id <APP_ID> \
  --capability-name ai-capability \
  --region <REGION> \
  --capability-config '{"aiConfig": {}}'

Querying Data (Chat)

Ask questions in natural language. The Agentic AI Assistant generates queries, executes them, and summarizes results.

Find data source ID

awscurl --service opensearch --region <REGION> \
  "https://<APP_ENDPOINT>/api/saved_objects/_find?fields=id&fields=title&type=data-source&search=<DATASOURCE_NAME>"

Where the AWS MCP server is available (run_script with make_request):

import json

resp = make_request(
    method='GET',
    url="https://<APP_ENDPOINT>/api/saved_objects/_find?fields=id&fields=title&type=data-source&search=<DATASOURCE_NAME>",
    service_name='opensearch',
    region_name='<REGION>',
    headers={'Content-Type': 'application/json'}
)
data = json.loads(resp.get('body', '{}'))
result = [obj['id'] for obj in data.get('saved_objects', [])]
result

Send chat message

awscurl --service opensearch --region <REGION> \
  -X POST "https://<APP_ENDPOINT>/api/chat/proxy?dataSourceId=<DATASOURCE_ID>" \
  -H "Content-Type: application/json" -H "Accept: text/event-stream" -H "osd-xsrf: true" \
  -d '{"threadId":"thread-001","runId":"run-001","messages":[{"id":"msg-001","role":"user","content":"<USER_QUESTION>"}],"tools":[],"context":[],"state":{},"forwardedProps":{}}'

Where the AWS MCP server is available (run_script with make_request):

import json

body = json.dumps({
    "threadId": "thread-1234567890000-abcd1234",
    "runId": "run-1234567890000-efgh5678",
    "messages": [{"id": "msg-1234567890000-ijkl9012", "role": "user", "content": "<USER_QUESTION>"}],
    "tools": [],
    "context": [],
    "state": {},
    "forwardedProps": {}
})

resp = make_request(
    method='POST',
    url="https://<APP_ENDPOINT>/api/chat/proxy?dataSourceId=<DATASOURCE_ID>",
    service_name='opensearch',
    region_name='<REGION>',
    headers={'Content-Type': 'application/json', 'Accept': 'text/event-stream', 'osd-xsrf': 'true'},
    body=body
)

raw = resp.get('body', '') if isinstance(resp, dict) else str(resp)
answer_parts = []
for line in raw.split('\n'):
    if line.startswith('data: '):
        try:
            event = json.loads(line[6:])
            if event.get('type') == 'TEXT_MESSAGE_CONTENT':
                answer_parts.append(event.get('delta', ''))
        except:
            pass

result = {"answer": ''.join(answer_parts)}
result

ID rules: Use literal unique strings. Reuse threadId for multi-turn. Fresh runId/id per request.

Investigating Incidents

Automated root cause analysis. Returns structured findings and hypotheses.

Discover investigation agent ID

awscurl --service opensearch --region <REGION> \
  -X POST "https://<APP_ENDPOINT>/api/investigation/ml/proxy?path=/_plugins/_ml/config/os_deep_research&method=GET" \
  -H "Content-Type: application/json" -H "osd-xsrf: osd-fetch"

Extract the agent ID from the response configuration.agent_id field.

Where the AWS MCP server is available (run_script with make_request):

import json

resp = make_request(
    method='POST',
    url="https://<APP_ENDPOINT>/api/investigation/ml/proxy?path=%2F_plugins%2F_ml%2Fconfig%2Fos_deep_research&method=GET",
    service_name='opensearch',
    region_name='<REGION>',
    headers={'Content-Type': 'application/json', 'osd-xsrf': 'osd-fetch'}
)

data = json.loads(resp.get('body', '{}'))
agent_id = data.get('configuration', {}).get('agent_id', '')
result = {"agent_id": agent_id}
result

Trigger investigation

awscurl --service opensearch --region <REGION> \
  -X POST "https://<APP_ENDPOINT>/api/investigation/agents/<AGENT_ID>/_execute?async=true" \
  -H "Content-Type: application/json" -H "osd-xsrf: osd-fetch" \
  -d '{"parameters":{"question":"Analyze anomaly in this dataset","context":"<CONTEXT>"},"dataSourceId":"<DATASOURCE_ID>"}'

dataSourceId MUST be at the same level as parameters. Capture memory_id from response.

Where the AWS MCP server is available (run_script with make_request):

import json

body = json.dumps({
    "parameters": {
        "question": "Analyze anomaly in this dataset, if there are major errors, find the root cause.",
        "context": "<INVESTIGATION_CONTEXT>"
    },
    "dataSourceId": "<DATASOURCE_ID>"
})

resp = make_request(
    method='POST',
    url="https://<APP_ENDPOINT>/api/investigation/agents/<AGENT_ID>/_execute?async=true",
    service_name='opensearch',
    region_name='<REGION>',
    headers={'Content-Type': 'application/json', 'osd-xsrf': 'osd-fetch'},
    body=body
)

result = resp.get('body', '') if isinstance(resp, dict) else str(resp)
result

Poll results

awscurl --service opensearch --region <REGION> \
  -X POST "https://<APP_ENDPOINT>/api/investigation/ml/proxy?path=/_plugins/_ml/memory_containers/investigation_memory_container_id/memories/working/_search&method=GET&dataSourceId=<DATASOURCE_ID>" \
  -H "Content-Type: application/json" -H "osd-xsrf: osd-fetch" \
  -d '{"_source":["structured_data_blob"],"query":{"bool":{"must":[{"term":{"namespace.session_id":"<MEMORY_ID>"}}],"must_not":[{"term":{"metadata.type":"trace"}}]}},"sort":[{"created_time":{"order":"asc"}}],"size":50}'

Poll until response field is non-empty (1-3 minutes). The document is created immediately with empty response, then updated once analysis completes.

Where the AWS MCP server is available (run_script with make_request):

import json

body = json.dumps({
    "_source": ["structured_data_blob"],
    "query": {
        "bool": {
            "must": [{"term": {"namespace.session_id": "<MEMORY_ID>"}}],
            "must_not": [{"term": {"metadata.type": "trace"}}]
        }
    },
    "sort": [{"created_time": {"order": "asc"}}],
    "size": 50,
    "from": 0
})

resp = make_request(
    method='POST',
    url="https://<APP_ENDPOINT>/api/investigation/ml/proxy?path=%2F_plugins%2F_ml%2Fmemory_containers%2Finvestigation_memory_container_id%2Fmemories%2Fworking%2F_search&method=GET&dataSourceId=<DATASOURCE_ID>",
    service_name='opensearch',
    region_name='<REGION>',
    headers={'Content-Type': 'application/json', 'osd-xsrf': 'osd-fetch'},
    body=body
)

data = json.loads(resp.get('body', '{}'))
hits = data.get('hits', {}).get('hits', [])
result = [hit.get('_source', {}).get('structured_data_blob', {}) for hit in hits]
result

make_request Reference

make_request(
    method='GET'|'POST',
    url='<FULL_URL>',
    service_name='opensearch',
    region_name='<REGION>',
    headers={...},
    body='<JSON_STRING>'
)
# Returns: dict with status_code, headers, body (string)

Constraints:

• MUST import json — no pre-imported modules
• random, time, inspect, uuid, datetime are blocked — use literal IDs
• No await — synchronous call
• URL query params MUST be percent-encoded (%2F for slashes)

Note: make_request is available via the AWS MCP server's run_script tool. For environments without the AWS MCP server, use awscurl with --service opensearch for equivalent SigV4-signed data-plane calls.

Security Considerations

IAM Least-Privilege Permissions

Minimum IAM permissions for this capability:

Note: After the application is created and its ID is known, scope es:GetApplication to arn:aws:es:<REGION>:<ACCOUNT>:application/<APP_ID> for least-privilege rather than using the wildcard.

Authentication and Transport

• The OpenSearch Application layer handles authentication via token exchange — no credentials are exposed in skill instructions
• All data-plane calls are SigV4-signed automatically (via make_request or awscurl)

Input Validation and Rate Limiting

• Validate and sanitize user input before sending to /api/chat/proxy to reduce prompt injection risk — avoid passing raw, unvalidated user text from untrusted sources directly into the content field
• Implement request throttling (via API Gateway, application-level rate limiting, or OpenSearch's built-in query limits) to prevent excessive Agentic AI Assistant invocations and runaway costs
• Consider enforcing maximum input length constraints on user questions
• The Agentic AI Assistant respects the domain's fine-grained access control (FGAC) — users can only query indices their IAM role or SAML identity has access to

Logging and Monitoring

• Enable CloudTrail logging for OpenSearch control-plane API calls (CreateApplication, RegisterCapability)
• Enable audit logging on the domain for data-plane access tracking
• Set CloudWatch alarms for unusual query error rates or latency spikes
• Monitor for unexpected patterns in Agentic AI Assistant usage

Sensitive Data Handling

• Chat responses and investigation results may contain PII or sensitive business data from your indices
• For AOSS collections, use data access policies to restrict which indices the Agentic AI Assistant can query
• Investigation results stored in memory indices are subject to the domain's encryption-at-rest policy (enabled by default on AOS/AOSS)
• If capturing Agentic AI Assistant interactions in CloudWatch Logs, always enable KMS encryption on the log group — responses may contain PII or sensitive business data from your indices

AWS Security Best Practices

• Security in Amazon OpenSearch Service
• Fine-grained access control
• Identity and Access Management