All skills
google avatar

/agent-platform-troubleshooting

@efcb6f0
by googlegoogle/skills21k stars
1,698

Troubleshoots Google Cloud Gemini Enterprise Agent Platform issues (Agent Gateway, Registry, Identity, Policies, Model Armor, Identity-Aware Proxy (IAP)). Use when agent requests fail with 403 (especially unauthorized egress), Agent Runtime queries return 500, or gateway/IAP logs show permission errors. Don't use for general Google Cloud Identity and Access Management (IAM) debugging or networking issues unrelated to the Agent Platform stack.

Use this Skill: https://skilld.dev/gh/google/skills/agent-platform-troubleshooting

This session only. Nothing lands on disk.

referencesagent-registry.md

≈11k tokens on demand. Your agent reads this file only when SKILL.md points to it.

Agent Registry — Debugging Reference

Curated reference for diagnosing why agents, MCP servers, and endpoints registered in Google Cloud Agent Registry are not accepting traffic through Agent Gateway / IAP.

Scope: Gemini Enterprise Agent Platform — Agent Registry (Preview). All commands use gcloud agent-registry.

Table of Contents


TL;DR for Debugging

Read these first when something is broken.

  1. Agent Registry is default-deny at the gateway. Agent Gateway uses Identity-Aware Proxy (IAP) to enforce authorization:

    • UAP (Policy V2): When the gateway uses iapPolicyVersion: "V2", Gatekeeper queries the attached registries. If the destination matches, Gatekeeper sets destination.is_registered = true and exposes rich CEL metadata under destination.agent_registry.*. If not cataloged, destination.is_registered = false.
    • Legacy IAM v1: When using legacy v1, if a target hostname is not registered as a Service (and a matching roles/iap.egressor binding does not exist on the registry entry for the calling agent's principal), egress is blocked. Source: docs.cloud.google.com/iap/docs/agent-overview.
  2. There are exactly three discoverable resource types, and they are projections of the same writable Service:

    • Agent (read-only Agent)
    • MCP server (read-only McpServer)
    • Endpoint (read-only Endpoint) You always create/update/delete via services. You list/describe via agents, mcp-servers, endpoints, or services. Source: docs.cloud.google.com/agent-registry/concepts.
  3. Hostname matching is exact. A single Google API can be reached at five+ distinct hostnames (base, mTLS, locational, locational-mTLS, REP regional). Each hostname is a separate Service registration. If your agent calls bigquery.us-central1.rep.googleapis.com but you only registered bigquery.googleapis.com, the call is denied. See the "Hostname Matrix" section.

  4. Manual registration is not supported in us or eu multi-region locations. Use a real region (e.g. us-central1) or global. Source: docs.cloud.google.com/agent-registry/register-endpoints.

  5. Two predefined IAM roles control the registry itself:

    • roles/agentregistry.viewer — discover (list/describe).
    • roles/agentregistry.editor — register/update/delete. These are project-scoped registry permissions, not traffic permissions.
  6. Traffic permissions live on IAP, bound to registry resources. The role that actually permits an agent to use a registered service is roles/iap.egressor (iap.webServiceVersions.egressViaIAP), bound via gcloud iap web set-iam-policy with one of: --resource-type=agent-registry (registry-wide), --agent=, --mcp-server=, or --endpoint= (per-resource). Source: docs.cloud.google.com/gemini-enterprise-agent-platform/govern/policies/assign-identity-iam, docs.cloud.google.com/iam/docs/roles-permissions/iap.

  7. Identifiers matter:

    • SERVICE_NAME / SERVICE_ID — your chosen short ID for the registry entry.
    • Agent identifier (URN) and MCP server identifier (URN) — globally unique, immutable. Used by discovery tools and policy bindings.
    • Resource URI — the actual runtime location (Cloud Run URL, Agent Runtime endpoint, GKE deployment). Different from the URN. Embedded in the agent principal for IAM.
  8. Specs have a hard 10 KB cap. --mcp-server-spec-content and --agent-spec-content files must be <= 10 KB. Larger specs silently fail validation.

  9. Automatic registration vs manual: Agent Runtime, Google Workspace, Gemini Enterprise, and annotated GKE deployments register automatically. Official Google/Google Cloud remote MCP servers register automatically when the API is enabled. Everything else (Cloud Run, on-prem, custom REST agents, third-party MCP) is manual.

  10. First debug command is almost always: list the resource type and confirm the hostname permutation you expect is actually present. gcloud agent-registry endpoints list --project=$PROJECT_ID --location=$LOCATION

  11. Boundary controls visibility. Agents are only discoverable if they are hosted in a project that is within the defined Agent Management Boundary of the Management Project. If an agent is missing from the list, check the boundary configuration.

  12. Agent Gateway Dual-Registry Binding: An Agent Gateway can link up to two registries in its registries array, provided that exactly one registry is global, and the second registry is either regional (e.g. us-central1) or multi-regional (e.g. us, eu). Two regional registries without a global are rejected by the control plane.

  13. REST API Path Extensions (.json): When registering REST APIs (like Zendesk), note that SDK clients append .json (e.g. /api/v2/tickets.json). Ensure your UAP CEL rules or registry endpoint interfaces account for exact paths, / subpaths, .json extensions, and ? query strings.


1. The Data Model in One Picture

                 WRITE side                          READ side (read-only projections)
              +--------------+                    +----------+
              |              |  endpoint-spec --> | Endpoint |
              |   Service    |  mcp-server-spec ->| McpServer|
              | (writable)   |  agent-spec -----> |  Agent   |
              +--------------+                    +----------+
                ^                                        ^
         create/update/delete                     get/list/search
         (services subcommand)                    (endpoints / mcp-servers /
                                                   agents subcommand)
  • The Service resource is the only thing you write.

  • Agent Registry automatically generates the corresponding Agent, McpServer, or Endpoint projection based on the *-spec-type you passed at create time.

  • The *-spec-type parameter you use at creation determines which projection appears:

    Flag Projection becomes
    --endpoint-spec-type=no-spec Endpoint
    --mcp-server-spec-type=tool-spec McpServer
    --agent-spec-type=... (e.g. A2A) Agent

Source: docs.cloud.google.com/agent-registry/overview, docs.cloud.google.com/agent-registry/concepts.

URN formats (memorize these)

  • Manually registered MCP server URN: urn:mcp:projects-PROJECT_NUMBER:projects:PROJECT_NUMBER:locations:REGION:agentregistry:SERVICE_ID
  • Google Cloud remote MCP server URN: urn:mcp:googleapis.com:projects:PROJECT_NUMBER:locations:global:SERVICE_NAME

If a discovery query returns no results for what you think is a registered server, check the URN format vs the location you registered against.

Source: docs.cloud.google.com/agent-registry/concepts.


2. Default-Deny + Hostname Matching

This is the single most common debugging failure mode. Read this whole section.

Why default-deny

Agent Gateway is the egress proxy for agents. It is built on IAP. IAP only forwards a request if there is an IAM allow policy granting roles/iap.egressor to the calling agent's principal on a registry resource that matches the request. If no Service matches the destination hostname, no policy can match, and the request is denied.

Source: docs.cloud.google.com/iap/docs/agent-overview.

Hostname Matrix — register every form an agent might call

A single Google Cloud API can be addressed via several distinct hostnames. Each one is a separate registry entry. The canonical set includes the following; if any are missing for an API your agent uses, expect denial.

Variant Hostname pattern Example (bigquery, us-central1)
Base / global SERVICE.googleapis.com bigquery.googleapis.com
mTLS SERVICE.mtls.googleapis.com bigquery.mtls.googleapis.com
Locational LOCATION-SERVICE.googleapis.com us-central1-bigquery.googleapis.com
Locational mTLS LOCATION-SERVICE.mtls.googleapis.com us-central1-bigquery.mtls.googleapis.com
REP (public) SERVICE.LOCATION.rep.googleapis.com bigquery.us-central1.rep.googleapis.com
REP (private/PSC) SERVICE.LOCATION.p.rep.googleapis.com bigquery.us-central1.p.rep.googleapis.com
  • Locational endpoints specify the region/multi-region in the URL prefix. Format: LOCATION-SERVICE.googleapis.com. Locational endpoints don't support data-residency-compliant connections from on-premises and don't support failure-domain isolation; REP is replacing them.
  • REP (Regional Endpoint) confines TLS termination and traffic to the specified region. Format: SERVICE.REGION.rep.googleapis.com. Private REP via PSC adds .p..
  • The base/global hostname terminates TLS at the nearest edge and provides no regional isolation.

Source: docs.cloud.google.com/docs/security/compliance/restrict-endpoint-usage.

Recommended Pattern: Consolidated Google APIs Service

[!CAUTION] MANDATORY GOOGLE APIS CONSOLIDATION RULE: NEVER recommend or register separate Service resources for individual Google APIs (such as Agent Runtime, Cloud Resource Manager, IAM, or Telemetry). Do NOT register each Google API as a separate service entry, as separate service entries cause resource clutter, complicate IAM policy management, and risk hitting registry quota limits. ALL Google APIs MUST ALWAYS be consolidated under a single Agent Registry Service named googleapis using multiple interfaces.

Instead of registering dozens of separate Service resources for each Google API and its regional/mTLS permutations, it is highly recommended to consolidate all Google APIs under a single Agent Registry Service (e.g., named googleapis) using multiple interfaces.

This reduces resource clutter, simplifies IAM policy management (you only need to grant roles/iap.egressor on one resource), and avoids hitting registry limits.

Recommended Base Interfaces

For a standard agent deployment, the consolidated googleapis service should define at least the following interfaces:

  • https://agentregistry.googleapis.com
  • https://aiplatform.mtls.googleapis.com
  • https://cloudresourcemanager.mtls.googleapis.com
  • https://iamcredentials.mtls.googleapis.com
  • https://telemetry.mtls.googleapis.com
  • https://{region}-aiplatform.mtls.googleapis.com (replace {region} with your deployment region, e.g., us-central1)
  • https://{region}-aiplatform.googleapis.com
  • https://aiplatform.{region}.rep.googleapis.com

When debugging "agent can call X but not Y", first check that the hostname Y is using matches one of the registered URLs exactly. A trailing slash, port, scheme, or extra subdomain mismatch will cause denial.

Creating Consolidated Service via gcloud
gcloud agent-registry services create googleapis \
    --project=$PROJECT_ID --location=$LOCATION \
    --display-name="Google APIs" \
    --endpoint-spec-type=no-spec \
    --interfaces=url=https://agentregistry.googleapis.com,protocolBinding=JSONRPC \
    --interfaces=url=https://aiplatform.mtls.googleapis.com,protocolBinding=JSONRPC \
    --interfaces=url=https://cloudresourcemanager.mtls.googleapis.com,protocolBinding=JSONRPC \
    --interfaces=url=https://iamcredentials.mtls.googleapis.com,protocolBinding=JSONRPC \
    --interfaces=url=https://telemetry.mtls.googleapis.com,protocolBinding=JSONRPC \
    --interfaces=url=https://$LOCATION-aiplatform.mtls.googleapis.com,protocolBinding=JSONRPC \
    --interfaces=url=https://$LOCATION-aiplatform.googleapis.com,protocolBinding=JSONRPC \
    --interfaces=url=https://aiplatform.$LOCATION.rep.googleapis.com,protocolBinding=JSONRPC

Quick "is the hostname registered?" check

gcloud agent-registry services list \
  --project=$PROJECT_ID \
  --location=$LOCATION \
  --format="table(name.basename(), interfaces.url)" \
  --filter="interfaces.url:HOSTNAME_FRAGMENT"

Replace HOSTNAME_FRAGMENT with e.g. bigquery.us-central1.rep.

Source: hostname patterns derived from docs.cloud.google.com/docs/security/compliance/restrict-endpoint-usage; registration flow from docs.cloud.google.com/agent-registry/register-endpoints.


3. Resource Type — Agents {#resource-type-agents}

Concept

Agents: Autonomous actors that possess specific skills. Skills represent an agent's high-level capabilities and are a primary mechanism for discovery. -- docs.cloud.google.com/agent-registry/concepts

Two kinds: - A2A-compliant agents -- implement Agent2Agent. Registry scans their agent-card.json to extract skills. - Standard REST agents -- registered as NO_SPEC type when no A2A spec exists.

Automatic registration sources: Agent Runtime (via SDK), Google Workspace built-ins, Gemini Enterprise built-ins, GKE deployments with the Agent Registry functional-type annotation. Everything else is manual.

List

gcloud agent-registry agents list \
  --project=PROJECT_ID \
  --location=REGION

Filter:

gcloud agent-registry agents list \
  --project=PROJECT_ID \
  --location=REGION \
  --filter="displayName='DISPLAY_NAME'"

Describe

gcloud agent-registry agents describe AGENT_NAME \
  --project=PROJECT_ID \
  --location=REGION

Output exposes core details, skills (if A2A), and Resource URI -- the runtime location of the agent, which is what gets embedded in the agent principal for IAM.

Update (via services, not agents)

Display name / description:

gcloud agent-registry services update AGENT_NAME \
  --project=PROJECT_ID \
  --location=REGION \
  --display-name="New name" \
  --description="Updated description"

Endpoint URL:

gcloud agent-registry services update AGENT_NAME \
  --project=PROJECT_ID \
  --location=REGION \
  --interfaces=url=ENDPOINT_URL,protocolBinding=PROTOCOL

Valid protocolBinding: HTTP_JSON, GRPC, JSONRPC.

Agent spec payload (max size 10 KB to satisfy gRPC control plane metadata header limits):

gcloud agent-registry services update AGENT_NAME \
  --project=PROJECT_ID \
  --location=REGION \
  --agent-spec-content=@AGENT_SPEC

Delete

For automatically registered agents: delete from the source runtime -- registry entry follows. For manually registered agents:

gcloud agent-registry services delete AGENT_NAME \
  --project=PROJECT_ID \
  --location=REGION

"This action removes the agent from search results and makes it undiscoverable to other tools."

IAM model for agents

  • Registry permission to manage: roles/agentregistry.editor on the project.

  • Traffic permission for an agent to talk to this agent (agent-to-agent egress): bind roles/iap.egressor to the source agent's principal on the target agent resource:

    gcloud iap web set-iam-policy agents-iap-policy.json \
      --project=PROJECT_ID \
      --agent=AGENT_ID \
      --region=REGION

    The policy file's members is the source agent's principal:

    • Agent Runtime / Gemini Enterprise: principal://TRUST_DOMAIN/AGENT_UNIQUE_IDENTIFIER (e.g. principal://agents.global.org-123456789012.system.id.goog/resources/aiplatform/projects/9876543210/locations/us-central1/reasoningEngines/my-test-agent)
    • DIY agents: principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/AGENT_SERVICE_ACCOUNT_IDENTIFIER

Common pitfalls

  • Agent registered but not discoverable:
    • Check you're querying the right project + location. Manual registration is rejected in us/eu multi-regions.
    • Verify the agent's hosting project is within the Agent Management Boundary of the Management Project. If it is outside the boundary, the registry will not expose it. Check/update via gcloud agent-registry boundary update.
  • Agent appears in services list but not agents list: wrong *-spec-type at creation time. The projection only appears for the matching spec type.
  • Agent "registered" in Agent Runtime but missing from registry: automatic registration only happens for Agent Runtime SDK-deployed agents. A custom Cloud Run agent does not auto-register.
  • A2A agent missing skills: registry couldn't reach the agent's agent-card.json endpoint. Check the agent's published Agent Card URL is publicly reachable from Google's registration crawler.

Sources: docs.cloud.google.com/agent-registry/manage-agents, docs.cloud.google.com/agent-registry/register-agents, docs.cloud.google.com/gemini-enterprise-agent-platform/govern/policies/assign-identity-iam.


4. Resource Type — MCP Servers {#resource-type-mcp-servers}

Concept

MCP servers: Providers of standardized data resources and tools. Tools are deterministic functions exposed by an MCP server that agents can invoke to perform specific actions. -- docs.cloud.google.com/agent-registry/concepts

Official Google and Google Cloud remote MCP servers are auto-registered on API enable. External / custom MCP servers (e.g. Cloud Run-hosted FastMCP) need manual registration with a tool spec.

Register

gcloud agent-registry services create SERVER_NAME \
  --project=PROJECT_ID \
  --location=REGION \
  --display-name="DISPLAY_NAME" \
  --mcp-server-spec-type=tool-spec \
  --mcp-server-spec-content=@toolspec.json \
  --interfaces=url=SERVER_URL,protocolBinding=PROTOCOL
  • PROTOCOL for MCP is typically JSONRPC.
  • toolspec.json must be <= 10 KB, conform to the MCP tool schema (name, description, optional annotations).

List

gcloud agent-registry mcp-servers list \
  --project=PROJECT_ID \
  --location=REGION

Useful filters: - displayName='NAME' - mcpServerId='urn:mcp:SERVER_URN' (filter by URN -- see URN format in section 1)

Describe

gcloud agent-registry mcp-servers describe SERVER_NAME \
  --project=PROJECT_ID \
  --location=REGION

The describe output includes the tool catalog and an ADK code snippet to wire the server into an agent.

Update tool spec

Tool spec changes are not auto-detected -- re-upload when the server adds/changes tools:

gcloud agent-registry services update SERVER_NAME \
  --project=PROJECT_ID \
  --location=REGION \
  --mcp-server-spec-content=@TOOL_SPEC

Delete

gcloud agent-registry services delete SERVER_NAME \
  --project=PROJECT_ID \
  --location=REGION

IAM model for MCP servers

  • Discover MCP servers/tools: roles/agentregistry.viewer.

  • Update tool definitions / register: roles/agentregistry.editor.

  • Use the Agent Registry remote MCP server itself: roles/mcp.toolUser for mcp.tools.call, plus the viewer role for discovery.

  • Agent-to-MCP egress (the actual traffic permission): bind roles/iap.egressor to the source agent's principal on the MCP server resource:

    gcloud iap web set-iam-policy agents-iap-policy.json \
      --project=PROJECT_ID \
      --mcp-server=MCP_SERVER_ID \
      --region=REGION

    CEL conditions can scope the binding to specific tools and to read-only or non-destructive calls. Documented condition attributes include:

    • iap.googleapis.com/mcp.toolName
    • iap.googleapis.com/mcp.tool.isReadOnly
    • iap.googleapis.com/request.auth.type (e.g. 'MCP')

    Example condition expression: api.getAttribute('iap.googleapis.com/mcp.toolName', '') == 'GitHubTool' && api.getAttribute('iap.googleapis.com/mcp.tool.isReadOnly', false) == true && api.getAttribute('iap.googleapis.com/request.auth.type', '') == 'MCP'

Common pitfalls

  • Agent calls a tool, gets denied: the IAP policy condition may restrict to isReadOnly == true while the tool is destructive, or mcp.toolName doesn't match.
  • Tool added to MCP server but agents can't see it: tool spec wasn't re-uploaded. services update --mcp-server-spec-content is mandatory after every server-side change.
  • Spec upload silently rejected: file > 10 KB.
  • Manual registration error in us / eu: not supported. Use a region or global.
  • MCP server registered but appears under services list only, not mcp-servers list: wrong spec type -- must be --mcp-server-spec-type=tool-spec.

Sources: docs.cloud.google.com/agent-registry/register-mcp-servers, docs.cloud.google.com/agent-registry/manage-mcp-tools, docs.cloud.google.com/agent-registry/use-agentregistry-mcp, docs.cloud.google.com/gemini-enterprise-agent-platform/govern/policies/assign-identity-iam.


5. Resource Type — Endpoints {#resource-type-endpoints}

Concept

Endpoints: Target URLs, typically REST APIs, accessed by an agent. By abstracting these destinations into manageable resources, Agent Registry lets you centrally govern which external services your agents can access. -- docs.cloud.google.com/agent-registry/concepts

Endpoints are how you authorize an agent to reach anything that isn't an agent or an MCP server: Google APIs (BigQuery, Agent Runtime, etc.), third-party REST APIs, internal HTTP services. Always manually registered.

Register

gcloud agent-registry services create SERVICE_NAME \
  --project=PROJECT_ID \
  --location=REGION \
  --display-name="DISPLAY_NAME" \
  --endpoint-spec-type=no-spec \
  --interfaces=url=ENDPOINT_URL,protocolBinding=PROTOCOL
  • --endpoint-spec-type=no-spec is mandatory -- this is what causes the read-only Endpoint projection to appear.
  • After creation, Agent Registry auto-generates the read-only Endpoint resource that agents discover.

Hostname registration is exact-match. You must register every hostname variant the agent might dial. See "Hostname Matrix" in section 2. The fix for "endpoint registered but traffic denied" is almost always: register the missing variant.

List

gcloud agent-registry endpoints list \
  --project=PROJECT_ID \
  --location=REGION

Filter:

gcloud agent-registry endpoints list \
  --project=PROJECT_ID \
  --location=REGION \
  --filter="displayName='NAME'"

For URL-substring search, list services instead and filter on interfaces.url:

gcloud agent-registry services list \
  --project=PROJECT_ID \
  --location=REGION \
  --format="table(name.basename(), interfaces.url, interfaces.protocolBinding)" \
  --filter="interfaces.url:HOSTNAME_FRAGMENT"

Describe

gcloud agent-registry endpoints describe ENDPOINT_NAME \
  --project=PROJECT_ID \
  --location=REGION

Update

gcloud agent-registry services update SERVICE_NAME \
  --project=PROJECT_ID \
  --location=REGION \
  --interfaces=url=ENDPOINT_URL,protocolBinding=PROTOCOL

Delete

gcloud agent-registry services delete SERVICE_NAME \
  --project=PROJECT_ID \
  --location=REGION

"This action immediately removes the endpoint from discovery search results." Google Cloud console UI requires typing DELETE to confirm.

IAM model for endpoints

  • Discover endpoints: roles/agentregistry.viewer.

  • Register / update / delete: roles/agentregistry.editor.

  • Agent-to-endpoint traffic: bind roles/iap.egressor to the source agent's principal on the endpoint resource:

    gcloud iap web set-iam-policy agents-iap-policy.json \
      --project=PROJECT_ID \
      --endpoint=ENDPOINT_ID \
      --region=REGION

    Per-endpoint policy bindings do not require a CEL condition; you can omit condition to grant unconditional egress to that one endpoint, or add a CEL condition for attribute-based scoping (e.g. by Name, ReadOnly, Destructive, Idempotent, OpenWorld properties).

Note on roles/iap.tunnelResourceAccessor

Some IAP-protected resources use roles/iap.tunnelResourceAccessor (iap.tunnelDestGroups.accessViaIAP, iap.tunnelInstances.accessViaIAP) -- that role governs classic IAP TCP tunnels to GCE instances and dest groups. Agent Gateway egress to Agent Registry resources does NOT use this role; it uses roles/iap.egressor (iap.webServiceVersions.egressViaIAP). If you see a tutorial suggesting tunnelResourceAccessor for agent egress, it's the wrong role -- confirm against docs.cloud.google.com/iam/docs/roles-permissions/iap.

Common pitfalls

  • Test connection fails in Google Cloud console for a private URL: expected. The Google Cloud console connection test only validates public URLs; it does not support private endpoints (Agent Runtime, internal HTTPS, etc.).
  • Agent gets denied on a Google API call: the API has a hostname variant you didn't register (commonly the REP regional form ${id}.${LOCATION}.rep.googleapis.com).
  • Endpoint registered in global but agent calls a regional URL: registry location doesn't have to match the destination URL location, but the IAM binding's --region must match where the resource is registered. Check that --region= on gcloud iap web set-iam-policy equals the location used at creation.
  • Endpoint not appearing in endpoints list: created with the wrong spec type. Must use --endpoint-spec-type=no-spec.

Sources: docs.cloud.google.com/agent-registry/register-endpoints, docs.cloud.google.com/agent-registry/manage-endpoints, docs.cloud.google.com/iam/docs/roles-permissions/iap, docs.cloud.google.com/gemini-enterprise-agent-platform/govern/policies/assign-identity-iam.


6. IAM Scope: Registry-Wide vs Per-Resource

There are two layers. Don't conflate them.

Layer A — Agent Registry permissions (manage the catalog)

Project-scoped, granted via standard IAM:

Role Use
roles/agentregistry.viewer List / describe / search agents, MCP,
: : endpoints :
roles/agentregistry.editor Create / update / delete Service resources
roles/mcp.toolUser Call tools via the Agent Registry remote MCP
: : server (mcp.tools.call) :

These let humans and CI register and inspect entries. They do not grant runtime traffic.

Layer B — Traffic Authorization (Legacy v1 vs. UAP Policy V2)

Depending on the gateway's iapPolicyVersion setting, registry resources authorize traffic through one of two mechanisms:

Mechanism B1: Unified Access Policy (UAP / Policy V2)

When the attached IAP Authz Extension specifies iapPolicyVersion: "V2", policies are decoupled from individual registry resources:

  • AccessPolicy and PolicyBinding (iam.googleapis.com/v3) attach to the Resource Manager hierarchy (Organization, Folder, Project) targeting numeric resource URIs (e.g. //cloudresourcemanager.googleapis.com/projects/${PROJECT_NUMBER}).
  • Rules grant FQDN permission iap.googleapis.com/resources.egressViaIAP.
  • Agent Registry entries dynamically populate the destination.agent_registry.* CEL attributes evaluated at runtime:
    • destination.is_registered (boolean): true if target matches a registry service.
    • destination.agent_registry.resource_type: 'ENDPOINT', 'MCP_SERVER', 'AGENT', 'SKILL'.
    • destination.agent_registry.location: region of registry resource.
    • destination.agent_registry.mcp_server.name: full MCP server resource name.
    • destination.agent_registry.mcp_server.method: 'tools', 'prompts', 'resources'.
    • destination.agent_registry.mcp_server.tool.name: specific tool name (e.g. 'updateTicket').
    • destination.agent_registry.mcp_server.tool.annotations.read_only_hint: boolean.
  • Mandatory CEL Guard: Always guard tool annotations with: (destination.agent_registry.mcp_server.method == 'tools') && (destination.agent_registry.mcp_server.tool.annotations.read_only_hint == true).
Mechanism B2: Legacy IAM v1 (Per-Resource Bindings)

In legacy v1, permissions are bound directly on Agent Registry resources via gcloud iap web set-iam-policy. The role is always roles/iap.egressor. The scope of the binding determines which target resources the source agent can reach:

Scope gcloud target flag Grants egress to
Registry-wide --resource-type=agent-registry All agents + MCP
: : : servers + endpoints :
: : : in the project's :
: : : registry :
Single agent --agent=AGENT_ID One target agent
Single MCP server --mcp-server=MCP_SERVER_ID One target MCP server
Single endpoint --endpoint=ENDPOINT_ID One target endpoint

All four take --project=PROJECT_ID and --region=REGION (or --folder / --organization for higher-scoped binding).

The policy JSON is uniform:

{
  "policy": {
    "bindings": [
      {
        "role": "roles/iap.egressor",
        "members": ["AGENT_PRINCIPAL"],
        "condition": { "title": "...", "expression": "..." }
      }
    ]
  }
}

Source: docs.cloud.google.com/gemini-enterprise-agent-platform/govern/policies/assign-identity-iam.

Agent principal formats (the members value)

  • Agent Runtime / Gemini Enterprise: principal://TRUST_DOMAIN/AGENT_UNIQUE_IDENTIFIER Example: principal://agents.global.org-123456789012.system.id.goog/resources/aiplatform/projects/9876543210/locations/us-central1/reasoningEngines/my-test-agent

  • DIY agents (Cloud Run, GKE, on-prem): principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/AGENT_SERVICE_ACCOUNT_IDENTIFIER Example: principal://iam.googleapis.com/projects/1234567890/locations/global/workloadIdentityPools/my-agent-identity/subject/ns/default/sa/my-agent

  • Agent hierarchy (whole platform container): principalSet://agents.global.org-ORGANIZATION_ID.system.id.goog/attribute.platformContainer/aiplatform/projects/1234

When debugging "wrong principal" errors: the principal is built from the agent's resource URI, not its display name and not its URN. View the resource URI via gcloud agent-registry agents describe.

Full IAP role reference (relevant subset)

Role Permission grants the role provides When to use
roles/iap.egressor (Beta) iap.webServiceVersions.egressViaIAP **Agent
: : : egress :
: : : through Agent :
: : : Gateway** :
roles/iap.httpsResourceAccessor iap.webServiceVersions.accessViaIAP Human/service
: : : ingress to an :
: : : IAP-protected :
: : : web app :
roles/iap.tunnelResourceAccessor iap.tunnelDestGroups.accessViaIAP, Classic IAP
: : iap.tunnelInstances.accessViaIAP : TCP tunnels :
: : : to GCE -- :
: : : not for :
: : : agent gateway :
roles/iap.admin Full IAP admin (iap.tunnel.*, all Manage IAP
: : getIam/setIam) : policies :
roles/iap.viewer Read IAP settings Audit

Source: docs.cloud.google.com/iam/docs/roles-permissions/iap.


7. Common Registration Errors and Where to Look

Symptom Likely cause Where to check
services create fails: "manual Used --location=us or Switch to a region (e.g.
: registration not supported" : --location=eu (multi-regions) : us-central1) or global :
services create succeeds but Wrong *-spec-type. Re-create with the correct
: resource missing from endpoints : --endpoint-spec-type=no-spec : spec-type, or checkservices list` :
: list : produces anEndpoint`. : :
`services update Spec > 10 KB Trim the tool spec
: --mcp-server-spec-content` rejects : : :
: file : : :
Agent calls API X, IAP returns 403 Hostname variant not registered `services list
: : (commonly REP : --filter="interfaces.url:FRAGMENT"` :
: : *.LOCATION.rep.googleapis.com) : :
Agent calls API X, IAP returns 403 Missing roles/iap.egressor `gcloud iap web get-iam-policy
: even though hostname is registered : binding for the source agent's : --project= --region= --endpoint=...` :
: : principal : (or --mcp-server=, --agent=, :
: : : --resource-type=agent-registry) :
MCP tool call succeeds for one IAP policy condition restricts Inspect the IAP policy on the MCP
: tool, denied for another : mcp.toolName or : server resource :
: : mcp.tool.isReadOnly : :
Auto-registered Agent Runtime Agent deployed via custom code, Re-deploy via Agent Runtime, or
: agent missing from registry : not Agent Runtime SDK : register manually as a Service :
A2A agent has no skills in agent-card.json unreachable Curl the Agent Card URL from a
: registry : from Google's crawler : public network; verify content type :
: : : and schema :
Agent principal mismatch in IAP Principal built from wrong `gcloud agent-registry agents
: policy : resource URI / project number / : describe` -> use Resource URI to :
: : SA identifier : construct :
roles/iap.tunnelResourceAccessor Wrong role -- agent egress Replace the binding
: granted, still denied : requires roles/iap.egressor : :
Endpoint Google Cloud console Connection test only works for Test via the agent itself or curl
: "Test connection" fails on private : public URLs (by design) : from a peered network :
: URL : : :
agents-iap-policy.json set, Wrong --region flag on Re-issue with the correct --region
: still denied : set-iam-policy -- must match : :
: : resource location : :
Cannot find URN for binding Use mcp-servers describe / See URN formats in section 1
: : agents describe and read the : :
: : name field; format per section : :
: : 1 : :

8. Discovery via the Agent Registry MCP Server

The registry itself is exposed as a remote MCP server at agentregistry.googleapis.com. Useful when debugging from inside an agent or via mcp inspector. Tools include:

Discovery: - search_agents, search_mcp_servers -- keyword/prefix search - get_agent, get_mcp_server, get_endpoint -- by resource name - get_service -- read the underlying writable spec - list_agents, list_mcp_servers, list_endpoints, list_services - list_bindings, get_binding, fetch_available_bindings - get_operation

Administrative: - create_service, update_service, delete_service - create_binding, update_binding, delete_binding

tools/list over MCP does not require auth:

POST /mcp HTTP/1.1
Host: agentregistry.googleapis.com
Content-Type: application/json

{ "jsonrpc": "2.0", "method": "tools/list" }

Source: docs.cloud.google.com/agent-registry/use-agentregistry-mcp.


9. Quick Reference — Every gcloud Subcommand {#quick-reference-gcloud}

# READ
gcloud agent-registry agents      list   --project=P --location=L
gcloud agent-registry agents      describe NAME --project=P --location=L
gcloud agent-registry mcp-servers list   --project=P --location=L
gcloud agent-registry mcp-servers describe NAME --project=P --location=L
gcloud agent-registry endpoints   list   --project=P --location=L
gcloud agent-registry endpoints   describe NAME --project=P --location=L
gcloud agent-registry services    list   --project=P --location=L
gcloud agent-registry services    describe NAME --project=P --location=L

# WRITE  (always 'services'; the projection follows from --*-spec-type)
gcloud agent-registry services create NAME \
    --project=P --location=L --display-name="..." \
    --endpoint-spec-type=no-spec \
    --interfaces=url=URL,protocolBinding=HTTP_JSON|GRPC|JSONRPC

gcloud agent-registry services create NAME \
    --project=P --location=L --display-name="..." \
    --mcp-server-spec-type=tool-spec \
    --mcp-server-spec-content=@toolspec.json \
    --interfaces=url=URL,protocolBinding=JSONRPC

gcloud agent-registry services create NAME \
    --project=P --location=L --display-name="..." \
    --agent-spec-content=@agent-card.json \
    --interfaces=url=URL,protocolBinding=HTTP_JSON

gcloud agent-registry services update NAME \
    --project=P --location=L \
    [ --display-name="..." | --description="..." \
      | --interfaces=url=URL,protocolBinding=... \
      | --mcp-server-spec-content=@spec.json \
      | --agent-spec-content=@card.json ]

gcloud agent-registry services delete NAME --project=P --location=L

# IAP egress policy bindings (separate API, separate gcloud surface)
gcloud iap web set-iam-policy POLICY.json \
    --project=P --region=L \
    [ --resource-type=agent-registry
    | --agent=AGENT_ID
    | --mcp-server=MCP_SERVER_ID
    | --endpoint=ENDPOINT_ID ]

gcloud iap web get-iam-policy \
    --project=P --region=L \
    [ same target flags ]

Spec size cap: 10 KB for --mcp-server-spec-content and --agent-spec-content.

Manual registration locations: any region or global. Not us or eu.


Source Index

Source: SKILL.md on GitHub

No alerts8d3 checks · Risk SAFE
  • Gen Agent Trust Hub8d

    This skill is a diagnostic tool designed to troubleshoot the Google Cloud Gemini Enterprise Agent Platform. It provides structured procedures and reference documentation for analyzing connectivity, identity, and policy issues within a Google Cloud environment. No security issues were detected during the analysis.

  • Socket8d

    No alerts

  • Snyk8d

    Risk: LOW · No issues

Signed by skilld at efcb6f0. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated last week
metadata
{
  "version": "1.0.0",
  "category": "AiAndMachineLearning"
}

README badge

README badge for google/skills/agent-platform-troubleshooting