/sap-ai-core

Orchestration Modules Reference

Complete reference for all SAP AI Core orchestration modules.

Documentation Source: https://github.com/SAP-docs/sap-artificial-intelligence/tree/main/docs/sap-ai-core

Orchestration V2 API

Endpoint

V2 Endpoint: POST {{deployment_url}}/v2/completion

V1 to V2 Migration

If migrating from V1 to V2:

1. Update endpoint from /completion to /v2/completion
2. Modify payload structure to use config.modules format
3. Test with existing orchestration configurations

V2 Request Structure

{
  "config": {
    "modules": {
      "prompt_templating": { /* template config */ },
      "llm": { /* model config */ },
      "grounding": { /* optional */ },
      "filtering": { /* optional */ },
      "masking": { /* optional */ },
      "translation": { /* optional */ }
    }
  },
  "placeholder_values": {
    "variable_name": "value"
  }
}

Key V2 Changes

V2 Module Naming

In the V2 API, modules use short keys within config.modules:

V2 Output with Citations

For Perplexity Sonar and Sonar Pro models, the V2 orchestration response includes a citations section with source URLs. This is the only model family that supports citations output.

Module Execution Order

The orchestration pipeline executes modules in this fixed order:

1. Grounding → 2. Templating → 3. Input Translation → 4. Data Masking →
5. Input Filtering → 6. Model Configuration → 7. Output Filtering → 8. Output Translation

Only Templating and Model Configuration are mandatory.

1. Templating Module (Mandatory)

Compose prompts with placeholders that get populated during inference.

Configuration

{
  "templating_module_config": {
    "template": [
      {"role": "system", "content": "You are {{?assistant_type}}"},
      {"role": "user", "content": "{{?user_message}}"}
    ],
    "defaults": {
      "assistant_type": "a helpful assistant"
    }
  }
}

Placeholder Syntax

Message Roles

• system: System instructions
• user: User input
• assistant: Assistant responses (for multi-turn)
• tool: Tool call results

2. Model Configuration Module (Mandatory)

Configure the LLM parameters.

Configuration

{
  "llm_module_config": {
    "model_name": "gpt-4o",
    "model_version": "latest",
    "model_params": {
      "max_tokens": 2000,
      "temperature": 0.7,
      "top_p": 0.95,
      "frequency_penalty": 0,
      "presence_penalty": 0,
      "stop": ["\n\n"]
    }
  }
}

Common Parameters

Model Version Options

• "latest": Auto-upgrade to newest version
• Specific version: e.g., "2024-05-13" for pinned version

3. Content Filtering Module

Filter harmful content in input and output.

Azure Content Safety Configuration

{
  "filtering_module_config": {
    "input": {
      "filters": [
        {
          "type": "azure_content_safety",
          "config": {
            "Hate": 2,
            "Violence": 2,
            "Sexual": 2,
            "SelfHarm": 2
          }
        }
      ]
    },
    "output": {
      "filters": [
        {
          "type": "azure_content_safety",
          "config": {
            "Hate": 0,
            "Violence": 0,
            "Sexual": 0,
            "SelfHarm": 0
          }
        }
      ]
    }
  }
}

Azure Content Safety Categories

Severity Scale:

• 0: Safe
• 2: Low severity
• 4: Medium severity (blocked by Azure global filter)
• 6: High severity (blocked by Azure global filter)

PromptShield Configuration

Detect prompt injection attacks:

{
  "filtering_module_config": {
    "input": {
      "filters": [
        {
          "type": "azure_content_safety",
          "config": {
            "PromptShield": true
          }
        }
      ]
    }
  }
}

Llama Guard 3 Configuration

{
  "filtering_module_config": {
    "input": {
      "filters": [
        {
          "type": "llama_guard_3",
          "config": {
            "categories": [
              "violent_crimes",
              "hate",
              "sexual_content",
              "self_harm"
            ]
          }
        }
      ]
    }
  }
}

Llama Guard 3 Categories (14)

4. Data Masking Module

Anonymize or pseudonymize PII before sending to LLM.

Pseudonymization Configuration

{
  "masking_module_config": {
    "masking_providers": [
      {
        "type": "sap_data_privacy_integration",
        "method": "pseudonymization",
        "entities": [
          {"type": "profile-person"},
          {"type": "profile-email"},
          {"type": "profile-phone"},
          {"type": "profile-credit-card-number"}
        ]
      }
    ]
  }
}

Anonymization Configuration

{
  "masking_module_config": {
    "masking_providers": [
      {
        "type": "sap_data_privacy_integration",
        "method": "anonymization",
        "entities": [
          {"type": "profile-person"},
          {"type": "profile-ssn"}
        ]
      }
    ]
  }
}

Complete Entity Type Reference (25)

Personal Identifiers:

Organizations:

Government/Financial IDs:

SAP-Specific:

Sensitive Attributes:

Custom Entity with Regex

{
  "masking_module_config": {
    "masking_providers": [
      {
        "type": "sap_data_privacy_integration",
        "method": "pseudonymization",
        "entities": [
          {
            "type": "custom",
            "pattern": "EMP-[0-9]{6}",
            "replacement": "EMPLOYEE_ID"
          }
        ]
      }
    ]
  }
}

5. Grounding Module

Inject external context from vector databases (RAG).

Basic Grounding Configuration

{
  "grounding_module_config": {
    "grounding_service": "document_grounding_service",
    "grounding_service_configuration": {
      "grounding_input_parameters": ["user_query"],
      "grounding_output_parameter": "context",
      "filters": [
        {
          "id": "<pipeline-id>",
          "search_configuration": {
            "max_chunk_count": 5
          }
        }
      ]
    }
  }
}

Grounding with Metadata Filters

{
  "grounding_module_config": {
    "grounding_service": "document_grounding_service",
    "grounding_service_configuration": {
      "grounding_input_parameters": ["user_query"],
      "grounding_output_parameter": "context",
      "filters": [
        {
          "id": "<pipeline-id>",
          "data_repositories": ["<repo-id>"],
          "document_metadata": [
            {
              "key": "department",
              "value": "HR"
            }
          ],
          "search_configuration": {
            "max_chunk_count": 10,
            "max_document_count": 5
          }
        }
      ]
    }
  }
}

Using Grounding Output in Template

{
  "templating_module_config": {
    "template": [
      {
        "role": "system",
        "content": "Answer questions using only the following context:\n\n{{$context}}"
      },
      {
        "role": "user",
        "content": "{{?user_query}}"
      }
    ]
  }
}

6. Translation Module

Translate input and output between languages.

Input Translation Configuration

{
  "translation_module_config": {
    "input": {
      "source_language": "auto",
      "target_language": "en"
    }
  }
}

Output Translation Configuration

{
  "translation_module_config": {
    "output": {
      "source_language": "en",
      "target_language": "{{?user_language}}"
    }
  }
}

Combined Translation

{
  "translation_module_config": {
    "input": {
      "source_language": "auto",
      "target_language": "en"
    },
    "output": {
      "source_language": "en",
      "target_language": "auto"
    }
  }
}

Complete Orchestration Example

All modules combined:

{
  "config": {
    "module_configurations": {
      "grounding_module_config": {
        "grounding_service": "document_grounding_service",
        "grounding_service_configuration": {
          "grounding_input_parameters": ["user_query"],
          "grounding_output_parameter": "context",
          "filters": [{"id": "<pipeline-id>"}]
        }
      },
      "templating_module_config": {
        "template": [
          {"role": "system", "content": "You are a helpful assistant. Use this context:\n{{$context}}"},
          {"role": "user", "content": "{{?user_query}}"}
        ]
      },
      "translation_module_config": {
        "input": {"source_language": "auto", "target_language": "en"},
        "output": {"source_language": "en", "target_language": "auto"}
      },
      "masking_module_config": {
        "masking_providers": [{
          "type": "sap_data_privacy_integration",
          "method": "pseudonymization",
          "entities": [
            {"type": "profile-person"},
            {"type": "profile-email"}
          ]
        }]
      },
      "filtering_module_config": {
        "input": {
          "filters": [{
            "type": "azure_content_safety",
            "config": {"Hate": 2, "Violence": 2, "Sexual": 2, "SelfHarm": 2}
          }]
        },
        "output": {
          "filters": [{
            "type": "azure_content_safety",
            "config": {"Hate": 0, "Violence": 0, "Sexual": 0, "SelfHarm": 0}
          }]
        }
      },
      "llm_module_config": {
        "model_name": "gpt-4o",
        "model_version": "latest",
        "model_params": {
          "max_tokens": 2000,
          "temperature": 0.5
        }
      }
    }
  },
  "input_params": {
    "user_query": "What are the company's vacation policies?"
  }
}

Documentation Links

• Orchestration Overview: https://github.com/SAP-docs/sap-artificial-intelligence/blob/main/docs/sap-ai-core/orchestration-8d02235.md
• Content Filtering: https://github.com/SAP-docs/sap-artificial-intelligence/blob/main/docs/sap-ai-core/content-filtering-f804175.md
• Data Masking: https://github.com/SAP-docs/sap-artificial-intelligence/blob/main/docs/sap-ai-core/data-masking-8b87002.md
• Grounding: https://github.com/SAP-docs/sap-artificial-intelligence/blob/main/docs/sap-ai-core/grounding-035c455.md