/sap-ai-core

Grounding and RAG Reference

Complete reference for SAP AI Core grounding capabilities (Retrieval-Augmented Generation).

Documentation Source: https://github.com/SAP-docs/sap-artificial-intelligence/tree/main/docs/sap-ai-core

Overview

Grounding integrates external, contextually relevant data into AI processes, enhancing LLM capabilities beyond general training material using vector databases.

Key Benefits

• Provide domain-specific context
• Access real-time data
• Reduce hallucinations
• Enable enterprise knowledge retrieval

Architecture

Indexing Pipeline

Documents → Preprocessing → Chunking → Embedding → Vector Database

1. Upload documents to supported repository
2. Pipeline preprocesses and chunks documents
3. Embedding model generates vectors
4. Vectors stored in managed vector database

Retrieval Pipeline

User Query → Embedding → Vector Search → Retrieved Chunks → LLM Context

1. User query converted to embedding
2. Vector similarity search in database
3. Relevant chunks retrieved
4. Chunks injected into LLM prompt

Supported Data Sources

Document Specifications

Supported Formats

Limits

• Maximum documents per pipeline: 2,000
• Refresh rate: Daily automatic refresh
• File size: Varies by format

Data Management APIs

Three primary APIs for document processing and retrieval:

Pipelines API

Creates data management pipelines that fetch documents from supported data sources.

Vector API

REST APIs for direct document ingestion and retrieval using vector embeddings.

Retrieval API

Performs similarity searches on the vector database.

API Comparison

Implementation Options

Option 1: Pipeline API

Automated document processing pipeline.

Create SharePoint Pipeline

curl -X POST "$AI_API_URL/v2/lm/groundingPipelines" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "AI-Resource-Group: default" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "hr-policies-pipeline",
    "configuration": {
      "dataSource": {
        "type": "sharepoint",
        "configuration": {
          "siteUrl": "https://company.sharepoint.com/sites/HR",
          "folderPath": "/Documents/Policies"
        }
      },
      "secretName": "sharepoint-credentials"
    }
  }'

Create S3 Pipeline

curl -X POST "$AI_API_URL/v2/lm/groundingPipelines" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "AI-Resource-Group: default" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "knowledge-base-pipeline",
    "configuration": {
      "dataSource": {
        "type": "s3",
        "configuration": {
          "bucket": "my-knowledge-base",
          "prefix": "documents/"
        }
      },
      "secretName": "s3-credentials"
    }
  }'

Create SFTP Pipeline

curl -X POST "$AI_API_URL/v2/lm/groundingPipelines" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "AI-Resource-Group: default" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "docs-sftp-pipeline",
    "configuration": {
      "dataSource": {
        "type": "sftp",
        "configuration": {
          "host": "sftp.company.com",
          "port": 22,
          "path": "/documents"
        }
      },
      "secretName": "sftp-credentials"
    }
  }'

Option 2: Vector API

Direct vector upload for custom chunking/embedding.

Create Collection

curl -X POST "$AI_API_URL/v2/lm/groundingCollections" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "AI-Resource-Group: default" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "custom-knowledge-base",
    "embeddingConfig": {
      "model": "text-embedding-3-small",
      "dimensions": 1536
    }
  }'

Note: Use text-embedding-3-small for 1536 dimensions or text-embedding-3-large with 3072 dimensions. Ensure model and dimensions align with OpenAI/SAP AI Core specifications.

Add Documents

curl -X POST "$AI_API_URL/v2/lm/groundingCollections/{collectionId}/documents" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "AI-Resource-Group: default" \
  -H "Content-Type: application/json" \
  -d '{
    "documents": [
      {
        "id": "doc-001",
        "content": "Document chunk text...",
        "metadata": {
          "source": "policy-manual.pdf",
          "page": 5,
          "department": "HR"
        }
      },
      {
        "id": "doc-002",
        "content": "Another chunk...",
        "metadata": {
          "source": "policy-manual.pdf",
          "page": 6,
          "department": "HR"
        }
      }
    ]
  }'

Add Pre-computed Vectors

curl -X POST "$AI_API_URL/v2/lm/groundingCollections/{collectionId}/documents" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "AI-Resource-Group: default" \
  -H "Content-Type: application/json" \
  -d '{
    "documents": [
      {
        "id": "doc-001",
        "content": "Document chunk text...",
        "vector": [0.123, -0.456, 0.789, ...],
        "metadata": {"source": "manual.pdf"}
      }
    ]
  }'

Creating Secrets

SharePoint Secret

curl -X POST "$AI_API_URL/v2/admin/secrets" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "AI-Resource-Group: default" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "sharepoint-credentials",
    "data": {
      "clientId": "<azure-app-client-id>",
      "clientSecret": "<azure-app-client-secret>",
      "tenantId": "<azure-tenant-id>"
    }
  }'

S3 Secret

curl -X POST "$AI_API_URL/v2/admin/secrets" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "AI-Resource-Group: default" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "s3-credentials",
    "data": {
      "AWS_ACCESS_KEY_ID": "<access-key>",
      "AWS_SECRET_ACCESS_KEY": "<secret-key>",
      "AWS_REGION": "us-east-1"
    }
  }'

SFTP Secret

curl -X POST "$AI_API_URL/v2/admin/secrets" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "AI-Resource-Group: default" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "sftp-credentials",
    "data": {
      "username": "<username>",
      "password": "<password>"
    }
  }'

Using Grounding in Orchestration

Basic Grounding Configuration

{
  "config": {
    "module_configurations": {
      "grounding_module_config": {
        "grounding_service": "document_grounding_service",
        "grounding_service_configuration": {
          "grounding_input_parameters": ["user_query"],
          "grounding_output_parameter": "context",
          "filters": [
            {
              "id": "<pipeline-id>",
              "search_configuration": {
                "max_chunk_count": 5
              }
            }
          ]
        }
      },
      "templating_module_config": {
        "template": [
          {
            "role": "system",
            "content": "Answer based on the following context:\n\n{{$context}}\n\nIf the answer is not in the context, say you don't know."
          },
          {
            "role": "user",
            "content": "{{?user_query}}"
          }
        ]
      },
      "llm_module_config": {
        "model_name": "gpt-4o",
        "model_version": "latest"
      }
    }
  },
  "input_params": {
    "user_query": "What is the vacation policy?"
  }
}

Grounding with Metadata Filters

{
  "grounding_module_config": {
    "grounding_service": "document_grounding_service",
    "grounding_service_configuration": {
      "grounding_input_parameters": ["user_query"],
      "grounding_output_parameter": "context",
      "filters": [
        {
          "id": "<pipeline-id>",
          "data_repositories": ["<specific-repo-id>"],
          "document_metadata": [
            {
              "key": "department",
              "value": "HR"
            },
            {
              "key": "document_type",
              "value": "policy"
            }
          ],
          "search_configuration": {
            "max_chunk_count": 10,
            "max_document_count": 5,
            "similarity_threshold": 0.7
          }
        }
      ]
    }
  }
}

Multiple Pipeline Sources

{
  "grounding_module_config": {
    "grounding_service": "document_grounding_service",
    "grounding_service_configuration": {
      "grounding_input_parameters": ["user_query"],
      "grounding_output_parameter": "context",
      "filters": [
        {
          "id": "<hr-pipeline-id>",
          "search_configuration": {"max_chunk_count": 3}
        },
        {
          "id": "<it-pipeline-id>",
          "search_configuration": {"max_chunk_count": 3}
        },
        {
          "id": "<finance-pipeline-id>",
          "search_configuration": {"max_chunk_count": 3}
        }
      ]
    }
  }
}

Search Configuration Options

Managing Pipelines

List Pipelines

curl -X GET "$AI_API_URL/v2/lm/groundingPipelines" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "AI-Resource-Group: default"

Get Pipeline Status

curl -X GET "$AI_API_URL/v2/lm/groundingPipelines/{pipelineId}" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "AI-Resource-Group: default"

Pipeline Statuses:

• PENDING: Initializing
• INDEXING: Processing documents
• READY: Available for queries
• FAILED: Error occurred

Delete Pipeline

curl -X DELETE "$AI_API_URL/v2/lm/groundingPipelines/{pipelineId}" \
  -H "Authorization: Bearer $AUTH_TOKEN" \
  -H "AI-Resource-Group: default"

Best Practices

Document Preparation

1. Clean content: Remove irrelevant headers, footers, boilerplate
2. Consistent formatting: Use clear headings and structure
3. Metadata tagging: Add useful metadata for filtering
4. Regular updates: Keep documents current

Chunking Strategy

1. Semantic chunks: Break at logical boundaries (sections, paragraphs)
2. Appropriate size: 200-500 tokens per chunk typically works well
3. Overlap: Consider 10-20% overlap between chunks
4. Context preservation: Include section headers in chunks

Query Optimization

1. Clear questions: Rephrase vague queries
2. Keyword inclusion: Include relevant technical terms
3. Context addition: Add domain context to queries

Retrieval Tuning

Troubleshooting

No Results Returned

1. Check pipeline status is READY
2. Verify documents were indexed successfully
3. Lower similarity threshold
4. Increase max_chunk_count
5. Check metadata filters match documents

Irrelevant Results

1. Increase similarity threshold
2. Add metadata filters
3. Review document chunking
4. Check embedding model matches query style

Performance Issues

1. Reduce max_chunk_count
2. Add specific metadata filters
3. Use multiple smaller pipelines
4. Consider pagination for large result sets

Documentation Links

• Grounding Overview: https://github.com/SAP-docs/sap-artificial-intelligence/blob/main/docs/sap-ai-core/grounding-035c455.md
• Pipeline API: https://github.com/SAP-docs/sap-artificial-intelligence/blob/main/docs/sap-ai-core/create-a-document-grounding-pipeline-using-the-pipelines-api-0a13e1c.md
• SharePoint Pipeline: https://github.com/SAP-docs/sap-artificial-intelligence/blob/main/docs/sap-ai-core/create-a-pipeline-with-microsoft-sharepoint-4b8d58c.md
• S3 Pipeline: https://github.com/SAP-docs/sap-artificial-intelligence/blob/main/docs/sap-ai-core/create-a-pipeline-with-aws-s3-7f97adf.md