/teams-app-developer

Automated Testing with playground-cli

Run programmatic integration tests against your bot using @microsoft/m365agentsplayground-cli — a headless, browser-free API that drives bot conversations via code. Use this for CI pipelines, multi-turn conversation testing, and verifying bot behavior without manual interaction.

Installation

npm install --save-dev @microsoft/m365agentsplayground-cli

Quick Start (TypeScript)

import { TestClient } from "@microsoft/m365agentsplayground-cli";
import { expect } from "chai";

describe("MyBot", () => {
  let client: TestClient;

  before(async () => {
    client = new TestClient({
      botEndpoint: "http://localhost:3978/api/messages",
      timeout: 15000,
      deliveryMode: "expectReplies",
    });
    await client.start();
  });

  after(async () => {
    await client.stop();
  });

  beforeEach(() => {
    client.newConversation(); // isolate each test
  });

  it("should greet the user", async () => {
    const [response] = await client.sendMessage("Hello");
    expect(response.text).to.include("Hello");
  });

  it("should return a card for /help", async () => {
    const responses = await client.sendMessage("/help");
    const cardResponse = responses.find((r) => r.attachments?.length);
    expect(cardResponse).to.exist;
  });
});

BotResponse fields

ConversationServer (HTTP API)

For non-Node test runners (Python, curl, any language):

import { createConversationServer } from "@microsoft/m365agentsplayground-cli";

const server = createConversationServer({ port: 9000 });
// server keeps running; call server.close() in teardown

# Verify health
curl http://localhost:9000/health
# → {"status":"ok"}

POST /run-conversation

{
  "config": {
    "botEndpoint": "http://localhost:3978/api/messages",
    "timeout": 30000,
    "deliveryMode": "expectReplies",
    "personas": {
      "alice": { "id": "user-alice", "name": "Alice", "email": "alice@example.com" }
    }
  },
  "scenario": "smoke-test",
  "input": {
    "turns": [
      { "test_id": "t1", "prompt": "Hello" },
      { "test_id": "t2", "prompt": "What can you do?", "turn_type": "chat" },
      { "test_id": "t3", "prompt": "", "turn_type": "install" },
      { "test_id": "t4", "prompt": "<html>Order shipped</html>", "turn_type": "sendEmail", "persona": "alice" }
    ]
  }
}

Response:

{
  "turns": [
    { "test_id": "t1", "status": "Completed", "actual_response": "Hello! I'm your assistant..." },
    { "test_id": "t2", "status": "Completed", "actual_response": "I can help you with..." },
    { "test_id": "t3", "status": "Completed", "actual_response": "Welcome! I'm installed..." },
    { "test_id": "t4", "status": "TimedOut",   "actual_response": "" }
  ]
}

Turn statuses: Completed | TimedOut | Errored | Skipped (skipped = previous turn failed)

Turn Types

Key Configuration Notes

Common Pitfalls

• No await client.start() → sendMessage() throws immediately
• deliveryMode missing for teams-ai bots → sendMessage() returns []
• No client.newConversation() between tests → bot state bleeds between test cases
• ConversationServer not started before HTTP tests → connection refused; verify /health first
• Parallel tests sharing one TestClient → mixed-up responses; use separate instances

References

• Manual interactive testing → playground.md
• npm package → @microsoft/m365agentsplayground-cli