skilld
All skills
launchdarkly avatar

/configs-variations

@3f246b4 official
by launchdarklylaunchdarkly/agent-skills25 stars
10

Experiment with configs by creating and managing variations. Helps you test different models, prompts, and parameters to find what works best through systematic experimentation.

Ask your Agent

Use this Skill: https://skilld.dev/gh/launchdarkly/agent-skills/configs-variations

This session only. Nothing lands on disk.

SKILL.md

≈49 tokens always: the name and description. ≈1.4k when used: this file. ≈304 more on demand in 1 file.

Config Variations

You're using a skill that will guide you through testing and optimizing configs through variations. Your job is to design experiments, create variations, and systematically find what works best.

Prerequisites

This skill requires the remotely hosted LaunchDarkly MCP server to be configured in your environment.

Primary MCP tool:

  • clone-ai-config-variation -- clone a baseline variation with selective overrides (recommended for experimentation)

Alternative MCP tools (for more control):

  • get-ai-config -- review existing variations before adding new ones
  • create-ai-config-variation -- create new variations from scratch

Optional MCP tools:

  • update-ai-config-variation -- refine a variation after creation
  • delete-ai-config-variation -- remove variations that didn't work out

Core Principles

  1. Test One Thing at a Time: Change model OR prompt OR parameters, not all at once
  2. Have a Hypothesis: Know what you're trying to improve
  3. Measure Results: Use metrics to compare variations
  4. Verify via Tool: The agent fetches the config to confirm variations exist

Workflow

Step 1: Identify What to Optimize

What's the problem? Cost, quality, speed, accuracy? How will you measure success?

Step 2: Design the Experiment

Goal What to Vary
Reduce cost Cheaper model (e.g., gpt-4o-mini)
Improve quality Better model or more detailed prompt
Reduce latency Faster model, lower max_tokens
Increase accuracy Different model family (Claude vs GPT-4)

Step 3: Create Variations (Recommended: Clone with Overrides)

Use clone-ai-config-variation to duplicate the baseline and override only what you're testing. The tool reads the source variation, merges your overrides, and creates the new variation. Everything you don't pass is inherited from the source automatically.

Required fields:

  • sourceVariationKey -- the baseline to clone from
  • key and name -- identifiers for the new variation (e.g., gpt4o-mini-cost-test)

Override ONLY the fields you are testing. Leave all other fields unset -- do not pass them even if you know their current values. The clone tool inherits them from the source. This enforces the one-variable-at-a-time principle:

  • Testing a cheaper model? Pass only modelConfigKey and modelName. Do NOT pass instructions, messages, or parameters.
  • Testing different instructions? Pass only instructions. Do NOT pass modelConfigKey or modelName.
  • Testing a parameter? Pass only parameters. Do NOT pass model or prompt fields.

The response returns both the source and created variation, so you can immediately verify the diff.

Step 3 (Alternative): Create from Scratch

If you need full control, use get-ai-config first to review the current state, then create-ai-config-variation with all fields specified manually. Always fetch before creating so you understand the existing config's mode, model, and parameters.

Step 4: Verify

If you used clone-ai-config-variation, the response includes both source and created variations for immediate comparison. Otherwise, use get-ai-config to confirm.

Report results:

  • Variations created with correct models and parameters
  • Only the intended variable differs between variations
  • Flag any issues

Note on API responses: After calling a creation or clone tool, treat a successful response as confirmation that the operation succeeded. The API response may not echo back every field you sent (e.g., model fields may show defaults). Do not retry or assume failure based on response field values alone -- verify with get-ai-config if needed.

modelConfigKey Format

Required for models to display in the UI. Format: {Provider}.{model-id}:

  • OpenAI.gpt-4o, OpenAI.gpt-4o-mini
  • Anthropic.claude-sonnet-4-5, Anthropic.claude-3-5-sonnet

Safety: Protect the Baseline

When the user wants to try a different model, prompt, or parameters, always create a new variation alongside the baseline. Never modify or delete the existing baseline variation. This applies even if the user says "replace" or "switch" -- the correct action is to create a new variation and let targeting/rollouts control traffic, not to edit the original.

  • Use clone-ai-config-variation or create-ai-config-variation to add the new variation
  • Do NOT use update-ai-config-variation on the baseline to change its model or instructions
  • Do NOT use delete-ai-config-variation on the baseline
  • Explain to the user that keeping the baseline enables comparison and safe rollback

What NOT to Do

  • Don't test too many things at once -- change one variable per variation
  • Don't pass unchanged fields when cloning -- let the tool inherit them from the source
  • Don't forget modelConfigKey (variations without it show as "NO MODEL" in the UI)
  • Don't make decisions on small sample sizes
  • Don't modify or remove the baseline variation -- create new variations alongside it
  • Don't use update-ai-config-variation to "replace" a baseline -- create a new variation instead

More resources

To learn more about creating and managing variations, read Create and manage config variations.

Related Skills

  • configs-create -- Create the initial config
  • configs-update -- Refine based on learnings

Source: SKILL.md on GitHub

Third-party checks

No alerts2mo3 checks · Risk SAFE
  • Gen Agent Trust Hub2mo

    The skill provides instructions for managing LaunchDarkly AI configurations and experiments using a set of dedicated MCP tools. It guides the agent in creating, cloning, and verifying configuration variations for model comparison and optimization. No security risks, malicious patterns, or unsafe practices were detected.

  • Socket2mo

    No alerts

  • Snyk2mo

    Risk: LOW · No issues

Provenance

Signed by skilld at 3f246b4. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub 3 days ago.

Activeupdated 2 months ago

Capability

compatibility
Requires the remotely hosted LaunchDarkly MCP server
metadata
{
  "author": "launchdarkly",
  "version": "1.0.0-experimental"
}

Topics

  • MCP
  • launchdarkly
  • experimentation
  • config-management
  • model-testing
  • prompt-optimization
  • ai-parameters
  • variations

README badge

README badge for launchdarkly/agent-skills/configs-variations

What it does

Creates and manages AI config variations within LaunchDarkly to test different models, prompts, and parameters one variable at a time. Use this to systematically experiment with cost-reduction, quality-improvement, or latency-optimization through the LaunchDarkly MCP server's clone and create tools, keeping a baseline variation intact for comparison and rollback.

Generated from the current SKILL.md.

Frequently asked

What MCP server does this skill require?
The remotely hosted LaunchDarkly MCP server. It must be configured in your environment before using this skill.
Can I test multiple variables at once in a single variation?
No. The skill enforces testing one variable per variation — change either model, prompt, or parameters, not combinations. This isolates what caused any performance difference.
Should I modify the baseline variation or create a new one?
Always create a new variation alongside the baseline. Never modify or delete the baseline — keeping it enables comparison and safe rollback if needed.
What fields should I pass when cloning a variation?
Pass only the fields you are testing: sourceVariationKey, key, name, and the override field (modelConfigKey, instructions, or parameters). Omit all other fields so they inherit from the source automatically.
How do I verify a variation was created correctly?
If you used clone-ai-config-variation, the response includes both source and created variations for immediate comparison. Otherwise, use get-ai-config to confirm the new variation matches your intent.

Generated from the current SKILL.md. These answers refresh after source changes.