Index of every page on this site: llms.txt
All skills
launchdarkly avatar

/launchdarkly-experiment-guide

@f0ee110 official
Run it yourself

Guide a developer from 'I don't know what to test' to a ready-to-create LaunchDarkly experiment. Use when someone wants to run an experiment or A/B test but doesn't have a metric, a flag, or a hypothesis yet, or isn't sure what to measure. Scans the codebase and the connected LaunchDarkly account, proposes a hypothesis and metric from real context, wires up the flag and tracking code, then drafts the experiment and offers to create it. Built for engineers without a data-science or product-analytics background.

Use this Skill: https://skilld.dev/gh/launchdarkly/ai-tooling/launchdarkly-experiment-guide

Nothing lands on disk. Nothing to clean up.

Fork this Skill

Edit a local copy. It keeps the author and licence.

referenceshypothesis-guide.md

≈854 tokens on demand. Your agent reads this file only when SKILL.md points to it.

Hypothesis guide

A hypothesis is a testable guess: if we change this, that measurable thing will move, and here's why. Most users freeze when asked for one. Don't ask cold — generate candidates from the feature they picked, show the shape, and let them edit.

The shape

Changing [what you'll change] will [increase / decrease] [the metric] for [which users], because [the reason].

A good one names four things:

  1. The change — the treatment vs. the control (what the flag switches).
  2. The metric — the single number you expect to move (the primary metric).
  3. The direction — up or down (and ideally a rough size, e.g. "~3%").
  4. The reason — the mechanism you believe in. This is what makes it a hypothesis and not a wish.

Turning a fuzzy goal into a hypothesis

User says… Reframe toward…
"I want more sales." "Which step loses people? Test a change there; metric = checkout completion or add-to-cart rate."
"The new page feels better." "Better how — faster, clearer, more clicks? Pick the one number that captures it."
"People don't sign up." "Test a change to the signup flow; metric = signup completion rate for new visitors."
"The app is slow." "Test the optimization behind a flag; metric = p95 latency or load time (numeric, lower is better)."

Worked examples by product type

E-commerce

Showing the redesigned checkout will increase checkout completion for logged-in shoppers, because it removes two form steps. Primary metric: checkout-completed (conversion).

SaaS onboarding

Adding an interactive product tour will increase activation (first key action within 24h) for new signups, because it shortens time-to-value. Primary metric: activated (conversion).

Content / media

Surfacing personalized recommendations will increase articles read per session, because readers find relevant content faster. Primary metric: article-opened (numeric count per session).

Performance

Enabling the new image pipeline will decrease p95 page load time without hurting engagement, because images are served pre-optimized. Primary metric: page-load-ms (numeric, lower is better). Guardrail: bounce-rate (should not increase).

Pricing / growth

Showing the annual plan first will increase annual-plan conversion for visitors to the pricing page, because it anchors on the better-value option. Primary metric: annual-plan-selected (conversion).

Sizing (keep it light for a first experiment)

You don't need a power calculation to start. Nudge toward:

  • A clear primary metric with a direction.
  • A realistic effect ("a few percent," not "double").
  • A decision rule: "if completion is up with high confidence after ~2 weeks, ship it." LaunchDarkly's stats engine handles significance; the user just needs to let it run long enough.

Anti-patterns to redirect

  • No metric ("see if people like it") → pin it to one measurable number.
  • Two changes at once ("new checkout and new colors") → split into two experiments so the result is attributable.
  • Vanity metric ("page views") → prefer a metric tied to the actual goal (completion, activation, revenue).
  • No reason → ask "why do you think it'll help?"; the reason shapes what to measure.

Source: SKILL.md on GitHub

No rule matched.

skilld matched fixed text patterns in SKILL.md and file names. Patterns miss obfuscated code.

skilld run checks every file with the same patterns. It asks for approval before it loads a Skill with a behavior marked Needs approval.

No alerts15d3 checks · Risk SAFE
  • Gen Agent Trust Hub15d

    This skill is a guided assistant for setting up A/B tests and experiments using LaunchDarkly. It scans the local codebase and git history to suggest relevant experiments and interacts with LaunchDarkly via MCP or an API token to manage flags and metrics. The skill follows best practices for credential handling and user consent.

  • Socket15d

    No alerts

  • Snyk15d

    Risk: LOW · No issues

Signed by skilld at f0ee110. This ties the file your Agent reads to that commit on GitHub. It does not review the instructions.

Last checked against GitHub yesterday.

Activeupdated 2 weeks ago
metadata
{
  "author": "launchdarkly",
  "version": "0.1.0-experimental"
}
Other metadata
compatibility
Uses the LaunchDarkly MCP server when it is configured; otherwise falls back to the LaunchDarkly REST API (needs a LaunchDarkly API token). Reads the local codebase and git history.

README badge

README badge for launchdarkly/ai-tooling/launchdarkly-experiment-guide