Copy variants for A/B tests

A/B testing copy is not just about changing a button label. It means serving one group of users a specific string, serving another group a different string, and ensuring that each user consistently sees the same variant. When copy variants are managed by returning different strings from application conditionals - if (experiment.variation === 'bold') return 'Get started' - the variants are invisible to reviewers, untested, and mixed into product code that should not own content decisions.

Messagevisor keeps copy variants in the translation layer, where they can be reviewed in YAML, tested against the evaluation engine, and changed without application deployments. If your runtime provides experiment variation resolvers, for example through a Featurevisor integration, experiment conditions can gate translation overrides directly.

The model for A/B copy variants#

Copy variants in Messagevisor are overrides on a message. Each override can target a specific experiment variation using a conditions expression. At runtime, the SDK evaluates which override applies given the active context - which includes the current experiment variation the user is assigned to.

This means:

Both the control and treatment copy live in one message definition
Reviewers see all variants in a single YAML file
Tests can assert each variant independently
Changing copy in a variant does not require touching application code

Targeting experiment variations#

When feature or experiment resolvers are configured, override conditions can reference experiment variations:

messages/auth/ctaSignup.yml

description: Primary sign-up call to action
translations:
  en: Sign up
  nl: Aanmelden
overrides:
  - key: cta-test-get-started
    conditions:
      experiment: cta-copy-test
      operator: hasVariation
      value: get-started
    translations:
      en: Get started
      nl: Aan de slag
  - key: cta-test-try-free
    conditions:
      experiment: cta-copy-test
      operator: hasVariation
      value: try-free
    translations:
      en: Try it free
      nl: Probeer het gratis

When a user is in the get-started variation of the cta-copy-test experiment, they see "Get started." When they are in try-free, they see "Try it free." Everyone else sees the control, "Sign up."

Combining experiment conditions with other targeting#

Experiment conditions compose with attribute conditions and segments. This lets you serve a copy variant only to the subset of an experiment audience that also matches other criteria:

messages/checkout/submitOrder.yml

description: Submit order button label
translations:
  en: Place order
  nl: Bestelling plaatsen
overrides:
  - key: express-checkout-new-user
    conditions:
      and:
        - experiment: checkout-copy-test
          operator: hasVariation
          value: express
        - attribute: daysSinceSignup
          operator: lessThanOrEquals
          value: 30
    translations:
      en: Place your first order
      nl: Plaats je eerste bestelling
  - key: express-checkout
    conditions:
      experiment: checkout-copy-test
      operator: hasVariation
      value: express
    translations:
      en: Express checkout
      nl: Snel afrekenen

New users in the express variation see a more personal message. Returning users in the same variation see the standard express copy. Everyone else sees the control.

Targeting a feature flag without an experiment#

When you want to serve different copy for users who have a feature enabled rather than for a specific experiment variation:

messages/billing/upgradePrompt.yml

description: Upgrade prompt shown to users who do not have the new billing UI
translations:
  en: Upgrade your plan
  nl: Upgrade je abonnement
overrides:
  - key: new-billing-ui
    conditions:
      feature: new-billing-ui
      operator: isEnabled
    translations:
      en: Explore new billing options
      nl: Ontdek nieuwe betaalopties

Users with the new-billing-ui feature enabled see the new copy. Everyone else sees the existing copy. Turning off the feature flag reverts the copy without any application change.

Passing experiment context from the application#

The SDK receives feature and experiment assignments through resolver functions. The exact implementation depends on your experiment system, but the pattern is:

JavaScript

import { createMessagevisor } from "@messagevisor/sdk";
import datafile from "./datafiles/messagevisor-web-en-US.json";
const m = createMessagevisor({
  datafile,
  locale: "en-US",
  context: {
    platform: "web",
    plan: currentUser.plan,
    daysSinceSignup: currentUser.daysSinceSignup,
  },
  resolveFlag: (featureKey, context) => featurevisor.isEnabled(featureKey, context),
  resolveVariation: (experimentKey, context) =>
    featurevisor.getVariation(experimentKey, context),
});
const ctaLabel = m.translate("auth.ctaSignup");

The exact integration shape depends on your application's experiment system. If you use Featurevisor, see your project's Featurevisor docs for how variation values flow into Messagevisor. If you do not, you can model the assigned variation as a normal Messagevisor attribute and use an attribute condition instead.

Testing copy variants#

Write explicit assertions for each variant to lock the copy and catch unintended changes during experiment rollout:

tests/messages/auth/ctaSignup.spec.yml

message: auth.ctaSignup
assertions:
  - description: Control - no experiment
    locale: en
    target: web
    expectedTranslation: Sign up
  - description: get-started variation
    locale: en
    target: web
    withVariations:
      cta-copy-test: get-started
    expectedTranslation: Get started
  - description: try-free variation
    locale: en
    target: web
    withVariations:
      cta-copy-test: try-free
    expectedTranslation: Try it free
  - description: Dutch get-started variation
    locale: nl
    target: web
    withVariations:
      cta-copy-test: get-started
    expectedTranslation: Aan de slag

Use withVariations to assert behavior per variation. These tests run in CI and catch any accidental change to variant copy before it reaches production.

Matrix for compact multi-locale variant coverage#

When you have many locales and several variants, the matrix assertion keeps the test file concise:

tests/messages/auth/ctaSignup.spec.yml

message: auth.ctaSignup
assertions:
  - matrix:
      locale: [en, nl]
    description: Control copy for ${{ locale }}
    locale: ${{ locale }}
    target: web
    expectedTranslation: ${{ locale == 'en' ? 'Sign up' : 'Aanmelden' }}

Reviewing variants in the catalog#

The catalog renders all overrides for a message in an expandable view. Reviewers can see:

The base (control) translation
Each override key and its conditions
The variant translations per locale

This makes experiment copy review accessible to product managers and content teams without requiring them to read YAML directly. Deep-linkable override rows let you point a stakeholder at a specific variant for sign-off before the experiment launches.

Rolling back a copy variant#

Because translation changes go through pull requests, rolling back a copy variant that performed poorly or contained an error is the same as any other Git rollback:

Command

$ git revert <commit-sha-that-added-the-variant>

After the revert merges and CI publishes updated datafiles, the variant copy is gone. No application deployment needed. The experiment can continue running with the control copy while the team decides whether to create a new variant.