A/B Test Bucketing

A/B test bucketing decides which variant of an experiment each user sees, deterministically, so the same user always lands in the same variant and only a chosen slice of traffic enters the experiment at all. Teams run experiments to compare a change against the current behavior on live users and read the metrics per variant. For those metrics to mean anything the assignment must be sticky — a user cannot drift between variants between page loads — and independent per experiment — being in the treatment group of one test must not drag the same user into the treatment group of the next. You will build the assigner at the center of this: a factory that reads one experiment config and answers two questions for any user.

Signature

type Experiment = {
  key: string;        // unique experiment id — also the salt for the hash
  traffic?: number;   // 0..100, default 100 — the share of users let into the experiment
  variants: Array<{ key: string; weight: number }>; // relative weights, need not sum to 100
};

function abBucketing(experiment: Experiment): {
  assign(userId: string): string | null;  // the user's variant key, or null if not in the experiment
  isInExperiment(userId: string): boolean; // did the user pass the traffic gate?
};

Examples

A 50/50 split that stays sticky per user:

const homepageCta = abBucketing({
  key: 'homepage-cta',
  variants: [
    { key: 'control', weight: 50 },
    { key: 'treatment', weight: 50 },
  ],
});

// traffic defaults to 100, so every user is in. u_530 draws 'treatment'
// and keeps it on every call — the assignment is sticky.
homepageCta.assign('u_530'); // 'treatment'
homepageCta.assign('u_530'); // 'treatment' — same answer, always
homepageCta.assign('u_203'); // 'control'

A 20% traffic experiment that excludes most users:

const betaSearch = abBucketing({
  key: 'beta-search',
  traffic: 20, // only about 20% of users enter the experiment
  variants: [
    { key: 'control', weight: 50 },
    { key: 'treatment', weight: 50 },
  ],
});

betaSearch.isInExperiment('u_042'); // false — outside the 20% gate
betaSearch.assign('u_042');         // null — no variant for an excluded user
betaSearch.assign('u_128');         // null
betaSearch.assign('u_035');         // 'control' — one of the ~20% let in

Notes

• Deterministic, never random — derive the assignment from a hash of experiment.key and userId, never Math.random or Date.now. The same user must get the same variant on every call and in every session, or the experiment cannot be measured.
• The traffic gate comes first — assign returns null for any user outside the gate, and isInExperiment is true for exactly the users assign would give a variant. A missing traffic lets the whole audience in; a traffic of 0 lets no one in.
• Independent across experiments — because the hash is salted with experiment.key, a user's bucket in homepage-cta is unrelated to their bucket in checkout-flow. Landing in treatment for one experiment must not correlate with treatment in another.
• Weights are relative — a variant's share is its weight over the total of every weight, so [{ a, weight: 1 }, { b, weight: 3 }] is a 25/75 split. Weights need not sum to 100.
• Do not worry about — where experiments are stored, network fetching, logging exposures, or validating the config. You are handed one experiment object and only assign users.