A/B Testing

A/B testing, also called split testing or controlled experimentation, is the practice of randomly assigning users (or sessions, or visits) to two or more variants of an experience and measuring the impact of the variant on a target metric — the operational form of hypothesis testing applied to digital products. The technique was pioneered in agricultural and clinical experiments in the early twentieth century, adapted to direct mail and catalog marketing in the mid-century, and exploded with web and mobile traffic in the 2000s when companies like Google, Microsoft, Amazon, Netflix, Booking, and Airbnb made experimentation a core operating discipline. The infrastructure: a randomization layer assigns each user a variant on entry (consistent across sessions via a sticky bucketing key like user ID), an instrumentation layer logs which variant each user saw and their subsequent actions, an analytics layer computes per-variant metrics with statistical tests, and a decision layer reads results against pre-registered hypotheses. Production platforms include Optimizely (the enterprise leader for marketing experiments), VWO, AB Tasty, LaunchDarkly (feature flagging with experimentation), Statsig, Eppo, GrowthBook (open-source), and built-in experiment platforms at major tech companies (Microsoft's ExP, Netflix's ABlaze, Booking's experiment platform). The proper discipline requires pre-registration (hypothesis and metrics defined before running), adequate power (sample size calculated for the minimum detectable effect), guardrail metrics (count traffic, latency, error rates to catch regressions), randomization validation (no imbalance in covariates), multiple-comparison adjustment when many metrics are tested, and sequential testing or fixed-horizon analysis (peeking at results before the planned end inflates false positives without correction). Common pitfalls: under-powered tests, peeking, post-hoc metric selection, novelty effects masquerading as durable improvements, and Simpson's paradox in segment analysis. For Digital Experience Platforms, A/B testing is the operational mechanism that turns hypotheses into evidence-driven experience improvements — every content variant, layout change, and personalization rule should be evaluated this way.

A/B-tested experiences under a Magic Quadrant DXP: Centralpoint runs A/B tests on content variants, audience treatments, and personalization strategies — translating Gartner Magic Quadrant DXP capabilities into measured engagement lift rather than theoretical capability. Twenty-five years of experiment discipline informs the served experience. Tests run on-premise, lineage is audit-graded, and A/B-validated experiences deploy through one line of JavaScript.


Related Keywords:
A/B Testing,A/B Testing,Oxcyon, AI, AI Governance, Generative AI, Inference, Inference, Inferencing, RAG, Prompts, Skills Manager,