Work / Vanguard A/B Test
VANGUARD A/B TEST
A redesign test: +3.71 points completion, short of the 5-point goal
A redesign can improve a number and still miss its goal. This experiment measured the difference honestly.
The question
Did a redesigned online process help more people finish, by enough to justify the change?
The result
Completion rose from 65.58% to 69.29%. That is a real improvement of 3.71 percentage points, and it stays below the 5-point goal. Backward transitions also rose from 6.82% to 9.19% of web events, which argues against an unconditional rollout.
Redrawn from the project's published results.
65.58%Completion, control group
69.29%Completion, redesign
+3.71 ptsLift, against a 5-point goal