Check your understanding
Quiz
You have built the experimentation toolkit end to end: sizing a test so it can see the effect it hunts, sharpening it with CUPED, keeping it honest against peeking and sample-ratio mismatch, designing around interference with clusters and switchbacks, earning during the test with multi-armed bandits and Thompson sampling, and valuing a new policy from a logged campaign. This quiz checks what stuck. The last two steps are live R you can run.
Question 1 of 10
What 80% power means
A checkout test is designed with 80% power against a lift from 4.0% to 4.6%. Which reading is correct?
Correct: power is conditional on a specific assumed effect, and answers how often this design detects it. A different assumed lift gives a different power.
Power rises with the true effect and the sample size. Halve the effect and the same design's power drops sharply.