Experiment Design
Export PDF (0)
Flagged
Completed
Export PDF (0)
0%
Coding
Theory
Quiz
Randomization Assignment
Deterministic A/B Variant Assignment
Junior
Deterministic Hash Bucketing
Junior
Arbitrary Two-Arm Traffic Split
Junior
Seeded Random Assignment
Junior
Vectorized A/B Bucket Assignment
Junior
Cluster Randomization Assignment
Mid
Weighted Variant Allocation
Mid
Salt Reuse Overlap Fraction
Mid
Session-Level Randomization Assignment
Mid
Global Holdout Over A/B Test
Mid
Realized Weighted Allocation Counts
Mid
Permuted-Block Randomization
Mid
Two-Step Variant Assignment
Senior
Cluster Rollup to Randomization Unit
Senior
Single-Hash Bucket Assignment (SRM Fix)
Senior
Orthogonal Overlapping Experiment Layers
Senior
Stratified Blocked Assignment
Senior
Exclusion Group Assignment
Senior
Sample Size Power Mde
Arcsine Two-Proportion Sample Size
Junior
Experiment Duration in Days
Junior
Power for Two-Proportion Test
Junior
Cohen's d With Pooled SD
Junior
Cohen's h Effect Size
Junior
Minimum Detectable Effect via Arcsine
Mid
Pooled Two-Proportion Sample Size
Mid
MDE for Two-Sample
t-Test
Mid
Beta By Alpha Trade-off
Mid
Unpooled Two-Proportion Sample Size
Mid
One- vs Two-Sided Sample Size
Mid
Unequal Allocation Sample Size
Mid
Sample Size for a Ratio Metric
Senior
Numeric MDE From Fixed Traffic
Senior
End-to-End A/B Experiment Planner
Senior
Power Curve Over Relative Lifts
Senior
Pooled Two-Proportion Power With
statsmodels
Senior
Metric Definition Aggregation
Pooled CTR by Variant
Junior
Triggered Conversion Rate by Variant
Junior
User-Level Conversion Rate by Variant
Junior
Latency Guardrail Regression Check
Junior
Metric Over Fixed Analysis Window
Junior
Pooled CTR vs Mean User CTR
Mid
Winsorize the Upper Tail
Mid
Winsorization Variance Reduction
Mid
Trigger Dilution: All vs Triggered Effect
Mid
Revenue Per User By Variant
Mid
Guardrail Error-Rate Check
Mid
Daily Metric Time Series by Variant
Mid
Delta-Method Ratio Metric Variance
Senior
Shared-Threshold Winsorization by Variant
Senior
Exposure-Relative Windowed Metric
Senior
Multi-Guardrail Non-Inferiority Evaluation
Senior
Composite OEC Score From Metrics
Senior
Experiment Analysis Lift
Relative Lift Calculation
Junior
Absolute Lift of Conversion Rate
Junior
Reject H0 From Confidence Interval
Junior
Conversion Rates Per Arm
Junior
Unpooled SE of Rate Difference
Mid
Pooled vs Unpooled Standard Errors
Mid
CI for Difference in Proportions
Mid
Welch's T-Test Metric Readout
Mid
Practically Significant Confidence Interval
Mid
Per-Variant Conversion Rate
Mid
Welch Confidence Interval for Mean Difference
Mid
Two-Proportion A/B Test Readout
Senior
Delta-Method SE for a Ratio Metric
Senior
Delta-Method Ratio Metric Readout
Senior
Delta-Method CI for Relative Lift
Senior
Log-Transform CI for Relative Lift
Senior
Continuous Metric Experiment Readout
Senior
Validity Checks Srm
SRM Chi-Square Goodness-of-Fit P-Value
Junior
SRM Alarm Decision
Junior
SRM Check For Unequal Split
Junior
SRM Chi-Square Goodness-of-Fit Statistic
Junior
SRM Report From Named Arms
Mid
Covariate Balance via SMD
Mid
Multi-Arm SRM Goodness-of-Fit Check
Mid
Per-Day SRM Alarm Detection
Mid
Flag Imbalanced Covariates by SMD
Mid
Filter Bots Before Mean Readout
Mid
Deduplicate Exposure Logs by First Touch
Mid
A/A Test False-Positive Rate
Senior
Detect Novelty Effect From Daily Lift
Senior
Detect Contaminated Users in Exposure Log
Senior
SRM Goodness-of-Fit vs Yates Contingency
Senior
A/A P-Value Uniformity Check
Senior
Unexposed Assigned Users Anti-Join
Senior
Variance Reduction Stratification
CUPED Theta Coefficient
Junior
CUPED Adjusted-Metric Transform
Junior
CUPED Expected Variance Reduction
Junior
ANCOVA Covariate-Adjusted Treatment Effect
Mid
CUPED-Adjusted Treatment Effect
Mid
End-to-End CUPED Metric Adjustment
Mid
Choose Best CUPED Covariate
Mid
CUPED Effective Sample-Size Multiplier
Mid
Post-Stratified ATE Estimator
Mid
Paired Difference Mean and SE
Mid
CUPED Effect And Standard Error
Senior
CUPED Large-Sample Readout
Senior
ANCOVA Treatment Effect and SE
Senior
Seeded CUPED Variance-Reduction Simulation
Senior
ANCOVA vs Unadjusted SE Reduction
Senior
Post-Stratified ATE and Standard Error
Senior
CUPED Theta by Scope
Senior
Sequential Testing Peeking
Pocock Alpha-Spending Function
Junior
Bonferroni Peeking Boundary
Junior
Apply Group-Sequential Stopping Boundary
Junior
Simulating Peeking Type-I Inflation
Mid
O'Brien-Fleming Alpha-Spending Function
Mid
mSPRT Likelihood-Ratio Statistic
Mid
Incremental Alpha Spent Per Look
Mid
Pocock Constant-Boundary Stop
Mid
O'Brien-Fleming Alpha Spent at Sample Sizes
Mid
Nominal Per-Look Z Cutoffs
Mid
Naive vs Bonferroni Peeking FPR
Senior
Always-Valid p-Value Sequence via mSPRT
Senior
Confidence Sequence Stopping Rule
Senior
Alpha-Spending Nominal Z-Boundaries
Senior
mSPRT Always-Valid Stopping Rule
Senior
Normal-Mixture Confidence Sequence
Senior
Group-Sequential Boundary Crossing Readout
Senior
Multiple Testing Multi Variant Designs
Bonferroni-Corrected P-Values
Junior
Family-Wise Error Rate for
m
Tests
Junior
Count Significant P-Values With Bonferroni
Junior
Benjamini-Hochberg FDR Correction
Mid
Compare Multiple-Testing Corrections
Mid
Benjamini-Hochberg Step-Up Reject Rule
Mid
Holm-Bonferroni Reject Decisions
Mid
A/B/n Readout With Holm Correction
Mid
BH-Correct a Named P-Value
Series
Mid
Corrected Alpha Thresholds From
multipletests
Mid
Hand-Rolled Benjamini-Hochberg FDR
Senior
Hand-Rolled Holm P-Value Adjustment
Senior
2x2 Factorial Interaction Effect
Senior
Seeded FWER Inflation Simulation
Senior
Benjamini-Yekutieli FDR by Hand
Senior
Staged Ramp Rollout Gate
Senior
Top 236 Experiment Design Interview Questions And Answers 2025 | TechPrep