Fundamentals & Components
What is the difference between a point estimate and an interval estimate, and why is an interval estimate often more informative?
How would you explain confidence intervals to a non-technical audience or business stakeholder?
What does the 95% confidence level actually mean?
Explain why a point estimate alone is often insufficient, and how a confidence interval addresses this limitation.
What is a confidence interval, and what is its purpose?
Describe the components of a confidence interval, specifically the point estimate, margin of error, and critical value.
How do you correctly read confidence intervals presented as error bars on a chart?
What is the margin of error, and how is it computed and interpreted?
Sampling & Estimation Foundations
What is a sampling distribution?
What is the difference between a population and a sample?
What is the standard error?
What is the difference between a parameter and a statistic in the context of interval estimation?
What is the difference between standard deviation and standard error, and why does the standard error typically decrease as sample size increases?
Why does the standard error shrink with the square root of n?
If you don't know the population standard deviation, how do you calculate the standard error for a sampling distribution?
How do estimator properties like unbiasedness, consistency, and efficiency motivate how we report uncertainty around an estimate?
Interpretation & Misconceptions
How do you interpret a 95% confidence interval? What does 'confidence' actually mean in this context, and what are common misinterpretations?
What are the common misinterpretations of a confidence interval, and why is it incorrect to say there's a 95% probability that the true parameter lies within this specific calculated interval?
What does the confidence attach to — the method or the single computed interval?
Why is it a mistake to interpret a 95% confidence interval as containing 95% of the data or 95% of future sample means?
Is a confidence interval a random or fixed quantity, and what is random once the interval has been computed from a sample?
What does 'coverage' mean for a confidence interval?
Interval Width & Sample Size
How does sample size affect the width of a confidence interval?
How would you reduce the width of a confidence interval?
What factors affect the width of a confidence interval, and how does each factor such as sample size, confidence level, and population variability influence it?
How can you determine the required sample size for an experiment given a target margin of error and a desired confidence level?
Can you explain the trade-off between the confidence interval width and the confidence level?
How does population variability affect the width of a confidence interval?
Hypothesis Testing & Inference
What is the relationship between a confidence interval and a hypothesis test, and how can you use a confidence interval to infer the result of a two-sided hypothesis test?
Why is a confidence interval often more informative than a bare p-value?
Why should confidence intervals be reported alongside effect sizes rather than just point estimates or p-values?
What is the overlapping confidence intervals fallacy, and if two confidence intervals for different groups overlap does that necessarily mean there is no statistically significant difference?
At a conceptual level, how do confidence intervals relate to non-inferiority or equivalence conclusions?
How exactly does a two-sided 95% confidence interval correspond to the set of null values a two-sided test at alpha 0.05 would fail to reject?
What is the problem of simultaneous or multiplicity when reporting many confidence intervals at once, and how would you adjust for it?
Distributions & Critical Values
What is a critical value, and how do you choose z-star versus t-star and the value corresponding to a given confidence level?
What is the success/failure rule of thumb, and why do we check it before using a normal-approximation interval for a proportion?
What is the Central Limit Theorem, and why is it fundamental to the construction of many confidence intervals? What are its key assumptions?
When would you use a z-interval versus a t-interval for estimating a population mean, and what is the role of the t-distribution and degrees of freedom?
Why is the t-interval wider than the z-interval, and at what point does the difference between them stop mattering?
Ab Testing & Decisions
How can you use a confidence interval to make a decision in an A/B test, such as what it means if an interval straddles zero?
If an A/B test shows a positive lift but the confidence interval for the treatment effect is very wide, how would you interpret this and what would you recommend next?
Two Sample & Proportion Intervals
What is the Wald interval for a proportion and why does it perform poorly at extreme probabilities or small sample sizes?
What is the difference between the pooled two-sample t-interval and the Welch interval when estimating the difference between two means, and when should you prefer each?
How would you construct a confidence interval for the difference between two proportions, and how do you read whether the interval indicates a real difference?
How does constructing a confidence interval for paired or matched data differ from the two-independent-sample case?
What are the challenges of constructing a confidence interval for a proportion, especially with small sample sizes or extreme probabilities, and what methods address these?
Can you explain how the Wilson score interval improves on the Wald interval for a proportion?
What is the Clopper-Pearson exact interval, and what are the tradeoffs of using it versus approximate proportion intervals?
How would you construct and interpret a confidence interval for a ratio or relative lift rather than an absolute difference?
Interval Types & Regression Bands
Why is a prediction interval always wider than a confidence interval?
What does the standard error of a regression coefficient represent, and how is it used to form a confidence interval around that coefficient?
What is a one-sided confidence interval, and when would you report an upper or lower confidence bound instead of a two-sided interval?
What is the distinction between a Confidence Interval, a Prediction Interval, and a Tolerance Interval, and when would each be appropriate?
Conceptually, how does a frequentist confidence interval differ from a Bayesian credible interval?
What is the difference between a confidence band around a fitted regression mean and a prediction band, and why is the prediction band wider?
Bootstrap & Resampling
What is bootstrapping, and when would you use it to construct a confidence interval, and what are its advantages and disadvantages?
What is the difference between the percentile bootstrap interval and the basic (reverse) bootstrap interval?
What is the BCa bootstrap interval, and what problems does the bias-correction and acceleration address?
Nonnormal & Robust Methods
How can small or uneven samples distort results, and how would you use stratified sampling or confidence interval checks to adjust for bias?
How would you identify the distribution of data from a sample, generalize it to the population, and find the confidence interval?
What happens to normal-based confidence intervals when the assumptions of the Central Limit Theorem fail, for example with heavy-tailed or highly skewed data?
How would you build a confidence interval for a median or another quantile, and why can this be harder than for a mean?
How would you build a confidence interval when data is skewed and you apply a log transformation, and how do you interpret the interval after back-transformation?