64 Confidence Intervals Interview Questions and Answers (2026)

A point estimate is a single best-guess value for a parameter, while an interval estimate gives a range of plausible values along with a stated confidence. The interval is often more informative because it quantifies the uncertainty around that guess.

• Point estimate:

  • A single number (e.g. sample mean of 5.2) that estimates the true parameter.

  • Says nothing about how precise or reliable it is.

• Interval estimate:

  • A range (e.g. 4.8 to 5.6) built around the point estimate with a confidence level.

  • Communicates both the estimate and the sampling uncertainty in one statement.

• Why intervals are more informative:

  • A wide interval warns you the estimate is imprecise; a narrow one signals confidence.

  • Two studies can share the same point estimate but very different intervals, and only the interval reveals that difference.

I'd frame it as a 'best guess plus a margin of uncertainty': instead of promising one exact number, we give a range we're fairly sure the true answer falls in, because we only measured a sample, not everyone.

• Use a relatable analogy:

  • "We estimate customer satisfaction is 78%, give or take 3 points, so realistically between 75% and 81%."

  • Like a weather forecast: not a single certainty but a confident range.

• Emphasize the business value:

  • The width tells them how much to trust the number before making a decision.

  • A tight range means act with confidence; a wide range means gather more data.

• Avoid jargon: Skip words like 'sampling distribution'; talk about 'how sure we are' and 'the range of likely values'.

A 95% confidence level refers to the method, not a single interval: if you repeated the sampling and interval-building process many times, about 95% of those intervals would contain the true parameter. It is not a 95% probability that the true value lies in your one specific interval.

• The correct (frequentist) interpretation:

  • Over many repeated samples, 95% of the constructed intervals cover the true value.

  • The randomness is in the interval (which shifts with each sample), not in the fixed true parameter.

• The common mistake: Saying "there's a 95% chance the true mean is in this interval" is technically wrong in frequentist terms: once computed, the interval either contains it or it doesn't.

• What 95% controls: The long-run success rate of the procedure, tied to the 5% chance of producing a miss.

A point estimate alone is insufficient because it hides how much it could vary from sample to sample: it presents a single number as if it were exact when it's really subject to sampling error. A confidence interval fixes this by attaching a quantified range of uncertainty.

• Why the point estimate falls short:

  • It varies from sample to sample but reports no measure of that variability.

  • It almost never equals the true parameter exactly, yet looks deceptively precise.

  • It can't distinguish a reliable estimate from a shaky one.

• How the interval addresses it:

  • Adds a margin of error reflecting sample size and variability.

  • Larger samples and lower variability produce narrower, more precise intervals.

  • Lets decision-makers judge whether the precision is adequate for the stakes involved.

A confidence interval is a range of plausible values for an unknown population parameter, computed from sample data with an associated confidence level. Its purpose is to express both an estimate and the uncertainty around it in a single, decision-friendly statement.

• What it is:

  • An interval (lower bound to upper bound) built around a point estimate.

  • Attached to a confidence level (commonly 90%, 95%, or 99%).

• Its purpose:

  • Quantify sampling uncertainty: how far the estimate might be from the truth.

  • Support inference: e.g. if a hypothesized value falls outside the interval, that's evidence against it.

  • Aid decisions by showing the range of outcomes that are consistent with the data.

• General form: point estimate ± (critical value × standard error).

A confidence interval is built from three parts: a point estimate that centers it, a critical value that sets the confidence level, and a standard error that measures variability; the critical value times the standard error forms the margin of error added to and subtracted from the estimate.

• Point estimate: The sample statistic (mean, proportion) at the center of the interval.

• Critical value:

  • Comes from the sampling distribution and the chosen confidence level (e.g. z = 1.96 for 95%, or a t-value for small samples).

  • Higher confidence means a larger critical value and a wider interval.

• Margin of error:

  • Equals critical value × standard error.

  • Defines the half-width of the interval on each side of the estimate.

• Putting it together: CI = point estimate ± margin of error.

Error bars showing a confidence interval represent the range of plausible values around each point estimate, so you read the length of the bar as precision and use overlap between bars as a rough (not definitive) guide to whether groups differ.

• Read the bar itself:

  • The dot/bar is the point estimate; the whiskers span the confidence interval.

  • Longer bars mean more uncertainty; shorter bars mean a more precise estimate.

• Compare groups carefully:

  • If one group's interval excludes another's point estimate, they likely differ meaningfully.

  • Non-overlapping intervals strongly suggest a difference; slight overlap does NOT guarantee no difference.

• Check what the bars actually represent: Error bars may show standard deviation, standard error, or a CI: always read the caption, since they have very different widths.

The margin of error is the half-width of a confidence interval: how far, in either direction, the true value might plausibly lie from the point estimate. It is computed as the critical value times the standard error and interpreted as the maximum expected estimation error at the chosen confidence level.

• How it's computed:

  • MoE = critical value × standard error (e.g. 1.96 × SE for a 95% z-interval).

  • Standard error shrinks as sample size grows (it scales with 1/√n).

• What drives it:

  • Larger samples reduce the margin; higher variability increases it.

  • Higher confidence levels widen it via a larger critical value.

• How to interpret it:

  • "Our estimate is 78% ± 3%" means the interval runs 75% to 81%.

  • It reflects only random sampling error, not bias from bad sampling or measurement.

A sampling distribution is the probability distribution of a statistic (e.g. the sample mean) computed over all possible samples of a given size from a population. It describes how the estimate varies from sample to sample and is the theoretical basis for confidence intervals.

• It is a distribution of a statistic, not of raw data: Imagine drawing many samples of size n, computing the mean of each; those means form the sampling distribution.

• Its spread is the standard error: A tighter sampling distribution means a more precise estimate.

• Central Limit Theorem: For large n the sampling distribution of the mean is approximately normal regardless of the population's shape, which lets us build intervals using normal or t critical values.

• It is usually theoretical: We rarely draw many samples; we infer its properties from one sample plus known math.

A population is the entire set of units you want to draw conclusions about; a sample is the subset you actually observe. We use samples because measuring the whole population is usually impossible or impractical, and inference bridges the gap.

• Population: All members of interest (e.g. every voter in a country); its summaries are parameters like the mean and standard deviation.

• Sample: A selected subset; its summaries are statistics used to estimate the parameters.

• Why the distinction matters for intervals:

  • A confidence interval quantifies uncertainty about the unknown population parameter using only sample information.

  • A representative, randomly drawn sample is what makes that inference valid; bias in sampling breaks it.

The standard error is the standard deviation of a statistic's sampling distribution: it quantifies how much an estimate would vary if you repeatedly resampled. It is the core precision measure used to build confidence intervals.

• Formula for the mean: SE = sigma / sqrt(n), estimated by s / sqrt(n) when the population SD is unknown.

• Interpretation: Small SE means your estimate is precise; large SE means it could easily have come out different.

• Role in intervals: A confidence interval is typically estimate ± (critical value × SE).

• Every statistic has its own SE: Proportions, regression coefficients, and differences each have their own SE formula.

A parameter is a fixed but unknown number describing the whole population; a statistic is a computed value from a sample used to estimate that parameter. Interval estimation is precisely the task of using a statistic to bracket the unknown parameter with a stated confidence.

• Parameter: The target of inference (e.g. population mean μ or proportion p); it is constant, not random.

• Statistic: A function of sample data (e.g. sample mean x̄); it is random because it depends on which sample you drew.

• How they connect in a confidence interval:

  • The interval is random (built from a statistic and its SE); the parameter is fixed.

  • So 95% confidence means 95% of such intervals across repeated samples cover the parameter, not that the parameter has a 95% chance of being in one specific interval.

Larger sample size narrows the confidence interval, because the standard error scales as 1/√n. The relationship has diminishing returns: it takes 4× the data to halve the width.

• Mechanism: Width is proportional to SE = σ / √n, so as n grows the margin of error shrinks.

• Square-root, not linear: Going from n=100 to n=400 (4×) halves the width; from n=100 to n=200 shrinks it only by about 30%.

• Secondary effect: For small samples the t critical value is inflated; as n grows t approaches z, giving a small extra tightening.

• Interpretation: More data means a more precise estimate of the parameter, not a change in the parameter itself.

CI half-width is roughly (critical value)·(standard error), so shrink it by increasing sample size, reducing variability, or accepting a lower confidence level. Sample size is usually the lever you control.

• Increase the sample size: Width scales with 1/√n, so quartering the width needs ~16× the data (diminishing returns).

• Reduce variability (σ): Better measurement, more homogeneous sampling, or stratification/blocking to remove nuisance variance.

• Lower the confidence level: A 90% interval is narrower than 95%, but you trade away coverage/certainty.

• Use a more efficient design or estimator: Paired designs and covariate adjustment (e.g. CUPED) cut the standard error without more samples.

A critical value is the multiplier that scales the standard error to set the interval's width for a chosen confidence level: it is the point on the reference distribution that leaves the right tail area. You pick z* when using the normal distribution and t* when using the t-distribution.

• Definition: For a two-sided level C, the critical value cuts off area (1 − C)/2 in each tail (e.g. 2.5% each for 95%).

• Common z* values: 90% → 1.645, 95% → 1.96, 99% → 2.576.

• When to use z*: Known σ, or large-sample proportion intervals where the normal approximation holds.

• When to use t*: Estimating a mean with σ unknown; t* depends on both the confidence level and df, and is always larger than the matching z*.

• How to find it: From tables or software: e.g. qnorm(0.975) gives 1.96; qt(0.975, df) gives t*.

The success/failure rule says you should have at least about 10 expected successes and 10 expected failures before trusting a normal-approximation (Wald) interval for a proportion. It is a check that the binomial's sampling distribution is symmetric and bell-shaped enough for the normal approximation to work.

• The condition: Require np̂ ≥ 10 and n(1 − p̂) ≥ 10 (some texts use 5 or 15).

• Why it's needed:

  • The count of successes is binomial; when p is near 0 or 1 (or n is small) that distribution is highly skewed, so a symmetric normal interval fits poorly.

  • Violations cause poor coverage (the true rate can be well below 95%) and intervals that spill past 0 or 1.

• What to do if it fails: Use a better interval: Wilson score, Agresti-Coull (add pseudo-counts), or exact Clopper-Pearson.

Standard deviation measures the spread of individual data points in a population or sample, while standard error measures the spread of a sample statistic (like the mean) across repeated samples. Standard error shrinks with larger samples because averaging more observations averages out random noise.

• Standard deviation (SD): Describes variability of raw observations; it is a property of the data and does not shrink as you collect more.

• Standard error (SE):

  • Describes variability of an estimate; for the mean, SE = SD / sqrt(n).

  • It is the SD of the sampling distribution of the statistic.

• Why SE decreases with n:

  • More observations give the average more information, so estimates from different samples cluster tighter around the true value.

  • Note SD stays roughly constant as n grows; only SE falls.

The standard error scales with 1/sqrt(n) because the variance of a sum of independent observations grows linearly with n, and taking the mean divides by n, leaving variance proportional to 1/n. Taking the square root to get the standard error gives 1/sqrt(n).

• The algebra:

  • Var(sum) = n·sigma² for independent observations.

  • Var(mean) = Var(sum)/n² = sigma²/n, so SD(mean) = sigma/sqrt(n).

• Practical consequence: diminishing returns: To halve the SE you must quadruple n, because precision improves with the square root, not linearly.

• Key assumption: Observations must be independent (or nearly so); correlated data shrinks slower than 1/sqrt(n).

When the population standard deviation is unknown (almost always), you estimate it with the sample standard deviation s and plug it into the formula, giving an estimated standard error. Because you're using an estimate of variability, you switch from the normal to the t distribution for the critical value.

• Estimate the SE: Use SE = s / sqrt(n), where s is the sample SD computed with the n-1 (Bessel) correction for unbiasedness.

• Use the t distribution: With s in place of sigma, the standardized statistic follows a t distribution with n-1 degrees of freedom, which has heavier tails to account for the extra uncertainty.

• Large samples: As n grows the t distribution converges to the normal, so the distinction matters most for small samples.

A 95% confidence interval is a range computed from data by a procedure that, over many repeated samples, captures the true parameter 95% of the time. 'Confidence' is a property of the method, not a probability about the one interval you happen to have.

• Correct interpretation:

  • If you repeated the sampling and interval-building process many times, about 95% of those intervals would contain the true parameter.

  • The 95% describes the long-run success rate of the procedure.

• What 'confidence' means: A frequentist reliability guarantee about the estimator, established before you see the data.

• Common misinterpretations:

  • Saying there is a 95% probability the true value is in this specific interval: once computed, it either does or does not contain it.

  • Thinking it contains 95% of the data points or 95% of future estimates.

  • Treating the interval endpoints as random after computation: they are fixed numbers.

The core error is treating a single computed interval as if it had a probability of containing the parameter. In frequentist statistics the parameter is a fixed unknown constant and the computed interval is a fixed set of numbers, so the true value is either inside it or not: there is no randomness left to assign a 95% probability.

• Why '95% probability in this interval' is wrong:

  • Probability was attached before sampling, to the random procedure. After you compute (2.1, 4.3) nothing is random anymore.

  • The parameter is fixed; the coverage probability of 0.95 refers to the collection of possible intervals, not this realization.

• Other frequent misreadings:

  • '95% of the data fall in the interval.'

  • '95% of future sample means fall in the interval.'

  • 'There is a 95% chance the next sample's estimate lands here.'

• The nuance: A Bayesian credible interval does let you say 'given the data, 95% probability the parameter is here', but that requires a prior and a different framework.

Confidence attaches to the method (the procedure that generates intervals), not to any single computed interval. The 95% is a long-run coverage rate of the estimator across repeated samples.

• The method has the 95% property: Across hypothetical repeated samples, 95% of the intervals the procedure produces contain the true parameter.

• The single interval does not: Once computed, it either contains the parameter or it does not: its success is 0 or 1, not 0.95.

• Analogy: Like a manufacturing process that produces 95% good parts: the process has the rate, but any one part in your hand is simply good or defective.

A confidence interval is a statement about the location of a single parameter (like a mean), not about the spread of data or the distribution of future estimates. Confusing it with those quantities mixes up fundamentally different intervals.

• Not 95% of the data:

  • That describes a range covering raw observations, closer to a tolerance interval or mean ± ~2 SD (using the standard deviation, not the standard error).

  • A CI shrinks toward the mean as n grows; a data-spread range does not.

• Not 95% of future sample means: That is closer to a prediction interval for a future estimate, which is a different construction.

• Key distinction: CI width uses the standard error (SE = SD / √n); data-coverage ranges use the standard deviation. They answer different questions.

Before you collect data the interval is random, because its endpoints are functions of the random sample. After you compute it from an observed sample, it is fixed: the numbers no longer vary, and the parameter was fixed all along.

• Before sampling: The endpoints are random variables; the procedure has a 95% chance of yielding an interval covering the parameter.

• After computing: The endpoints are fixed numbers; coverage is now 0 or 1, just unknown to us.

• What is never random: The true parameter: it is a fixed (though unknown) constant in the frequentist view.

• So the randomness lives in the sampling: This is exactly why probability statements apply to the procedure, not to the realized interval.

Interval width is driven by three main levers: sample size, confidence level, and population variability (plus the resulting standard error). A typical mean CI is estimate ± (critical value × standard error), so anything that changes the critical value or the standard error changes the width.

• Sample size (n): Larger n shrinks the standard error by 1/√n, so the interval narrows. Quadrupling n roughly halves the width.

• Confidence level: Higher confidence (99% vs 95%) uses a larger critical value, widening the interval. More certainty costs precision.

• Population variability (σ or s): More spread in the data increases the standard error and widens the interval. Less variability narrows it.

• Bundled into the standard error:

  • SE = σ / √n, so n and σ act through it; the critical value scales it up for higher confidence.

  • Small samples also use t critical values (fatter tails), adding a bit more width.

Invert the margin-of-error formula for the mean or proportion, solving for n. You pick the confidence level (which fixes the critical value), the target margin of error, and an estimate of variability, then compute the sample size needed.

• Formula for a mean:

  • Margin E = z·(σ/√n), so n = (z·σ / E)².

  • Needs a prior estimate of σ (pilot study, historical data, or range/4 rule).

• Formula for a proportion:

  • n = z²·p(1−p) / E².

  • If p is unknown, use p = 0.5: it maximizes p(1−p) and gives the safest (largest) n.

• The three inputs you must fix:

  1. Confidence level → critical value z (e.g. 1.96 for 95%).

  2. Margin of error E → the half-width you can tolerate.

  3. Variability estimate → σ or p.

• Always round n up, and use a t-based/iterative version if σ is unknown and n is small.

There is a direct tension: higher confidence requires a wider interval. To be more sure the interval captures the true parameter, you must cast a wider net, given fixed data.

• Confidence level sets the critical value: 90% → z≈1.645, 95% → z≈1.96, 99% → z≈2.576; larger multiplier means wider interval.

• Precision vs. certainty: A narrow interval is precise but less likely to contain the truth; a wide one is more certain but less informative.

• The escape hatch: The only way to get both high confidence and narrow width is to collect more data (or reduce variance).

• Degenerate extremes: 100% confidence gives an infinitely wide (useless) interval; 0% gives a single point.

More population variability widens the interval directly: the standard error is proportional to the standard deviation, so a noisier population produces a less precise estimate for the same sample size.

• Width is driven by the standard error: SE = σ/√n, and half-width = critical value · SE, so width ∝ σ.

• Intuition: When observations scatter widely, any single sample tells you less about the true mean.

• Interaction with n: High σ can be offset by a larger n, since only the ratio σ/√n matters.

• Practical takeaway: reducing variance (cleaner measurement, stratification) is often cheaper than gathering enough data to overpower it.

A (1−α) confidence interval and a two-sided hypothesis test at level α are duals: the interval is exactly the set of null values that would not be rejected. So if the null value falls outside the CI, you reject; if it falls inside, you fail to reject.

• The rule for a two-sided test:

  • H₀ value (e.g. 0 for a difference, 1 for a ratio) outside the 95% CI → reject at α=0.05.

  • H₀ value inside the CI → fail to reject.

• Why they match: Both use the same point estimate, standard error, and critical value; the CI just inverts the test.

• The CI tells you more: It shows the direction and plausible magnitude of the effect, not just a reject/fail-to-reject verdict.

• Caveat: the confidence level and the test's α must correspond (95% CI ↔ two-sided 0.05 test); a one-sided test pairs with a one-sided interval.

A p-value collapses everything into a single significance verdict, while a confidence interval reports the estimated effect and a range of plausible values on the actual scale of the quantity. It answers not just "is there an effect?" but "how big, and how precise?"

• Magnitude and direction: The CI is expressed in real units (e.g. +2 to +8 ms), so you see practical importance; a p-value does not.

• Precision: A wide CI signals a noisy/underpowered estimate; a tiny p-value alone hides this.

• Contains the test result: Whether it excludes the null value gives you the significance decision anyway.

• Guards against misreading: Highlights the difference between statistical significance and practical significance (a trivial effect can be significant with huge n).

An effect size tells you how large the effect is, and the CI tells you how uncertain that estimate is; together they convey practical meaning and precision that a point estimate or p-value alone cannot. A point estimate hides uncertainty, and a p-value hides magnitude.

• Point estimate alone is incomplete: It gives a single number with no sense of how much it might vary across samples.

• P-value alone is incomplete: "Significant" says nothing about whether the effect is large enough to matter.

• Effect size + CI together:

  • Shows the estimated magnitude and the range of plausible values, enabling a business/clinical judgment.

  • Lets readers assess whether even the most optimistic or pessimistic end of the interval is meaningful.

• Supports meta-analysis and reproducibility, since effect sizes with intervals can be pooled and compared across studies.

The Central Limit Theorem (CLT) says the sampling distribution of the sample mean (or sum) of independent observations approaches a normal distribution as sample size grows, regardless of the population's shape. This is what lets us build normal- and t-based intervals even for non-normal data.

• Statement: For iid data with mean μ and finite variance σ², the standardized mean (x̄ − μ)/(σ/√n) converges to a standard normal as n → ∞.

• Why it matters for CIs: It justifies the estimate ± z*·SE form: the estimator is approximately normal even when the underlying data are skewed, so we can use normal (or t) critical values.

• Key assumptions:

  1. Independence: observations don't influence each other (random sampling; ideally n < 10% of population if sampling without replacement).

  2. Identically distributed / finite variance: σ² must exist (heavy-tailed distributions like Cauchy break the CLT).

  3. Adequate sample size: how large depends on skewness (n ≈ 30 is a rough guide; strongly skewed data need more).

• Caveat: The CLT is about the distribution of the statistic, not the raw data: larger n makes the sampling distribution normal, it does not make the data normal.

Use a z-interval when the population standard deviation is known (or n is large enough that estimating it barely matters); use a t-interval when σ is unknown and estimated from the sample, which is the usual real-world case. The t-distribution accounts for the extra uncertainty from estimating σ.

• z-interval:

  • Uses x̄ ± z*·(σ/√n) with the known population σ.

  • Rare in practice for means: σ is seldom truly known.

• t-interval:

  • Uses x̄ ± t*·(s/√n) with the sample SD s.

  • This is the default for a single mean when σ is unknown.

• Role of the t-distribution: It is bell-shaped but has heavier tails than the normal, giving wider intervals to compensate for the randomness in s.

• Degrees of freedom:

  • For one mean, df = n − 1; fewer df means heavier tails and a larger t*.

  • As df grows the t-distribution converges to the normal, so the two intervals coincide for large n.

• Assumption for both: Roughly normal data or large enough n (via the CLT); the t-interval is fairly robust to mild non-normality but sensitive to strong skew or outliers at small n.

The t-interval is wider because it uses the t-distribution, whose heavier tails produce a larger critical value than z* to account for the extra uncertainty of estimating σ with the sample SD. That extra width shrinks as the sample grows, and by roughly n ≥ 30 (df ≈ 30+) the difference is negligible.

• Source of the extra width: Using s in place of the true σ adds variability; the t-distribution's fatter tails widen the interval to keep coverage honest.

• Magnitude at small n:

  • At df=5, t* for 95% is ~2.57 vs z*=1.96: a substantially wider interval.

  • At df=30, t* ≈ 2.04, very close to 1.96.

• When it stops mattering:

  • t converges to the normal as df → ∞; practically the gap is trivial once n is a few dozen.

  • Rule of thumb: default to t regardless, since it self-corrects and equals z in the large-sample limit anyway.

In an A/B test you build a confidence interval for the treatment effect (difference in means or proportions) and read whether zero (no effect) is a plausible value: if the interval excludes zero the result is statistically significant, and its sign tells you the direction.

• Interval straddles zero (contains 0):

  • You cannot rule out "no difference" at that confidence level: the effect is not statistically significant.

  • Equivalent to a two-sided test failing to reject the null at the matching alpha.

• Interval entirely above zero: Treatment plausibly beats control across the whole range: a significant positive lift.

• Interval entirely below zero: Treatment plausibly hurts: significant negative effect, don't ship.

• Look at the whole range, not just significance:

  • Compare the interval to your practical/minimum meaningful effect: significant but tiny may not be worth shipping.

  • An interval that excludes zero but includes trivially small values means "real but maybe not useful."

The Wald interval is the textbook normal-approximation interval for a proportion, centered on the observed proportion with a standard error estimated from that same proportion. It is simple but has poor and erratic coverage at small n or extreme p.

• Formula: p̂ ± z·√(p̂(1−p̂)/n), where p̂ is the sample proportion and z the normal critical value.

• Why it fails at extremes / small n:

  • Uses p̂ in the standard error: when p̂ is near 0 or 1, p̂(1−p̂) shrinks toward 0, making the interval artificially narrow.

  • Degenerate at 0 or n successes: SE becomes 0, so the interval has zero width and false certainty.

  • Symmetric and unbounded: can produce limits below 0 or above 1.

  • Relies on the normal approximation, which needs a fairly large n and p not near the boundaries.

• Consequence: actual coverage often falls well below the nominal 95%, so prefer Wilson or Agresti-Coull.

Both estimate the difference between two group means, but the pooled t-interval assumes equal variances and combines them into one estimate, while Welch does not and adjusts the degrees of freedom. Welch is the safer default; pooled is only marginally tighter when variances truly are equal.

• Pooled two-sample t-interval:

  • Assumes both groups share a common variance and pools them into one estimate.

  • Uses df = n1 + n2 − 2.

  • If the equal-variance assumption is wrong (especially with unequal sample sizes), coverage can be badly off.

• Welch interval:

  • Keeps each group's variance separate, so no equal-variance assumption.

  • Uses the Welch-Satterthwaite approximation for (fractional) degrees of freedom.

  • Robust when variances or sample sizes differ.

• When to prefer each:

  • Default to Welch: it costs little when variances are equal and protects you when they aren't.

  • Use pooled only with a genuine reason to believe variances are equal (e.g. designed experiment) and roughly balanced groups.

You estimate the difference p̂1 − p̂2 and put an interval around it; if that interval excludes zero, the two proportions plausibly differ, and its sign shows which group is higher. As with single proportions, prefer a method better than plain Wald.

• Basic (Wald) construction:

  • (p̂1 − p̂2) ± z·√(p̂1(1−p̂1)/n1 + p̂2(1−p̂2)/n2).

  • Standard errors add because the two samples are independent.

• Better methods:

  • Newcombe's method combines two Wilson intervals for far better coverage at small n or extreme rates.

  • Agresti-Caffo (add a bit to each cell) is a simple improvement over Wald.

• Reading the interval:

  • Contains 0: no significant difference detected.

  • Entirely positive or negative: a real difference in that direction.

  • Also judge magnitude against what is practically meaningful, not just exclusion of zero.

With paired data you reduce each matched pair to a single difference and build a one-sample interval on those differences, whereas the two-independent-sample case treats the groups as unrelated. Pairing removes between-subject variability, usually giving a tighter interval.

• Paired approach:

  • Compute d_i = x_i − y_i for each pair, then a one-sample t-interval: d̄ ± t·(s_d/√n), with df = n − 1 (n = number of pairs).

  • It uses only the within-pair variability of the differences.

• Why it differs from two-sample:

  • Observations aren't independent: pairs are correlated (same subject, matched twins, before/after).

  • Pairing cancels shared nuisance variation, so s_d is often much smaller than the pooled between-group spread.

  • Positive correlation within pairs is what makes the paired interval narrower and more powerful.

• Practical note: only valid when data are genuinely paired; treating paired data as independent overstates uncertainty and wastes power.

A prediction interval must account for two sources of uncertainty (uncertainty in estimating the mean plus the natural scatter of an individual observation), whereas a confidence interval only accounts for the first. That extra variance term makes it strictly wider.

• Confidence interval variance:

  • Only the standard error of the estimated mean: roughly σ²/n.

  • This vanishes as n → ∞, so the CI can become arbitrarily narrow.

• Prediction interval variance:

  • Adds the irreducible variance of a single point: σ² + σ²/n.

  • Even with infinite data the σ² term remains, so the PI never shrinks below the spread of the data.

• Intuition: knowing the mean perfectly still doesn't tell you where one random draw lands.

The standard error of a regression coefficient measures how much that estimated coefficient would vary across repeated samples: it's the estimated standard deviation of the coefficient's sampling distribution. You use it with a t-quantile to form the coefficient's confidence interval.

• What it captures:

  • Grows with residual variance σ² and shrinks with more data and more predictor spread.

  • Inflated when predictors are collinear (high variance inflation factor).

• Forming the interval:

  • CI = β̂ ± t(α/2, df)·SE(β̂), with df = n - p.

  • The t-statistic β̂ / SE(β̂) tests whether the coefficient is 0.

• Interpretation:

  • If the interval excludes 0, the predictor is significant at that level.

  • Relies on the usual OLS assumptions; use robust or clustered SEs when errors are heteroscedastic or correlated.

A one-sided confidence interval bounds a parameter from only one direction (giving just an upper or just a lower limit), putting the entire error allowance on that one side. You use it when only one direction of the parameter matters for the decision.

• How it differs from two-sided:

  • A two-sided 95% CI splits 5% into two 2.5% tails; a one-sided 95% bound puts all 5% in one tail.

  • So the one-sided bound is tighter in its direction (uses z(0.95) instead of z(0.975)).

• When to report an upper bound: When you care only that something isn't too large: e.g. an upper bound on contaminant level or defect rate.

• When to report a lower bound: When you care only that something is at least some value: e.g. minimum strength, minimum reliability, or non-inferiority.

• Caveat: choose one-sided in advance based on the question, not after seeing the data, or you inflate the effective error rate.

These properties describe how good an estimator is, which directly justifies why and how we attach uncertainty to an estimate. Unbiasedness says the estimate is centered right, consistency says it converges to the truth with more data, and efficiency says it has the smallest variance, and variance is exactly what a confidence interval reports.

• Unbiasedness: The estimator's expected value equals the parameter, so the interval is correctly centered on average and coverage claims hold.

• Consistency: As n grows the estimator converges to the true parameter and its SE shrinks toward zero, so intervals get narrower with more data.

• Efficiency: Among unbiased estimators, the efficient one has the lowest variance, hence the narrowest (most informative) interval at a given confidence level.

• The connection to reporting uncertainty:

  • A confidence interval is meaningful only when the point estimate is trustworthy; these properties are what license quantifying its precision via the SE.

  • A biased estimator produces intervals that systematically miss, so no width of interval fixes it.

Coverage is the probability that the interval-generating procedure produces an interval containing the true parameter. A method with 'nominal' 95% coverage should capture the parameter in 95% of repeated samples.

• Nominal vs actual coverage:

  • Nominal is the stated level (e.g. 95%); actual is the true long-run rate the method achieves.

  • They can differ when assumptions are violated or samples are small.

• Under- vs over-coverage:

  • Under-coverage (actual below nominal) means intervals are too narrow and overstate precision: a real risk.

  • Over-coverage means intervals are wider than necessary, sacrificing precision.

• It is a property of the method: Coverage is evaluated over the sampling distribution, not for one realized interval.

No: two overlapping confidence intervals do not imply the difference is non-significant. The correct test looks at the CI for the difference, which uses the combined standard error, not whether the two individual intervals touch. Judging significance by overlap is overly conservative and often wrong.

• Why overlap misleads:

  • The SE of a difference is √(SE₁²+SE₂²), which is smaller than the sum of the two half-widths that overlap would require.

  • So intervals can overlap moderately while the difference is still significant.

• The correct approach: Build a CI for the difference (or run the two-sample test) and check whether it excludes 0.

• What the two directions do imply:

  • Non-overlapping intervals → the difference IS significant (safe conclusion).

  • Overlapping intervals → inconclusive; could go either way.

Non-inferiority and equivalence conclusions are made by checking where a confidence interval sits relative to a pre-specified margin, not just relative to zero: you compare the CI for the treatment difference against a clinically meaningful threshold.

• Set a margin first: Define a margin (delta) that represents the largest difference considered clinically unimportant, before looking at data.

• Non-inferiority:

  • Conclude the new treatment is not meaningfully worse if the whole CI stays on the acceptable side of the margin (e.g. the lower bound of the difference is above -delta).

  • Only one direction matters, so a one-sided framing (or one-sided CI bound) is typical.

• Equivalence:

  • Conclude two treatments are practically the same if the entire CI falls within (-delta, +delta), both bounds inside the margin.

  • This is the CI analog of the two one-sided tests (TOST) procedure.

• Key intuition: A non-significant difference (CI containing zero) does NOT prove equivalence: the CI could still be wide enough to include clinically large differences. You must show the CI excludes the margin, not just zero.

A two-sided 95% CI is exactly the set of null values that would NOT be rejected by a two-sided test at alpha 0.05: the interval and the test use the same standard error, point estimate, and critical value, so they are two views of the same computation (this is called duality).

• The rejection rule: You reject a null value θ0 when the standardized distance |estimate − θ0| / SE exceeds the critical value (e.g. 1.96).

• The interval: The CI is estimate ± (critical value × SE): precisely the θ0 values where that distance is ≤ the critical value.

• Correspondence:

  • θ0 inside the CI ⇔ fail to reject; θ0 outside the CI ⇔ reject.

  • So the CI is the collection of all null hypotheses consistent with the data at that alpha level.

• Caveat: The duality is exact only when the CI and the test use the same variance estimate and pivot; some tests (e.g. score vs Wald for proportions) use slightly different SEs, so the match can be approximate.

Multiplicity is the problem that each 95% CI has a 5% chance of missing its target, so when you report many intervals at once the chance that at least one misses grows quickly: your family-wide confidence is much lower than the nominal per-interval level.

• Why it happens: For k independent 95% intervals, the chance all k simultaneously cover is roughly 0.95^k (e.g. ~36% for k=20), far below 95%.

• What you want instead: Simultaneous (family-wise) coverage: the probability that ALL intervals cover their parameters is at least 1 − alpha.

• Adjustment methods:

  1. Bonferroni: use per-interval level 1 − alpha/k (simple, conservative, works for any dependence).

  2. Tukey HSD: for all pairwise mean comparisons, tighter than Bonferroni.

  3. Scheffé: for arbitrary contrasts, most general but widest.

  4. Šidák: 1 − (1 − alpha)^(1/k), slightly less conservative than Bonferroni under independence.

• Trade-off: Adjusted intervals are wider: you buy simultaneous confidence at the cost of precision, so only correct for the family you actually intend to interpret jointly.