What is a Normal (Gaussian) distribution, and what are its key characteristics, including its shape and the 68-95-99.7 rule?
Junior
Can you explain the 68-95-99.7 rule and its significance for the Normal distribution?
Junior
Tell me a distribution other than the normal distribution.
Junior
What are the standard normal distribution and Z-scores?
Junior
When would you use a normal distribution, and what are some real-world applications?
Junior
Describe the Log-normal distribution and the scenarios where it frequently arises.
Mid
Explain the Exponential distribution, its memoryless property, and its common applications.
Mid
Explain the Gamma distribution and its relationship to the Exponential and Erlang distributions.
Mid
Describe the Beta distribution and its use in modeling probabilities.
Mid
When would you use a Weibull distribution, and what does its shape parameter signify?
Mid
Why is the Normal distribution considered so fundamental and ubiquitous in statistics and data science?
Mid
Can you explain the Student's t-distribution?
Mid
What is the Pareto distribution and the power-law, and how does the Pareto principle relate to it?
Mid
What is the Laplace (double exponential) distribution, and when would it be an appropriate model?
Mid
What is the logistic distribution and how does it compare in shape to the normal?
Mid
What is the Chi-square distribution as an object, and what are its typical uses?
Mid
Which common distributions have the memoryless property, and what does memorylessness actually mean?
Mid
Why does the Beta distribution serve naturally as a distribution over a probability?
Mid
What are the unique characteristics of the Cauchy distribution, especially regarding its mean and variance?
Senior
What is the Gumbel / extreme-value distribution, and what kind of quantities does it model?
Senior
Discrete Distributions
What is the difference between the Bernoulli and Binomial distributions?
Junior
What is a distribution in statistics, and can you give examples of common discrete distributions such as Bernoulli, Binomial, and Poisson?
Junior
Explain the Binomial distribution, its parameters, and when it is an appropriate model.
Junior
Explain the Poisson distribution, its parameters, and the types of events it models.
Junior
What is the Bernoulli distribution, and what real-world scenario does it model?
Junior
Explain the discrete Uniform distribution and its characteristics.
Junior
What is the relationship between the mean and standard deviation in a Poisson distribution, and can you give an example scenario?
Junior
Describe the Geometric distribution, its memoryless property, and its typical applications.
Mid
Differentiate between the Binomial and Hypergeometric distributions, particularly concerning sampling with and without replacement.
Mid
What is the Multinomial distribution, and how does it generalize the Binomial?
Mid
What is the Negative Binomial distribution, and what does it model in terms of waiting for the k-th success?
Mid
What is overdispersion, and why might the Negative Binomial distribution be preferred over the Poisson distribution in such cases?
Senior
Foundations & Random Variables
What is a probability distribution, and how does it describe a random variable?
Junior
Differentiate between discrete, continuous, and mixed random variables, providing examples for each.
Junior
What are probability distributions and why are they important in machine learning?
Junior
Can you define random variables, probability distributions, PDF, and CDF?
Junior
What is a uniform distribution?
Junior
What is the difference between discrete and continuous probability distributions?
Junior
What are the main ideas of the Law of Large Numbers?
Junior
What is the support of a distribution, and why does it matter when choosing a family to model bounded data?
Junior
What is a sampling distribution, and what are its types?
Mid
What is the difference between a probability distribution and a sampling distribution?
Mid
What defines a location-scale family of distributions, and which common distributions form one?
Mid
Pmf Pdf & Cdf
Explain the concepts of Probability Mass Function (PMF) and Probability Density Function (PDF), highlighting their key differences.
Junior
Define the Cumulative Distribution Function (CDF) and list its essential properties.
Junior
What are the properties of a Probability Mass Function (PMF)?
Junior
How do you determine if a given function represents a valid probability mass function, and why?
Junior
Why is the value of a PDF at a specific point not considered a probability?
Mid
How can you derive the PMF or PDF from a given CDF?
Mid
How would you check if a given function is a valid PDF?
Mid
What is the survival function of a distribution, and in what situations do you prefer working with it over the CDF?
Mid
What is the quantile function (inverse CDF), and when would you use it rather than the PDF or CDF?
Mid
Shape Tails & Mixtures
What is the difference between positive and negative skewness, and how does it affect the mean-median relationship?
Junior
For a right-skewed distribution, how do the mean, median, and mode order themselves, and why?
Junior
What is skewness, what does it tell you about a distribution's shape, and how does it affect the mean-median-mode relationship?
Mid
Explain skewness and kurtosis as measures of a distribution's shape.
Mid
Differentiate between leptokurtic, mesokurtic, and platykurtic distributions.
Mid
What is multimodality, and what does it suggest about the underlying data generation process?
Mid
Describe what a mixture distribution is.
Mid
What is kurtosis and what does it tell you about a distribution's tails?
Mid
What is a mixed (part-discrete, part-continuous) random variable, and can you give an example?
Mid
What are heavy tails (or fat tails), why are they important in practice, and how do they differ from light tails?
Senior
Explain why distributions with heavy tails, like the Cauchy distribution, can break mean-based reasoning and the Central Limit Theorem.
Senior
What is the difference between the fourth central moment and excess kurtosis, and how do you interpret them?
Senior
What does it mean for a distribution to have infinite or undefined variance, and what practical problems does this create?
Senior
Limit Theorems & Approximations
Explain the Central Limit Theorem and its real-life applications.
Mid
What are the essential conditions that must be met for the Central Limit Theorem to apply?
Mid
Explain the concept of continuity correction when approximating a discrete distribution with a continuous one.
Mid
Can you explain the relationship between the binomial and normal distributions, including the conditions for approximation?
Mid
How can a statistician or data scientist use the central limit theorem to their benefit?
Mid
When would you use the normal approximation to the Binomial or Poisson distribution, and what are the conditions under which these approximations are valid or might fail?
Senior
Moments Expectation & Mgf
What is expected value, and how do you use it?
Junior
For a random variable X that follows a Uniform distribution between -3 and 7, how do you find E(X)?
Junior
Explain the concepts of expectation and variance for a named probability distribution, and how they are derived.
Mid
Can you define the different central moments (zeroth, first, second, third, fourth)?
Mid
What are the different types of parameters that characterize a probability distribution (e.g., location, scale, shape)?
Mid
What is the Moment Generating Function (MGF), and what is its primary purpose in probability theory?
Mid
How can the MGF be used to derive moments of a distribution and to prove properties of sums of independent random variables?
Senior
What is the difference between the moment generating function and the characteristic function, and why does the characteristic function always exist?
Senior
Fitting Estimation & Goodness Of Fit
What is a Q-Q plot and how do you interpret it to assess if a dataset follows a specific theoretical distribution?
Mid
What are the differences in interpretation for various patterns observed in a Q-Q plot (e.g., S-shape, curved, straight line)?
Mid
Explain the concept of the empirical CDF and its role in assessing distributional fit.
Mid
What is kernel density estimation, and how does it help in understanding the shape of a distribution from data?
Mid
How do you estimate probability distributions from data?
Mid
What is a PP plot, and how does it differ from a QQ plot for assessing distributional fit?
Mid
What is the Kullback-Leibler (KL) divergence and how is it used in probability and information theory?
Senior
Compare and contrast the Method of Moments and Maximum Likelihood Estimation as procedures for fitting a named probability distribution to data.
Senior
What are some common goodness-of-fit tests (e.g., Kolmogorov-Smirnov, Shapiro-Wilk, Anderson-Darling, Chi-square) and when would you use each to check distributional assumptions?
Senior
Why can over-relying on a normality test like Shapiro-Wilk be misleading with very large sample sizes?
Senior
Normal Distribution Deep Dive
Explain the impact of the mean and standard deviation on the normal distribution.
Junior
Given X ~ N(3, 2²) and Y ~ N(1, 2²), what is the distribution of Z = 2X - Y?
Mid
Why does the Student's t-distribution converge to the normal distribution as its degrees of freedom increase?
Mid
Explain why a linear combination of independent normal random variables is also normally distributed.
Senior
If you can draw from a normal distribution with known parameters, how do you generate draws from a uniform distribution?
Senior
Applied Modeling & Assumptions
Can you describe the characteristics and applications of common distributions like Normal, Binomial, and Poisson?
Junior
What is the difference between a uniform and a normal distribution?
Junior
When would you choose the mean versus the median as a metric for average performance?
Junior
When would you use a binomial, Poisson, normal, or exponential distribution, and can you give real-world scenarios?
Mid
Given a real-world scenario, how would you choose an appropriate probability distribution to model it?
Mid
When and why would you consider applying a log transformation or other variance-stabilizing transformations to data, particularly for skewed distributions?
Mid
What distributional assumptions distinguish the Gaussian, Multinomial, and Bernoulli variants of Naive Bayes?
Mid
Discuss the distributional assumptions commonly made in various statistical models (e.g., Gaussian errors in OLS, Poisson or Negative Binomial for count data in GLMs).
Senior
Why are normality assumptions often applied to residuals of a model or to sampling distributions rather than directly to the raw data itself?
Senior
Explain the concepts of zero-inflation and overdispersion in count data, and how they influence the choice of a probability distribution for modeling.
Senior
What is distribution shift between training and production data, and why does it matter?
Senior
What is the exponential-family response and link function in a GLM, expressed as a distributional assumption?
Senior
How does the log-normal distribution get confused with a power-law, and how would you tell them apart?
Senior
When is applying a variance-stabilizing transform inappropriate, and what are the pitfalls of transforming data?
Senior
Multivariate Distributions & Dependence
Explain the differences between joint, marginal, and conditional probability distributions.
Mid
How is independence between random variables expressed in terms of their joint, marginal, and conditional distributions?
Mid
What is a covariance matrix, and how does it capture the relationships between multiple random variables?
Mid
Describe the Multivariate Normal distribution, including its mean vector and covariance matrix.
Senior
Is it always true that if each component of a random vector is normally distributed, then the vector itself follows a multivariate normal distribution? Explain.
Senior
What is the significance of the covariance matrix in the context of a Multivariate Normal distribution?
Senior
Differentiate between correlation and dependence in the context of random variables.
Senior
You have two normally distributed random variables X and Y with correlation ρ — what's the distribution of X+Y?
Senior
What are the marginal and conditional distributions of a multivariate normal, and what shape are its contours?
Senior
What is a convolution of two independent random variables, and how does it relate to the distribution of their sum?
Senior
What is a copula, and what problem does it solve in modeling dependence between random variables?
Senior
Distribution Relationships
Describe the relationships between the Bernoulli, Binomial, and Poisson distributions.
Mid
How is the Log-normal distribution related to the Normal distribution?
Mid
How is the sum of squared independent standard normals distributed, and why?
Mid
Explain how the Exponential distribution is a special case of the Gamma distribution, and how sums of exponentials relate to the Gamma/Erlang distribution.
Senior
How are the Chi-square, Student's t, and F distributions derived from the Normal distribution, and what are their degrees of freedom?
Senior
Explain the relationship between the Beta distribution and the Gamma distribution.
Senior
What does it mean for a family of distributions to be "closed under addition," and provide an example?
Senior
How is the Chi-square distribution derived from Normal distributions, and how does it relate to the Gamma and Exponential distributions?
Senior
How is the F distribution constructed as a ratio of chi-square distributions?
Senior
Explain the duality between the Exponential distribution for inter-arrival times and the Poisson distribution for counts.
Senior
State the conjugacy relationships where Beta is the conjugate partner of the Binomial and Gamma of the Poisson, as a relationship between families.
Senior
What assumptions define a Poisson process, and how do they lead to the Poisson and Exponential distributions?
Senior
Sampling & Random Generation
Describe the Inverse Transform Sampling method for generating random variates from a given distribution.
Mid
Explain the Probability Integral Transform and its implications for generating random numbers.
Senior
How does the Box-Muller transform work to generate standard normal random variates from uniform random numbers?
Senior
Explain the concept of rejection sampling.
Senior
Explain the change-of-variables (Jacobian) technique for finding the distribution of a transformed random variable.