Continuous Distributions
What is a Normal (Gaussian) distribution, and what are its key characteristics, including its shape and the 68-95-99.7 rule?
Can you explain the 68-95-99.7 rule and its significance for the Normal distribution?
Tell me a distribution other than the normal distribution.
What are the standard normal distribution and Z-scores?
When would you use a normal distribution, and what are some real-world applications?
Describe the Log-normal distribution and the scenarios where it frequently arises.
Explain the Exponential distribution, its memoryless property, and its common applications.
Explain the Gamma distribution and its relationship to the Exponential and Erlang distributions.
Describe the Beta distribution and its use in modeling probabilities.
When would you use a Weibull distribution, and what does its shape parameter signify?
Why is the Normal distribution considered so fundamental and ubiquitous in statistics and data science?
Can you explain the Student's t-distribution?
What is the Pareto distribution and the power-law, and how does the Pareto principle relate to it?
What is the Laplace (double exponential) distribution, and when would it be an appropriate model?
What is the logistic distribution and how does it compare in shape to the normal?
What is the Chi-square distribution as an object, and what are its typical uses?
Which common distributions have the memoryless property, and what does memorylessness actually mean?
Why does the Beta distribution serve naturally as a distribution over a probability?
What are the unique characteristics of the Cauchy distribution, especially regarding its mean and variance?
What is the Gumbel / extreme-value distribution, and what kind of quantities does it model?
Discrete Distributions
What is the difference between the Bernoulli and Binomial distributions?
What is a distribution in statistics, and can you give examples of common discrete distributions such as Bernoulli, Binomial, and Poisson?
Explain the Binomial distribution, its parameters, and when it is an appropriate model.
Explain the Poisson distribution, its parameters, and the types of events it models.
What is the Bernoulli distribution, and what real-world scenario does it model?
Explain the discrete Uniform distribution and its characteristics.
What is the relationship between the mean and standard deviation in a Poisson distribution, and can you give an example scenario?
Describe the Geometric distribution, its memoryless property, and its typical applications.
Differentiate between the Binomial and Hypergeometric distributions, particularly concerning sampling with and without replacement.
What is the Multinomial distribution, and how does it generalize the Binomial?
What is the Negative Binomial distribution, and what does it model in terms of waiting for the k-th success?
What is overdispersion, and why might the Negative Binomial distribution be preferred over the Poisson distribution in such cases?
Foundations & Random Variables
What is a probability distribution, and how does it describe a random variable?
Differentiate between discrete, continuous, and mixed random variables, providing examples for each.
What are probability distributions and why are they important in machine learning?
Can you define random variables, probability distributions, PDF, and CDF?
What is a uniform distribution?
What is the difference between discrete and continuous probability distributions?
What are the main ideas of the Law of Large Numbers?
What is the support of a distribution, and why does it matter when choosing a family to model bounded data?
What is a sampling distribution, and what are its types?
What is the difference between a probability distribution and a sampling distribution?
What defines a location-scale family of distributions, and which common distributions form one?
Pmf Pdf & Cdf
Explain the concepts of Probability Mass Function (PMF) and Probability Density Function (PDF), highlighting their key differences.
Define the Cumulative Distribution Function (CDF) and list its essential properties.
What are the properties of a Probability Mass Function (PMF)?
How do you determine if a given function represents a valid probability mass function, and why?
Why is the value of a PDF at a specific point not considered a probability?
How can you derive the PMF or PDF from a given CDF?
How would you check if a given function is a valid PDF?
What is the survival function of a distribution, and in what situations do you prefer working with it over the CDF?
What is the quantile function (inverse CDF), and when would you use it rather than the PDF or CDF?
Shape Tails & Mixtures
What is the difference between positive and negative skewness, and how does it affect the mean-median relationship?
For a right-skewed distribution, how do the mean, median, and mode order themselves, and why?
What is skewness, what does it tell you about a distribution's shape, and how does it affect the mean-median-mode relationship?
Explain skewness and kurtosis as measures of a distribution's shape.
Differentiate between leptokurtic, mesokurtic, and platykurtic distributions.
What is multimodality, and what does it suggest about the underlying data generation process?
Describe what a mixture distribution is.
What is kurtosis and what does it tell you about a distribution's tails?
What is a mixed (part-discrete, part-continuous) random variable, and can you give an example?
What are heavy tails (or fat tails), why are they important in practice, and how do they differ from light tails?
Explain why distributions with heavy tails, like the Cauchy distribution, can break mean-based reasoning and the Central Limit Theorem.
What is the difference between the fourth central moment and excess kurtosis, and how do you interpret them?
What does it mean for a distribution to have infinite or undefined variance, and what practical problems does this create?
Limit Theorems & Approximations
Explain the Central Limit Theorem and its real-life applications.
What are the essential conditions that must be met for the Central Limit Theorem to apply?
Explain the concept of continuity correction when approximating a discrete distribution with a continuous one.
Can you explain the relationship between the binomial and normal distributions, including the conditions for approximation?
How can a statistician or data scientist use the central limit theorem to their benefit?
When would you use the normal approximation to the Binomial or Poisson distribution, and what are the conditions under which these approximations are valid or might fail?
Moments Expectation & Mgf
What is expected value, and how do you use it?
For a random variable X that follows a Uniform distribution between -3 and 7, how do you find E(X)?
Explain the concepts of expectation and variance for a named probability distribution, and how they are derived.
Can you define the different central moments (zeroth, first, second, third, fourth)?
What are the different types of parameters that characterize a probability distribution (e.g., location, scale, shape)?
What is the Moment Generating Function (MGF), and what is its primary purpose in probability theory?
How can the MGF be used to derive moments of a distribution and to prove properties of sums of independent random variables?
What is the difference between the moment generating function and the characteristic function, and why does the characteristic function always exist?
Fitting Estimation & Goodness Of Fit
What is a Q-Q plot and how do you interpret it to assess if a dataset follows a specific theoretical distribution?
What are the differences in interpretation for various patterns observed in a Q-Q plot (e.g., S-shape, curved, straight line)?
Explain the concept of the empirical CDF and its role in assessing distributional fit.
What is kernel density estimation, and how does it help in understanding the shape of a distribution from data?
How do you estimate probability distributions from data?
What is a PP plot, and how does it differ from a QQ plot for assessing distributional fit?
What is the Kullback-Leibler (KL) divergence and how is it used in probability and information theory?
Compare and contrast the Method of Moments and Maximum Likelihood Estimation as procedures for fitting a named probability distribution to data.
What are some common goodness-of-fit tests (e.g., Kolmogorov-Smirnov, Shapiro-Wilk, Anderson-Darling, Chi-square) and when would you use each to check distributional assumptions?
Why can over-relying on a normality test like Shapiro-Wilk be misleading with very large sample sizes?
Normal Distribution Deep Dive
Explain the impact of the mean and standard deviation on the normal distribution.
Given X ~ N(3, 2²) and Y ~ N(1, 2²), what is the distribution of Z = 2X - Y?
Why does the Student's t-distribution converge to the normal distribution as its degrees of freedom increase?
Explain why a linear combination of independent normal random variables is also normally distributed.
If you can draw from a normal distribution with known parameters, how do you generate draws from a uniform distribution?
Applied Modeling & Assumptions
Can you describe the characteristics and applications of common distributions like Normal, Binomial, and Poisson?
What is the difference between a uniform and a normal distribution?
When would you choose the mean versus the median as a metric for average performance?
When would you use a binomial, Poisson, normal, or exponential distribution, and can you give real-world scenarios?
Given a real-world scenario, how would you choose an appropriate probability distribution to model it?
When and why would you consider applying a log transformation or other variance-stabilizing transformations to data, particularly for skewed distributions?
What distributional assumptions distinguish the Gaussian, Multinomial, and Bernoulli variants of Naive Bayes?
Discuss the distributional assumptions commonly made in various statistical models (e.g., Gaussian errors in OLS, Poisson or Negative Binomial for count data in GLMs).
Why are normality assumptions often applied to residuals of a model or to sampling distributions rather than directly to the raw data itself?
Explain the concepts of zero-inflation and overdispersion in count data, and how they influence the choice of a probability distribution for modeling.
What is distribution shift between training and production data, and why does it matter?
What is the exponential-family response and link function in a GLM, expressed as a distributional assumption?
How does the log-normal distribution get confused with a power-law, and how would you tell them apart?
When is applying a variance-stabilizing transform inappropriate, and what are the pitfalls of transforming data?
Multivariate Distributions & Dependence
Explain the differences between joint, marginal, and conditional probability distributions.
How is independence between random variables expressed in terms of their joint, marginal, and conditional distributions?
What is a covariance matrix, and how does it capture the relationships between multiple random variables?
Describe the Multivariate Normal distribution, including its mean vector and covariance matrix.
Is it always true that if each component of a random vector is normally distributed, then the vector itself follows a multivariate normal distribution? Explain.
What is the significance of the covariance matrix in the context of a Multivariate Normal distribution?
Differentiate between correlation and dependence in the context of random variables.
You have two normally distributed random variables X and Y with correlation ρ — what's the distribution of X+Y?
What are the marginal and conditional distributions of a multivariate normal, and what shape are its contours?
What is a convolution of two independent random variables, and how does it relate to the distribution of their sum?
What is a copula, and what problem does it solve in modeling dependence between random variables?
Distribution Relationships
Describe the relationships between the Bernoulli, Binomial, and Poisson distributions.
How is the Log-normal distribution related to the Normal distribution?
How is the sum of squared independent standard normals distributed, and why?
Explain how the Exponential distribution is a special case of the Gamma distribution, and how sums of exponentials relate to the Gamma/Erlang distribution.
How are the Chi-square, Student's t, and F distributions derived from the Normal distribution, and what are their degrees of freedom?
Explain the relationship between the Beta distribution and the Gamma distribution.
What does it mean for a family of distributions to be "closed under addition," and provide an example?
How is the Chi-square distribution derived from Normal distributions, and how does it relate to the Gamma and Exponential distributions?
How is the F distribution constructed as a ratio of chi-square distributions?
Explain the duality between the Exponential distribution for inter-arrival times and the Poisson distribution for counts.
State the conjugacy relationships where Beta is the conjugate partner of the Binomial and Gamma of the Poisson, as a relationship between families.
What assumptions define a Poisson process, and how do they lead to the Poisson and Exponential distributions?
Sampling & Random Generation
Describe the Inverse Transform Sampling method for generating random variates from a given distribution.
Explain the Probability Integral Transform and its implications for generating random numbers.
How does the Box-Muller transform work to generate standard normal random variates from uniform random numbers?
Explain the concept of rejection sampling.
Explain the change-of-variables (Jacobian) technique for finding the distribution of a transformed random variable.