Correlation and Multicollinearity
What is correlation and what does a correlation coefficient tell you?
What is the difference between covariance and correlation? Why is correlation often preferred?
What is the range of a correlation coefficient, and what do values like -1, 0, and +1 signify?
Can you write down the Pearson correlation formula and explain its terms?
How is correlation related to covariance?
What is the difference between r and r-squared?
What is the correlation of a variable with itself?
What happens to the correlation coefficient if one of the variables is constant or has zero variance?
What is the difference between uncorrelated variables and independent variables?
What is the difference between correlation and the slope in a linear regression?
Explain why covariance is scale-dependent and how correlation addresses this.
Why is correlation invariant to linear rescaling of the variables?
What mathematical properties does a correlation matrix always have (symmetry, unit diagonal, positive semi-definiteness)?
When you compute a correlation matrix in pandas, how are non-numeric columns and missing values handled, and why does that matter?
Explain the concept 'correlation does not imply causation' and why it's important in data analysis.
What is a spurious correlation and what causes it?
How can a lurking or confounding variable produce a misleading correlation?
What is reverse causation and how does it fit into the correlation-causation trap?
What is ecological correlation and why can aggregated data mislead?
Pitfalls & Limitations Of Correlation
What does Anscombe's quartet demonstrate about correlation?
Why does the Pearson correlation coefficient only measure linear association, and what happens if the true relationship is curved?
How sensitive is the Pearson correlation to outliers, and what effect can a single outlier have?
What is Simpson's paradox and how can it reverse a correlation?
What does the Datasaurus dozen demonstrate about relying on a correlation coefficient or summary statistics alone?
What is restriction of range and how can it impact the observed correlation between two variables?
What is attenuation of correlation due to measurement error?
Can you correct an observed correlation for attenuation due to measurement error, and what does the correction require?
Rank Based Correlation Methods
When would you use Spearman's rank correlation coefficient instead of Pearson's? What are the key differences and assumptions?
Explain the differences between Pearson, Spearman, and Kendall Tau correlation coefficients, and when would you use each?
What is Kendall's tau and how is it defined in terms of concordant and discordant pairs?
How do rank-based correlation coefficients handle ties?
Which correlation coefficient is more robust to outliers and non-normality, and why?
How does applying a monotonic but non-linear transformation to a variable affect its Pearson versus Spearman correlation with another variable?
Explain Kendall's Tau. When is it preferred over Spearman's correlation?
Categorical & Binary Correlation
How do you measure the association or correlation between two categorical variables?
How do you measure the association or correlation between a continuous variable and a categorical variable?
How do you measure correlation when one variable is binary — what is the point-biserial correlation?
Why is using a plain Pearson correlation coefficient on one-hot encoded or nominal categorical data generally inappropriate or misleading?
How do you calculate and interpret correlation when one or both variables are binary or categorical (e.g., point-biserial, phi, Cramer's V)?
What is the correlation ratio (eta), and how does it capture non-linear or categorical-to-continuous association that Pearson's r misses?
What is the difference between biserial and point-biserial correlation?
What is the difference between statistical and practical significance of a correlation?
How does sample size affect the statistical significance of a correlation coefficient? Can a very small correlation be statistically significant?
How do you test whether a correlation coefficient is statistically significant?
What assumptions underlie the significance test for Pearson's correlation coefficient?
What is the Fisher z-transformation and why is it needed for inference on a correlation?
How do you construct a confidence interval for a correlation coefficient?
Multicollinearity Fundamentals
What is multicollinearity in the context of regression analysis?
What is multicollinearity, and what are its common sources in a regression model?
Differentiate between perfect multicollinearity and imperfect (or near) multicollinearity.
What is the dummy variable trap and how does it cause perfect multicollinearity? How do you avoid it when encoding categorical features?
What is the difference between collinearity and multicollinearity?
How do polynomial and interaction terms introduce multicollinearity?
What are common sources of multicollinearity in a dataset, beyond just high pairwise correlations?
What is the difference between structural and data-based multicollinearity?
How can a variable be a near-perfect linear combination of several others without any large pairwise correlation?
Why does perfect collinearity make X-transpose-X singular so OLS has no unique solution?
Consequences Of Multicollinearity
Is high correlation between a predictor and the target variable a problem, and how does it differ from correlation among the predictors themselves?
What are the main consequences of multicollinearity on the estimated coefficients and standard errors in a linear regression model?
Why is multicollinearity a problem in linear regression?
Does multicollinearity affect the overall predictive power or the fitted values of a regression model?
How does multicollinearity affect the interpretability of individual predictor effects in a regression model?
Explain how multicollinearity can lead to wide confidence intervals for regression coefficients.
Why does multicollinearity make coefficient estimates extremely sensitive to adding or removing a few observations or a single predictor?
Why do regression coefficients become unstable and prone to sign-flipping in the presence of severe multicollinearity?
If the overall F-test for a regression model is significant, but individual t-tests for several predictors are not, what might you suspect and why?
Does multicollinearity bias the OLS coefficient estimates? What about the fitted values and predictions?
How does multicollinearity impact the p-values of individual coefficients, the overall F-test, and the R-squared value of a regression model?
Does multicollinearity affect logistic regression the same way it affects OLS?
Detecting Multicollinearity
How do you detect multicollinearity in a regression model?
What does a VIF of 1 mean?
Why is simply looking at a pairwise correlation matrix not sufficient to fully detect multicollinearity?
Explain the Variance Inflation Factor (VIF). How is it calculated, and what does a VIF value of 1, 5, or 10 signify?
What is Tolerance in the context of multicollinearity, and how does it relate to VIF?
How do you detect and handle correlation between variables in linear regression?
What values of VIF did you use to remove multicollinearity, and why?
How do you interpret VIF values, and what are common rules of thumb (e.g., VIF > 5 or VIF > 10)?
Why are the VIF thresholds of 5 and 10 conventions rather than hard rules?
Why does statsmodels VIF require an explicitly added constant column, and what happens if you forget it?
How does the VIF relate to the inflation of a coefficient's standard error — why is the square root of VIF the relevant multiplier?
How can eigenvalues and condition indices of the correlation matrix be used to detect multicollinearity? What are typical thresholds?
What are other diagnostic tools for multicollinearity besides VIF (e.g., condition number, eigenvalues, tolerance)?
Where does the 'inflation' interpretation of VIF come from via the auxiliary regression?
What is generalised VIF and when do you need it for categorical terms?
What is the condition number and how is it used to diagnose multicollinearity?
Why does scaling matter before computing condition number or eigenvalue diagnostics?
How can the determinant of the correlation matrix be used as a diagnostic for multicollinearity?
Remedies For Multicollinearity
What are some common strategies or remedies to address multicollinearity in a regression model?
When considering dropping one of several highly correlated predictors, how would you decide which one to remove?
How can combining or averaging correlated variables help mitigate multicollinearity?
What is multicollinearity and how will you handle it in your regression model?
How would you explain the tradeoff of a multicollinearity remedy to a non-technical stakeholder?
Does collecting more data or data with better spread help with multicollinearity, and why?
In what situations might you choose to do nothing about multicollinearity, and why?
How do regularization techniques like Ridge regression help mitigate multicollinearity?
Why is centering important for polynomial and interaction terms when dealing with multicollinearity?
Can Principal Component Analysis (PCA) be used as a remedy for multicollinearity? What are the advantages and disadvantages of this approach?
Why is lasso's arbitrary pick among correlated features a caveat when using it as a remedy?
Partial Correlation & Model Specific Effects
What is partial correlation and how does it control for a third variable?
Does multicollinearity affect tree-based models in the same way it affects linear regression? Why or why not?
How can multicollinearity impact feature importance scores such as permutation importance or SHAP values in tree-based or other complex models?
What is the difference between partial and semi-partial correlation?