Why do linear and distance-based models require feature scaling while decision trees do not?
Which supervised model families natively handle missing feature values without imputation?
Which supervised model families require explicit encoding of categorical inputs?
Why are linear models often preferred when data is high-dimensional and sparse?
When is adopting a non-linear ensemble justified over a linear baseline?
Why can tree-based models capture non-linear relationships out of the box while linear models need explicit basis expansions?
What is a minimal, defensible baseline for an imbalanced supervised classification task?
Why do tree ensembles handle mixed categorical and numerical features better than linear models?
When is a deep neural network justified over a gradient-boosted tree ensemble for tabular data?
Why do deep neural networks typically require more labelled data than tree ensembles on tabular datasets?
Problem Framing & Target Design
What is the difference between multi-label classification and multi-output regression?
What are the trade-offs of binning a continuous target into discrete buckets?
When should an ordered target be framed as ordinal regression rather than continuous regression?
What are the tradeoffs of training separate models for multiple targets versus a single multi-output model?
How should you frame a supervised target that is a rate, such as claims per policy-year, rather than a raw count?
What is structured prediction, and how does it differ from predicting multiple outputs independently?
What is the consequence of training with squared error on a highly skewed or multi-modal target distribution?
When would you frame a ranking problem as a supervised regression task versus a pairwise classification task?
When should a pricing or bidding problem be framed as predicting a probability of conversion rather than predicting price directly?
How do censored labels in survival or churn prediction violate standard supervised learning assumptions?
When would you frame a forecasting problem as tabular supervised regression instead of using an autoregressive time series model?
Labels Annotation & Noise
What is weak supervision?
What is a proxy label in supervised learning?
What is inter-annotator agreement?
What is distant supervision?
How does active learning decide which unlabeled examples to send for labeling?
Why do clear annotation guidelines matter for the quality of supervised training labels?
What risks arise when training a model on proxy labels instead of direct ground truth?
How does random label noise affect linear models compared to high-capacity non-parametric models?
How does random label noise affect a supervised model differently from systematic label bias?
How would you train a supervised model when you know a meaningful fraction of the training labels are wrong?
With a fixed annotation budget, how would you decide between labelling more new examples and relabelling existing noisy examples?
Distribution Shift Leakage & Deployment
What is the difference between interpolation and extrapolation in supervised prediction?
What is the difference between covariate shift and label shift?
What is training-serving skew in target definitions?
How does target leakage differ from train-test contamination?
How do delayed labels impact continuous model evaluation in production?
How can target leakage be detected in feature pipelines before deployment?
How does covariate shift affect a supervised model's predictions if the conditional label distribution remains fixed?
What happens to a supervised model's generalization guarantees when the i.i.d. assumption fails?
How does a feedback loop degrade a supervised model once deployed?
How do selective labels, where outcomes are only observed for cases that were acted upon (such as approved loans), bias a supervised model?
How would you adjust a deployed classifier's predicted probabilities when class priors shift but the class-conditional feature distributions stay the same?
Loss Functions & Predictive Outputs
What is the difference between a point prediction and a probabilistic prediction?
How does quantile (pinball) loss allow a supervised model to produce prediction intervals instead of single point predictions?
Why does minimising log loss yield conditional probabilities rather than hard class assignments?
What are the trade-offs of using point predictions versus full predictive distributions in production?
Why is standard multiclass cross-entropy loss suboptimal when predicting ordinal labels?
What are the tradeoffs of training on a surrogate loss instead of the target evaluation metric?
Error Diagnosis & Data Needs
How do you estimate how much labelled data a supervised model needs?
How do you diagnose whether an underperforming model is bottlenecked by data volume versus model capacity?
How do you distinguish whether validation errors stem from noisy labels or insufficient feature signal?
How can you diagnose whether model error is driven by label noise versus irreducible data noise?
Learning Theory Foundations
What is Empirical Risk Minimization (ERM)?
What is the difference between empirical risk and true (generalization) risk?
What is the unknown joint probability distribution P(X, Y) in the supervised learning framework?
What is the core intuition behind the PAC learning framework?
What is the difference between approximation error and estimation error?
Why does a hypothesis class with infinite VC dimension fail to guarantee generalization?
How does the capacity of the hypothesis class impact sample complexity?
What is the difference between the realizable and agnostic settings in PAC learning?