74 Exploratory Data Analysis Interview Questions and Answers (2026)

EDA is the practice of investigating a dataset to summarize its main characteristics, uncover patterns, spot anomalies, and check assumptions, mostly through visualization and summary statistics, before formal modeling begins.

• What it involves:

  • Examining distributions, relationships between variables, missing values, outliers, and data quality issues.

  • Uses tools like histograms, box plots, scatter plots, and descriptive stats.

• Why it matters:

  • Builds intuition about the data so you choose appropriate features, transformations, and models.

  • Catches problems early (leakage, bad encodings, skew) that would otherwise corrupt a model.

  • Guides hypothesis generation and shapes the questions worth asking.

• Mindset: Open-ended and iterative: you let the data reveal surprises rather than confirming a predetermined answer.

The core philosophy of EDA, pioneered by John Tukey in his 1977 book Exploratory Data Analysis, is that you should let the data speak first: explore openly and visually to discover what is there, rather than jumping straight to confirming preconceived hypotheses.

• Tukey's contribution:

  • Reframed statistics to value exploration and discovery, not just formal inference.

  • Invented practical tools like the box plot and stem-and-leaf display to see data quickly.

• The famous quote:

  • "The greatest value of a picture is when it forces us to notice what we never expected to see."

  • It captures EDA's spirit: visualization surfaces surprises that hypothesis-driven analysis would overlook.

• Underlying attitude: Curiosity, skepticism, and iteration: question the data, look at it many ways, and stay open to being surprised.

The phrase warns that just because two variables move together does not mean one causes the other. During EDA you constantly find correlations, but treating them as causal without controlled evidence leads to false conclusions.

• Why correlation isn't causation:

  • Confounding: a hidden third variable drives both (ice cream sales and drownings both rise with summer heat).

  • Reverse causation: the direction may be opposite to what you assume.

  • Coincidence: with enough variables, spurious correlations appear by chance.

• Why it matters in EDA:

  • EDA reveals associations, not mechanisms; it can suggest causal hypotheses but not prove them.

  • Establishing causation needs controlled experiments (A/B tests) or careful causal inference methods.

• Practical stance: Report correlations as leads to investigate, and consciously ask what could confound the relationship.

The main goal of EDA is to understand the data deeply enough to make sound modeling decisions: understand its structure, quality, and relationships while checking the assumptions your later methods will rely on.

• Understand distributions: Central tendency, spread, skewness, and shape of each variable.

• Assess data quality: Detect missing values, outliers, duplicates, and inconsistent encodings.

• Explore relationships: Correlations and interactions between variables, and how features relate to the target.

• Check assumptions: Linearity, normality, or independence that downstream models assume.

• Generate hypotheses and guide modeling: Surface candidate features, transformations, and questions worth testing.

Visualization turns raw numbers into shapes your eye can read, revealing patterns, anomalies, and relationships that summary statistics alone can hide. The goal in EDA is to build intuition about the data before formal modeling.

• Reveals structure summaries miss: Anscombe's quartet: four datasets share the same mean, variance, and correlation yet look completely different when plotted.

• Surfaces problems early: Outliers, skew, multimodality, gaps, and missing-data patterns pop out visually (boxplots, histograms).

• Guides hypotheses and next steps: Scatter plots and pair plots suggest relationships worth testing or features worth engineering.

• The goal is understanding, not presentation: EDA plots are fast, iterative, and for you: they inform cleaning, transformation, and modeling decisions.

They differ by how many variables you examine at once: univariate looks at one, bivariate at two, and multivariate at three or more. You escalate as you move from understanding individual variables to understanding how they interact.

• Univariate (one variable):

  • Purpose: describe distribution, center, spread, outliers of a single feature.

  • Tools: histogram, boxplot, value counts, summary stats.

  • Use when first profiling each column.

• Bivariate (two variables):

  • Purpose: measure relationship or association between a pair.

  • Tools: scatter plot, correlation, grouped boxplots, crosstabs.

  • Use to check feature-target relationships or redundancy between two features.

• Multivariate (three or more):

  • Purpose: capture interactions, confounding, and joint structure.

  • Tools: pair plots, correlation heatmaps, color/facet-encoded plots, PCA.

  • Use when relationships depend on a third variable (e.g. Simpson's paradox) or for dimensionality reduction.

The five-number summary is a compact description of a distribution using five values: the minimum, first quartile (Q1), median (Q2), third quartile (Q3), and maximum. Together with percentiles it tells you where data concentrates, how spread it is, and where outliers might lurk, without assuming any shape.

• The five numbers:

  • Min and Max give the range; Q1, median, Q3 split the sorted data into four equal-count parts.

  • The middle 50% of values lie between Q1 and Q3.

• Quartiles vs percentiles:

  • Quartiles are just the 25th, 50th, and 75th percentiles; percentiles generalize to any cut point (e.g. p95, p99).

  • Tail percentiles are how you reason about extremes (latency SLAs, income tails).

• What it reveals in EDA:

  • Skew: if the median sits closer to Q1 than Q3, the distribution is right-skewed.

  • Spread via the IQR (Q3 - Q1), which is robust to outliers unlike the range.

  • Outlier flags using the 1.5 x IQR fence, the basis of a boxplot.

• Robustness: these are order statistics, so they resist extreme values (unlike mean and standard deviation).

Qualitative (categorical) data describes qualities or categories, while quantitative (numerical) data measures amounts you can do arithmetic on. The distinction determines how you summarize, visualize, and model each variable.

• Qualitative (categorical):

  • Represents labels or groups: color, gender, product category.

  • Subtypes: nominal (no order) and ordinal (ordered ranks).

  • Summarized by counts, proportions, and mode.

• Quantitative (numerical):

  • Represents measurable amounts: age, price, temperature.

  • Subtypes: discrete (counts) and continuous (measurements); interval vs ratio distinguishes whether zero is meaningful.

  • Summarized by mean, std, quartiles, and distribution plots.

• Why it matters: arithmetic (means, correlations) is valid on quantitative data but meaningless on qualitative codes, even when categories are stored as numbers.

Numbers only have meaning inside their business context: the data dictionary tells you what each column actually measures, its units, how it was collected, and what special values mean. Without it you can compute statistics that are technically correct but analytically wrong, misreading codes, sentinels, or definitions.

• Defines meaning and units: Is revenue gross or net? Is the amount in dollars or cents? The same number means different things.

• Explains encodings and sentinels:

  • Values like -1, 999, or "N/A" may mean missing, not a real quantity, and would skew a mean.

  • Category codes map to labels you can't guess from the data alone.

• Frames what's plausible and important:

  • Context tells you whether an outlier is an error or a genuine, meaningful extreme.

  • It focuses EDA on questions the business actually cares about, not random correlations.

• Reveals collection process: How and when data was captured explains gaps, biases, and structural missingness.

Sanity checks confirm the data is what you think it is before you trust any analysis: check shape, types, ranges, and whether values are physically/logically plausible. The goal is to catch loading errors and corruption early.

• Structural checks:

  • Row/column counts match expectations; inspect dtypes with .info() and eyeball .head()/.tail().

  • Check for duplicate rows and duplicate IDs.

• Plausibility checks (does it make sense?):

  • Ranges: no negative ages, percentages within 0-100, dates not in the future.

  • Units and scale look correct (prices, currencies, magnitudes).

  • Categorical values are from an expected set (no typos or stray codes).

• Missing and sentinel values: Quantify nulls per column; watch for disguised missings like -999, 0, or "N/A".

• Consistency checks: Cross-field logic (start date before end date), and totals that should reconcile.

Exploration plots exist to answer a question in your own head as fast as possible, so speed and quantity beat polish: you make dozens, glance, and discard. Presentation visuals exist to communicate one conclusion to others, so clarity and correctness of framing matter most.

• Exploration plots are a thinking tool:

  • Defaults are fine: no titles, custom colors, or annotations needed if only you are reading them.

  • Time spent styling is wasted because most will be thrown away.

  • Volume matters: many quick views reveal patterns a single polished chart won't.

• Presentation visuals are a communication tool:

  • Need clear labels, titles, appropriate scales, and a single obvious takeaway.

  • Must avoid misleading framing (truncated axes, cherry-picked ranges).

• Practical implication: Only invest in polish for the few charts that made the cut, after you know the conclusion.

A box plot summarizes a distribution through its five-number summary: it shows the median, the middle 50% of the data (the box), the overall spread (whiskers), and flagged outliers. It is excellent for comparing groups and spotting skew, but it hides the shape of the distribution.

• What the box shows:

  • The line inside is the median (Q2), the 50th percentile.

  • Box edges are Q1 (25th) and Q3 (75th); the box height is the IQR = Q3 - Q1.

  • A median off-center in the box indicates skew.

• What the whiskers show:

  • Typically extend to the last point within 1.5 * IQR of the quartiles.

  • Points beyond are drawn individually as outliers.

• What it hides:

  • Multimodality: a bimodal distribution looks identical to a unimodal one.

  • Sample size and density within regions (a violin or histogram shows these).

• Best use: Comparing a numeric variable across categories side by side.

Skew describes which side of a distribution has the long tail. Right-skewed (positive) has a long tail toward high values; left-skewed (negative) has a long tail toward low values. The direction of the tail tells you where extreme values live and how mean and median relate.

• Right-skewed (positive skew):

  • Long tail on the right; most mass at low values with a few large outliers.

  • Mean > median (large values pull the mean up).

  • Common for bounded-at-zero, unbounded-above quantities: income, wait times, counts.

• Left-skewed (negative skew):

  • Long tail on the left; most mass at high values with a few small outliers.

  • Mean < median.

  • Common when there's a ceiling: exam scores near a max, age at death.

• Why it matters:

  • Prefer the median over the mean to summarize skewed data.

  • Skew can violate normality assumptions; a transform (log, sqrt) often symmetrizes right-skewed data.

Impossible values are entries that violate domain logic or physical/business constraints (negative age, a percentage of 150, a future birthdate); you find them by comparing observed ranges and categories against the rules you expect the data to obey.

• Numeric range checks:

  • Inspect df.describe() min/max against plausible bounds (age in [0,120], counts \ 0, ratios in [0,1]).

  • Flag violations explicitly: df[df['age'] < 0].

• Categorical validity: Use value_counts() to catch typos, unexpected categories, or inconsistent casing ("USA" vs "usa").

• Cross-field and temporal logic:

  • Check relations that must hold: start_date <= end_date, discount <= price.

  • Dates in the future or before the business existed are invalid.

• Format and type checks: Numbers stored as strings, malformed emails/zip codes, or unit mismatches (kg vs lb) surface as parse failures or bimodal scales.

• Practice tip: codify these as reusable assertions/validation rules (e.g. great_expectations or simple assert statements) so checks are repeatable, not one-off.

A constant column has a single value everywhere, and a near-zero-variance column is almost constant (one value dominates or it barely varies); you spot them via variance, cardinality, and dominant-value frequency, and they matter because they carry little to no information and can trip up models.

• How to spot them:

  • Unique-count check: df.nunique() equal to 1 flags true constants.

  • Dominant-frequency check: if the top value_counts(normalize=True) exceeds ~95-99%, it's near-zero-variance.

  • For numeric features, very low df.var() or scikit-learn's VarianceThreshold.

• Why they matter:

  • No discriminative power: a constant can't help separate outcomes.

  • They add dimensionality/noise and can cause numerical issues (zero-variance breaks standardization, singular matrices in linear models).

  • They inflate one-hot dimensionality if a rare-category dummy is near-constant.

• Caveat before dropping:

  • A rare positive class in an imbalanced target is near-zero-variance but still crucial: don't drop the label.

  • A column constant in your sample may vary in production, hinting at a filtering/leakage issue worth investigating.

EDA is open-ended exploration to discover patterns and generate hypotheses; CDA is rigorous testing to confirm or reject a pre-specified hypothesis. They are complementary stages, not competitors.

• Purpose:

  • EDA: generate ideas, explore what the data might say.

  • CDA: evaluate a specific claim with statistical rigor (hypothesis tests, confidence intervals, p-values).

• Approach:

  • EDA is flexible, visual, and iterative with no fixed hypothesis.

  • CDA is structured: hypothesis, model, and analysis plan are set in advance.

• Outcome:

  • EDA yields candidate hypotheses and insights.

  • CDA yields formal evidence and quantified uncertainty.

• Key caution: Ideally use separate data: a hypothesis found via EDA should be confirmed on fresh data to avoid circular reasoning.

Anscombe's Quartet is a set of four datasets constructed by Francis Anscombe that have nearly identical summary statistics (mean, variance, correlation, regression line) yet look completely different when plotted. Its lesson: always visualize your data because summary numbers can hide the real structure.

• What they share: Same mean and variance of x and y, same correlation (~0.816), same fitted regression line.

• What they hide: One is a clean linear relationship, one is clearly non-linear (a curve), one is linear but dragged by a single outlier, one has a vertical cluster plus one influential point.

• The lesson:

  • Summary statistics compress and can mislead; plots reveal outliers, non-linearity, and leverage points that numbers alone miss.

  • Modern echo: the Datasaurus Dozen makes the same point dramatically.

EDA sits after data collection and cleaning but before feature engineering and modeling, acting as the bridge that turns raw data into an understanding that informs every downstream decision. It is also revisited iteratively as the project evolves.

• Where it fits:

  • Problem framing → data collection → cleaning → EDA → feature engineering → modeling → evaluation.

  • Overlaps with cleaning (you find dirty data during EDA) and loops back after modeling.

• What it informs: Feature choices, transformations, model selection, and validation strategy.

• If you skip it:

  • You miss data quality issues (outliers, leakage, mis-encoded values) that silently break models.

  • You may pick the wrong model or misread results, and "garbage in, garbage out" takes over.

  • Debugging happens late and expensively rather than early and cheaply.

Exploratory analysis is open-ended discovery to understand what the data contains; explanatory analysis is focused communication of a specific, already-found insight to an audience. The goal shifts from finding the story to telling it.

• Exploratory: for you, to learn:

  • Many quick, rough plots and summaries; you don't yet know what matters.

  • Optimizes for coverage and speed, not polish.

• Explanatory: for others, to convince:

  • A few carefully designed visuals that highlight one clear conclusion.

  • Optimizes for clarity: annotations, labels, removing distractions.

• How the goal changes:

  • Exploratory asks "what is going on here?"; explanatory asserts "here is what matters and why."

  • You typically explore first, then distill the findings into an explanatory narrative.

Central tendency describes the "typical" value of a variable: the mean is the arithmetic average, the median is the middle value, and the mode is the most frequent value. Choosing among them depends on distribution shape, data type, and outliers.

• Mean:

  • Uses every value, so it's efficient for symmetric, outlier-free numeric data.

  • Sensitive: a few extreme values drag it away from the bulk of the data.

• Median:

  • The 50th percentile; robust to outliers and skew.

  • Preferred for skewed data like income or house prices.

• Mode:

  • Only sensible option for categorical/nominal data.

  • Useful for spotting multimodality (more than one peak).

• Reading the gap between mean and median:

  • Mean > median suggests right skew; mean < median suggests left skew; near-equal suggests symmetry.

  • With outliers, prefer the median (or a trimmed mean) as the summary.

Central tendency tells you where the data sits, but dispersion tells you how much it varies around that center; two datasets can share a mean yet behave completely differently. Without spread, a summary is misleading because it ignores risk, reliability, and the presence of extremes.

• Same center, different spread: Mean = 50 with values tightly at 48-52 is very different from values ranging 0-100.

• Range: Max minus min; quick but dominated by extremes.

• Variance and standard deviation:

  • Average squared deviation from the mean; std is in the original units, so it's interpretable.

  • Sensitive to outliers because deviations are squared.

• IQR and MAD: IQR (Q3 - Q1) and median absolute deviation are robust measures for skewed or outlier-heavy data.

• Why it matters in EDA: Spread flags data-quality issues, informs outlier detection, and drives decisions like scaling or transformation.

Skewness measures the asymmetry of a distribution, and kurtosis measures the heaviness of its tails (propensity for extreme values). Together they describe distribution shape beyond center and spread, and both are easy to spot with plots plus their numeric coefficients.

• Skewness (asymmetry):

  • Positive/right skew: long tail to the right, mean > median.

  • Negative/left skew: long tail to the left, mean < median.

  • Near 0 means roughly symmetric.

• Kurtosis (tailedness):

  • High (leptokurtic): heavy tails, more outliers than a normal curve.

  • Low (platykurtic): light tails, fewer extremes.

  • Excess kurtosis is measured relative to the normal distribution (0).

• How to identify during EDA:

  • Visually: histograms, KDE plots, and Q-Q plots against a normal reference.

  • Numerically: pandas .skew() and .kurt(), or scipy.stats functions.

  • Why it matters: strong skew/heavy tails may warrant transformations (log) or robust methods.

Non-graphical EDA summarizes data with numbers (statistics and tables), while graphical EDA shows data visually (plots). They are complementary: numbers give precision and objectivity, plots give pattern recognition and context.

• Non-graphical (quantitative):

  • Summary stats, frequency tables, correlation coefficients, skewness/kurtosis values.

  • Strengths: precise, reproducible, easy to compare and automate at scale.

  • Weakness: can hide structure (Anscombe's quartet).

• Graphical:

  • Histograms, boxplots, scatter plots, heatmaps.

  • Strengths: reveals shape, clusters, outliers, and relationships instantly.

  • Weakness: subjective, harder to compare exactly or scale to many variables.

• When to prefer which:

  • Use plots to discover and understand shape; use numbers to confirm, quantify, and report.

  • In practice you interleave both rather than choosing one.

The interquartile range (IQR) is the spread of the middle 50% of the data, computed as the third quartile minus the first quartile (Q3 - Q1). Unlike standard deviation, it's a robust measure that ignores extreme values, making it better suited to skewed or outlier-heavy data.

• Definition:

  • Q1 = 25th percentile, Q3 = 75th percentile, IQR = Q3 - Q1.

  • It's the width of the box in a boxplot.

• Robustness vs standard deviation:

  • IQR is based on ranks/positions, so outliers barely affect it.

  • Standard deviation squares deviations from the mean, so a single extreme value inflates it heavily.

• Outlier detection: Common rule flags points below Q1 - 1.5*IQR or above Q3 + 1.5*IQR.

• When to use each:

  • IQR for skewed distributions or when outliers are present.

  • Standard deviation for roughly symmetric, normal-like data where every value should count.

The coefficient of variation (CV) is the standard deviation divided by the mean, usually expressed as a percentage. It is a unitless measure of relative spread, so it lets you compare variability across variables that have different units or very different scales, where raw standard deviation would mislead.

• Definition:

  • CV = std / mean (often x 100%).

  • Because units cancel, it answers "how big is the spread relative to the typical value?"

• When it beats standard deviation:

  • Different units: comparing variability of height (cm) vs weight (kg) is meaningless with raw std but fair with CV.

  • Different magnitudes: salaries in the millions vs the thousands may have similar CV even with wildly different std.

• Cautions:

  • Only meaningful for ratio data with a true zero and positive values; useless if the mean is near zero (it explodes) or can be negative.

  • Sensitive to outliers through both mean and std, so pair it with robust measures.

The unit of observation is what a single row represents (a customer, a transaction, a customer-month), and each column's data type governs what operations and summaries are valid. Nailing both early prevents nonsense analysis: aggregating at the wrong grain, or computing a mean on something that isn't truly numeric.

• Unit of observation defines the grain:

  • It tells you what a count of rows means and how to join, group, and avoid double-counting.

  • Mixing grains (e.g. order-level and customer-level rows) silently inflates aggregates.

• Data type dictates valid operations:

  • Nominal/categorical: only counts and modes; no ordering or arithmetic.

  • Ordinal: order is meaningful but gaps aren't, so use medians, not means.

  • Interval/ratio: full arithmetic; only ratio supports meaningful ratios and CV.

  • Datetime: ordering, differences, and resampling, not naive averaging.

• Catches common traps:

  • An ID or zip code stored as an integer must be treated as categorical, not averaged.

  • Encoded categories (1,2,3) look numeric but aren't ordered quantities.

Data type is the single biggest driver of which summary statistics and plots are appropriate: categorical data calls for counts and bar charts, numerical data for distributions and correlations, and datetime for time-ordered views. Matching technique to type keeps your EDA honest and readable.

• Nominal / categorical:

  • Summaries: frequency counts, proportions, mode.

  • Plots: bar charts, count plots; avoid implying order.

• Ordinal:

  • Summaries: median, mode, ordered frequency tables.

  • Plots: ordered bar charts; boxplots if treating levels carefully.

• Interval / ratio (numerical):

  • Summaries: mean, std, quartiles; CV only for ratio.

  • Plots: histograms, KDE, boxplots univariate; scatter and correlation bivariate.

• Datetime:

  • Summaries: min/max range, gaps, counts per period after resampling.

  • Plots: line/time-series charts, seasonality and trend views.

• Combinations drive bivariate choices: Numeric vs numeric: scatter; categorical vs numeric: grouped boxplots or violin plots; categorical vs categorical: crosstabs or heatmaps.

The primary key is the column (or combination) that uniquely identifies each row, and it should match the unit of observation. Checking uniqueness early confirms you understand the grain and surfaces duplicates or accidental fan-out from joins before they corrupt every downstream count and aggregate.

• Confirms the grain: If the intended key isn't unique, your actual unit of observation is finer than you think (e.g. one row per order-item, not per order).

• Prevents inflated metrics:

  • Duplicate rows overcount sums, means, and distinct counts.

  • Duplicates often reveal bad joins, ETL replays, or source errors worth fixing at the root.

• How to check:

  • Compare row count to distinct key count; inspect any rows where they differ.

  • Consider composite keys when no single column is unique.

I move from context to structure to content: first understand what the data is supposed to represent, then verify its shape and types, then profile quality and distributions, and only then explore relationships. The goal of initial EDA is to understand and trust the data before analyzing it.

• Understand the context first:

  1. Read the data dictionary; identify the unit of observation and what each column means.

  2. Clarify the business question the data should answer.

• Check structure and types:

  1. Inspect shape, column names, and dtypes with df.info() and df.head().

  2. Fix mis-typed columns (IDs as ints, dates as strings) and classify each as categorical, numerical, or datetime.

  3. Verify the primary key is unique; check for duplicate rows.

• Assess data quality:

  1. Quantify missingness per column and look for patterns.

  2. Scan for sentinels, impossible values, and inconsistent categories.

• Univariate exploration:

  1. Numerical: df.describe(), histograms, boxplots for spread, skew, and outliers.

  2. Categorical: value counts and bar charts for cardinality and balance.

• Bivariate / multivariate exploration:

  1. Correlations and scatter plots among numerics; grouped summaries against key categories.

  2. Time trends if a datetime exists.

• Document findings: Note data-quality issues, assumptions, and hypotheses to investigate next.

EDA outputs are the distilled understanding of the data: a documented picture of its quality, structure, and key relationships, plus concrete actions for the next phase. They range from artifacts (notebook, cleaning spec) to decisions (features, hypotheses).

• Data quality report: Missingness, duplicates, outliers, invalid values, and type issues found.

• Distribution and relationship summary: Key univariate distributions, correlations, target relationships, and notable segments.

• Cleaning and preprocessing plan: How to handle nulls, encode categoricals, transform skewed features, cap outliers.

• Hypotheses and feature ideas: Candidate features, suspected drivers, and leakage risks to test in modeling.

• Artifacts to share: A reproducible notebook, a short summary/deck for stakeholders, and a curated set of key plots.

Good EDA communication tailors depth to the audience, leads with the decision-relevant finding, and stays reproducible. The aim is that a teammate can trust, act on, and re-run your conclusions.

• Know the audience: Stakeholders want implications and a clear narrative; engineers/DS want methods, caveats, and code.

• Lead with findings, not process: State the insight and its "so what" first, then supporting evidence.

• Make it reproducible: Share the notebook/scripts, data version, and environment so results can be regenerated.

• Be honest about uncertainty: Note sample sizes, biases, assumptions, and what you did NOT check.

• Curate the visuals: Show a few clean, labeled plots that make the point; keep exploratory clutter out.

• Keep a living document: A shared summary of findings and open questions that updates as analysis evolves.

A describe table is a fast anomaly scanner: read it column by column comparing count, central tendency, spread, and extremes against what you expect. Discrepancies flag missingness, outliers, skew, or wrong types before you plot anything.

• count row: Counts below the row total reveal missing values; unequal counts across columns hint at structure.

• mean vs median (50%): A large gap signals skew or outliers; mean far above median means a right tail.

• min and max: Impossible values (negatives, zeros, sentinels like -999) or absurd extremes.

• std and the quartile spread:

  • std near 0 means a near-constant (useless) feature; huge std vs IQR suggests outliers.

  • Compare max to 75% (Q3): a max far beyond Q3 flags a long tail.

• For categoricals (include='all'): unique, top, and freq reveal cardinality and dominant categories.

Explore categoricals mainly through frequency counts: look at how many distinct levels exist (cardinality), which levels are rare, and whether the target/classes are imbalanced. Bar charts of value counts do most of the work.

• Assess cardinality: nunique() tells you the number of levels; high cardinality (IDs, free text) needs special encoding, not one-hot.

• Find rare categories:

  • value_counts() or value_counts(normalize=True) surfaces levels with tiny support.

  • Consider grouping them into an "Other" bucket to reduce noise and overfitting.

• Check class imbalance (for the target): Compare class proportions; heavy skew affects metric choice (use precision/recall, not accuracy) and sampling.

• Data quality within categories: Watch for inconsistent labels ("USA" vs "usa"), whitespace, and unexpected codes.

• Relationship to target: Group the target by category (mean rate or crosstab) to see which levels are predictive.

Bin width controls how much you smooth the data: too wide hides real structure, too narrow amplifies noise into spurious peaks. The same variable can look unimodal, bimodal, or ragged depending purely on this one choice.

• Too wide (few bins): Over-smooths: merges distinct modes, hides gaps, and can make a skewed or multimodal distribution look deceptively simple.

• Too narrow (many bins): Under-smooths: sampling noise shows up as jagged spikes and empty bins that look like real structure but aren't.

• Considerations to guide the choice:

  • Sample size: more data supports narrower bins reliably.

  • Try several widths rather than trusting one default; the pattern that persists across widths is the real one.

  • Use principled rules as a starting point: Freedman-Diaconis (robust to outliers), Sturges, or Scott.

• Also watch bin alignment (anchor/edges): Shifting where bins start can move a value across a boundary and change the apparent shape, independent of width.

• Best practice: pair the histogram with an ECDF or KDE to confirm the story isn't a binning artifact.

A bimodal distribution has two distinct peaks and a multimodal one has several; each peak marks a value range where data concentrates. Multiple modes often signal that the sample is really a mixture of different subgroups pooled together.

• What a mode is:

  • A local peak in the density: values cluster there more than in surrounding ranges.

  • Unimodal = one peak; bimodal = two; multimodal = three or more.

• What multiple modes suggest:

  • Hidden subgroups: e.g. heights mixing men and women, or response times split by device type.

  • A lurking categorical variable you should split on (segment and re-plot per group).

  • Distinct data-generating processes or regimes combined in one column.

• Cautions:

  • Confirm the peaks are stable across bin widths / bandwidths, not a small-sample or binning artifact.

  • Implication for stats: a single mean/std poorly summarizes a multimodal variable; model the subgroups instead.

Use a histogram when you want to see raw counts and honest, discretized data (including gaps and spikes); use a KDE when you want a smooth, continuous estimate of shape that's easy to overlay and compare across groups. Both depend on a smoothing parameter: bin width for histograms, bandwidth for KDE.

• Histogram:

  • Shows actual frequencies; good for spotting gaps, spikes, and discreteness.

  • No distributional assumptions, but shape depends on bin width and edge alignment.

• KDE:

  • Smooth curve; clean for overlaying multiple distributions on one axis.

  • Bandwidth is the key knob: too small = wiggly/noisy, too large = washes out real modes.

  • Can mislead near hard boundaries (e.g. smears mass below zero for a non-negative variable) and implies continuity the data may not have.

• Rule of thumb:

  • Single variable, want fidelity to raw data: histogram.

  • Comparing several groups or presenting shape cleanly: KDE (or both overlaid).

  • Always vary the smoothing parameter to check that features are real.

Reading a distribution's shape means describing its symmetry, number of peaks, tail behavior, and any irregularities, then asking what data-generating reality could produce it. Each feature is a clue about the process, the population, or data-quality issues.

• Symmetric: Balanced around center; mean ≈ median. Suggests a single stable process (often approximately normal).

• Skewed: Long tail one side; a natural bound or multiplicative process. Consider median and transforms.

• Unimodal vs bimodal/multimodal: One peak = one homogeneous group; multiple peaks = likely mixed subgroups or regimes to segment.

• Heavy tails: More extreme values than a normal predicts; outliers/rare large events matter, and variance may be unstable.

• Gaps: Empty ranges: separate populations, filtering/exclusion rules, or impossible values.

• Spikes: Sharp overrepresentation at one value: default/placeholder values, rounding, caps, or coded missingness (e.g. 0 or 999).

• Overall takeaway: Shape guides which summary stats are honest and which models/assumptions are valid, and often surfaces data-quality problems early.

An ECDF plots, for each value x, the fraction of observations less than or equal to x, producing a monotonic step function from 0 to 1. Unlike a histogram, it uses every data point directly with no binning, so it shows the exact distribution without a smoothing choice.

• How to read it:

  • Height at x = proportion of data ≤ x; read percentiles straight off the y-axis (median at y = 0.5).

  • Steep regions = dense data; flat regions = gaps with few or no values.

• What it reveals better than a histogram:

  • No bin-width choice, so no binning artifacts: the shape is deterministic.

  • Precise quantiles and the fraction above/below any threshold.

  • Clean comparison of two groups by overlaying curves (basis of the KS test).

  • Spikes appear as sharp vertical jumps at repeated values.

• Trade-off: Modes/peaks are harder to see (they appear as changes in slope), so histograms remain more intuitive for shape.

A QQ-plot plots the quantiles of your data against the quantiles expected from a normal distribution; if the variable is normal, the points fall along a straight diagonal line. Systematic departures from that line reveal how and where normality breaks.

• How to read it:

  • Points on the reference line: consistent with normality.

  • Both ends curving up/down (S-shape or inverted): heavier or lighter tails than normal.

  • A convex/concave bend: skew (points sagging below then rising indicates right skew).

  • Isolated points off the ends: outliers.

  • Discrete horizontal steps: rounding or granular data.

• Why use it:

  • Far more sensitive to tail behavior than a histogram, which is where normality assumptions usually fail.

  • Diagnoses the type of deviation (skew vs tails), not just presence.

• Caveats:

  • Small samples wander from the line by chance; don't over-read minor wiggles.

  • It's an informal visual check; pair with judgment rather than relying solely on a formal test.

Use a violin plot when the shape of the distribution matters, especially to reveal multiple modes that a box plot would hide; use a box plot when you want a compact, robust summary of quartiles and outliers. The trade-off is richness of shape versus simplicity and reliability.

• Violin plot:

  • Shows a mirrored KDE, so you see full shape: modality, skew, and where density concentrates.

  • Best when comparing distributions across groups and shape (not just spread) is the story.

• Box plot:

  • Shows median, quartiles (IQR), whiskers, and outliers: compact and hard to misread.

  • Its weakness: it can't show bimodality, so two very different distributions can share the same box.

• The trade-off you're making:

  • Violins rely on a bandwidth (a smoothing choice), so they can imply structure that isn't there and mislead with small samples.

  • They're also busier and less familiar to some audiences.

  • Practical compromise: overlay a box or quartile markers inside the violin to get shape plus robust summary.

A skewed variable has a long tail that compresses most of the data into a small range and lets extreme values dominate. Applying a transformation like log pulls in the tail, spreads out the crowded values, and often reveals structure (linear relationships, symmetry) that was hidden.

• Why skew is a problem in EDA:

  • Summary stats (mean, SD) get distorted by the tail and stop describing the bulk of the data.

  • Plots become unreadable: most points bunch near one edge while a few outliers stretch the axis.

• What a transformation does:

  • A log transform compresses large values more than small ones, turning right-skew toward symmetry.

  • It converts multiplicative/exponential patterns into additive/linear ones, so relationships become easier to see and model.

• Choosing a transform:

  • log for right-skewed positive data (income, counts); needs a shift like log(x+1) if zeros are present.

  • sqrt for milder skew or counts; Box-Cox / Yeo-Johnson to search for an optimal power.

• What it reveals:

  • Hidden linear/proportional structure, clusters, or approximate normality; sometimes exposes that apparent outliers were just tail members.

  • Caveat: results are now on a log scale, so interpret and label axes accordingly.

To relate a numerical variable to a categorical one, you compare the numeric distribution across the categories: non-graphically by computing grouped summary statistics, and graphically with plots that show each group's distribution side by side.

• Non-graphical:

  • Grouped summaries: mean, median, SD, quartiles, count per category (df.groupby('cat')['num'].describe()).

  • Compare central tendency and spread; note group sizes since small groups are noisy.

• Graphical:

  • Box plots: quick view of medians, IQR, and outliers across categories.

  • Violin or overlaid density/histograms: show full distribution shape (bimodality that a box plot hides).

  • Strip/swarm plots: show individual points, good for small samples.

  • Bar plot of group means with error bars when you want a single summary comparison.

• What to look for:

  • Do groups differ in level (shifted center) or in spread/shape?

  • Any category with tiny n, extreme outliers, or different variance that would break later assumptions.

A scatter plot maps two numerical variables to x and y, revealing the form, direction, and strength of their relationship plus clusters, outliers, and gaps. Its main enemy is overplotting: when points overlap you can no longer see density, so you need techniques to recover it.

• What it reveals:

  • Form: linear, curved, or none.

  • Direction: positive or negative association.

  • Strength: how tightly points hug a trend.

  • Anomalies: outliers, clusters, subgroups, or heteroscedasticity (fanning spread).

• The overplotting problem: With many or discrete points, overlap hides how many observations sit in each region.

• Fixes for overplotting:

  • Transparency (alpha) so density shows through stacking.

  • Smaller markers, or jitter for discrete/rounded values.

  • 2D binning: hexbin or 2D histograms; contour/density plots for large n.

  • Sampling a subset of rows when data is huge.

• Enhancements: Add a trend/LOESS line to summarize the pattern; encode a third variable via color or size.

A correlation matrix (often shown as a heatmap) summarizes pairwise linear/monotonic association among many numerical variables in one view, helping you spot strong relationships, redundant features, and candidates for further study. Pearson measures linear association on the raw values; Spearman measures monotonic association on ranks, making it robust to outliers and non-linear but monotonic patterns.

• Purpose of the matrix/heatmap:

  • Scan many pairs at once; color encoding makes strong +/- blocks pop.

  • Flag multicollinearity/redundant features and shortlist pairs to scatter-plot.

• Pearson:

  • Captures strength of a linear relationship using actual values.

  • Sensitive to outliers and assumes roughly linear, continuous data.

• Spearman:

  • Correlates the ranks, so it detects any monotonic relationship (even curved).

  • Robust to outliers and works with ordinal data.

• Choosing for exploration:

  • Use Pearson when relationships look linear and data is fairly clean.

  • Use Spearman with skew, outliers, ordinal variables, or suspected monotonic-but-nonlinear links.

  • A big Pearson/Spearman gap itself is a clue: non-linearity or outlier influence.

• Caveat: Correlation is not causation, and both miss non-monotonic patterns (e.g. U-shaped), so always back it with plots.

Plot choice is driven by two things: the data types involved (categorical vs numerical, and how many variables) and the specific question you're asking (distribution, comparison, relationship, composition, or trend). You match the encoding to what you want the reader's eye to extract.

• By number and type of variables:

  • One numerical: histogram, density, or box plot (shape, spread, outliers).

  • One categorical: bar chart of counts.

  • Numerical vs numerical: scatter plot (+ trend line).

  • Numerical vs categorical: box/violin plots by group.

  • Categorical vs categorical: grouped/stacked bars or a heatmap of the cross-tab.

• By the question you're answering:

  • Distribution: histogram, density, box.

  • Comparison across groups: grouped bars, box plots.

  • Relationship/correlation: scatter, hexbin, heatmap.

  • Trend over time: line chart.

  • Composition of a whole: stacked bar (prefer over pie for comparison).

• Practical adjustments:

  • Large n or overlap: switch to hexbin, density, or aggregation.

  • Add a third variable via color, size, or faceting rather than cramming one plot.

  • Guiding rule: pick the encoding that makes the answer readable at a glance and honest (undistorted axes).

To explore two categorical variables you build a contingency table (cross-tabulation): a grid of counts for every combination of their levels. It shows how the categories co-occur, and converting counts to proportions reveals whether the distribution of one variable depends on the other (association).

• What a contingency table tells you:

  • Joint counts of each level pair, plus row/column totals (margins).

  • Row or column percentages show conditional distributions: does the split of Y change across levels of X?

  • If conditional distributions are similar across categories, the variables look independent; if they differ, there's association.

• Building it: pd.crosstab(x, y) for counts; add normalize='index' for row proportions.

• Graphical views:

  • Grouped bar chart: compare counts side by side.

  • Stacked/100% stacked bars: compare composition across groups.

  • Heatmap of the table or a mosaic plot for many cells.

• Quantifying association:

  • A chi-square test checks whether observed counts deviate from independence; Cramer's V measures strength.

  • Watch for sparse cells (low counts), which make patterns unreliable.

Grouping, aggregation, and pivoting let you collapse rows into summaries along dimensions, turning raw records into comparisons that reveal how a metric changes across categories, time, or combinations of both.

• Grouping partitions rows by a key:

  • e.g. df.groupby('region') splits data so each subgroup can be compared side by side.

  • Surfaces between-group differences: which segment converts more, spends more, fails more.

• Aggregation reduces each group to statistics:

  • mean, median, count, std, quantiles per group expose central tendency and spread.

  • Comparing mean vs median per group hints at skew or outliers inside a group.

• Pivoting cross-tabulates two dimensions:

  • a matrix (rows x columns) makes interactions visible: does the effect of A depend on B?

  • Great feeder for a heatmap to spot hot/cold cells.

• Patterns they surface: imbalanced categories, dominant segments, trends over time, and interaction effects that a single-variable summary would hide.

Plot the series over time first, then use smoothing, resampling, and decomposition to separate the slow trend, repeating seasonal cycles, and abrupt level shifts from the noise.

• Start with the raw line plot: a proper datetime x-axis reveals the overall shape, gaps, and obvious spikes.

• Trend: overlay a rolling mean (df.rolling(window).mean()) to smooth noise and see long-run direction.

• Seasonality:

  • resample to a coarser grain, or plot by cycle (month-over-month, day-of-week) to expose repeating patterns.

  • An autocorrelation (ACF) plot confirms periodicity at fixed lags.

• Level shifts / structural breaks: look for sudden step changes in the mean; annotate known events (launches, policy changes) to explain them.

• Decompose to separate components: seasonal_decompose or STL splits the series into trend, seasonal, and residual for a clean side-by-side view.

Encoding extra variables with colour, size, and shape works for a few added dimensions, but breaks down once channels overload the viewer's perception: overlap, indistinguishable levels, and cognitive load make the plot unreadable rather than informative.

• Where each channel fails:

  • colour: hue works for ~7 categories max; continuous colour is hard to read precisely.

  • size: only conveys rough magnitude, and large points occlude small ones.

  • shape: only a handful of shapes are distinguishable, and they vanish under overplotting.

• General breaking point: beyond roughly 3 to 4 encoded dimensions the channels interfere and the reader can't decode them reliably.

• Alternatives:

  • small multiples / faceting: split into a grid of panels by category instead of piling encodings onto one plot.

  • pair plots or a scatterplot matrix to see many variable pairs.

  • dimensionality reduction (PCA/UMAP) to summarise many features into 2D.

  • interactivity: tooltips and filtering move detail off the static canvas.

An outlier is a question, not a verdict: I try to explain where it came from before deciding whether to fix it, keep it, or investigate it, because the same extreme point could be a typo, a real rare event, or the exact signal the analysis exists to find.

• Check plausibility against the domain: is the value physically/logically possible (age 200, negative price)? Impossible values point to errors.

• Look for data-entry fingerprints: shifted decimals, unit mix-ups (cm vs m), swapped fields, or sentinel codes like 999 or -1.

• Cross-reference other columns and records: a consistent supporting story across fields argues for a genuine event; a lone contradiction argues for error.

• Judge genuine rare events on their own terms: in fraud, fault detection, or extreme finance, the tail is the point: don't discard it reflexively.

• Decide and document: correct if clearly wrong, keep and flag if real, and run analyses with and without it to gauge influence.

Start by quantifying missingness per column and row, then study where and how the gaps cluster: the pattern (random vs systematic) tells you both how to handle them and whether they encode bias.

• Detecting missingness:

  • Count nulls with df.isna().sum() and null rates per column/row; visualize with a missingness matrix or heatmap (missingno).

  • Remember disguised nulls: sentinels like -999, empty strings, or "unknown" won't show as NaN.

• Three canonical patterns:

  • MCAR (Missing Completely At Random): gaps unrelated to any variable; safe to drop but rare.

  • MAR (Missing At Random): missingness depends on other observed columns (e.g. income missing more for one region).

  • MNAR (Missing Not At Random): missingness depends on the unseen value itself (high earners hide income); most dangerous for bias.

• What the patterns suggest:

  • Correlated gaps across columns hint at a broken pipeline, an optional form section, or a join that failed.

  • Missingness concentrated in a subgroup signals sampling/collection bias that will skew models and conclusions.

  • Test it: create an is_missing flag and check if it correlates with the target or other features to distinguish MAR/MNAR from MCAR.

Sentinel values are real numbers or tokens standing in for "missing," so they blend into the data and quietly distort statistics; they usually betray themselves as an implausible spike or an outlier far from the rest of the distribution.

• How they reveal themselves:

  • An isolated spike in a histogram at a round or extreme value (-999, 9999, 0).

  • A min/max in describe() wildly out of range, or a suspiciously high count of a single value in value_counts().

  • Text placeholders like "N/A", "missing", "-" appearing as categories.

• Why they are dangerous:

  • They corrupt aggregates: a -999 drags means and inflates variance; 0-as-missing understates a true average.

  • They are treated as legitimate signal by models, learning spurious relationships.

  • They hide the true missingness rate, so you underestimate a data-quality problem.

• Defense: convert known sentinels to NaN early (na_values=[-999,'N/A']) and confirm with domain experts what each code means before analysis.

Visuals mislead when their encoding distorts the underlying numbers: truncated axes exaggerate differences, wrong scales compress or stretch patterns, and cherry-picked bins or ranges manufacture a story; you avoid fooling yourself by choosing honest defaults and stress-testing your charts.

• Common distortions:

  • Truncated y-axis: not starting bar charts at zero makes small gaps look dramatic.

  • Inappropriate scale: linear axis for exponential data (should be log), or dual axes implying false correlation.

  • Cherry-picked bins: histogram bin width chosen to hide or invent modes; time windows chosen to show a favorable trend.

  • Aspect-ratio and area tricks: bubble sizing by radius not area, over-stretched charts.

• How to avoid it:

  • Start value axes at zero for magnitude comparisons; label scales clearly and disclose transforms.

  • Try multiple bin widths and views (histogram + KDE + ECDF) to confirm a pattern is robust.

  • Show the full data range and context, not a favorable slice.

  • Cross-check the visual against summary statistics so the eye and the numbers agree.

A confounding (lurking) variable is a third factor that influences both variables in a relationship you observe, creating or distorting an association that isn't the direct causal link you think it is. It fools you because you never plotted or controlled for it, so its effect masquerades as a relationship between the two variables you happened to look at.

• What it is:

  • A variable correlated with both your predictor and your outcome, sitting 'behind' the observed link.

  • Classic example: ice cream sales correlate with drownings, but temperature (season) drives both.

• How it fools you:

  • You attribute causation to the wrong variable, or see a spurious link where none exists.

  • It can also mask a real relationship or reverse its apparent direction (Simpson's paradox).

• How to catch it in EDA:

  • Segment or stratify by suspected confounders and re-check the relationship within groups.

  • Use faceted plots and color-coding by a third variable; check correlations among all candidates.

  • Lean on domain knowledge to enumerate plausible lurking variables before concluding.