Funnel Fundamentals
How do you calculate step-conversion rate, overall end-to-end conversion rate, and drop-off rate in a funnel, and how are they related?
What is a conversion funnel, and why is analyzing conversion in stages more informative than a single end-to-end conversion rate?
How do the definition and structure of a conversion funnel vary across different business models such as e-commerce checkout, SaaS onboarding, or content consumption?
When defining a conversion funnel, what are the key considerations for choosing the steps and the underlying event taxonomy?
What does a funnel analysis actually show you, and what crucial information does it not directly provide?
What is the difference between macro and micro conversions in a funnel, and why is it important to track both?
How do you identify the 'bottleneck' step in a conversion funnel, and what's the difference between the biggest percentage drop and the biggest absolute user loss?
Funnel Metrics & Calculation
Describe the concept of a 'conversion window' in funnel analysis. How does changing this window impact the reported conversion rates, and why is it important to define it clearly?
What is the difference between a 'strict order' funnel and an 'any order' funnel, and when is each approach appropriate?
What are 'first-touch' and 'last-touch' attribution within a conversion funnel, and what are their respective uses?
What are the implications of counting 'users' versus 'sessions' versus 'events' when calculating funnel conversion rates, and when would you choose each approach?
Why might a real-world conversion funnel not be strictly monotonic, with users appearing to skip steps or enter mid-funnel, and how should this be handled in analysis?
How do you handle users who re-enter a funnel multiple times, and what are the considerations for deduplicating or counting repeat attempts?
What is a 'time-to-convert' distribution, and why is the mean often a misleading metric for it, especially when conversions have a long tail?
What are survivorship effects deeper in a funnel, and how can they distort the interpretation of later-stage conversion?
Funnel Diagnosis & Optimization
How would you analyze a conversion funnel to identify the primary drop-off points and potential friction areas?
How would you use user segmentation in your analyses for funnels or cohorts, and what types of insights would you expect to gain from it?
What are the common mistakes in funnel analysis, and how can they be avoided?
What is the difference between funnel analysis and path or journey analysis, and when is each more appropriate?
How do you ensure each stage of a funnel is aligned with a company's revenue and growth goals?
Cohort Analysis Foundations
Explain what cohort analysis is and why it is important for understanding user behavior and retention patterns, especially compared to aggregate metrics.
How do you define the cohort date for each user when structuring a retention analysis?
What is the difference between a cohort and a plain user segment, and when does grouping become a cohort?
When reading a cohort retention grid, explain what it means to read 'down a column,' 'across a row,' and 'along the diagonal,' and what insights each view provides.
What is the difference between an 'acquisition cohort' and a 'behavioral cohort'? When would you use each type?
How do you decide on the appropriate cohort grain such as daily, weekly, or monthly for a product?
What is the difference between reading a metric across calendar time versus across cohort age, and why does cohorting exist to separate the two?
What are common pitfalls in interpreting cohort retention data, and how do you account for confounding factors when analyzing retention?
Explain how cohort analysis can be used to compare the quality of different acquisition channels rather than just volume or cost per install.
Cohort Construction & Interpretation
Why is it crucial to normalize each row in a cohort table by its own 'cohort size' when comparing retention across different cohorts?
Why do small or young cohorts move noisily, and why shouldn't a single small cohort's movement be read as a trend?
Discuss the issue of 'incomplete' or 'immature' cohorts in the lower-right of a retention grid. Why should these not be interpreted as a decline in retention?
In what sense is a retention curve a survival curve, and what does it mean that the newest cohorts are right-censored?
Retention Metrics & Curves
What is user retention, and why is it considered a critical metric for a product's sustainable growth?
What are the pitfalls of relying on the day-1, day-7, and day-30 retention conventions?
Explain the characteristic shape of a typical retention curve. What does a curve that flattens to a stable asymptote signify, versus one that decays to zero?
What is a smile curve in retention, and what might it signify about user behavior or product changes?
Define 'N-day retention,' 'unbounded (or rolling) retention,' and 'range (or bucketed) retention.' How do these different definitions impact the reported retention numbers for the same product, and when is each appropriate?
When defining 'active' for retention analysis, why is a simple 'login' event often a poor choice, and what are better alternatives?
What does it mean to 'shift the retention curve up' versus 'flatten the retention curve,' and how do these relate to different product strategies?
How would you measure retention for a product with a natural weekly or monthly usage frequency, as opposed to a daily-use product?
How can you use retention analysis by activation state to identify a product's aha moment?
Why are retention benchmarks across different companies or products often not directly comparable?
Why is a long-horizon retention metric hard to use as a success metric, and what trade-offs does choosing it involve?
Churn & User States
Explain the 'leaky bucket' metaphor in the context of user acquisition and retention. Why can't acquisition alone solve a fundamental retention problem?
What is the difference between voluntary and involuntary churn, and why does the distinction matter when interpreting a churn number?
How is measuring churn different from predicting churn, and why shouldn't they be conflated?
What is resurrection or win-back, and why is it treated as a distinct motion from acquisition and retention?
Explain the relationship between churn rate and retention rate. What is the difference between 'logo churn' and 'revenue churn,' and why can 'net revenue retention' exceed 100 percent?
How do you determine the appropriate period of inactivity after which a user is considered churned, and what factors influence this judgment?
Explain the user-state framework of new, current, resurrected, dormant, and churned users, and the growth-accounting identity that ties them together.
What is the 'quick ratio' as a growth-quality measure, and how do you interpret it?
Engagement & Activity Measurement
Define DAU, WAU, and MAU. What are the limitations of each, and how does the DAU-over-MAU 'stickiness' ratio help provide a more nuanced view of engagement?
What is the difference between session-based and user-based metrics?
How do you define what constitutes an active user for different types of products or services?
When would it be more appropriate to use a WAU/MAU ratio instead of a DAU/MAU ratio to measure stickiness?
What is RFM (recency, frequency, monetary) segmentation, and how is it used to understand engagement and value?
How do you distinguish feature adoption in terms of breadth, depth, and frequency, and why does each matter?
How is retention measured at social platforms using an event-driven data model, considering challenges like multiple sessions per day and billions of user events?
Why is the distribution of engagement often heavily skewed, and what does this imply about using average engagement metrics versus medians or power-user curves?
What are L7 and L28 engagement histograms, and what do they tell you that DAU/MAU cannot?
Cohort Economics & Monetization
Explain the concepts of LTV and CAC at a high level. How are they typically used together to evaluate acquisition channels?
What is the CAC payback period, and why is it an important metric especially for SaaS businesses?
Why is it dangerous to extrapolate an LTV estimate from immature cohorts?
What are common challenges or pitfalls when calculating LTV, particularly with non-constant churn or heavy-tailed lifetimes?
How do you build a cumulative-revenue or LTV curve from a cohort's retention curve?
What is the simple ARPU-over-churn approximation for LTV, and under what conditions does it break down?
Product Metrics Strategy
How would you measure the success of a new feature or product launch, considering both early signals and long-term outcomes, and what guardrail metrics would you establish?
What metrics do you track for product success, especially for early-stage versus growth-stage products?
What metrics would you use to measure success for a subscription product, and how would you use cohort analysis to track retention for such a product?
What is a vanity metric in the context of engagement and retention analysis, and how do you guard against relying on one?
Data Quality & Instrumentation
What is the difference between server-side and client-side event capture, and how does the choice affect funnel and retention accuracy?
Why must bot and internal traffic be excluded from these analyses, and how does their presence distort funnels and retention?
How does client-side data loss from ad blockers, offline sessions, or failed beacons bias event-based metrics?
Describe the essential instrumentation and tracking-plan prerequisites needed to perform robust funnel, cohort, and retention analysis.
How do you differentiate between a genuine drop in funnel conversion and a problem with the underlying event tracking or instrumentation?
What is 'sessionization' in event data, including the typical inactivity-timeout convention, and how is it relevant to funnel analysis?
What are some common event-data-specific quality problems, such as missing events, duplicate events, and identity stitching issues, that can break funnel analysis, and how do they manifest in the data?
Why is a metric definition silently changing over time a serious risk for cohort and retention analysis, and how do you protect against it?
How do late-arriving or backfilled events distort a funnel or retention number computed too soon, and how do you mitigate this?
Investigating Metric Changes
A core onboarding workflow experiences a significant drop-off in user completion. How would you investigate and diagnose the root cause of this drop-off?
A stakeholder says conversion fell 20 percent last week — walk through how you would investigate it.
You notice a significant drop in a key product metric such as conversion or retention for the latest cohort. Walk through your systematic approach to investigate what happened.
How do you approach a situation where funnel, cohort, or retention numbers reported by different tools or dashboards disagree?
Analytical Pitfalls & Causal Inference
Why might two different analysts, given the same raw event data, arrive at different conversion rates for the same funnel?
How can Simpson's Paradox affect funnel analysis when segmenting by different user groups or channels, and what does it imply about interpreting aggregate metrics?
Why is an observed correlation between using a feature and higher retention usually selection rather than a causal effect?
Why is funnel, cohort, and retention analysis fundamentally observational, and what can it not tell you on its own about causation?