How do you calculate step-conversion rate, overall end-to-end conversion rate, and drop-off rate in a funnel, and how are they related?
Junior
What is a conversion funnel, and why is analyzing conversion in stages more informative than a single end-to-end conversion rate?
Junior
How do the definition and structure of a conversion funnel vary across different business models such as e-commerce checkout, SaaS onboarding, or content consumption?
Mid
When defining a conversion funnel, what are the key considerations for choosing the steps and the underlying event taxonomy?
Mid
What does a funnel analysis actually show you, and what crucial information does it not directly provide?
Mid
What is the difference between macro and micro conversions in a funnel, and why is it important to track both?
Mid
How do you identify the 'bottleneck' step in a conversion funnel, and what's the difference between the biggest percentage drop and the biggest absolute user loss?
Mid
Funnel Metrics & Calculation
Describe the concept of a 'conversion window' in funnel analysis. How does changing this window impact the reported conversion rates, and why is it important to define it clearly?
Mid
What is the difference between a 'strict order' funnel and an 'any order' funnel, and when is each approach appropriate?
Mid
What are 'first-touch' and 'last-touch' attribution within a conversion funnel, and what are their respective uses?
Mid
What are the implications of counting 'users' versus 'sessions' versus 'events' when calculating funnel conversion rates, and when would you choose each approach?
Senior
Why might a real-world conversion funnel not be strictly monotonic, with users appearing to skip steps or enter mid-funnel, and how should this be handled in analysis?
Senior
How do you handle users who re-enter a funnel multiple times, and what are the considerations for deduplicating or counting repeat attempts?
Senior
What is a 'time-to-convert' distribution, and why is the mean often a misleading metric for it, especially when conversions have a long tail?
Senior
What are survivorship effects deeper in a funnel, and how can they distort the interpretation of later-stage conversion?
Senior
Funnel Diagnosis & Optimization
How would you analyze a conversion funnel to identify the primary drop-off points and potential friction areas?
Mid
How would you use user segmentation in your analyses for funnels or cohorts, and what types of insights would you expect to gain from it?
Mid
What are the common mistakes in funnel analysis, and how can they be avoided?
Mid
What is the difference between funnel analysis and path or journey analysis, and when is each more appropriate?
Mid
How do you ensure each stage of a funnel is aligned with a company's revenue and growth goals?
Senior
Cohort Analysis Foundations
Explain what cohort analysis is and why it is important for understanding user behavior and retention patterns, especially compared to aggregate metrics.
Junior
How do you define the cohort date for each user when structuring a retention analysis?
Junior
What is the difference between a cohort and a plain user segment, and when does grouping become a cohort?
Junior
When reading a cohort retention grid, explain what it means to read 'down a column,' 'across a row,' and 'along the diagonal,' and what insights each view provides.
Mid
What is the difference between an 'acquisition cohort' and a 'behavioral cohort'? When would you use each type?
Mid
How do you decide on the appropriate cohort grain such as daily, weekly, or monthly for a product?
Mid
What is the difference between reading a metric across calendar time versus across cohort age, and why does cohorting exist to separate the two?
Mid
What are common pitfalls in interpreting cohort retention data, and how do you account for confounding factors when analyzing retention?
Senior
Explain how cohort analysis can be used to compare the quality of different acquisition channels rather than just volume or cost per install.
Senior
Cohort Construction & Interpretation
Why is it crucial to normalize each row in a cohort table by its own 'cohort size' when comparing retention across different cohorts?
Mid
Why do small or young cohorts move noisily, and why shouldn't a single small cohort's movement be read as a trend?
Mid
Discuss the issue of 'incomplete' or 'immature' cohorts in the lower-right of a retention grid. Why should these not be interpreted as a decline in retention?
Senior
In what sense is a retention curve a survival curve, and what does it mean that the newest cohorts are right-censored?
Senior
Retention Metrics & Curves
What is user retention, and why is it considered a critical metric for a product's sustainable growth?
Junior
What are the pitfalls of relying on the day-1, day-7, and day-30 retention conventions?
Junior
Explain the characteristic shape of a typical retention curve. What does a curve that flattens to a stable asymptote signify, versus one that decays to zero?
Mid
What is a smile curve in retention, and what might it signify about user behavior or product changes?
Mid
Define 'N-day retention,' 'unbounded (or rolling) retention,' and 'range (or bucketed) retention.' How do these different definitions impact the reported retention numbers for the same product, and when is each appropriate?
Senior
When defining 'active' for retention analysis, why is a simple 'login' event often a poor choice, and what are better alternatives?
Senior
What does it mean to 'shift the retention curve up' versus 'flatten the retention curve,' and how do these relate to different product strategies?
Senior
How would you measure retention for a product with a natural weekly or monthly usage frequency, as opposed to a daily-use product?
Senior
How can you use retention analysis by activation state to identify a product's aha moment?
Senior
Why are retention benchmarks across different companies or products often not directly comparable?
Senior
Why is a long-horizon retention metric hard to use as a success metric, and what trade-offs does choosing it involve?
Senior
Churn & User States
Explain the 'leaky bucket' metaphor in the context of user acquisition and retention. Why can't acquisition alone solve a fundamental retention problem?
Mid
What is the difference between voluntary and involuntary churn, and why does the distinction matter when interpreting a churn number?
Mid
How is measuring churn different from predicting churn, and why shouldn't they be conflated?
Mid
What is resurrection or win-back, and why is it treated as a distinct motion from acquisition and retention?
Mid
Explain the relationship between churn rate and retention rate. What is the difference between 'logo churn' and 'revenue churn,' and why can 'net revenue retention' exceed 100 percent?
Senior
How do you determine the appropriate period of inactivity after which a user is considered churned, and what factors influence this judgment?
Senior
Explain the user-state framework of new, current, resurrected, dormant, and churned users, and the growth-accounting identity that ties them together.
Senior
What is the 'quick ratio' as a growth-quality measure, and how do you interpret it?
Senior
Engagement & Activity Measurement
Define DAU, WAU, and MAU. What are the limitations of each, and how does the DAU-over-MAU 'stickiness' ratio help provide a more nuanced view of engagement?
Junior
What is the difference between session-based and user-based metrics?
Junior
How do you define what constitutes an active user for different types of products or services?
Junior
When would it be more appropriate to use a WAU/MAU ratio instead of a DAU/MAU ratio to measure stickiness?
Mid
What is RFM (recency, frequency, monetary) segmentation, and how is it used to understand engagement and value?
Mid
How do you distinguish feature adoption in terms of breadth, depth, and frequency, and why does each matter?
Mid
How is retention measured at social platforms using an event-driven data model, considering challenges like multiple sessions per day and billions of user events?
Senior
Why is the distribution of engagement often heavily skewed, and what does this imply about using average engagement metrics versus medians or power-user curves?
Senior
What are L7 and L28 engagement histograms, and what do they tell you that DAU/MAU cannot?
Senior
Cohort Economics & Monetization
Explain the concepts of LTV and CAC at a high level. How are they typically used together to evaluate acquisition channels?
Mid
What is the CAC payback period, and why is it an important metric especially for SaaS businesses?
Mid
Why is it dangerous to extrapolate an LTV estimate from immature cohorts?
Mid
What are common challenges or pitfalls when calculating LTV, particularly with non-constant churn or heavy-tailed lifetimes?
Senior
How do you build a cumulative-revenue or LTV curve from a cohort's retention curve?
Senior
What is the simple ARPU-over-churn approximation for LTV, and under what conditions does it break down?
Senior
Product Metrics Strategy
How would you measure the success of a new feature or product launch, considering both early signals and long-term outcomes, and what guardrail metrics would you establish?
Mid
What metrics do you track for product success, especially for early-stage versus growth-stage products?
Mid
What metrics would you use to measure success for a subscription product, and how would you use cohort analysis to track retention for such a product?
Mid
What is a vanity metric in the context of engagement and retention analysis, and how do you guard against relying on one?
Mid
Data Quality & Instrumentation
What is the difference between server-side and client-side event capture, and how does the choice affect funnel and retention accuracy?
Mid
Why must bot and internal traffic be excluded from these analyses, and how does their presence distort funnels and retention?
Mid
How does client-side data loss from ad blockers, offline sessions, or failed beacons bias event-based metrics?
Mid
Describe the essential instrumentation and tracking-plan prerequisites needed to perform robust funnel, cohort, and retention analysis.
Senior
How do you differentiate between a genuine drop in funnel conversion and a problem with the underlying event tracking or instrumentation?
Senior
What is 'sessionization' in event data, including the typical inactivity-timeout convention, and how is it relevant to funnel analysis?
Senior
What are some common event-data-specific quality problems, such as missing events, duplicate events, and identity stitching issues, that can break funnel analysis, and how do they manifest in the data?
Senior
Why is a metric definition silently changing over time a serious risk for cohort and retention analysis, and how do you protect against it?
Senior
How do late-arriving or backfilled events distort a funnel or retention number computed too soon, and how do you mitigate this?
Senior
Investigating Metric Changes
A core onboarding workflow experiences a significant drop-off in user completion. How would you investigate and diagnose the root cause of this drop-off?
Mid
A stakeholder says conversion fell 20 percent last week — walk through how you would investigate it.
Mid
You notice a significant drop in a key product metric such as conversion or retention for the latest cohort. Walk through your systematic approach to investigate what happened.
Senior
How do you approach a situation where funnel, cohort, or retention numbers reported by different tools or dashboards disagree?
Senior
Analytical Pitfalls & Causal Inference
Why might two different analysts, given the same raw event data, arrive at different conversion rates for the same funnel?
Senior
How can Simpson's Paradox affect funnel analysis when segmenting by different user groups or channels, and what does it imply about interpreting aggregate metrics?
Senior
Why is an observed correlation between using a feature and higher retention usually selection rather than a causal effect?
Senior
Why is funnel, cohort, and retention analysis fundamentally observational, and what can it not tell you on its own about causation?