Imagine you've invested in building a programming education app. Your acquisition campaign is working, and installations are increasing. Yet a closer look at user behaviour reveals that many new users never complete their first lesson.
Marketing wants more reminders. Design thinks the initial experience is too complicated. The content team suggests making the opening lesson shorter.
Each proposal might help. None has yet established why people are leaving.
As the Product Manager, what should you investigate before deciding what to change?
That is the challenge behind the fourth question in our series:
"What do you do if users install a learning tool but don't finish the first lesson?"
What is the interviewer actually assessing?
This is not simply a question about designing a better interface. It tests whether you can recognise a product problem, investigate user behaviour and make defensible decisions using incomplete information.
The interviewer wants to see whether you distinguish a symptom from its underlying causes.
Failure to complete lesson one is a symptom. The cause might be a confusing interface, unsuitable content, technical problems, a mismatch between expectations and reality, or limited time.
A strong answer should identify a particular user segment, establish the problem worth solving, compare viable approaches and propose a measurable experiment.
1. Find out who is dropping off and where
A common mistake is treating everyone who installs the application as a single group.
Someone who installs the app but never opens it has a different problem from someone who begins a lesson, studies for ten minutes and then leaves.
Before proposing features, I would map the actual user journey.
That journey might include installation, first launch, selection of a learning goal, viewing a lesson, starting it, interacting with the material and completing it.
At each stage, we need to understand how many users arrive, how many progress and where participation declines.
There is an important distinction here: installing an app does not necessarily indicate a serious intention to learn.
Some people install applications out of curiosity. We should not automatically attribute every failure to progress from installation to lesson completion to a weakness in the learning experience.
My initial questions would be:
- What proportion of installers actually start the first lesson?
- What proportion of starters complete it?
- At what point do users typically leave?
- Do they return and finish the lesson later?
- Are the patterns consistent across different user groups?
The last question is particularly useful.
Beginners might struggle more than experienced learners. The problem might disproportionately affect Android users, learners studying in a particular language or people with unreliable internet connections.
Without segmentation, we risk building a universal solution to a problem affecting only a particular group.
2. Separate behavioural signals from their causes
Suppose the analytics show a substantial drop-off near the beginning of the first lesson.
That observation tells us where something is happening, not why.
I would combine quantitative analysis with qualitative research.
On one side, I would examine the journey, time spent at each stage, technical errors, loading performance, content choices and exit points.
On the other, I would speak to users who abandoned the lesson and observe representative learners attempting the same tasks in usability sessions.
The investigation should distinguish several competing hypotheses.
Hypothesis one: Getting started requires too much effort.
Users might have to register, answer several questions and configure their learning preferences before seeing anything useful. The difficulty could begin before the lesson itself.
Hypothesis two: The content does not match expectations.
The acquisition message might promise practical, rapid learning, while the opening lesson consists of lengthy theoretical explanations.
Hypothesis three: The initial learning experience is too difficult.
A beginner may encounter unfamiliar terminology without enough support or any sense of progress. That can undermine their motivation to continue.
Hypothesis four: There is a technical or accessibility barrier.
Perhaps a video repeatedly buffers, progress is not saved, or a quiz cannot be completed using assistive technology.
Hypothesis five: The learner intends to finish in multiple sessions.
Leaving a lesson unfinished during the first session does not always mean failure. Users may expect to pause and resume when it suits them.
The W3C's guidance on providing sufficient time reinforces the importance of allowing people different amounts of time to read and interact with content, particularly where accessibility needs are involved.
These hypotheses must be evaluated against actual user evidence, rather than treated as conclusions.
3. Choose an intervention that addresses the cause
Once the strongest explanation becomes clearer, we can compare possible interventions.
If onboarding is unnecessarily complicated, we might remove non-essential steps or postpone configuration until after the user has experienced some value.
If learners cannot see why the lesson matters, we might make the intended outcome clearer and help them achieve an early, meaningful success.
If lesson length or cognitive load is the barrier, we could organise the material into smaller units, introduce manageable exercises or support resuming unfinished work.
The usability principle of progressive disclosure is relevant here: present essential information first, and reveal additional complexity when it becomes useful. Applying it successfully requires understanding what users need for their actual tasks, not merely hiding functionality.
Research published by Nielsen Norman Group also highlights the limitations of intrusive onboarding tutorials. Instructions presented before they are needed can interrupt users and may not help them perform a task later. Contextual assistance can be more appropriate in some situations.
However, we should not conclude that shorter lessons are automatically better.
A lesson that becomes easier to complete but teaches very little may improve the dashboard while weakening the product.
The decision needs to balance ease of starting, educational value and implementation cost.
4. A hypothetical software-product example
Hypothetical scenario
Consider a language-learning application whose opening lesson combines an introduction, audio playback, practice exercises and a final assessment.
The initial analytics suggest that a substantial proportion of new learners leave when they reach the listening exercise.
The team has three options.
The first is to send reminders. Notifications might encourage some users to return, but they will not resolve unclear instructions or broken audio playback.
The second is to simplify every lesson. That would involve a much larger investment and could damage educational quality without sufficient supporting evidence.
The third is to investigate the particular step and test a narrowly targeted improvement.
I would choose the third option.
Working with design, research and engineering colleagues, I would first establish whether the audio plays reliably, whether the instructions are understandable and whether the exercise matches beginners' existing knowledge.
Suppose the research reveals that novice users cannot work out what they are expected to do after listening.
An experimental version could introduce a short, contextual explanation and one worked example without lowering the educational standard.
We could then compare it with the existing experience and observe whether more learners complete the exercise successfully.
If subsequent evidence instead reveals that audio playback fails on weaker connections, I would change the decision and prioritise the technical defect.
The value of this approach is that we test a specific explanation at comparatively low cost rather than committing to a full redesign.
5. How should success be measured?
One of the biggest mistakes in this situation is choosing a metric that can improve without producing a genuinely better learning experience.
If lesson completion is our only measure, the team might improve the number by removing important material or making the assessment trivially easy.
I would therefore use three layers of measurement.
Primary outcome: Successful first-lesson completion
For the hypothetical product, we could define the seven-day first-lesson completion rate as:
New users who complete lesson one within seven days of starting it ÷ new users who started lesson one.
This is a proposed analytical definition for our scenario, not a universal industry standard.
Separately, I would measure the proportion of eligible new installers who start the lesson.
These metrics answer different questions: one measures the transition into learning; the other measures progress once learning has begun.
Diagnostic metrics
I would examine time to first meaningful learning success, step-level abandonment, exercise errors, use of contextual guidance and rates of returning to resume unfinished lessons.
Event-based analytics can help structure these measurements.
Google Analytics recommends events such as tutorial_begin and tutorial_complete for measuring onboarding tutorial activity.
Importantly, those events describe onboarding processes. We should not assume that completing an app tutorial is equivalent to completing an actual educational lesson. Lesson events need their own clear definitions.
Guardrail metrics
We must ensure that improved completion does not come at the expense of learning, reliability or satisfaction.
Useful safeguards might include performance on a short comprehension exercise, progression to lesson two, subsequent return behaviour, technical failures and qualitative feedback.
Before running an experiment, I would agree on event definitions, comparison groups, observation periods and success criteria.
If the available sample is too small to support a confident conclusion, I would describe the findings as preliminary rather than claim an established improvement.
6. What if you only had two weeks?
This is one of the two follow-up questions included in the original interview exercise.
With two weeks available, I would not promise a complete redesign. My aim would be to reduce the most consequential uncertainty.
During the opening days, I would validate the analytics, establish a reliable funnel and identify a user segment experiencing a significant, actionable problem.
I would then conduct a small number of targeted interviews and usability sessions to understand what prevents those users from progressing.
Next, I would work with design and engineering to introduce one limited change and expose it to an appropriate test group.
At the end of the fortnight, I would not claim to have proved long-term learning outcomes.
A credible result might be a validated explanation, the removal of an obvious usability barrier or early evidence supporting the next experiment.
If the sample is insufficient, the test should continue beyond the initial two-week investigation.
7. What evidence would make you reject your proposed solution?
This is the second follow-up question from the source material, and it tests an essential aspect of product judgement.
Suppose our recommendation is to divide the first lesson into shorter sections.
If the experimental version increases completion but learners perform worse on comprehension exercises or are less likely to begin lesson two, I would not automatically consider the change successful.
Likewise, if abandonment turns out to be concentrated on devices experiencing playback failures, I would shift the priority from educational redesign to technical reliability.
A Product Manager should not become attached to a proposed solution.
The important capability is knowing what evidence would support it, contradict it or justify changing direction.
8. How I would answer in an interview
Illustrative spoken answer from a Product Manager candidate:
"First, I'd check that we're defining the problem properly. Are users failing to start the first lesson, or starting it and dropping out partway through? Those are different problems and probably need different solutions.
"I'd look at the funnel and segment users by experience level, content and technical conditions. I'd also speak to a few learners to understand what is actually getting in their way.
"If the evidence suggested that beginners were struggling with the first exercise, for example, I'd try some contextual guidance at that point rather than redesign the entire course.
"I'd measure whether more people complete the lesson, but I'd also check whether they've genuinely learnt something and whether they return for the next one.
"If the evidence doesn't support my hypothesis, I'd change the approach. The goal isn't simply to improve a conversion figure. It's to help learners reach their first meaningful learning outcome with less unnecessary friction."
9. What makes an answer weak?
"I'd make the first lesson shorter." This selects a solution before establishing a cause. Lesson length might not be the problem.
"I'd send notifications to everyone." This ignores user differences and risks increasing interruptions or prompting people to disable notifications altogether.
"I'd increase the completion rate." Without a meaningful definition of success and safeguards for learning quality, the metric could improve while the real experience deteriorates.
"I'd add all the features users need." This provides no evidence of prioritisation, no clear target group and no justification for development effort.
A convincing answer should explain not only what you would try but also the assumptions, constraints, risks and evidence that would lead you to reconsider it.
Conclusion
When people install a learning application but fail to complete the first lesson, the right starting point is not automatically a redesign, more reminders or additional features. It is to discover which learners are struggling, where their progress stops and what prevents them from reaching an initial meaningful learning outcome.
I would map the journey, identify a specific affected segment, investigate the most plausible cause and test the smallest intervention capable of challenging that hypothesis. At the same time, I would monitor learning quality and subsequent engagement so that an attractive completion figure does not conceal a weaker experience.
If the evidence points to a different cause, or the intervention damages educational value, I would change course.
Good Product Management is not about building a solution faster. It is about making sure the solution addresses the right problem.
