korrents

korrents · Lenny's Podcast

Finding hidden growth opportunities in your product | Albert Cheng (Duolingo, Grammarly, Chess.com)

Albert Cheng · 1h 25m · youtube.com ↗

26 korrents from this recording

1h
Albert Cheng did not write this page.

Every claim below is a statement made in this recording, quoted word for word and linked to the second it was said, so you can hear it rather than take our word for it. The wording comes from the transcript published alongside the recording; the sentence above each quote is our reading of the claim, not their wording.

  1. 0:10:32 · watch on youtube.com↗

    Our reading

    A growth team that only explores turns scattershot, and one that only exploits saturates — the job is to keep oscillating between the two.

    Their wordsIf you do too much exploration, you can have your team feel a little bit too scattershot, just trying a 100 different random ideas. What's the through line? What's the strategy? How do you pattern match, you know, successes across them? And if you do too much in exploitation, which is often the MMO of growth teams, it can lead to this like saturation and stagnation where you're just locally maximizing a thing.
    Interesting
  2. 2 min later

  3. 0:12:03 · watch on youtube.com↗

    Our reading

    After a defeat people want encouragement, not a list of their mistakes — so a learning product should lead with what the user did well.

    Their wordsWhen you lose a game now as opposed to surfacing your blunders and your like horrible stuff that you did, we flip it on its head. And so we show you your brilliant moves, your best moves. And we have coach say something encouraging. You know, losing just part of learning like keep it up, that type of thing. That change alone was pretty dramatic for us. It grew game reviews by 25%, subscriptions by 20%
    Interesting
  4. 2 min later

  5. 0:13:34 · watch on youtube.com↗

    Our reading

    Only 30 to 50 percent of product experiments win, because consumer behaviour is too unpredictable to be predicted from a hypothesis.

    Their wordsI mean in my experience the typical win rate and I hate to use that term for experiments is is often something like 30 to 50%. Like usually you're not actually get like you're trying a bunch of things a lot of hypotheses turn out not to be true. you know, consumer products are very unpredictable like that.
    Interesting
  6. 3 min later

  7. 0:16:13 · watch on youtube.com↗

    Our reading

    A rising share of statistically insignificant experiments is the signal that a team has exploited an area too far and should go back to exploring.

    Their wordsAnd if I'm starting to see like more and more experiments that are not statistically significant, that may be a signal to me to say, okay, we might have kind of tried to exploit a little bit too far. Like there might not be as much juice to squeeze.
    Interesting
  8. 1 min later

  9. 0:17:36 · watch on youtube.com↗

    Our reading

    Most of the questions people have about a company's data are never asked, and giving them a machine to ask instead of a colleague uncovers them.

    Their wordsAnd I think what's also kind of interesting is that just human nature is that if you have a question that you feel like, you know, you might be a bit embarrassed to ask or you don't want to bother someone, you just don't ask the question, right? And so by the nature of having these tools, you get actually a pretty large explosion of questions being asked.
    Interesting
  10. 2 min later

  11. 0:19:48 · watch on youtube.com↗

    Our reading

    AI has made prototyping fast but has not yet joined it to the workflow, so at any real scale the handoffs between functions are still there.

    Their wordsI would say though that something that is kind of annoying to me is that we haven't yet figured out the bridging from the tinkering to the workflow quite as seamlessly as I would like.
    Interesting
  12. 4 min later

  13. 0:24:10 · watch on youtube.com↗

    Our reading

    A freemium product's free tier should sample everything the paid product can do, because a limited free experience teaches users the product is limited.

    Their wordsand so I think this is interesting just modernization learning that especially if you work on a premium product try to have your free product be a reflection of like everything that your product can offer you. Obviously, to an extent, there's some costs involved with some of the paid features and things like that, but it generally will pay it for itself if you're able to put your best foot forward um and go do that.
    Interesting
  14. 3 min later

  15. 0:27:30 · watch on youtube.com↗

    Our reading

    Reverse trials work where a product accumulates lock-in, which is why they suit B2B tools and rarely beat an ordinary free trial in consumer products.

    Their wordsI think if you have especially a B2B feature where you may have some lock in reverse trials can be super powerful. You just want to get people in there. You don't need to ask for their credit card because they're using your CRM or they're investing quite a lot of time in like building out, you know, material and content. And so by the time that window drops, you actually like feel, oh man, I probably should keep this and and start paying. I think for a lot of consumer products, it's a little bit harder for that to work. And so I've typically seen more just normal free trials be be the norm.
    Interesting
  16. 1 min later

  17. 0:28:30 · watch on youtube.com↗

    Our reading

    Retention is what makes a consumer subscription business possible: without it the whole burden falls on getting a stranger to pay on day one.

    Their wordsYeah, I mean user retention is gold for consumer subscription companies. If you don't retain your users, then a lot of the onus is on getting them to pay on like day one. That's super hard, right?
    Interesting
  18. 1 min later

  19. 0:29:48 · watch on youtube.com↗

    Our reading

    A consumer app with 30 to 40 percent day-one retention is in solid shape, and much below that calls the users' intent into question.

    Their wordsI think when you have your D1 retention somewhere around like the 30 or 40% mark, like that's quite solid, I think, for for a consumer app.
    Interesting
  20. 1 min later

  21. 0:30:47 · watch on youtube.com↗

    Our reading

    For a daily-frequency product it is existing-user retention, not new-user retention, that compounds — and mature companies should put most of their energy there.

    Their wordsI think especially if you have a product that has daily frequency like that's actually the retention that matters the most is that like of your existing user base that has developed a habitual pattern how sticky is your product and it's that retention rate that really compounds and build that builds that daily habit.
    Interesting

  23. 0:31:13 · watch on youtube.com↗

    Our reading

    For a product you install rather than open, activation is the decisive moment and it carries the user for years.

    Their wordsOne exception is that Grammarly was a different type of product in that you install it and you don't proactively open it every day. So that was kind of interesting to me because I assume that you should always just focus on existing user retention. But for a product like Grammarly, it's actually the activation, installation, aha moment that's really really critical and will carry the user for a very very long time.
    Interesting
  24. 2 min later

  25. 0:33:27 · watch on youtube.com↗

    Our reading

    A mature product's growth comes from resurrecting dormant users, so the returning-user experience deserves to be designed as deliberately as onboarding.

    Their wordsAnd it's actually worth spending some time making sure that that kind of resurrected, for lack of a better word, experience inside the product is really excellent and that you find novel ways to try to bring them back.
    Interesting
  26. 6 min later

  27. 0:39:33 · watch on youtube.com↗

    Our reading

    Duolingo's real competence is motivation and habit formation, and language learning was only the first thing it applied that to.

    Their wordsI actually in many ways see language learning as like their first vehicle. And what they have a superpower in is that again, the motivation, the habits, etc.
    Interesting
  28. 3 min later

  29. 0:42:09 · watch on youtube.com↗

    Our reading

    Brand marketing and growth experimentation are not rivals — a product personality carried into social media becomes fuel for the measured growth work.

    Their wordsYeah, I mean I used to think it was versus, but now I realize that they combine really well. It could be rocket fuel for for your growth.
    Interesting
  30. 2 min later

  31. 0:43:42 · watch on youtube.com↗

    Our reading

    Steady experimentation is what a growth team controls, but the step changes come from outside waves the company did not create and would be foolish to ignore.

    Their wordsAnd it's like the growth experimentation is more the the the slow and steady or fast and steady, I should say, uh approach where you're just continually iterating. You're making the product experience better, but then every so often, right, there's a big wave that comes in. you can quadruple your, you know, registrations overnights and yeah, you'd be a fool not to take advantage of that.
    Interesting
  32. 6 min later

  33. 0:49:44 · watch on youtube.com↗

    Our reading

    Large language models are bad at chess: pattern recognition is not the same capability as searching one position very deeply.

    Their wordsYou know, interestingly, LM themselves are quite bad at playing chess. Like, they hallucinate moves. They look at patterns, right? They're they're very good at pattern recognition, but not so good at going super super super deep on a specific chess thing.
    Interesting
  34. 2 min later

  35. 0:51:30 · watch on youtube.com↗

    Our reading

    Growth is not metrics hacking: the job is to connect users to the value of the product.

    Their wordsI like to describe growth as the the job is to connect users to the value of your product.
    Interesting
  36. 5 min later

  37. 0:56:07 · watch on youtube.com↗

    Our reading

    Working at a company turns you into a power user, which is why pattern matching fails and even an experienced growth leader is wrong all the time.

    Their wordsI've worked a lot of companies right but I'm wrong all the time and I think consumer behavior can be very fickle and especially when you work at a company you become a power user naturally. So sometimes you you may forget like what the actual user experience is for a brand new user.
    Interesting
  38. 2 min later

  39. 0:58:16 · watch on youtube.com↗

    Our reading

    The value of an ambitious experiment target is not hitting it but the conversation about what would have to be true to hit it.

    Their wordsAnd a thousand experiments by itself, like if you just did that, but you didn't learn, you didn't make an impact, and that's kind of a waste of time, right? The whole point of setting a goal is that you can have conversations about what would need to be true to actually hit that goal.
    Interesting
  40. 3 min later

  41. 1:01:30 · watch on youtube.com↗

    Our reading

    The experimentation system — a growth model and correct instrumentation — matters more than any individual experiment.

    Their wordsThe system matters just as much as any given experiment. Probably even more. Right. I think starting with a growth model so you have an understanding of how your company grows in the first place and which channels you're going to leverage is critical. You need to make sure that you are instrumenting your product in and out otherwise you're going to run experiments and have wonky results.
    Interesting
  42. 3 min later

  43. 1:04:06 · watch on youtube.com↗

    Our reading

    You cannot manufacture virality, but you can find the moments users already screenshot and make those moments far better.

    Their wordsSo as opposed to going against I guess human intuition and trying to get them to share stuff that they otherwise wouldn't on the margins want to share like lean into it more actually like grab the moments where users are already organically screenshotting and make those much much much better and you can kind of 5x or 10x and and drive a lot of growth that way too.
    Interesting
  44. 1 min later

  45. 1:05:34 · watch on youtube.com↗

    Our reading

    Gamification needs three things at once — a tight core loop, a metagame to strive at, and a profile that accumulates the user's investment.

    Their wordsThen you need a metag game, which for Duolingo is kind of like the path, but it's also the leaderboard, achievements, kind of long-term things that you're going to strive to such that you have like long-term, I guess, motivation uh to continue doing the thing. And then the profile is also critical because you build up a profile over time. It's a reflection of your investment inside the product experience. And so when you nail those three things, you can end up with a long-term learning journey that can be quite successful.
    Interesting

  47. 1:06:03 · watch on youtube.com↗

    Our reading

    Most newcomers to a skill lose their first encounter with it, and losing costs retention — so beginners need a route around the real game.

    Their wordslike what we see is that over 75% of our new users, they classify themselves as like, I'm completely new to chess or I'm a beginner. And unfortunately, if you're new to chess and you're a beginner, you're not going to have that fun of a time playing live games. We see this in the data. It's like less than a third of those users actually win their first game. And when you lose a game, user retention is 10% worse than when you win a game.
    Interesting
  48. 3 min later

  49. 1:08:35 · watch on youtube.com↗

    Our reading

    Agency and clock speed beat domain experience in a hire, and in a fast-moving field experience is often a crutch of habits that need discarding.

    Their wordsI saw some of the highest performers just being people that had very high agency, had that clock speed, had that energy. Um, yes, they they cared about the mission, but they didn't necessarily need to have deep experience on that matter. And in fact, sometimes that experience could be a crutch in certain ways. Especially in this world where the grounds are shifting so fast with AI, a lot of your like learn habits actually need to be intentionally discarded. You know, you need to have a beginner's mind on this type of stuff.
    Interesting
  50. 7 min later

  51. 1:15:49 · watch on youtube.com↗

    Our reading

    Doing PR before you have evidence customers want the thing is risky, because the publicity creates a sunk cost that makes the team see a failing product through.

    Their wordsAnd the third one is like we did a lot of actually PR uh prior to the service going out just to get the word out. Um and you know PR has has its time and place. Uh but I think doing it before you have validation that customers definitely want the thing is quite risky. It can lead to a lot of sunk cost uh once you get it out because you're you're just you know you need to see it through.
    Interesting