Experimentation was never historically in science about winners and losers. Experimentation is always very hypothesis driven. It's about, what are you learning?
Ramesh Johari
Professor, Stanford University
8 quotes across 1 episode
Marketplace lessons from Uber, Airbnb, Bumble, and more
Learning isn't free. When you don't know something, it seems not even a question that you would allocate some of your samples to all options. After the fact you're like, 'Treatment was better. What the heck were we thinking? Why'd we give all those samples to control?'
A marketplace business never starts as a marketplace business, because what we think of as a marketplace business is something which at scale is removing the friction of the two sides finding each other. But when you start, you don't have that scale.
Maybe we shouldn't talk about the concept of a marketplace founder. Really there's founders. Every founder is a marketplace founder. It'll be a choice they make after they grow as to whether they want to build a platform.
You can't experiment your way out of everything. What people decide to test in a world that has promoted experimentation for everything tends to be more incremental by design.
What AI has done for us is it's massively expanded the frontier of things we could think about our problem, hypotheses we could have, maybe things we could test. I really think actually what that does is puts more pressure on the human, not less.
Prediction is inherently about correlation. But when we ask people to make decisions, we're asking them to think about causation. The distinction between causation and correlation matters a lot.
What AI has done for us is it's massively expanded the frontier of things we could think about our problem, hypotheses we could have, maybe things we could test. I really think actually what that does is puts more pressure on the human, not less.