Lenny Distilled

Brendan Foody

CEO and Co-founder, Mercor

7 quotes across 1 episode

Why experts writing AI evals is creating the fastest-growing companies in history

I think the people that can leverage AI to do whatever their day-to-day workflows are... We'll give people interviews where we say, 'Use ChatGPT and Kodak. Use Claude code. Use whatever tool cursor and whatever tools are available to build a website and let's see what product you're able to build in an hour.'

It's some combination of being stubborn with respect to your thesis around how the world will change, but also very open-minded with respect to exactly what form that takes and how the market's developing.

If the model is the product, then the eval is the product requirement document.

Sometimes you just need to sheer it from the market and know that it's there, the poll, to know the right places to focus. Because if it's difficult to sell, if it's extremely difficult to sell the marginal customer, you're not going to be able to grow a huge business.

You don't want to fight against them using the models. It's similar to when the calculator came out, you don't want to give people all of this arithmetic work of how do you get them to do it and not use the calculator. You want to tell them, 'Use the tools and let's see what you can do.'

The wealthiest companies in the world are willing to spend whatever it takes to improve model capabilities.