The better models are at following instructions, the more valuable PMs become

Lenny's Podcast's latest episode features Tulsee Doshi, who leads product for Google's frontier AI models, covering Gemini, Nano Banana, Genie, Veo, and Lyria. The episode runs 58 minutes and focuses on what PMs actually do on model teams, and why the role has become critical again. The original video is below.

Day-to-day model development doesn't look like pure engineering. Tulsee compares it to Snape's Potions class: someone has to watch the recipe, the heat, and the failure modes. The model itself has UX—output quality, instruction following, shot control—all product problems. PMs need to break vague 'better' into trainable, evaluable goals. One commenter summed it up in reply: for Gemini and Veo, 'better' might mean reliable answers, or the camera following instructions; a good PM turns user needs into concrete goals the model team can train and evaluate.

This also explains why writing evals keeps coming up. One commenter said writing evals is the highest-priority work for PMs now; AI can produce what you ask for and still get it wrong. Another reply was more direct: if you can't write an eval, you haven't thought the requirements through clearly. It wouldn't be unreasonable to use that as a hiring standard.

The PM role is changing. The material mentions that PMs are moving from writing requirements to creating things with 'heft,' judging the complete experience rather than just staring at model scores. Google's full-stack frontier AI strategy gives this role more room, while Tulsee uses a 60-20-20 framework to balance capabilities that work now and those close to working. The specific meaning of the ratios is expanded on in the episode; I won't fill it in for her here.

There's another detail readers with development backgrounds should note: the stronger model capabilities get and the cheaper implementation becomes, the faster bad decisions reach users. One commenter said this actually raises the value of PM judgment. Another reply joked that PMs didn't suddenly become valuable—they've been buried in evals all along. It's a bit harsh, but close to the truth.

The episode also discusses avoiding AI slop, how PMs adapt to model iteration, and how to write your first eval. The original video is on YouTube, and the transcript is at Lenny's Newsletter. For general readers, this episode can be seen as an industry snapshot: when models do what they're told, the product manager's job shifts from passing messages to defining what 'doing it right' means.