The argument in one line.
As AI lets nearly anyone prototype, code, or write a PRD, the scarce and defensible skill shifts from functional expertise to systems thinking — the habit of stepping outside your own ticket to question what you're assuming about the broader business, while craft accountability stays entirely human.
Read if. Skip if.
READ IF YOU ARE…
- A product, engineering, or design leader whose team is confused about who owns what now that AI lets everyone prototype.
- A manager or founder who wants a concrete framework for building a high-talent-density, high-accountability culture.
- Someone deciding whether to double down on a narrow specialty or broaden into systems-level thinking.
- Anyone curious how a company the size of Netflix operationalizes 'AI fluency' without a rigid mandate.
SKIP IF…
- You want hands-on prompt engineering or specific AI tool tutorials — this is a leadership/strategy conversation, not a how-to.
- You're looking for hot takes on AI replacing jobs wholesale — the guest's position is deliberately more measured.
- You want tactical detail on Netflix's actual tech stack or ML systems — this stays at the organizational and cultural level.
The full version, fast.
AI now lets PMs ship code and designers write PRDs, which has thrown product and engineering teams into a confusing 'storming' phase over who's responsible for what. Netflix's CPTO argues this doesn't dissolve functional expertise — craft excellence in engineering, data science, and design stays scarce — but it does raise the value of 'systems thinkers' who can abstract across business domains, build shared paved paths, and question their assumptions before shipping. The same logic extends to Netflix's culture: traits like high agency, autonomy, and top pay aren't goals in themselves, they're an 'excellence operating system' built on non-negotiable talent density, comfort with risk, and resisting the urge to add process after every failure.
Free for members
Chat with this breakdown — free.
Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.
Create a free account →Who's talking.
01:30guestElizabeth Stone
00:00hostLenny Rachitsky
Where the time goes.
00:00 – 02:25
01 · Introduction
Lenny opens with a pulled-quote teaser on role confusion under AI, then introduces Elizabeth Stone, Netflix's Chief Product and Technology Officer, framing this as her second appearance on the show.
02:25 – 07:36
02 · AI and role confusion: the storming phase before the forming phase
Elizabeth frames the current disorientation over blurred PM/designer/engineer roles as a normal 'storming' period that precedes a new equilibrium, not a reason to restrict AI use.
07:36 – 11:55
03 · How roles have changed in the past two and a half years
PMs, designers, and data scientists now push further into the product lifecycle — prototyping and writing code — before engineering has to get involved, but only once the business problem is already validated.
11:55 – 13:26
04 · Will functions survive? The case for craft specialism
Despite blurring, Elizabeth argues deep craft excellence in engineering, data science, and design remains scarce and valuable, and specialized disciplines aren't disappearing.
13:26 – 17:22
05 · What Netflix is hiring more of—and less of
Netflix is hiring more 'systems thinkers' who can abstract across business domains into shared infrastructure and design systems, rather than narrow local-feature builders.
17:22 – 20:20
06 · Why systems thinking is the rising skill across every function
As AI-driven agents operate across more systems at higher velocity, the risk of inconsistent output rises — making shared paved paths, guardrails, and systems thinking essential scaffolding.
20:20 – 22:08
07 · Is the design process dead?
Responding to a rival thesis that 'design process is dead,' Elizabeth pushes back — design moves faster with AI tools, but deep design thinking remains core to Netflix's product quality.
22:08 – 28:33
08 · Skills trending down
Narrow, deep specialization is trending down in favor of adaptable generalists who can move across the engineering stack, except in a handful of irreplaceable deep-technical niches.
28:33 – 31:00
09 · AI fluency and Netflix's career ladder overlay
Rather than rewriting career ladders level by level, Netflix layered a company-wide 'AI fluency' aspiration on top of existing levels, now factored into hiring interviews too.
31:00 – 35:12
10 · AI use cases beyond coding
Beyond prototyping, Netflix's biggest AI wins are distilling decades of institutional data and content-production tools — localization, pre-visualization, and the newly-acquired InterPositive's re-shoot capabilities.
35:12 – 38:36
11 · Netflix's AI history
Netflix has used ML/AI for over a decade, including the $1M Netflix Prize recommendation contest, giving it a head start applying newer generative techniques to personalization and content creation.
38:36 – 41:11
12 · Excellence as an operating system
Elizabeth reframes Netflix's famous culture traits — high agency, autonomy, top-of-market pay — not as ends in themselves but as deliberate mechanisms engineered to produce excellence.
41:11 – 46:41
13 · The pillars of the excellence OS
Talent density is non-negotiable; the rest is comfort with risk-taking, resisting the urge to add process after failures, and letting people make and learn from their own decisions.
46:41 – 50:21
14 · The keeper's test—and why it's mostly a positive conversation
The keeper's test — 'would I fight to keep this person if they told me they were leaving?' — is framed as usually an affirming conversation about strengths, not just an exit mechanism.
50:21 – 52:54
15 · Attracting top talent in the age of frontier AI labs
Netflix competes for talent not by matching frontier-lab compensation optics but by appealing to people who love the specific intersection of entertainment, consumer product, and technology at global scale.
52:54 – 56:25
16 · Junior talent, craft mastery, and the mentorship question
Netflix still runs intern and new-grad programs; junior hires bring native fluency in new work styles, but mentors still must teach craft mastery and accountability for AI-assisted output.
56:25 – 59:45
17 · Where engineering goes in 5 to 10 years
Elizabeth expects engineers to need less raw language syntax but more systems understanding — able to diagnose and fix what an agent builds, even without having hand-written it.
59:45 – 1:02:18
18 · The future of entertainment: beyond film and TV
Netflix's entertainment offering is expanding past film and TV into games, live events, and podcasts, requiring far more seamless, personalized discovery across formats and moments of day.
1:02:18 – 1:06:15
19 · AI in Hollywood: Netflix's creator-enablement position
Netflix positions itself as enabling creators across the full spectrum from AI-refusal to AI-embracing, while maintaining that human storytelling stays at the center of entertainment.
1:06:15 – 1:12:07
20 · Lightning round and final thoughts
Book recommendations, a favorite recent watch, Eight Sleep, a family life motto about the 'last 5%,' and Elizabeth's Tour de France cycling trip close out the conversation.
Lines worth screenshotting.
- PMs, designers, and data scientists now push further into the product lifecycle — prototyping, even writing code — before engineering has to unblock anything.
- Blurred roles only pay off when the business problem is already validated; without that clarity, prototyping just produces spaghetti no one asked for.
- Comparative strengths persist even as roles blur: data scientists still own 'can we trust this data,' PMs still own 'is this the right problem,' engineers still own 'how does this scale.'
- Netflix is hiring more people who can abstract across business domains into shared infrastructure, not just people who can move fast locally.
- As more agents operate across more systems at higher velocity, guardrails can't live in one person's head — they have to be encoded into shared, paved infrastructure.
- A concrete habit for building systems-thinking skill: on any problem, take one deliberate step back and question what you're assuming about the broader business context.
- Deep, narrow specialization is trending down in favor of people who can flex across the stack or across functions, except in a handful of genuinely rare technical niches.
- Netflix layered a single company-wide 'AI fluency' aspiration on top of existing career ladders instead of rewriting every level's criteria, because the underlying tech changes monthly.
- The single highest-value personal AI use case for Netflix's CPTO isn't coding — it's distilling years of institutional experiments and research into an instant, usable answer.
- Netflix's newly acquired InterPositive (founded by Ben Affleck) can relight, reframe, and change dialogue after a scene is already shot, still directed by the filmmaker's creative call.
- Netflix's AI credibility predates the generative wave by nearly two decades — the $1M Netflix Prize contest to improve the recommendation algorithm ran in the mid-2000s.
- Culture traits like high agency, autonomy, and top-of-market pay were never the goal themselves — they were mechanisms deliberately chosen to produce excellent outcomes.
- Talent density is the non-negotiable starting point; the rest of the culture (risk tolerance, minimal process, decision autonomy) only works once that bar is met.
- When something goes wrong, the instinct to bolt on more process has reliably cost Netflix time without improving planning, leveling, or comp outcomes.
- The keeper's test — 'would I fight to keep this person if they told me they were leaving?' — is most often used to open a positive conversation about someone's strengths, not to threaten an exit.
- Netflix recruits against frontier AI labs not on compensation optics but on the durable appeal of entertainment-plus-technology-plus-consumer-scale as a problem space.
- Junior hires remain deliberate, not a casualty of AI — they bring native fluency in current tools and consumer behavior that senior staff often lack.
- AI-generated code that performs well but can't be explained is a real, current discomfort for Netflix's own leadership, not a solved problem.
- Netflix's entertainment offering is deliberately expanding past film and TV into games, live events, and podcasts, which makes personalized discovery a harder problem, not an easier one.
- Netflix's stated AI-in-Hollywood position is non-prescriptive: some filmmakers refuse it entirely, others lean in heavily, and the studio supports the full spectrum rather than mandating a house style.
Systems thinking, not specialization, is what AI-era teams actually need
WHAT TO LEARN
As AI collapses the line between PM, designer, and engineer, the scarce skill isn't a job title — it's the ability to step back from your ticket and reason about how it fits the wider system, while craft accountability stays entirely human.
02AI and role confusion: the storming phase before the forming phase
- Confusion over 'what is my job now' is a normal storming phase that precedes a new stable way of working — it's not a signal to restrict AI use.
- The benefit of blurred roles only pays off when the business problem is already clear; prototyping without that clarity just produces spaghetti no one asked for.
03How roles have changed in the past two and a half years
- PMs, designers, and data scientists can now carry an idea further, prototyping and even writing code, before engineering has to unblock anything.
- AI is especially strong at distilling years of institutional experiments into a usable starting hypothesis, instead of relying on the one veteran who remembers every past test.
04Will functions survive? The case for craft specialism
- Comparative strengths persist even when roles blur: data scientists still own 'can we trust this data,' PMs still own 'have we framed the right problem,' engineers still own 'how does this scale.'
- Craft excellence in engineering, data science, and design is still scarce, AI makes execution faster, it doesn't manufacture judgment.
05What Netflix is hiring more of—and less of
- Netflix is hiring more people who can abstract across business domains into shared building blocks, not just people who can move fast locally.
- The same shift shows up in design: senior designers are needed less for pixel-level feature work and more for the design systems that keep the whole product coherent.
06Why systems thinking is the rising skill across every function
- As more agents operate across more systems at higher velocity, an organization can't rely on one person knowing where the guardrails are, that has to be encoded into shared, paved infrastructure.
- Velocity, not AI hype, is the real driver: more bets and more different kinds of people building things multiply the cost of not having shared paved paths.
- A concrete habit for building the skill: on any problem, take one deliberate step back and ask what you're assuming about the broader business context, without spiraling into solving the whole company's strategy.
07Is the design process dead?
- Faster tooling doesn't excuse skipping design thinking on high-stakes work, deep design judgment is still what keeps complexity invisible to the end user.
- The work of design changes shape, with more options and faster iteration, without the underlying mindset disappearing.
08Skills trending down
- Deep, narrow specialization is trending down in favor of people who can flex across the stack or across functions.
- Specialists aren't obsolete, they're expected to keep asking whether their old way of solving a problem is still the right one.
- A handful of genuinely rare technical niches, like video encoding or playback systems, still require deep, hard-to-replace specialists.
09AI fluency and Netflix's career ladder overlay
- Instead of rewriting every job level's criteria for AI, Netflix laid a single company-wide 'AI fluency' expectation on top of existing career ladders, deliberately vague because the tech changes monthly.
- That fluency expectation now extends to hiring: interviews probe how candidates use AI day to day, and candidates can use AI tools during coding interviews.
10AI use cases beyond coding
- The single highest-value AI use case for Netflix's CPTO personally isn't coding, it's data distillation, turning 'what did we learn from that old experiment' into an instant answer.
- On the content side, AI already powers localization, subtitles and dubs, and 'pre-visualization' that lets a filmmaker test a creative vision before using a physical set.
- Netflix's newly acquired InterPositive can relight, reframe, and change dialogue after a scene is shot, still directed by the filmmaker's creative call, not automated end to end.
11Netflix's AI history
- Netflix's AI credibility didn't start with generative AI, the $1M Netflix Prize predates the current AI wave by close to two decades.
- That history gives Netflix a head start, since the problem of personalizing an ever-expanding catalog was already understood before the tools got more powerful.
12Excellence as an operating system
- Culture traits like high agency, autonomy, and top-of-market pay were never the goal themselves, they were mechanisms chosen to produce excellent outcomes.
- The same traits that built Netflix's early culture deck are now what people associate with how top AI labs operate.
13The pillars of the excellence OS
- Talent density is the non-negotiable starting point; without it, no amount of autonomy or process design gets you consistent good decisions.
- When something goes wrong, resist the reflex to bolt on more process; more process has reliably cost time at Netflix without improving planning, leveling, or comp outcomes.
- Letting someone make a decision you'd have made differently, and staying out of it unless it's truly material, is deliberately uncomfortable but is how people learn to own outcomes.
14The keeper's test—and why it's mostly a positive conversation
- The keeper's test is most often used to open a positive conversation about someone's strengths, not to threaten an exit.
- It also works as an early-warning check: if you'd feel relief rather than urgency at someone's resignation, you should have had the harder conversation sooner.
15Attracting top talent in the age of frontier AI labs
- Netflix doesn't try to out-bid frontier AI labs on compensation optics, it recruits on the specific, durable appeal of entertainment plus technology plus consumer scale.
- The pitch to candidates is explicit: if foundational model research excites you, Netflix isn't that place, but few companies offer this combination at this scale.
16Junior talent, craft mastery, and the mentorship question
- Junior hires remain a deliberate part of the talent strategy, not a casualty of AI, bringing native fluency in new tools and current consumer behavior.
- Accountability for code and product quality doesn't transfer to the AI, so mentorship now has to explicitly teach 'what good looks like.'
17Where engineering goes in 5 to 10 years
- Writing a language's syntax and understanding how a system actually works are two different skills, and the second one doesn't go away even as agents do more of the first.
- AI-generated code that performs well but can't be explained is a real, current discomfort, not a solved problem, because you can't fix or trust what you can't diagnose.
18The future of entertainment: beyond film and TV
- Entertainment is deliberately not staying one thing at Netflix, with games, live events, and podcasts folding in alongside film and TV.
- More format variety makes discovery harder, not easier, so personalization has to work harder to keep a sprawling catalog from feeling fragmented.
19AI in Hollywood: Netflix's creator-enablement position
- Netflix's stated AI position is deliberately non-prescriptive: some filmmakers refuse it entirely, others use it heavily, and the studio supports the full spectrum.
- Even with AI doing more of the work, storytelling stays anchored to human performance and humanity, a bet on what audiences will keep responding to.
20Lightning round and final thoughts
- A recurring personal operating principle worth borrowing: 'the last 5% of effort usually makes all the difference,' a bias toward finishing rather than stopping at good enough.
- Watching for something good happening every day, even in stressful periods, is presented as a deliberate practice, not a passive mood.
Terms worth knowing.
- Systems thinking
- Stepping back from a specific problem to consider how it fits the broader set of business domains and shared building blocks, instead of solving it in isolation.
- Storming vs. forming
- A period of role confusion and friction a team works through before settling into a new stable way of working after adopting a transformative technology.
- Paved path
- A company-sanctioned, well-supported set of tools and infrastructure that teams default to, so they don't rebuild the same building blocks from scratch.
- Keeper's test
- A Netflix framework where managers ask themselves whether they would fight to keep a given person if that person said they were leaving — used to open both praise and hard-feedback conversations.
- Excellence as an operating system
- Netflix's framing that cultural traits like high agency, autonomy, and top pay aren't goals in themselves — they're mechanisms engineered to consistently produce excellent outcomes.
- AI fluency
- Netflix's cross-functional, level-agnostic expectation that every employee develops judgment about when and how to use AI, rather than a fixed skills checklist.
- Member of technical staff
- An emerging title trend, associated with frontier AI labs, that collapses traditional role boundaries between PM, engineer, and designer into one flexible technical-contributor title.
Things they pointed at.
Lines you could clip.
02:25
“Everyone can be everything now. PMs can ship code. Designers can write PRDs, and engineers can product—and there's this confusion and frustration of what is my job anymore.”
12:40
“I still find great engineering to be scarce, great data science to be scarce, great creativity to be scarce.”
17:22
“We need more systems thinkers in a world with AI.”
21:40
“Each problem you're trying to solve, step out one click to the what am I assuming is true about the broader space.”
38:40
“Excellence as an operating system.”
41:20
“Talent density is the nonnegotiable. You have to start with that.”
42:40
“We don't try to avoid failures. We try to recover quickly when we have them.”
47:10
“The lion's share of the time, my response is, I would fight so hard to keep you.”
57:30
“I know I'm getting better performance from this, but I have no idea why, and if this thing breaks, I'm gonna have no idea how to fix it.”
1:09:40
“The last 5% of effort usually makes all the difference.”
Where the conversation goes.
00:00 – 11:55denseAI and role confusion across PM, design, and engineering
11:55 – 22:08denseWhat Netflix is hiring for: systems thinkers vs. specialists
22:08 – 31:00denseTrending-down skills and AI fluency in career ladders
31:00 – 38:36steadyAI use cases beyond coding and Netflix's AI/ML history
38:36 – 46:41denseExcellence as an operating system and its pillars
46:41 – 56:25denseKeeper's test, hiring, and junior talent
56:25 – 1:06:15steadyEngineering's future and the future of entertainment
1:06:15 – 1:12:07sparseLightning round
Word for word.
Read-along
Don't just watch it. Burn it in.
See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.
metaphoranalogy
00:00Everyone can be everything now. PMs can ship code. Designers can write PRDs.
00:04Engineers can product, and there's this confusion and frustration of what is my job anymore. Anytime
00:09a new technology comes along, you go through a storming phase before you go through the forming phase of things. We are in the middle of that right now.
00:18I don't think that means we should put AI back into the box and say, let's not use it. If we all become builders, will we still need separate functions? I still see craft excellence that's really important that I don't think is going away anytime soon.
00:31I still find great engineering to be scarce, great data science to be scarce, great creativity to be scarce. If you look at the early culture deck of Netflix, high agency, autonomy, paying top of market, this is what I hear constantly now from how the top AI labs operate.
00:47Netflix's culture has always been excellence as an operating system. It's a resistance to do the thing that a lot of bigger companies would do and to feel comfortable in that discomfort
00:58very often. What are the ingredients to make this happen? Talent density is the nonnegotiable,
01:03being very comfortable with risk taking. In cases where things are not going well, not assume that process is gonna fix What have you added to the career ladders within this AI world? We need more systems thinkers, people who can look across all the business domains and abstract that to here's the building blocks we're gonna need.
01:22How do people learn this? Small trick. Each problem you're trying to solve, step out one click to the what am I assuming is true about the broader space.
01:34Today, my guest is Elizabeth Stone, product and technology officer at Netflix. This is Elizabeth's second visit to the podcast. Her first visit when she was just the CTO was, for the longest time, one of the most popular episodes of this podcast.
01:47You'll soon see why. This is such a killer conversation because when we chatted 2.5 ago, AI was only starting to emerge. And as a long time head of engineering and product and data science, Elizabeth has such a unique perspective on where things are heading and what's worth paying attention to.
02:04Prior to Netflix, Elizabeth was VP of science at Lyft, chief operating officer at Nuna, an economist at the analysis group, and a trader at Merrill Lynch. Before we get into it, don't forget to check out lenny'sproductcast.com for an entire year free of the hottest and best crafted AI products in the world available exclusively to Lenny's newsletter subscribers.
02:23With that, I bring you Elizabeth Stone.
02:29Elizabeth, thank you so much for being here. Welcome back to the podcast.
02:33Thank you. I'm honored to be here once and now twice. That's right.
02:37That's a rare a rare treat for me. I don't know if you know this, but your first visit to the podcast, your episode ended up being my second most popular episode. You're right behind Brian Chesky for the longest time.
02:49Wow. I I'm pleasantly surprised and also mildly competitive
02:55of how do I get to the first spot, but I'll set that aside for now. That's this is our this is our shot. Brian's amazing, so I'll let that one go.
03:04Yeah. He is and then there's just, all these fancy AI people that are just coming, you know, coming in hot. So it's been two and a half years at this point.
03:13A lot's changed. Obviously, AI. Something AI is allowing, uh, people to do is everyone can kinda be everything now.
03:21This idea of PMs can ship code, designers can write PRDs, and engineers can product, and everyone's everything. There's a bunch of elements of this conversation.
03:31One is that I've heard from people that there's also this kind of confusion and frustration of, like, what is my job anymore? Like, what am I responsible for as a PM, as a designer?
03:41Is that something you've experienced? I hear it within Netflix for sure.
03:47I think anytime a new technology comes along, especially one that's as transformative as Gen AI, you go through a storming phase before you go through the forming phase of things, and I think we are in the middle of that right now.
04:03I don't think that means we should put AI back into the box and say, let's not use it because this is kinda this is complicating all of our preconceived notions about our roles.
04:14But I do think it means we have to be much more thoughtful about how do we get the benefits while reducing the costs. I think it's a great thing that people are experimenting with how can I develop an idea faster, prototype an idea, put together an initial set of code that would allow us to test it?
04:32Do I believe that means anyone should be shipping code to production, that everyone should actually be doing everything? Probably not.
04:41But I think that it's good for people to be exploring what's possible. And then, like I mentioned earlier, the benefit of having product and tech teams together is that if the business problem is clear, I think it's okay and it's healthy for there to be some fluidity in the roles that people play because instead of having to wait for the engineering team to be ready to be able to prototype something, product and design can move faster on it, but they should still work with their engineering partner to think through how should we productize this?
05:10How do we scale it? What are the guardrails for it? So I don't think it makes the functional expertise obsolete.
05:17I think it means that teams have to be more comfortable with maybe this helps us move faster in a certain direction. From an organizational perspective, things I think about to make this more coherent or less frustrating are some of the things that have to be in place for us to get the benefits rather than the cost.
05:36So that includes clarity on source of truth data, guardrails on shipping code to production or testing before we make large changes, thinking about opportunities where we can trust the output of AI versus we should have a process or review that helps us check that we're getting high quality outcomes, and the importance of reiterating that humans are still responsible for what happens.
06:02So it can be that an agent wrote the code or I helped to do an analysis when that's not really my background, but it doesn't make it doesn't make people not have the responsibility that comes with what they've created.
06:15So I think investing in some of those core infrastructure and practices and reiterating the accountability and responsibility for the outcomes helps to balance some of, like, what's possible with what we should actually be doing.
06:27This episode is brought to you by our season's presenting sponsor, WorkOS. What do OpenAI, Anthropic, Cursor, Vercel, Replit, Sierra, Clay, and hundreds of other winning companies all have in common?
06:40They are all powered by WorkOS. If you're building a product for the enterprise, you felt the pain of integrating single sign on, SCIM, RBAC, audit logs, and other features required by large companies. WorkOS turns those deal blockers into drop in APIs with a modern developer platform built specifically for b to b SaaS.
06:59Literally every startup that I'm an investor in that starts to expand upmarket ends up working with WorkOS, and that's because they are the best. Whether you are a seed stage startup trying to land your first enterprise customer or a unicorn expanding globally, WorkOS is the fastest path to becoming enterprise ready and unblocking growth.
07:17It's essentially Stripe for enterprise features. Visit workos.com to get started or just hit up their Slack where they have actual engineers waiting to answer your questions.
07:27Work o s allows you to build faster with delightful APIs, comprehensive docs, and a smooth developer experience. Go to workos.com to make your app enterprise ready today.
07:37What's really awesome about having you back on the podcast is we chatted, like, before AI was a massive transformation in the world. So it's a really cool arc that we can explore here, the the shift that we've all gone through.
07:50Mhmm. Coming back to the roles of the product and eng teams, I'm curious how much these roles have changed in the last two and a half years.
07:59If you think about product engineering, uh, design, data science, user research, which roles have changed most?
08:07Which roles have changed least? Like, what's most different in the last two since two and half years ago? So you've mentioned some of the things.
08:14So I'll I'll reiterate them and then maybe build. So I have found that PMs, designers, data scientists are able to get farther in the product development life cycle before engineering really needs to be front of the line in unlocking things than was true a couple years ago.
08:37I say that with some caution because, like we were talking about, I don't think it's great to all of a sudden have thousands of prototypes if they're not aimed at this is an important problem to solve for the business, and the engineering partners are aware that we're solving that problem and that designers and product managers are gonna take lead in starting to shape the idea, but it's not working in a vacuum, and it's not to see what sticks.
09:03But when it's the right problem, approach in a thoughtful way with some alignment on that. I've seen product design data science move faster in the direction of let's get to something that's testable on this hypothesis. So that's prototyping, that's writing code.
09:18The other thing I've seen as being very valuable is we have a lot of information running around in the virtual walls of Netflix. We have experiments we've run over decades.
09:28We have insights from consumers. We have input from stakeholders across the business, and that was a problem that really presented a challenge of, like, how do we get the most out of that long history of knowledge and learnings to say, let's apply that to the problem we've got now to move faster in this is a promising path, or this is something that we've learned something about and we could leverage here.
09:52And AI is very powerful at distilling information, looking across a broad set of things, doing an analysis around it, getting to the core of here's some insights to start with. I I would hesitate to rely on that exclusively, but I think it's a head start.
10:08And I find even in my own work day to day, instead of sending an email that disrupts someone of, like, remind me what research did we do in what year and what was the question and what was the test we ran? I can find that almost instantly, then I can form my own, here's what I find interesting about this, and I have now skipped a couple steps towards, is there something actionable here?
10:30So that's data analysis, it's modeling, it's distillation of information, and I'm seeing more people do that to your original question. So instead of that needing to be only the experts who were here for twenty years and saw every experiment or know where to find it, we're now able to do that faster within product and tech across all functions, and a big unlock for us is our business stakeholders sitting in finance and content and advertising can do that as well, and then bring back an initial hypothesis where they wanna work more deeply with the data scientist and engineer and so on.
11:03So there's something there about the the hypothesis generation, prototyping, thinking deeply about problems that feels like it's accelerating, and that functions are able to do that in a more fluid way.
11:16But I still see comparative strengths. So data scientists are still gonna be experts at, can we trust this data? Are we interpreting it the right way?
11:25What's the data versus judgment that we should be applying here? A product manager is still gonna be exceptional at saying, have we really framed the what of this, like, the problem we're solving in the right way?
11:36An engineer still has a craft around the how. How does this scale? What does high quality look like?
11:42What problems is this gonna create for us based on how we build and deploy something? So I still see the nuggets of that comparative advantage. It's just that we're able to move more fluidly in a lot of steps that normally we would have blockers on.
11:55There's so much interesting stuff here. Uh, one is this last point you made. It's something I've been thinking about.
12:00If we all become builders, will we still need separate functions? There's this, like, member of technical staff trend that is happening at PressAI where it's like, alright. We don't have a title.
12:09You could be anything. Uh, you don't have to be in a bucket. What you're saying here is you believe we will continue to have specialties, product person, engineer, data science designer.
12:19While they do more of other functions, there's still a lot of value. And tell me if I'm hearing you correct, in having this specific discipline and skill and background. I still see a craft excellence that's really important in the disciplines that I don't think is going away anytime soon, even if there's fluidity or blurring of the work across the functional lines.
12:38It goes back to what I mentioned earlier of you still have humans who have to make sure that what we're doing makes sense. We're solving the right problems in a way that is best for Netflix members or business stakeholders. And that if I talk to an engineer, a data scientist, a designer, yes, they speak more languages now than they used to because they have the benefit of these AI tools, but there's still something that is not replaceable when I think about the craft and how they think about what good looks like.
13:11And that feels true across all levels, and, you know, I still find great engineering to be scarce, great data science to be scarce, great creativity to be scarce. So I yes.
13:23Some things are easier, but that hasn't dissolved in my mind.
13:27Are there functions that you are finding you are hiring more of, like the pie chart pie expanding, say, for engineering or PM or design or something, and then functions you're need less of with AI tooling and LLMs rising?
13:43I'm not sure that it matches exactly to functions, but I can tell you what we're having we're seeing more of, we need more of. We need more systems thinkers in a world with AI.
13:57That looks a little bit different across functions, but I could play out a couple examples. So in our core infrastructure team at Netflix and central engineering, a lot of what made Netflix successful over time was that local teams with specific business problems could move fast to deliver.
14:20They very often were not feeling like they needed to be on a central paved path. They built the stack that they needed to solve the problem and have the impact. In a world of AI with agents operating across multiple systems, wanting source of truth data, the importance of having preferred paved paths that get the most of the benefits and produce some guardrails so we can make sure we're doing good work, common infrastructure, common paved paths, solving problems once with a core set of capabilities becomes more important.
14:53So we are hiring more people who can look across all the business domains and abstract that to here's the building blocks we're gonna need in a world with AI, so that's one of the lenses, but also just with a lens of what got Netflix here doesn't get Netflix there, and we're gonna have to have a stronger set of infrastructure to move quickly in this future.
15:14So that means that engineering profiles are more distributed systems, more infrastructure, more of that system thinking mindset than a a local business expertise. Though, of course, we still have people who are deep in personalization and advertising and content delivery, so it's more something additive for us to have that core infrastructure and systems thinking.
15:35If I take another example, like design, it's extremely important that our experienced design team is developing templates and, again, systems thinking for what does great user design look like at Netflix so that they can enable lots of people, including those who are not designers by training, to develop products that are coherent, that fit into the end to end member experience.
16:01I get really nervous about having different design languages or different types of user interactions and .
16:09So designers need to then be the people we're hiring, Again, for design systems thinking, how do we think about templates and expression of the brand and what a good user experience looks like, and what is Netflix and, like, the Netflix differentiated special sauce. So there's more people on our design team that have to think that way now than could I help to design a specific feature for a specific product?
16:33So there's this stepping back to look at the big picture that I think is happening in every single function, and that requires some, yeah, reorientation of skills among the existing team and also hiring people who've got that that type of expertise.
16:49And across all of it, it's a mindset shift. So we are not hiring people who are not excited to explore, try new things, understand lots is changing, and feel comfortable with that ambiguity, be comfortable that there's a blurring of how we work and how we partner.
17:09It that's true for people who are already at Netflix and people who we are adding to the team, that that curiosity innovation mindset has not it's not been more important, at least in the time that I've been working in this field.
17:22On the systems thinking piece, is the reason this is becoming more important
17:26that it is people are moving so fast that you need to invest in platforms and frameworks and and design language and basically teach people to phish so they cannot be blocked, or is there are there other reasons? I think it's probably velocity.
17:40So platforms
17:41do have a benefit of leverage. So in general, that that's an opportunity with or without AI for a platform to get most teams 80% of the way there, and then they don't have to reinvent those building blocks.
17:54We have more bets that we're making across the business, more things we're trying to build, so platform mindsets are good, and it's something that is relatively more recent for Netflix to think about that being a real critical enabler. There is also the sense of a scaffolding in a world of AI, so not just the higher velocity, but you have more people doing more types of work that are different or new, like we were talking about, and there's risk that comes with how do you think about access and identity in that situation?
18:27How do you think about security in that situation? How do you think about shipping high quality code and design and user experiences? And so I I don't think it scales well to have each person who's building something have to go figure out, could you remind me what good looks like here and what are the bumpers or guardrails I should keep in mind?
18:45I think we need to encode that in our and our ways of working, and for a data science or analytical field to encode, here's the source of truth data, here's how to interpret it, here's how to access it, here's what to do with it or not to do with it, and to be careful with certain types of data. I don't an organization that has thousands of people can no longer rely on tribal knowledge, I'm gonna find the one person who knows this.
19:10So this was a challenge that was there before AI. It's probably a more urgent challenge with AI, and I like the idea of using AI or any new tech to motivate
19:20like, we knew this is work we needed to do. No time like the present to invest in that more heavily across the team. I wonder if another reason for this becoming more valuable is because agents are now doing a lot of work and giving them the context, giving them the scaffolding, giving them the design language just speeds all that up.
19:38Yeah, and one of the visions we have at Netflix is
19:41we will have so many agents that are contributing to doing work that you need to be able to reason and rationalize throughout that. You know, the humans are the ones guiding.
19:53What's the problem we need to solve? Do I feel like what we're producing is impactful and high quality output? But the work will be done by both humans and agents, And that creates velocity and benefits, and it creates risks.
20:07And I think that's important from especially from an engineering perspective that we figure out how to manage that in a way that lets people move quickly but doesn't create undue
20:18downside or risks for the company. This connects so, uh, directly with, um, Ginny Wen was on the podcast. She was head of design for Clock Code and Cowork and had this whole design process is dead kind of thesis.
20:30And And the pitch there is just there's no time for design, the design process. And instead, as a designer, you're just kind of steering people and pointing in the direction and adjusting and also thinking big picture as when you have the time.
20:43And it feels like that's kinda what you're describing here is, like, create the platform for people to move fast, then there's no time for, like, design process of a specific new feature. I have mixed feelings about that because I
20:54we do wanna enable with infrastructure and systems thinking more people to do great work with strong design as part of it.
21:04Why not take that opportunity that the new tech provides? But for our most important priorities, design is critical to solve things in the right way.
21:16So we do still make time for important design work. We it can move faster. The designers themselves have more tools in their toolkit so they can do incredible work at a faster velocity, show more options, learn, iterate, test more quickly.
21:32But I think it would be a mistake to say design and deep design expertise and thinking gets squeezed out just because we can write code faster, we can do data analysis faster, that feels like at least for a large scale consumer product like Netflix, I feel like we would lose one of the things that makes Netflix great, which is the product technology and design makes a lot of complexity invisible and makes for a seamless customer experience.
21:59That that's a design mindset that has to be core to it. So the work itself might look different, but I don't think we lose the mindset.
22:06That's an awesome counterpoint. So what I'm hearing is kind of trending up skills, attributes you look for, systems thinking, and this kind of mindset of being comfortable and excited about change and what's coming and not being stuck in your own ways.
22:20What are you finding is trending down? What are you less looking for that you used to value more highly?
22:28The days of very narrow, deep specialization feel more limited to me.
22:35I can come up with examples where we still need it because there's an industry or technology expertise where there's only a few people in the world who really know how things work. We have examples of that on the team for encoding or how our playback systems work and things that have been incredibly innovative and novel for Netflix.
22:55I I still believe we need specialized practitioners in those spaces. But as a general rule, compared to five or ten years ago, I I would believe we have fewer specialists and more people who are generalist or adaptable in multiple directions, and that could be adaptable across functional expertise.
23:17It could be adaptable across flavors of engineering. So can I navigate both back end and front end systems?
23:25Can I hook into infrastructure with a lot of expertise? I think the the mindset now needs to be, I can learn that quickly, and that goes back to the systems thinking.
23:34So I think specialists can learn to have a broader array of tools more easily than was true in the past. So it we need fewer of them perhaps because talent's able to grow in that direction, and there's something about sticking to a narrow specialty that maybe triggers for me a concern about what about the mindset of growing in different directions and exploring?
24:00And I don't wanna be too narrow even in my own assessment of that, but I it's important that people who are specialists still have that sense of I wanna try a new way of solving these problems versus the way we have in the past. And when you say specialist, are you thinking, like, front end?
24:15I'm a front end engineer versus a back end, or are there other Yeah. Or it could be a domain set of knowledge of, uh, you know, I'm, deep a payments expert. I'm a payments expert.
24:24I'm an ads marketplace design expert. I am an an expert in this very specific tooling that studio productions use.
24:34Mhmm. So their specialist and subject matter expertise is an advantage provided that person is willing to grow and extend into, is this really still the right tool or the right way to think about the problem?
24:49So I think it's the layers of the stack from an engineering perspective that there's less specialty, and then tools that are unlikely to be static or, like, to have a lot of inertia around them.
25:01I would think, like, we would want people who are able to innovate and imagine, like, what's the future version of this? And so we want more talent like that. Awesome.
25:10So coming back to the systems thinking piece, people hearing this are like, okay, I gotta work on my systems thinking skill set. How do people develop the skill?
25:18Other is it just do it for a long time, work at a lot of complex projects? Like, I think of this book that, uh, everyone always references with the slinky on the front, thinking in systems. Uh, Yeah.
25:29How do people learn this?
25:31Small trick. Each problem you're trying to solve step out one click to the, like, what am I assuming is true about the broader space in solving this problem?
25:46So I was given a task to build some new feature for the Netflix member experience. Let me take one beat and think about what is the bigger consumer problem we're trying to solve here?
25:59What's the type of content that this feature is gonna be able to support? Do I think that the way I was planning to build this is gonna make sense in a way that scales across multiple content types, or it could be something that's a capability that then is contributed to a platform set of offerings from multiple areas.
26:20Is the consumer problem that I'm solving with this feature going to be one of the most important consumer problems that Netflix is going to need to solve as we have an expanding world of entertainment, and we wanna make it more personalized and immersive? Those are all questions that, like, you don't have to .
26:38You don't have to solve for Netflix's overall strategy and who are we relative to competition, but you take the thing you're responsible for and you just do one zoom out of the problem you're solving and question that.
26:51I wouldn't spend too long in the questioning state because then you're stuck, then you're not making forward progress, but I think that helps people to think in terms of systems and question, are we solving the right problem in the right way that matters for the end consumer?
27:06Another way as you describe it, another way I'm thinking about it is, like, think, uh, if you were your manager, how would they what's their broader perspective across not just your one team and problem and KPI, but the larger picture? I've got advice over years that is similar to that, which is,
27:22are there ways that I can do my job that helps my manager do their job?
27:30And so if I thought about all the things I'm directly responsible for, but I thought about it from the perspective of my manager, so not just product and tech, but finance and content and other parts of the business, I would naturally zoom out and think about how all these component pieces need to come together and how the whole could be greater than the sum of the parts.
27:48I think that's useful thinking. And for engineers to think about how do I leave a better version of these systems, how do I think about the thing that's going to be high quality and scale for others, there's both a how do I help my manager and there's how do I help my colleagues, which is a core part of some of our engineering principles of do the thing that is right for the broader organization instead of just what's right for you locally.
28:10That's systems thinking as well. So it's not just seniority, but it's breadth of the way I solve this problem and I build this, is it gonna be useful to my colleagues, and am I gonna leave a stronger version of things for the future set of innovations that we wanna make?
28:24That is awesome tactical advice. Uh, making your manager's life easier is always a good a good tactic career wise.
28:31Several reasons. Yeah. Following the thread a little bit, I know you all added career ladders and levels recently.
28:38Was like a new thing. You guys used to not have these things. So kind of all on that thread, what have you added to the career ladders within this AI world, if anything, that you find you want people to lean into more, you're looking to more, or or not?
28:54Like, did you not change your career ladders and performance, you know, criteria?
28:58So the way we've approached this so far is instead of trying to articulate at each level exactly how AI changes those expectations to instead put an overlay across all of the talent at Netflix, people on the team and those who are hiring to talk about an aspiration for AI fluency.
29:20And what that looks like is gonna vary by function, it's gonna vary based on where you are in your career, that could be what level you're in or what type of role or persona, work you're doing, but the aspiration for AI fluency, which is a tough thing to define. So does it mean that I have an experimentation mindset?
29:39Does it mean that I know where AI is useful and not useful? Does it mean that I have actually built things using AI? I feel like the the way that has shown up in career ladders and how we talk about it evolves almost by the quarter, if not month or day, because the tech itself is advancing so much.
29:57So the most useful thing is not to make it level specific or role specific, but to encourage everyone towards the expectation on AI fluency, which doesn't mean use it as a tech for the sake of tech. It's tech where it's useful, to have good judgment about that, and to have the mindset to be open minded to explore and try new things.
30:15That's the nonnegotiable for all roles, and that's true at the senior most levels of of Netflix where we talk about we too need to have deep fluency in AI even if we're not writing code as part of our day jobs.
30:27So that's that's changed, and then that's showing up in our hiring practices as well, getting comfortable within interviews, exploring how are people thinking about AI or technology, what are they using in their day to day or their current job, how comfortable are they with change and exploration, and even for things like coding interviews, allowing candidates, of course, to use AI tools because that's gonna be part of what the work requires now.
30:52So those have been shifts that we've made, but I I doubt it's a shift that's done versus we're right in the middle of it. I'm I'm just gonna keep following this thread. Obviously,
31:01AI is transformative for coding. It's a big unlock for prototyping.
31:08Are there other use cases of AI at Netflix that have been really impactful that people may not think about or not realize?
31:16So there's two that come to mind. So the first is data analysis, distillation of information modeling, which is, you know, get using the tools to get our arms around all the insights we have, similar to what I mentioned before.
31:29What experiments have we run? What are the metrics that I should be looking at for a certain problem? What's the consumer research that we've done?
31:36And that is much higher velocity and much higher quality contingent on you check that the results are valid, you work with your local data scientist on am I using the source of truth data on this?
31:51But that's been a great one, and that's one personally that I would say I most use some of these tools for. So that goes beyond prototyping and coding to general analytical thinking and translating data to action and insight.
32:05The other one is on the content production creation part of the business, which has lots of applications.
32:13This was true before GenAI. So ML and AI were deeply used in a lot of the production tools. We've used them to think about how to create promotional assets at scale, how to localize in subtitles and dubs.
32:27So Gen AI is a big step function in where the impact can be in creative ideation. We call those things like pre visualization or basically bringing a creator's vision to life before you even get into the you bring people to a set and start to actually go through the production itself.
32:44There's lots of use cases in post production. So we recently acquired a company, Interpositive, that was started by Ben Affleck that built a set of models and capabilities that allow you, after you've shot something, to relight, reframe, reshoot, change dialogue in ways that are very impactful to get higher quality content are still led by the filmmaker or creator saying, you know what?
33:08I would like to try something else to bring this vision to life, but that impact is extremely promising, and we're seeing lots of productions leverage different tools, some of them built in house, some of them that we enable through other vendors for those content creation use cases.
33:23And then as we think about how content comes to the product, I mentioned localization, subtitles, and dubs, but also how we create high quality trailers, images, artwork at scale that then we can use to help make sure that titles find their audiences around the world, those all are huge levers when we think about the AI impact.
33:44So that that, again, goes well beyond prototyping or coding to some of the creative use cases. And you can imagine that just like they work for studio productions for film and TV, they work for advertising, they work for marketing, off service campaigns, and so those are all areas that we're exploring.
34:01This episode is brought to you by Mercury, radically different banking loved by over 300,000 entrepreneurs and now with Command.
34:09I've been a customer of Mercury's for over six years. I have never once thought about leaving. Mercury is basically what happens when banking is built by product people, not by bankers.
34:19They make it so easy,
34:21dare I say fun, to send invoices, move money around, set up virtual cards for folks on my team. Does your bank have an API, a terminal native CLI, or an AI ready MCP server?
34:33I don't think so. And just recently, they launched Command, a conversational interface built directly into Mercury which acts as your financial operator.
34:42I've been using Command to transfer money around to figure out what categories I've been spending the most money in, analyze my cash flows. And just today, I used it to find out how much I've made from specific sponsor over the past year. I just ask, how much have I made from x over the past year?
34:58Ten seconds later, have an answer. It is so freaking cool. Visit mercury.com
35:03to learn more and apply online in minutes. Mercury is a fintech company, not an FDIC insured bank, banking services provided through Choice Financial Group and column NA, members FDIC.
35:13You mentioned how Netflix has been very early to AI and ML for a long time. Younger people may not remember this, but y'all had this contest to optimize Netflix prize.
35:26Yeah. Yeah. The Netflix prize.
35:27Like like, just show an example of how early you were to AI and ML. People there was I think it was a million dollar prize to optimize the Netflix ranking algorithm a little bit. Like, whoever can optimize it the most, And I think the winner optimized it by a few percentage points, something like that, and it was like a huge deal.
35:43All these super smart people got around around the world. Yeah. And it happened a few times.
35:48Right? I mean, you said it on my behalf. Um, often when there's questions about how is Netflix thinking about AI,
35:56it's great to remind people of exactly that point, that this is not new to us, that especially for personalization, it's been central to delivering a great experience to members. It's impossible to take the breadth of content that we have.
36:10There's ever more content. That's one of the challenges we face, and make discovery easier and easier and easier, which is one of the challenges that Netflix has, and using AI and ML has been a way to do that.
36:23You wanna personalize right title for the right person at the right moment. That problem gets harder. The the more exciting our catalog gets, the greater breadth of content we have, not just film and TV, but games and live and podcasts, personalization becomes even more important and what that experience is.
36:40So we can take a lot of that history and say, okay. Well, now how do we solve this problem? Because the tech is even more powerful, but it gives us a running head start in being clear about the problem to solve, how important it is that Netflix solve that for our members.
36:54And then the same is true, as I was mentioning on the creative side of the house, AI and ML have been in things like visual effects or in localizing language for a long time. Now we say, what's the next era of that when the tech is more powerful?
37:08And in in both cases, it ends up taking a strength that Netflix has, which is marrying entertainment and technology and making sure we stay ahead of the game to deliver things that are even better.
37:20So I I love that it it's part of our history. It still continues to be a strength, and it's gonna have to be a strength given the size of the challenges we're facing around the breadth of entertainment while keeping a great experience.
37:32Yeah. And I I love that back then it was called machine learning, and AI was like, no. No.
37:37This is not AI. AI is never never never that's never gonna happen. It's just machine learning.
37:41Well, then all of a sudden, you call everything AI, and some of it's machine learning. That's right. So I, like, I try try to you know, it depends, like, the thing that is of the moment to describe.
37:52So I think we bucket all of it as AI now. Yeah. And there's a lot of AI use cases that are not generative use cases, so we could go down a deep, dark hole of all the specific things.
38:02But in general, like, I don't think it would surprise anyone that Netflix is using a broad array and it with so much excitement about what's possible. The fun thing at Netflix for the people who work here is that if you're really passionate about the applications of tech for creative outlets, for consumer products, for infrastructure, we have all of those problems, and AI is at the center of them.
38:26And it's good not to forget that that that's true even if Netflix isn't branded as an AI company. AI is a tool that we're very comfortable using to get these great entertainment and technology outcomes.
38:37The other really interesting thing, just to kind of keep, uh, complementing Netflix here, if you look at the early culture deck of Netflix and also our conversation last time, things that emerge from that are things like high agency. This was like something core to Netflix in the beginning.
38:53High agency, autonomy, high talent density, very bottom buzz up thinking, super quick experiments and launching, paying top of market.
39:03Uh, this is all stuff that every AI like, is what I hear constantly now from how the top AI labs operate. So we're all ending here, and this is where Netflix has been forever.
39:13Yeah. It's a little prescient in understanding
39:16what makes talent incredible. I've thought about all those aspects of the culture at Netflix as this is gonna sound a little bit nerdy, but excellence as an operating system.
39:28So the goal of all those cultural elements wasn't the end bone themselves. It wasn't, let's just make sure people have as much responsibility as possible, or let's you know, we don't like process, so let's make sure that we don't have any of that.
39:42It was instead a very strongly held opinion that that you get to excellence by giving people a lot of agency and accountability, by pushing decisions as deep in the organization as possible, hiring great people who can be trusted to have good judgment and make good decisions.
40:00And that ends out driving incredible outcomes plus a lot more motivation and sense responsibility. It means every person on the team can feel like I'm being given a lot of keys and a lot of accountability for what happens here.
40:16And I myself feel like when you know you're carrying that level of trust and accountability, you wanna do your best work. And so there's something that feels very intuitive about Netflix's culture has always been aiming at excellence.
40:31And when you have great talent and you give them the ability to do their best work without micromanaging it or drowning it in process, you actually get much better outcomes.
40:41And so I do think that the newer era companies are picking up on something that is feeling very familiar to us, and it it's not something that comes easily. So having culture is not a static thing.
40:52Culture needs to grow and evolve as the company gets bigger, the types of problems you're solving change, but the notion that, like, we're going for excellence and trusting that exceptional talent needs to be able to do their best work, that's unchanged and something that I think continues to be a special sauce for us.
41:10I love this concept. Excellence as an operating system. It's very systems thinking, you might say, for how to set up a company.
41:18Exactly, Lenny. So for people that like, everyone listening to this will want, uh, excellence as an operating system. Like, who would not want this?
41:26Uh, it'd be helpful for people to hear what are kind of the ingredients to make this happen. One is obviously high talent density, just hiring only the best. Two is accountability.
41:36Kind of there's, like, the input and the output, essentially. Uh, input amazing people, top the top people, keep them make them accountable, give them autonomy.
41:43What would you say kind of, like, the pillars of creating this excellence as an operating system if people if founders are listening to this, like, I want them to do that? Well, the talent density is the nonnegotiable.
41:53You have to start with that. If you don't have that, you can't get to a place where you have confidence in decision making at all levels of the organization, allowing people to take risks and innovate quickly.
42:06That's a big part of excellence in the Netflix culture, which is being very comfortable with risk taking. We don't try to avoid failures.
42:14We try to recover quickly when we have them. I think there's been great examples of that. Our foray into live was a wonderful example of being comfortable taking a ton of risk, knowing it would be imperfect, knowing we would learn fast, and we would be better for it.
42:29I've never been prouder of the team seeing how we worked through that. So you have to be talent density, comfortable that people are gonna take the context that you give them, strong judgment and risk taking, and fight for the things that are the best outcomes for the business.
42:48You have to be very clear that what you're doing is driving outcomes for consumers and Netflix. So it's Netflix matters. Netflix members matter.
42:58It's not about my own personal success or what I prefer, so there's a selflessness that is part of this excellence operating system. And then the other thing I would say is some of the things that are they're really unnatural for humans to do.
43:13So I could give a couple examples of things to get comfortable with, which is there are certainly days where I see decisions happening, and I think, I would make a different decision.
43:26Like, is that really gonna be the best thing? But my job, especially in the Netflix culture, is not to step in in every one of those cases and overrule or veto or question someone, especially if it's it's not material, it's not gonna burn the place down, let people make that decision and learn from it and ask for those reflections afterwards of, like, how did it go?
43:52Maybe I was wrong, maybe the decision was a great one, but that it's related to the risk taking and the, like, help people learn how to feel comfortable making their own decisions, especially when they're not all gonna be the right decisions and they're gonna learn something tough from it.
44:08I felt that myself from my boss and my peers saying, this is your decision. You know, I can provide input, I can help you brainstorm, it's yours in the end, and it it just doesn't come naturally.
44:19When the stakes are high, when I feel responsible for what the org's doing to let people lean into risk can be uncomfortable. And I think that also means in cases where things are not going well, as another example, to not assume that process is gonna fix it. So if oh, something I've learned over the past few years that when planning is difficult, I've never heard someone say, like, oh, we figured out the perfect way to plan or the perfect way to go through feedback and leveling and compensation.
44:51But every time we saw that and we added more process, we spent more time without getting better outcomes. And so it's another unnatural thing that I think everyone's inclination when things are hard and complicated is you think you're simplifying the problem by putting a lot of constraints around it, but it actually goes against the, like, is there a more creative way to plan or to make people decisions or to make prioritization decisions that actually get us to better outcomes.
45:22And so it's a resistance to do the thing that a lot of bigger companies would do and to feel comfortable in that discomfort very often.
45:30So that's something I feel in my role, and I I would believe a lot of people at Netflix feel it because you try not to do the thing that is standard.
45:39It's easy to say that and hear that, but I so know what you mean, where somebody screws up and you're like, okay, what was the thing that went wrong? Let's put a process in place to avoid this from happening. And what you're saying is, like, you need to resist that, uh, because that slows things down, and the best people don't wanna be working in a place with all these checklist and process and gates and things like that.
45:58I think the best people wanna know there's gonna be a blameless retro,
46:02and they're gonna feel so individually responsible that they're gonna say, how do I make sure this doesn't happen again?
46:10Not with process, but with, like, how could I share these learnings? How could I do work differently to make sure that I get to a better outcome next time? When you are trusting people to take those reflections and learn and grow, I think you get much better outcomes over time, and you get a much stronger team, which I think is part of our role as leaders of, like, you're you're trying to grow a team that is resilient and durable and knows how to have great impact.
46:38You're not trying to control everything.
46:41Which is a key to building a team with high talent density. There's kinda there's two sides of this that I wanna chat about briefly. One is the hiring, and the other is, uh, keeping the people.
46:51So, uh, you're famous for the keepers test. We talked about this last time. Another unnatural thing for people.
46:56Uh, people that wanna understand what this is, they can listen to the first conversation. But has that how has that evolved over the last couple of years? That's still a core part of the culture, this idea of the keeper's test?
47:05It's often cited in a way where you think of keeper's test as
47:10that moment where you decide to let someone go, that they're not the right fit for the role and the conversation about that. But it's equally commonly used to have a conversation about how extraordinary someone is, how well they're doing in a role.
47:26Because it the entry point is for me to say to one of my direct reports or for them to say to me, how am I doing on your keeper test? And the lion's share of the time, my response is, I would fight so hard to keep you.
47:41Let me go through a set of things that I think you're doing such a great job at, what your strengths are, where you're having a lot of impact. Here's how you could be even better. So it's it's an entry into a conversation that is very positive and uplifting for people, but the framing is, do I pass the keeper test?
47:56And then, of course, there's the harder situations where I'm evaluating, does someone pass the keeper test or they're asking me? And it's is the toughest thing to say.
48:06To be honest, you're not passing that right now. I think you could get there in some cases, and that comes with feedback and what are those milestones. Or in some cases, you're saying, we've really tried, and I don't see the path to success.
48:19So it it's just it's an anchor and an entry point for a conversation that can go lots of different directions. And the thing I like about it is it's good hygiene on feedback and checking in on how things are going and forcing a tough conversation sometimes instead of shying away from it.
48:37Or to keep great talent, you do need to say you're doing great. Like, that that's an important part of making p o people feel recognized and valued. So I don't want it to come across that we just have this
48:48very negative view of it. I think there's this positive side of the coin as well. Awesome.
48:53I guess just to explain to people what this is so they don't have to go listen to a whole other podcast, I'll try to briefly explain it. The idea here, a part of the Netflix culture is that, uh, when you, uh, have people reporting to you, uh, you should always be thinking if I were to would I hire this person today knowing what I know about them?
49:11And if not, then I should probably let them go.
49:13And the idea there is to keep the high bar, to not ever just, like, settle. Okay. This person, they're here.
49:18I guess we'll keep them around. Is that is that roughly the way to understand it? Yeah.
49:21Oh, and the way it it can it's sort of a corollary to that, if that person came to me today to say they were leaving, would I fight to keep them or not? Or would I say if my sense is relief of, Oh, yeah, it probably would be better to have someone else in this role, I should have taken action in having that conversation sooner.
49:39I love as you said, it's such an so many uncomfortable things you have to do to maintain
49:45Yeah. Well, it's the well, the keeper test is one, maintaining talent density, context, not control among leaders.
49:53We talk about being highly aligned but loosely coupled, which is where light process, you know, the minimum to make sure we're clear on the priorities and we can execute them is what we're solving for. All of these things are not things that human beings or organizations at scale
50:09tend to do. So it's constant diligence to try to maintain the thing that's made Netflix a special place. Because in the end, it's the work and the culture that attracts people and retains people, and we need that to be a successful business.
50:22So that's exactly where I was gonna go. Uh, so to make this work, you need to attract the best people. It's always been very hard to attract the best people.
50:30Feels insanely hard these days with the amount of dollars flying around, the fancy AI labs, so much competition. There's like everyone's just you know, it's it's crazy. What have you found to be, uh, effective in convincing the top people to still come to Netflix and and join versus all the other fancy places they can go?
50:49Yeah. We've always had a lot of competition for talent. It might feel more pronounced right now, but we we have great talent on the team.
50:57Maybe that goes without saying, but I feel like I should say it out loud because I believe it. We have incredible talent in Netflix, recent hires, long tenured people.
51:06I'm always impressed by the work that the team is doing, so I don't feel like we've suffered or, like, other companies are vacuuming up all the good people because so many of them, I do think, sit at Netflix.
51:20It does feel like we have to be more more explicit about the types of people and talent that tend to thrive at Netflix versus other companies like some of the Frontier Labs.
51:35So people at Netflix have to be passionate about the application of technology and the application or building products to solve a certain set of problems.
51:45You have to love entertainment. You have to love consumer products at scale. You have to love the global nature of that.
51:51There are a lot of incredibly talented people who love that sweet spot. I am one of them between tech and product and entertainment, and how do you make those things come together in a way that's remarkable?
52:04And you use AI to do it. You use other technologies and products to do it, but that has to be something that drives you to be really excited about a lot of the roles at Netflix.
52:15If instead you're inspired by some of the foundational work that the frontier model companies are doing, which is exciting in its own way, it's a different persona, it's a different, like, here's the problem space that I want to work in, but I don't think there's a shortage of people who get really excited about the applications of the technology and see the connection to that, to things that they love and use every day, like Netflix.
52:39And so that, you know, that gets me up in the morning, and I think it gets a lot of the team members up, and we have conversation about, like, that's something special that only talent at Netflix can do or fill in the blank for another industry that's deep in the application of it. I think that's inspiring.
52:54I wanna kinda touch on a couple things that I've been thinking about in this world of AI that we're approaching. One is, uh, junior people.
53:03It feels like everyone's just like this is a good example. You're hiring a lot of awesome senior people that have proven they're awesome and, you know, high talent density, high bars. Also, just AI makes it so easy to do stuff that people may not be learning how to do anything.
53:19They're, like, junior engineers, I'm thinking, or junior PMs, junior designers. Like, there's just like, how do new people become these awesome senior people?
53:28Is there anything you've you think about? Are you hiring junior people?
53:32How do you think about this if this hap what happens with junior people not necessarily learning or having a path to learn to become the senior person? We are still hiring junior people, and they're really important to our talent strategy.
53:44So we still have an intern program. We still have a new grad program, which is was new for us as of a few years ago. So prior to a few years ago, we were only hiring more experienced talent across all the functions.
53:56Now we do hire people straight from undergrad and graduate programs, and we'll continue to do that. So even in a world of AI where some things are easier, we were talking earlier about mindset, AI fluency.
54:12From my experience, younger folks are more open minded. They tend to be more native in some of these new ways of working.
54:23For a company net like Netflix, they're also very fluent in how entertainment is changing, how consumer behaviors are changing, how product and tech is influencing that in the products that they're using.
54:35That's really important to have on our team. So there's the part of the persona which is who are you as a new grad who's an engineer, but there's also who are you as someone who's in their early twenties and has a perspective on the world that is highly valuable and a comfort with the way the world is changing.
54:52So that's why I say it's a critical part of our talent strategy. To the, okay, so you step into the role and you have AI tools that didn't exist five or ten years ago, I would say mastery of the craft is still very important.
55:05So going back to as the team member, I am responsible for the quality of code that I am submitting for production. I'm responsible for the quality of products that I'm building, how they are designed, what that user consumer experience is. None of that is going away.
55:21So if I think about more junior or earlier career talent on the teams, we need to be investing just as much in the mentorship of this is what good looks like. This is how you use these tools, but you still take accountability for what the outcomes are, what the quality of the output is.
55:36And I think I mentioned this earlier, I find that mastery and that craft excellence scarce still, so we want to make sure we're teaching that. I think it's a valid concern of, like, how do I get that if I'm not as hands on as I would have had to be? But we still carry responsibility for reviewing code, testing code, being able to diagnose problems, knowing what a good product looks like.
55:57Like, I think that's a very scarce skill to say, this is excellence in in a product that solves a problem that matters and in how it's designed. So I don't think that craft mastery, the importance of it is going away.
56:10Probably the way we train and grow talent has to change because they're gonna use different tools, and I can guarantee you that earlier career talent is going to be teaching older folks like me many new things too.
56:24So I think it goes in both directions. Where do you think engineering goes in the, I don't know, five, ten years? Do you think
56:31people need to still understand code, or do you think there's this abstraction layer that sits on top where you don't even have to learn C plus plus Java, Python, whatever?
56:42I think there's a difference between
56:44being able to write lines of code in a particular language like Python or C plus plus and understanding how code, computer systems, products work.
56:57And I don't think the latter is going away because if we trusted agents to know all the languages and write all the code, we're not gonna know why is something is it a good product?
57:10Is it a bad product? Is it working as we expected when it doesn't, like I mentioned earlier, we take a lot of risk. We fail fast.
57:16We recover fast. That requires an understanding of how are these systems working.
57:21I might use an agent to help me understand those things, help me detect an anomaly or something that's broken faster and triage it, but I still need to have a fluency of, like, what is this thing that we're building and how does it work so I know if it's good and I know how to fix it.
57:37I don't know, Ada. I hope that doesn't go away because it's, you know, that that's like a do we make the world a better place through the stuff that we're building, I think requires some understanding of what we've built. What I'm hearing, which makes sense, is you may not have to write the code, you have to understand it and what's happening,
57:51but it's so much harder to just, as a person not writing it, to actually,
57:56you know, have that instilled in you? I think that's one of the the things that the learning curve is very steep on right now. So looking at some of the code that some of these models or agents are writing, they're very hard to follow.
58:10It's like, I know I'm getting better performance from this, but I have no idea why, and if this thing breaks, I'm gonna have no idea how to fix it. That that makes me uncomfortable. You know, maybe that's because I'm still on that learning curve of, like, how do we operate in that world?
58:23Like, what's the set of tests or rationalization and understanding that we need to have to get comfortable with it? But at first glance, it looks very unfamiliar and very unsettling.
58:34So I think engineering over time will evolve to be comfortable with that and have fluency in it and know how to guide new tech and agents and new capabilities to make sure that we feel really good about what the output is.
58:48I wonder what the metaphor is for this where this like, it's I continue to be astounded by how much engineering has transformed in, like, two years. It's like a completely different drop down.
58:58You're just you used to sit there in an ID and write code, and now you're just talking to agents and reviewing code and shipping a 100 PRs a day. It feels like it's a it's an acceleration
59:08of how much engineering has changed, but if you looked over the last ten years or twenty years, you would say the same thing. Mhmm.
59:17So it there's just something that's moving faster, and it's hard to wrap our heads around how quickly it's moved in the past couple of years, but it's not it's not totally unfamiliar that engineering or data science or product would have these big shifts just like how filmmaking works.
59:36If you look over the last a hundred years, it's unbelievably different because of technology and new tools that we brought to it.
59:44Just feels like the cycle is speeding up. Okay. I wanna talk about entertainment for for a brief moment.
59:49Just I'm curious just, like, how entertainment will change over time, say, next five, I don't know, five, ten years. Just, you know, today, we open up Netflix, check out some shows, watch some videos.
1:00:00It hasn't changed in a while, just that idea of like, oh, I'm gonna watch The Pit and just watch it all. I'm gonna watch a movie. I got TikTok.
1:00:07I got Instagram, feeds of stuff. Like, how much different do you think this will be in, I don't know, five years the way we entertain ourselves? Already changing at Netflix
1:00:16because entertainment is not gonna be one thing in the future, and it's already not one thing now. So part of the reason that we are going beyond film and TV in our offering is because there's there's an expectation that consumers have of much greater variety across formats, devices, moments of the day that Netflix needs to be able to serve well in order to meet consumer expectations and hopefully exceed them over time.
1:00:43So when we think about the addition of mobile and TV or cloud games, live content, podcasts, working with a broader set of creators who are now on the Netflix service.
1:00:56All of those things create a greater breadth of what entertainment is, and Netflix is able to define and expand that. And it puts a higher bar expectation on how do we make sense of that for a Netflix member.
1:01:11So how do we show you this very seamless journey from I listen to the Bill Simmons podcast to I watch quarterback because I love that as one of the Netflix offerings in the more, you could say, traditional film or TV space to I play the most recent FIFA cloud game, and I wanna be able to do that in both TV and on my mobile phone because now I'm on the move, and I wanna be able to discover and engage with the content at different moments of the day.
1:01:40That's already a journey that we're building into Netflix, which I think will become stronger and stronger over time. So the future of entertainment isn't gonna be one thing, and it's gonna have to be more personalized, more immersive, more interactive with this sense of this is a world that I can explore in lots of different directions depending on what I'm looking for in the moment.
1:02:00And the the challenge Netflix has is we've got to make discovery and engagement much easier than it feels today. We have tons of content.
1:02:07It can feel very fragmented, especially when you consider all the services or offerings out there, And I I think Netflix is very well positioned to understand how to solve that problem across entertainment, product, and tech.
1:02:19The other, um, element of this is AI, obviously.
1:02:23As an outside observer, it's, like, so interesting to see how in tech, it's like, AI, I love it. It's the future. It's the best.
1:02:28In Hollywood, it's like, no.
1:02:31Shut it down. There's a There's a mix. There's a very wide array.
1:02:35So we Netflix's role in this is to enable creators with whatever tools they wanna use to bring their vision to life.
1:02:44There are gonna be some creators or filmmakers who are on the end of the spectrum that says, absolutely not. No AI. That is not how I do production.
1:02:53It's not it's not consistent with my vision. That's fine. We work with those creators.
1:02:58There's other creators, a growing number of them, I would say, who are very interested in exploring, wait, can these Gen AI tools make something possible that wasn't possible before?
1:03:08Can I tell a story in a new way? Can I make that story higher quality and more resonant for audiences? Can I do things that are extra creative in how I think about bringing a story to life?
1:03:20And we support them as well, and we support all the folks who are in the in between. Mean, that's a really important position for us to be in, again, because entertainment is not gonna be one thing. There's not gonna be one format.
1:03:31I think there's gonna be types of film and TV that feel traditional, and then there's gonna be entirely new formats that unbelievable creators help to bring to life, and Netflix wants to participate in that, which means we need to have a flexibility in the tools that we provide and the types of partnerships we have and to really have a creator enablement view rather than a prescriptive, like, we only do this one way.
1:03:54I think people are gonna be surprised by just how good AI content is. Like, uh, Spencer Pratt's videos are just like, everyone's like, wow. This is entertaining.
1:04:02Obviously, AI, but it's so interesting.
1:04:04Do you think do think we'll get to a place where it's just like the whole TV shows are AI and people love it? I have a hard time picturing
1:04:10entertainment that doesn't have humans at the heart of it. So that that's humans in the creation of the storytelling, which I think is a scarce and valuable skill.
1:04:22You know, storytelling is one and the same with humanity and, like, knowing what connects with people.
1:04:29So I think humans will be part of the will always be a core part or a critical part of the story. And I think watching watching characters on screen who don't have that humanity feels less compelling to me.
1:04:46And what the power of storytelling really is, to, like, see another human and to watch how they perform a role or, like, bring an emotion to life. It's such a human element.
1:04:57Will AI help to bring that to life? Will it play a material part in some of those productions or how we get them to look and feel a certain way? Yeah.
1:05:07Definitely. But I don't see the version of it that doesn't have the human as the backbone.
1:05:12There's a quote that I think is misattributed to Salman Rushdie, which is, uh, when a child is born, they first ask for food and water and projection, and then they ask for, tell me a story.
1:05:26It's a thing going back since the beginning of time that storytelling has been a key part of community and social networks and human feeling and connection.
1:05:40So I love the idea that technology can amplify that and can bring that to life in very new, novel, exciting ways. But if to say storytelling wouldn't have that humanity at the center feels like something would be missing.
1:05:56We're gonna see some wild shit over the years coming on. I'm there's no question about that. And a lot of it could be very entertaining, you know, at the I I don't debate that either, but I think there's gonna be a broad range, and I think Netflix needs to be at the center of shaping that and bringing that to life, which is our plan.
1:06:15Amazing. Well, we covered a lot of ground, Elizabeth. Uh, before we get to a very exciting lightning round, is there anything else that you wanted to share, leave listeners with, maybe double down on from things we've talked about?
1:06:27It probably came across throughout, but I I would underscore that this is a really exciting time to be building products and entertainment.
1:06:34Everything we talked about of, like, what's changing in the tech and consumers and, like, what is entertainment? We're at this unbelievable high velocity innovation period, so it's what keeps me at Netflix.
1:06:46I think it's a fun place to be. I would be missing something if I didn't reinforce that I think that's true. I also think that as an industry, we spend a lot of time sometimes talking about the the pure tech or the the capability, and we sort of lose the forest for the trees.
1:07:04We're trying to build great consumer products that people love. We're trying to make great entertainment that people love, and it's their favorite thing that I don't want that to be lost in.
1:07:14Of course, there's amazing tech and product stuff that sits underneath, but in the end, the thing that's most inspirational is what do we bring to people around the world. And along those lines, there's been such a, uh,
1:07:26the opposite of glut and drought of a consumer, new consumer products, consumer experiences. Like, there's very few success. Like, almost no consumer startup works, and AI feels like an opportunity for something else to work, and I feel like Netflix is one of the rare companies and brands that continues to deliver an awesome consumer product and business.
1:07:45There's just not that many of them. Yeah. Well, we're gonna keep that up.
1:07:49Well, with that, we've reached our very exciting lightning round. I've got five questions for you. Are you ready?
1:07:54Okay. I'm ready. Alright.
1:07:56What are two or three books that you find yourself recommending most to other people?
1:08:01I have to come up with different books than I said last time. I don't remember that, but that sounds great. I still like a good throwback, so two that are coming to my mind, Into Thin Air, John Krakauer, and Liars Poker, Michael Lewis.
1:08:17So I I worked on Wall Street, and I like reminding people what it was like in the way back time. Favorite recent movie or TV show you really enjoyed, which is maybe too hard for someone working at Netflix, but I'm curious what comes up. The list is very long.
1:08:31The most recent I watched, Remarkably Bright Creatures, after a recommendation from my mom. It's a tearjerker.
1:08:38Talk about the human part of storytelling.
1:08:41Favorite product you recently discovered that you really love? Critical for my health and well-being? Eight Sleep.
1:08:48Do you have a favorite life motto that you often come back to in work or in life? I often go back to the things that my parents instilled in me in very early times. So the the risk of repeating maybe, first, something good happens every day.
1:09:06Watch for it even in the most stressful times. And second, that the last 5% of effort usually makes all the difference.
1:09:17These are awesome. I they hit they hit me. Final question.
1:09:21Uh, I don't know what anything about this, but you mentioned you're doing some kind of cycling event. Oh, yeah. Tell us tell us what's going on.
1:09:28What are you doing here?
1:09:30So my husband and I are doing a trip where we ride alongside the Tour de France for the last week of the race. So the tour is three weeks.
1:09:40The last week has a lot of mountain stages, so we get to ride part of the route each morning and then watch the race in the afternoon, not for the faint of heart.
1:09:51So I'm trying to train up so I can enjoy those rides. It's It's supposed to be a vacation after all.
1:09:57My god. I love this vacation. We're just gonna race.
1:10:00So is it like a race? I love professional sports. It's fun to be able to participate in it.
1:10:06So is this like racing, or you just kind of try to go nonchalantly through the course? You go nonchalantly.
1:10:12That's the bare hair. Think I mentioned. Yeah.
1:10:14It's on the hills. It's physically and mentally challenging. And I you know, it's not a race, but I don't wanna be at the back of the pack.
1:10:22So I gotta be comfortable enough to hold my own. Wow. I love how different this is from your job.
1:10:28And it feels like someone a it's a good balance, and it gets me outdoors and gives me some nice perspective, so I'm looking forward to it. Elizabeth, you are awesome. Two final questions.
1:10:38Where can folks find you online if they wanna follow you, reach out for maybe anything that came up? And how can listeners be useful to you? The best place to find me and some of the work we're doing or reach out is the Netflix tech blog, actually, where we're putting a lot of things that I've been talking about up there.
1:10:53We're trying to do a better job communicating about the fun stuff we're working on, so that's a good first stop, usually. Um, and then how listeners can be useful, try all the new stuff that we're putting out there.
1:11:08Um, watch the live events, play the games, have fun with the new vertical video feed that we have on mobile called clips, send us feedback.
1:11:17So we wanna make it better, and a lot of these things are new zero to one efforts for us, so we're trying to get to
1:11:23great and excellent as quickly as possible. I love that the homework is go watch Netflix, and
1:11:29You can also watch other things. Tell us how we can be better, but I'm I'm definitely interested in how can we be better at Netflix. I love it.
1:11:36I'm gonna go I'm gonna go do that. Elizabeth, thank you so much for being here and being here again. Thank you for having me.
1:11:42Always fun. Hi, everyone.
1:11:44Thank you so much for listening. If you found this valuable, you can subscribe to the show on Apple Podcasts, Spotify, or your favorite podcast app.
1:11:52Also, please consider giving us a rating or leaving a review, as that really helps other listeners find the podcast. You can find all past episodes or learn more about the show at lenny'spodcast.com. See you in the next episode.
The Hook
The bait, then the rug-pull.
Lenny opens with a rapid-fire teaser of pulled quotes before introducing Elizabeth Stone, Netflix's Chief Product and Technology Officer, back for her second appearance after her first became one of the show's most popular episodes. Two and a half years and one generative-AI wave later, the conversation is about what actually changed inside Netflix — and what, deliberately, hasn't.
How they asked for the click.
MENTIONED ON CAMERA
