How Netflix builds a culture of excellence | Elizabeth Stone (CTO)

We really can't have any of the other aspects of

culture

, including sincere learning, the pursuit of

excellence

, and the freedom and responsibility to improve, if it doesn't start with a high density of talent and, in some ways, very much reflect Reed Hastings as the founder of Netflix, so when he founded Netflix and grew the company over time, with the belief that there could be a different approach to building a company that would make it a place that people thrived and loved to be in. and it felt different from other places, both in quality. from that density of talent, but even more importantly, you like

excellence

and results, and that's where people would get a great sense of satisfaction, so it's very ingrained in Netflix since its original days and to achieve that, there really is You have to stand firm.

My guest is Elizabeth Stone. Elizabeth is CTO of Netflix and, as far as I can tell, the first economist to be named CTO of a Fortune 500 company prior to this role Elizabeth was VP of Data and Insights prior to Netflix, she was VP of Science at Lyft , coo at Nuna, Trader at Marl Lynch and economist in a group of analysts. In our conversation we covered many topics, we talked about how she has a background in economics. she has helped Elizabeth in her career and why she hopes to see more economists rise through the ranks of technology companies.

She shares some of her secret sauce for moving up so quickly at so many companies so consistently that we delve into Netflix's unique

culture

of high density of radical talent. Cander, freedom and responsibility. We also talked about the structure Netflix has for its data and user research teams, which she believes is part of the secret to Netflix's success. We also look at what cycling and triathlons have taught Elizabeth about life and how she brings that to the table. in her work and much more, many thanks to Ali Ral for introducing me to Elizabeth. If you enjoy this podcast, don't forget to subscribe and follow it on your favorite podcasting app or on YouTube.

This helps tremendously and I really appreciate it. I bring you Elizabeth Stone after a few brief words from our sponsors. Vanta brings you this episode. When it comes to insurance, her company has top-notch security practices. Things get complicated quickly. Now you can assess risk and ensure your customers' trust. and automate compliance to sock 2 ISO 2701 Hippa and more with a single platform. vanta V's market-leading trust management platform helps you continuously monitor compliance along with reporting and risk tracking. Plus, you can save hours completing security questionnaires with Vana AI. Join thousands of global companies using V to automate evidence collection unify risk management and streamline security reviews get $1,000 off vanta when you visit v.com Lenny, that's vana.com SL Lenny, let me tell you about of a product called sber, the all-in-one communications API platform designed for web and mobile applications in a world saturated with multi-channel communication, product teams are discovering the effectiveness of in-app communication with Sen Bird.

Businesses can enhance their in-app experience with uncluttered, branded communication that includes AI-powered chatbots. calling and live streaming capabilities, all designed for business marketing and top-level support. Forward-thinking companies like hinge patreon Yahoo Accolade and more use senir to create in-app communication experiences that drive conversion and engagement retention. Communication in a communication has the highest conversion, the highest engagement and the highest satisfaction. of any communication channel and when it comes to investing in this channel, trust sber to take your inep communication experience to the next level. Get started today with Sen Bird's free plan and as a listener to Lenny's podcast you'll get an extra 2 months of unlimited usage. and access to all premium features including creating your own AI generative chatbot, visit bird.com Lenny to start your free journey that is sg.com Lenny Elizabeth thank you so much for being here and welcome to the podcast thank you thank you for having me , so when we scheduled this conversation you were VP of Data and Netflix and you've since gotten a promotion, now you're the CTO of Netflix, which sounds a lot more sophisticated to you.

A question for you, what is life like now that you are a technology director versus data vpf, how? Is it more different? I'm imagining more meetings. I would say the biggest thing that changed is probably the amount of context switching and the degree to which I feel behind or I have a lot to learn and I felt like I had a lot to learn. in the vpa data and insights role i was in before in the park because we cover a lot of different areas of the business and there are always people to learn from but the engineering organization just takes that to 100 basically so more people know . more problem spaces, aspects of technical expertise that I'm just not as familiar with, um and yes, a lot more meetings.

I imagine there will be a lot of higher stakes meetings as well. Yeah, so luckily there's not a lot of meetings on Netflix like you're actually in it now. This room is scary, but it feels like the role has more consequences, which is actually kind of exciting, similar to what you just talked about about being CTO. Your background is really unusual. You are a trained economist. You have a PhD in economics. and as far as I can tell, you were the first CTO of a Fortune 500 company who is an economist with a background in economics. First of all, it's true.

I don't know if I think it's true, but it tells me I haven't checked the list. That was one of the things I didn't do after I got my degree. It may be unusual. I've heard a lot of comments about that so I don't know if I'm the only one, but I definitely will be, it's probably unusual, yeah. So I guess the question is: do you think this is an anomaly and will remain really rare? Do you think this is something we're going to see more of in tech companies, and in general, do you think tech companies should hire more economists?

Yes To the last question, that's the easiest one, but one of the things that I observed, even with the focus on data science, where I've delved into it over a period of time, is that economics is a variant of data science. data, so it is a set of technologies. Technical skills, certainly, are a way of framing certain problems or solving challenges, so when I first moved from economics to technology it was before there was much of the kind of frenzy around data science that we've seen more recently and was It's harder to argue that economics is a version of data science and perhaps complementary to other versions of data science, and I feel that more strongly now that I've seen it up close and, by extension , I would say just as I think.

Data science can be useful for many different problems that you may not immediately think of. Oh, this is something we should add data to. I think economics is generally valuable for many different challenges and is a useful perspective to add things to especially. in a business context and especially how we want to simplify problems in a way that makes them easy to handle, so I feel like it's been a benefit for me to have had that kind of formal training and then bring that perspective or way of thinking to different roles, so I don't know if a lot of people at Netflix consider me an economist, but I find that reflected in the way I think about things, and to the extent that that's true generally, I think it's helpful in a lot of companies and I feel like even since I made the move to Tech, I've seen it become a lot more common to think about the value of having Economist on teams just to take advantage of that threat a little bit more.

Is there anything very tactical or concrete that you can I can share that was useful to you in that experience that was useful to you in his career, in addition to the dry, grim science of it all. So an example would probably be understanding incentives and thinking about unintended consequences, and I think that's true for both. In terms of internal leadership, being part of a management team that thinks about how to clarify priorities or motivate a company or define the problems we want to solve and then part of it is more externally oriented, how do we want to think about what What does Netflix do? is for consumers and how we want to think about competition and there may be a form of rational thinking, which is a version of economics, shouldn't smart, rational people behave in the following way?

And then there's the good thing, if given certain incentives, what could they do? You see, we didn't think it was optimal or we didn't expect it to happen, but it could be a consequence or repercussion here, so I think that kind of framing, I don't know if it's unique to economics in some way. because it also has elements of psychology, and forward planning has become really useful for thinking about kind of cause and effect, so it's come up in a lot of different spaces at Netflix and other roles I've been in. I was looking at your LinkedIn and looking at your career over the years and it seems like you've had a meteoric rise at four different companies and I'll go over them briefly, so at your first job you went from associate to VP in three years at the next company, Nuna, you went from data science manager to CTO in two years at Netflix, you went from VP to CTO in three years.

I think that's very strange. I'm curious what your secret sauce is for being so successful at something like this. In a lot of places and especially in the context of what advice you can share with people early in their career, this is one of those thought-provoking questions that you wouldn't normally do, so that's great, I don't really consider it as a secret sauce, but maybe I can go over some of the things that I think have been critical as you listed them, it sounds like the two to three year point is the real sweet spot, so maybe there's something about that timeline , but I think some things almost feel trite in the way I would say them, which is that I'm very dedicated to the work and the teams that I'm a part of, it's been a part of who I am for a long time and I give it my all. that I have. to the job that I'm in and I think the dedication and the joy that I get from that has mattered, it's important because I enjoy what I'm doing.

I do the best job I can, less for myself and my own ambition and more so because I consider myself part of a team and therefore I really need to perform for that team and I think that framing in my mind and that motivation has helped me. on some fronts, which is the way I build. Partnerships with people I work with and truly care about setting other people up for success and being someone people want to work with, so I learn from them, they learn from me, and together we get better results for the business. I've found that part of that is being someone that people can leverage to translate from technical to non-technical and non-technical to technical, so I think that's been a relative advantage in my role, so, Although I was often sitting in more technically oriented teams, much of my career advancement was toward roles that required that type of fluency in communication and came from being able to partner with people across companies who didn't necessarily have the same experience. , but where we really needed to connect spaces so that we could be more effective and that was something that I think really the training for came from the analysis group where it was a very quantitative set of work that we had to find a way to communicate to the judges and juries of economic cases, so that was something that was I trained in other roles and I think I have been able to take advantage.

I'm a relatively introverted only child, so I observe a lot, which means I learn from other people, and in each of these roles I've put a lot of effort into observing. What other people do they think about how I could learn something from them, whether it's what I want to be able to do myself or what I think, oh, that doesn't quite fit or feel authentic to me and my style. I do a lot of introspection, so I've been around amazing people in all of these roles and I feel like I learned a lot through osmosis and observation and then was able to leverage that to become stronger in the roles I was sitting in. so I took some notes here, so some things you mentioned are like dedication, essentially working very hard and taking your job seriously, being part of a team and setting other people up for success, translating complex tech language and problems to people not related to technology and So, be really good at observing and learning from other people around you.

Are there one or two examples you could share of some of these to make it even more concrete for people? Dedication is like working long hours, being part of a team. anyWhatever Lin does to share a story, maybe to help people put this into practice, no, and it's a good clarification because the dedication is really not about long hours of work, but more about how much I I care about excellence, I guess, so I give it. I do the best I can in those situations and that may not mean that I work really crazy hours or that I work on weekends or that I'm the one willing to sacrifice vacations.

In fact, I've tried to avoid setting that as an expectation, but rather hold it to myself. at a very high level, so an example would be, especially as I get into more senior roles, there may be an expectation that it's okay for other people to attend to me, whether it's meeting time or providing information about something o Review a document or follow through on something I said I was going to do and I really try to avoid it, meaning if someone sends me something I try to be very responsive about it. I know I said I was going to do something.

I'm following it on the schedule I said I was going to do. If I have a meeting, I try to be on time for that meeting and those are all types of dedication. the work that shows up and oh it seems like Elizabeth works really hard but the motivating factor is that other people trust me and I want to perform for them and that's when I say dedication and it's related to the second point about showing up. Well, for the team, those would all be examples of me feeling urgency in responding to people and doing high-quality work for other parties, from technical to non-technical.

I think a great example is actually very timely at Netflix, which is what we're doing. moving forward to offer live types of content, live events, live television shows, we announced this week that we will be presenting WWE starting later this year and early next year 2025, that's easier said than done. I know there are many entertainment companies. that have live content, but Netflix has really been in the streaming content business, so live content is something new for us and it's something that will require a very close partnership between our content organization and our product organization. and technology because there is a content strategy.

There's a business strategy, there's a technology strategy and a big part of my role is can I explain how we're going to address those technical issues in a way that

builds

trust with the content team and can I try to understand their content strategy? in a way that sets the technical teams up for success and we understand what we need to be able to deliver here in terms of requirements and I don't think I would be able to do my current role well if it wasn't. able to do that kind of translation for something that's going to be a big bet for the business and something that we want to invest in together and then set my partners up for success in that, so I like, I'm going to do everything I can. to make sure we deliver good results to my content partners because I feel like that's what's best for Netflix in the business.

Incredible examples in terms of Life content. I think of Love is Blind. I think it was for Premiere, whatever meeting we have. I got sucked into that show, very good job and I think there were some problems with that reunion broadcast, yeah, that was a little less than a year ago, so the surprising thing about the failure is that you learn a lot, we learned a lot. I've taken notes on that and we had a couple of successful events after that, including the Netflix Cup last October, and we have some exciting events ahead of us, so I think that's something that strengthened us, but it revealed that we are addressing an issue. difficult. yeah Twitter feeds during that love is blind Premier hilarious people are angry okay and then in terms of keeping the bar high for yourself I love it I think of a quote that there's a VC and Wall in Floodgate and she did this interview with Tim Ferris and she shared that her father always asked her: are you doing a world-class job with this?

Are you doing a world-class job with your homework? Are you doing a world-class job with your piano recital? And that's the bar that he always had for her and I feel like that's a really good way to think about work and if you can, you know life in general, yeah, my mom used to describe it to me, she probably still does, even though it was necessary repeat it more when I was younger than the last 5 years. The % is the 5% that really matters and so is the framing of the thing, the extra effort you put into something to make it world class or to make it excellent, so I like to push myself in that sense and hope that it establishes itself. .

It's also a good example for other people and it's very consistent, especially with the company cultures that I tend to thrive in, where that's the general expectation of the culture, so you don't feel like you're doing it just because then I think that you can start. feel frustrated by that. I know this is a big part of Netflix culture and I want to get into that, but before that I'm curious what that looks like with the people who report to you, how do you help them level up? this ability to have a very high bar and I'll give you an example while you're thinking, maybe an example is the way I describe it to my PMS: you want to have this aura that you have and that if you give something to Lenny he has this, He is going to continue, he is going to close the cycle, he is going to achieve it, if he can't do it, he will tell me.

I feel like this red thread is not going to go away, you are not going to drop this ball, is there something that What you have learned is a good way to help someone learn this type of skill and understand why it is so important. It is shown to the people who report to me and it is a part of the example, so if I don't do it, why would they? and I take it very seriously, that we should all be held to the same standards, and secondly, I give feedback when it's not up to standard, so I think some of the things that I've observed, especially with people in my equipment. is that expectations aren't always clear and you can't assume they are if you don't share them and when something doesn't meet expectations or doesn't really come across as Excellence, I think it's a combination of both, giving feedback on that and being direct about it and specific about what it would take to bring this to the bar that I expect or to show up the way that I hope and then the third thing, and probably the most important thing, is to help them fill that gap, so that would mean, let's take an example , it's certainly happened frequently in a lot of work, like a document, okay, it's not great, it's not going to be easy for people to follow, it's not as clear as it could be, there are things that would strengthen it.

I can give feedback on that to make sure that yes, it's going to take another round of iteration, yes, we're going to have to work another week on this and no, yes, not finish this, but push people to get there and then jump into the document and help, so I feel very strongly about it and that's what I mean by setting an example of working together on this and then helping people improve so that next time they know the expectations and they've had help getting there. there in the past, so that's probably happened a thousand times in my career where I jump in with both feet because something needs to be better and I think teams are better for it right after or I.

I hope they are. I think that's a really good framework just to reflect what you said. Set expectations that the bar will be very high and this is what I expect from you: give them very specific feedback about where the gap is and then help them fill that gap and I think a lot of people can feel this and hear this and say oh man, I don't want a manager who is a person with such high expectations and feels very stressful, but I have had these managers and I feel like that's when I learned the most and improved when I had someone who gave me very high expectations and then helped me understand that This is where you're not doing as well as I can.

I know you could do better, go back and work on this. I know it sounds annoying, but I think in practice it ends up helping you more in your career. I imagine you've seen a similar result. I think so. I mean, you would have. ask some people on my team so I can see it differently than I see it, it's a difficult skill because it's not always easy to give feedback, especially if you feel like you know someone has put a lot of effort into something and So I think a lot about how I deliver that feedback to make it seem like we're on the same team and I'm trying to help them succeed, not help them encourage failure, and that's where I think that third part of the helping framework can make people feel as if I were in a safe space.

My boss wants me to be successful. my boss is helping me here and I often do it behind the scenes, so maybe that's another variant of this, which is that I don't do it live in the big meeting in front of all the people where the presentation doesn't go very well. . I do it afterwards, where it feels like a safer space to say here's a way this could have been better, let's think about this differently. Next time, it'll give the people you know a little bit of Grace, you know, and a little bit of ability to absorb that feedback without feeling like they're on a stage, another thing that someone might feel when they hear this is like Oh my God, this will take me many hours to get to a place that Elizabeth is happy with and I know you said this doesn't necessarily mean many hours.

Do you have any tips or ideas on how to avoid burnout and burnout? working all the time but also keeping this really high bar and high expectations, it's really not time. I even found myself in a meeting today saying that if we are clarifying the objectives of something, it could be that the last 20% of the polish is in the document is a very bad use of time, so if we are going to meet to talk about the quarterly review business was the example, what were the highlights, what were the lowlights, what were the learnings from the quarter, where are there places of misalignment.

The reason we're doing the quarterly Business Review is to have a really candid conversation about how we think things are going to have a debate about things that maybe we're stuck on, it's not to have a perfectly polished document for that. conversation, so my comments. In that case, I'd rather someone spend the time thinking about what conversation we really want to have. How can I prepare it? Couldn't you spend another 20 hours to make it look like everything is perfect on this document? I think in that sense it's not just Excellence like you write the perfect document.

I should probably be careful not to use that as the only example, but we actually got to the result we wanted because we thought it through and put a lot of energy and time and iteration into making sure we got to that result and this example is an example where you gave feedback to someone because they spent too much time on polish or is this before and you think about the pyramid of this framework. of shared set expectations give specific feedback, help them fill the gap and then do it privately, is it like setting expectations in this example or is it like feedback you spend too much time on?

This is setting expectations, so one of the things in My New Role is that there are some practices that the team has had where they are trying to understand if we are still going to have those practices, what will be the same and different about those things and understand my expectations, so it's great that people ask that question so I can be clear, oh wait, if you're in the last 20% of this polishing the dock. I'd rather spend time here and that's how I'd like the conversation to go so we all get something out of it. get out of this instead of feeling like it's just a leadership reviewer on my behalf, so in this specific situation I was setting expectations ahead of time so we can set everyone up for success.

Brilliant. Well, we've been talking about this. but this is an important part of the Netflix culture, so overall Netflix has a really special and unique culture even though it's been around for over 25 years, I think now it seems like the culture has emerged a lot Sometimes, there is that initial cultural platform that emerged. That surprised everyone, there's a recent book, no rules, rules, I think it's called and it seems like Netflix has done a great job maintaining their culture and I feel like there are three important elements and maybe there are more, one is uh. Very high talent density and focus on high performing people, two is sincerity and being really direct and then three is giving people freedom and responsibility and getting rid of useless processes like time and things like that, so maybe just dive into the first of high talent. density which is high and this focus on high performance.

I guess the question is what does this actually look like, what does this look like on Netflix and imagine part of this is hiring, part of this is performance reviews and then why is that? important why there is so much focus on Netflix what happens when you have such a high talent density is sointrinsic to who Netflix is ​​as a company and in some ways very reflective of Reed Hastings as the founder of Netflix So when he founded Netflix and grew the company over time, he did so with the belief that there could be a different approach to building a company to make it a place where people thrived and loved being and felt different from other places in the world. quality of that density of talent, but even more importantly, like excellence and results, and that's where people would get a great sense of satisfaction, so it's very deeply ingrained in Netflix since its original days and a large part of that talent density is definitely hiring, so who are the people that come in and join the team, but a big part of it is that we can't really have any of the other aspects of the culture, including sincere learning, seeking excellence and the freedom and responsibility to improve if you don't start at the top?

The density of talent and in some ways that's not the end, it's the means to the end of what Reed and the rest of the leadership team have been trying to build and to do that you really have to work hard. of things that don't feel like natural human behavior and what I mean by that is giving feedback. This falls into the second group, so giving feedback is being honest about your expectations when they are not met, which might be better to help people to improve and to be able to receive that type of feedback yourself to keep the talent density high because no one is coming.

Netflix as a perfect human being and remains a perfect human being all the time, we all have ways that we can grow and improve and so in order to keep that bar high, you have to be willing to have those kinds of conversations very uncomfortable an uncomfortable amount of sincerity and feedback to keep that bar high and then the other part is another thing that doesn't come naturally to humans, which is making a call in a fairly timely manner if someone isn't able to meet the bar to say no. I think the role you're playing is the right one or I don't think Netflix is ​​the right place for you and make it something that's part of best practices to get to a point where you can we could make that decision and that's where we We refer to the gatekeeper test, which is really just a mental framework to make sure that we hold ourselves accountable for this, where I do wonder if this person on my team came to me and said I'm leaving today.

I have a different opportunity and I would like to take advantage of it. Would you do everything you could to keep them on Netflix? If not, then I should have that difficult conversation about whether you would really be here. Are you in the right role? If I could feel a little bit relieved if you said you're leaving and the reason the gatekeeper test and that question are useful is because no one wants to think that way, it's very hard to tell someone that I think this is not the right thing to do. I think you should leave the company, so we need to introduce some of those Reflections to encourage behavior and we also want to get to a place where, when you have that difficult conversation, people aren't surprised.

It's easier said than done, but you can only get to that conversation. I don't think Netflix and you are a perfect fit for each other if you've been providing feedback along the way and therefore it feels like you know. Its most ideal state is a mutual observation in practice, it is not always so fluid, obviously we are human, but everything feeds on itself to make sure that we really stick to what we say, our norms of behavior as part of the culture, How do you operationalize that, is it like a mental model that you have to keep in mind or is it like every quarter you should do this exercise, is it part of the performance review process, how do you actually operationalize the network?

It's definitely a mental model. model, so when we talk to managers about what it means to be a manager at Netflix, it would mean that you should ask yourself this quite often about the people on your team. People often ask me if I'm passing your guardian test to be a part of. from a regular manager who reports directly to you one-on-one and is just another way of saying: Am I meeting your expectations? What's going well What's not going well How you think about things and sometimes it can be a very uncomfortable conversation to have in the middle of many things like we need to talk about this project or that deliverable or this thing that's happening, taking the time to give a Stepping back and just saying how I am can sometimes feel loaded and the gatekeeper test, although it feels like a very heavy concept. it creates a sense of levity around being able to have that conversation regularly, so if we put it into practice, it's a point that you raised.

I'll just clarify that we don't have performance reviews, oh wow, so we don't have a practice that many other companies do what we would think of reflecting as a rating of how things are going. We have an annual 360-degree feedback cycle where you ask for and receive feedback from many people, but it's not a contribution to some outcome, it's just for value. of feedback and to make sure that we maintain that strength and have an annual compensation cycle where we reflect on how people are doing and therefore you think about performance as part of both promotions like compensation, but that's the way it is.

It has to be part of the day to day and part of the operational rhythm because we don't create a process in which that comes to the surface. Interesting, I didn't know that, so the idea goes on like it's all a lot of people dream of not having performance reviews, we're going to give them continuous feedback in real time, we don't have to wait 6 months. I feel like people talk about this but very rarely do it, but this is how you guys operate in ideal practice. So you have to keep reminding yourself that this is our ideal because it's very easy to rely on the 360 ​​year cycle and all of a sudden I can get about 300 comments and some of those things are things that happened in six months. ago and I'm like, oh, I wish you'd told me this at the time, that's what it would have been like to live more of the Netflix culture, so we have to work on doing it that way, but yeah, if it works well, it's very timely direct feedback. of the 360 ​​cycle.

It's a kind of annual check-in, let me have a complete overview, let me be able to distill some topics, let me prepare for a conversation with my manager and then it takes away the kind of crutch out of six. monthly performance review or something like that when you talked about this example of someone asking you often if I'm passing your gatekeeper test, it makes me feel like someone is super nervous, they're like, I'm passing your gatekeeper test and that makes me I feel like it could create a culture of a lot of stress and worry and this Hunger Games mentality that I have to compete and worry and I could die or get fired any day.

I guess the solution to that is partly cultural. This is how we work with you, you don't need to stress all the time, but you may be fired if you don't meet this gatekeeper test. How can you avoid this? As is the constant worry that you could be fired any day and that you may not be meeting the bar in my personal experience. I've felt much more comfortable having these conversations than not having them, so in many roles I've had I haven't been sure how I was doing or the things I could do. I was doing better and I didn't really know how to get that information and that made me feel a lot more stressed or nervous or at risk than having those conversations as part of the culture, so I think it can be cheeky. -heartbreaking and I myself feel like it's the high bar for excellence at Netflix and you're surrounded by if we're doing it right, you're surrounded by amazing people and that can fuel a sense of if I'm doing it well enough compared to how everyone everyone else is doing it and I know that high bars for the most part can motivate people and in a good way, but in some ways it makes them nervous and that's where I think it's helpful to know that we look forward to having these conversations so that they can just kind of you know, let your shoulders relax a little bit.

Yes, expectations are high, but my manager says I'm doing a great job or my manager says I'm not doing a great job, but they gave me concrete things I could do. better and that's why I think knowing is better than not knowing and in that sense it's culture combined with conversations about performance. I hope to take some of that stress away, but I've certainly heard a lot that without that conversation people can be nervous, that's such a good point and such a good example that I feel like every company wants to have a high bar and only have high-quality employees. high performance and keep the bar high for every person they hire.

I'm curious. I know this could be its own podcast and book, but just in terms of hiring people who are amazing and maintaining this bar of excellence, is there anything you can point out that might be helpful to other companies hiring to help identify amazing people and make sure that bar stays high? What I know is that you pay the best salaries on the market. I think one unique thing about Netflix is ​​that we only pay people, so maybe that's part of this answer, but what advice would you give people to keep a really high salary? bar on your talent, yeah, I mean, at the point of compensation, we pay what we call top-of-the-market staff, which means we want to be highly competitive in pay, but we don't want the pay to be like the golden handcuffs of the Netflix TVs on the market instead of paying.

People have very competitive compensation, so I think that's important for attracting and retaining talent and it's been a big part of the culture, but almost more important is that we hope we don't have to rely on that to want people to want to be here. on Netflix. or so we can evaluate whether people are going to thrive at Netflix and the way I've thought about hiring in that context is we know we're going to offer very competitive compensation, but are they going to make it to Netflix and help us identify the right problems to solve or new ways to solve existing problems and that's a different way to hire than you might think, especially at scale, where you say, "This person has this skill, this skill, this skill, check." They will go into this box and they will deliver the work that I need them to do.

I'm being intentionally simplistic. I recognize that many people don't hire that way, but at Netflix we try very hard to say that. We're looking from a new perspective or at the person who is really going to make us stronger as a team, so thinking about additive skills, additive perspectives, people who are going to drive our thinking about something and that tends to help us think about talent density because You're constantly introducing people to the team who are moving up a level, so the questions you have to ask in an interview may be different because yes, we're trying to assess whether you have the basic skills to be successful here, but we're also looking for the things that they make people exceptional or even stronger than the team we have and then you think about forming magical teams made up of all those incredible minds and what you can get from that and that feels more like the density of talent and practice understood it, so the advice essentially is don't look for someone who's just great, look for someone who raises the bar for the whole team, brings a whole new perspective, yeah.

I think the cool thing about this idea of ​​just maintaining excellence consistently is that the best people want to work with the best people and as soon as there's one person who sucks and the company allows it, it just brings everyone down because they know we can be okay, we'll tag no one who's going to do it. anything about it and when you make it clear that we only want the best and we only hire the best, we only keep the best, it keeps the best there. That's part of this strategy, it's definitely the goal and I think understanding that there are gaps in the team. and people's abilities or behavior can be really toxic to other people on the team, so it's a cost that this episode is brought to you via explo.

A game-changer for customer-facing analytics and data reporting. Do your users crave more reporting and dashboard analysis within your product, are you tired of trying to build it yourself as a product leader? You probably have these requests on your roadmap, but the struggle to prioritize them is real. Creating analytics from scratch can be time-consuming, expensive, and a really challenging process. A fully wh-labeled integrated analytics solution, designed entirely with the user in mind. Getting started is easy to explore. It connects to any database or relational store and with its low-code functionality you can create and design dashboards in minutes, once you're ready simply embed the dashboard orreport in your app with a small snippet of code the best part your end users can use explosive AI features for their own reporting and dashboards eliminating customer data requests for your support team create and embed an experience of fully labeled analysis in days try it for free at the expo. co/ Lenny that's ex.

Lenny asked on Twitter what questions to ask him and there was a great question that came from Nonu, he is the head of product at Linear and he asks what practices Netflix does that other companies shouldn't try to do because their talent level is so high. higher than other companies, that's freedom and responsibility in a nutshell, so let me explain that it's a good question and it's related to what I said before: talent density is a prerequisite for many of the other ways we operate . if we want to create a work environment where we're not prescriptive about how people solve problems or the scope of problems they might address assuming they have a big impact for the business and we don't have a lot of processes around that work so think about being able to make big innovations in our engineering systems or introducing new ways of thinking about metrics and experimentation, we get a lot of those things because we give people the freedom and space to explore and question things and experiment in a way Solutions and I think that would be very difficult, if not dangerous, if we did not have a high density of talent.

It's not really a top-down, then B, then C process, even in the way that we carry out some of our planning processes or how to think about how. We think about priorities, there's a lot of room to contribute at all levels of the team and that requires talent density and then there are things like you have to have amazing people if you're not going to have really strict guardrails that influence the consumer experience or in the experience of business stakeholders and we give people a lot of responsibility for those things, so I think the lack of process and prescriptiveness depends on what we have.

Amazing people who are smart but, better yet, have judgment strong. This is what you always hear from people who give advice to Founders: just hire amazing people, get out of their way, and let them do their job, which is often not a successful experience. What are the examples? that things come that arose from this freedom. I don't know of products, features or ideas that came out of this or policies or processes that you don't have and that everyone else could have, so I know there is no vacation time, there is unlimited vacation. time, if that I guess, it's still a thing unlimited vacation time, okay, yeah, okay, cool, and I know the performance reviews you talked about that, so I guess either way it's an example of something that came up of this Freedom or something like that. process that would surprise people that you don't have or a framework or system that we've been able to offer, you know, I'll talk to my own team about Innovations in our content delivery network or Innovations in coding or Innovations in discovery and personalization.

They weren't driven by any leader saying "I think this is a priority," in many cases they were driven by individual contributors who had great ideas for innovation, so a lot of the things that Netflix has had success came from creating space for people on the team, so there are probably thousands of examples of product features and things like that came from creating this space and now the trick is finding the sweet spot so that we can operate efficiently at this kind of scale without turning off some of what it was. the core beauty of culture maybe one last question about culture we talked about a little bit about cander.

I'm just curious if there's an example that comes to mind of a cander example that you saw recently or had to be honest. person that might be interesting to share uh where it's like oh wow that's what you mean when you say a culture of sincerity and openness there's a couple of things that come to mind I'm generally a transparent leader, which means I share information freely and it's openly part of the culture of context, not control, which means that part of my job is to make sure people have the context they need to do their jobs well, and in practice that means I take notes on leadership meetings and I share them with the entire organization and that Sometimes includes canders around Reflections on things that are not going well or problems that we need to solve.

Sometimes it's just letting people know what leadership is talking about so they have an idea of ​​what's most important on their mind, but it's a version of transparency that I feel strongly doesn't exist in many other places and I think It is also a version of Cander by being able to share. I can't, we can share every detail of everything we're talking about, but I try to share things that probably cross the boundaries a little bit on the team, feeling like they understand what's going on across the company and what I'm thinking, and then I A second example comes to mind, which until two years ago individual taxpayers did not do so. we had levels at Netflix, so all the engineers were just senior engineers, all the data scientists were senior data scientists, and we didn't have a leveling system.

We introduced IC levels almost exactly two years ago and it was a big change because it was seen as something that was sacred, a lot of people came to Netflix because we didn't have it, we didn't have a process around promotions, that's probably part of the reason. why we never had performance reviews, because promotions really weren't. at stake and it gave people a sense of freedom of not having to worry about that kind of structure, but when you get to a St scale of an organization, we needed some kind of scaffolding to say that we want to talk about how we make up teams, like When do we need a person who has 30 years of experience?

When do we want to have a new graduate? Because that's what the job requires. We didn't have a language for it, so I introduced levels a couple of years ago and we had a big change. The rollercoaster is the only way I can describe how it was a good line, uh, and yeah, it was like being like the dryer for a few months and actually talking about it with the team, that's just context and backdrop for a example. from cander recently, which is that we had a kind of autopsy or retrospective on how it's been with the IC levels, so it's like Wells, not Wells, and normally I would think that in many cultures it would be like we overcome that change.

We are experiencing that change as if they did not reflect on it because that opens part of that initial debate and I felt differently about it. I think that's a good example of being honest about how this was a big change for us. Not everything went perfectly, there are many things we can do better in the implementation of levels on Netflix and I would rather share that information than pretend that everything went well and we achieved all the goals, so I think I try to build examples like that because I think that That level of openness and reflection helps create a sense of community and trust across the team.

It's an incredible example in the sense of that, but also of this category of freedom and responsibility, something that Netflix innovated a long time ago and I'm curious if this There is still this idea of ​​Chaos Monkeys, which is essentially a show that it runs on your infrastructure and it just kills processes and random things and just to see what breaks and make sure things are stable when things really start to break down, so, it's chaos. monkeys is that what is still like that is called and then it is that there is still one thing there are still some chaos monkeys running through the servers not unbridled chaos monkeys it is not okay chaos contained there is no car we have too much responsibility talking about freedom and responsibility for the member experience to inject pain, although we do a lot of experiments to test resilience and that probably means injecting things where we're not quite sure if a is better than b and that happens in all engineering systems really at scale, but it's not for pure chaos, it's for intentional learning so we can avoid making bigger mistakes and then as we pursue new endeavors like cloud gaming, we have a beta version that's now available.

It would be another example where we try to create intentionally low-key examples where we can test the bals of our systems in a way that is unlikely to harm the member experience, but that's less randomness and more by design, so that we're doing it in some places that feels mostly like good engineering practice so we can understand when and when. It's really show time and we're really going to test our systems: can they work the way we want them to break the chaos monkey? Yeah, kind of in these lines of data, something that data itself has always been at the heart of Netflix and As I understand it, the way that the data team and the insights team are structured has been one of the reasons why Netflix has been so successful and that is the team you led before taking on this new role.

Can you talk to us about how these teams are structured and why? the structure is very effective, yes, I certainly like to think that it is special, it is unusual. I can explain why, at the scale of company that Netflix has now, very often data-oriented teams are embedded in other parts of the business, so they could either be embedded in a line of business like ads or games or that are organized more functionally by separating data engineers from data scientists from analytics engineers from consumer researchers and we have resisted that and have maintained a centralized team that is functionally diverse across all those types of functions. that I just described and they work in almost every area of ​​the business from within the team and I understand why a lot of companies shy away from this because it really requires a basically extraordinary partnership that we would have people working on.

Data issues that are not reported to the teams that depend on them, but the benefit we get is that we can think about our functional experience. Are we really the best data engineers in the world? The best data scientists in the world and how we keep getting better. From a functional and technical perspective, it gives people better career paths because there is more mobility between teams. It feels like a team that has functional expertise with a lot of different problems to solve, so I think it allows for more cross-pollination of ideas. in a way and it also allows us to be really objective, that's probably the most important thing, our job is not to tell the story that someone wants to hear with the data or solve the problem that someone thinks is most important, it is for us to have our own perspective on things and I think that raises the level of the entire organization because it means that we can tell the truth or be curious in a way that might not fit if we had a different organizational structure. balancing that with being a good partner delivering on the things we agreed were priorities being flexible without wasting our time but it gives us agency and responsibility beyond that and I feel like the team takes that very seriously so I've seen examples of that in how we bring data into many spaces, including how we partner with engineering on data-related topics or how we partner with content that I'm not sure we would have gotten to if it weren't for having that core that is a kind of center of excellence around it and, uh, it's data and ideas that the team executed.

Insights are what describe user research or what that feature actually is, so part of the data and insights is a consumer insights team that includes many different types of research. So in some ways consumer is even a misnomer because there are parts of the team that do internal research, for example, on tools and products for our Productions studio, so it's more user research oriented than consumer, and then the parts of that team that are consumer facing. doing all kinds of things, from content screenings to make titles the best version of themselves before they're on the service to a more traditional ux research search to think about how we can offer the best title discovery experience or how we can think of things that improve. accessibility and then that team has a global remit, so there are also teams that have more local or regional experience in understanding consumer needs and entertainment, so consumer insights and a team formerly known as is still known as abbreviated data science and engineering combined. to create data and knowledge, it was probably a fewtwo years, that's another piece that is unusual and becomes a really comprehensive experience in data and research, so we could address a problem like what is the right way to think about recommendations and what is the best way to get them out? the light. which combines qualitative and quantitative attitude research with behavioral research in more aspects of data science, data engineering and analytics, is great because I think it's really weird that what people think of us in research of users is within the data or and I think that could be a solution to some of the negative reactions that a lot of user research teams get where they are like I don't know what you guys are doing all this anecdotal evidence if it's under the same organization uh, I feel like that leads to a lot more credibility and avoids this like, oh, the data is telling me this, this user research team is telling me this, what should we do?

Yeah, I mean, consumer insights was one of the newest teams for me, you know, it wasn't in my experience to lead a team like that and not in my individual training, but they are critical to making sure that we stay consumer-oriented. and members on things and I've loved seeing teams collaborate on problems because we talk about it as a superpower internally in combining those skill sets, so I think the Netflix consumer insights team has had a lot of credibility in a certain area of ​​expertise and we take it to the next level by combining it with another fun functional experience, so it's not like that.

It's not necessary in all problem spaces, so we try not to go overboard and say that we need to collaborate everywhere because it seems like the wrong expectation, but we try to make the most of it in spaces where we really benefit, so yes, it worked. really good, amazing, okay, I'm GNA. Ask two more questions before we get to our exciting lightning round and both are skills that the little birds have told me you are very good at one is that you are very, very intentional and thoughtful. about staying close to individual teams and individuals within the company even though you are rising higher in the organization.

I'm curious to know how you do that, how you actually practice this skill of staying very close to the teams at the bottom of the ladder. and the people who are working on things on the ground, basically, a lot of that is how I spend my time and fight to preserve opportunities to connect with people, so, for example, I still have biweekly office hours, the people sign up for spaces, um. and I understand it may seem a bit like speed dating for 20 minute slots but I get to meet a ton of people and hear about the work here which is the most important thing and people book them many months in advance and it's just a chance to stay at Toco and then I do anything in sessions with different sized teams depending on how intimate we want it to feel, but really anything is fair game as a way to get to know me as a person to listen to questions and try to be sincere. about what I can answer, I can't answer and those things have helped me stay connected, but both examples are about taking the time for it, so what I discovered as my role changed is that it just wouldn't happen if I didn't. made it a priority and then through those types of sessions I think I become or hope to become more accessible so that people know that you can send me a slack message, you can send me an email like I mentioned before.

I will respond to you as quickly as I can because I want to stay on that bar and that creates a flow of communication between me and the team that I really value. I don't think I would want to do my job if I didn't have those connection points, that helps You also send that email to everyone after every leadership meeting, so they say, oh yeah, Elizabeth, yeah, do you hear from me, yeah, something related to this, so we have a mutual friend, that's how we connected. Ali Ral, she was a data scientist. on Herb B is now on Uber and she had a question she wanted me to ask her and it's about how good are you at being present so her question is something she noticed about something I noticed about her is how 100% present she no matter who you talk to, do you have any advice for people to get better at this because it's so hard in the age of email and iPhones and slack?

Her question is, when do you respond to things, if not sometimes in meetings? In fact, I think I'm the most present when I have conversations like this and I make a lot of time to have one-on-one conversations where I'm genuinely curious about how someone is doing, how I can help them with what they need. I'm excited because it's authentic, and while my EA would probably be embarrassed to say that I like to spend time doing a lot of those one-on-ones, it's relatively easier for me to say that human connection is part of what I enjoy. I think this is true for a lot of people in what we get out of work in life, but I try to live that in those meetings, I'm probably not as good when we're talking about meetings of 30 people and I'm multitasking.

Well, I'll admit that I did that for sure, but I think I treat one-on-one conversations as pretty sacred and one of the things that I've noticed that helps me continue to invest in that and maybe be helpful to other people. They are some of my best friends and connections, including people like Ali, they are people I met on the professional path, so I worked very closely with Alli's husband, Keith Enwood, in various places, both in an analysis group and in Lyft, and that means it's created opportunities and it's been connection points and so you get what you give basically. um there are people in my life that are a part of my life because I worked with them or because I crossed paths with them and I like to think that if I can do a positive like. bookmark them, you know it's going to come back and be a benefit at some point too, so I think to sum that up is I really enjoy it, it's what I get out of it, especially from work, and then it's my community and that has served me very well.

Over time, I've given people advice that this is a small community. Think about what you are investing in other people because that will also be important to you in the future and try to live it myself. That good advice is like two things. That comes to mind when you treat people the way you want to be treated. Someone once said maybe, and I think you come back to this a couple of times, this idea of ​​just paying attention to what gives you energy and what you're good at. and almost doubling that, just making it more and more of a superpower, yeah, that last part resonates.

It's been a big part of my personal and professional practice to reflect on how I feel, what excites me, what excites me. I'm enjoying it and I think it helps me become more grounded, which maybe helps me be more present or a better manager or leader. That could be part of the secret sauce too, but it's part of my practice, I can't help it. but ask, is this like real practice? Do you do this regularly or is it just something you think about? I should reflect. I wish I was so advanced to say that I meditate and create this whole structure.

It's more than I think. I mentioned that maybe I'm an introvert so I spend some time alone, that's how I recharge and early in the morning, especially people who know me, are sometimes horrified at the time of day I send emails, but the first few Morning hours are a quiet time for me where I do it. I try to keep a daily check on how things are going. Why do I feel anxious? Why do I feel excited? It's kind of a muscle that develops, so until it does, I don't journal. I have a meditation practice. I have a time of day where I try to keep it protected from other things so I can think for a second.

What I think is that Jeff Bezos has this approach in the morning, he just calls it Putters. he around he doesn't have meetings until I think 10 or something he just wants to hang out read the paper see what's going on email I'm trying to do. I really like that, it feels really good. I'm just GNA. no responsibilities in the morning I like I never heard I'm going to adopt that language butter spread um Elizabeth, is there anything else you'd like to touch on or leave with the listeners before we get to our exciting lightning bolt R?

I'm ready for the exciting lightning round, well that brings us to our very exciting lightning round, first question, what are the two or three books that you most recommended to other people? It's probably a little recent, but I've been recommending what I talk about when I talk about running um de muracami, which talks about introspection about the similarities between running and writing as sort of flow states and very meditative things, so I had read some of his fiction books and autobiographical reflection on these types of professions or hobbies, I think it's very revealing, so that's one of my favorite books for a long time, it's a delicate balance between mystery and that's just a great story about human complexity, challenges and relationships, so I am drawn to both books and television and movies that deal with humans.

Speaking of television and movies, this is perhaps a high-stakes question for someone who works at Netflix. Do you have a favorite recent movie or TV show? I'm not going to name all the ones that look too much like an ad, a cinematic triangle of sadness is phenomenal if you haven't seen it and then I'll go to Netflix to watch TV. I thought he was hysterical. I'm a fan of Ali Wong, but it's also a pretty unique story and I think they just won a bunch of Emmys. awesome good picks next question question Do you have any favorite interview questions that you would like to ask the candidates you are interviewing?

High talent density. I usually look for the person who would be better at my position than I would be at my position, so I often ask people what their priorities would be, what they would do differently if they had my job. Next question, do you have any favorite products that you have recently discovered and really like? While I carry the title of CTO, I live a fairly analog life. The newest product is a pour over coffee maker which is actually part of my morning ritual which I will now call entertaining the places I drink Great Longitudes in my coffee making process because I find it relaxing and also it's not a recent find but I have to do it.

Shout out that my pelaton is probably the favorite product. I have the bicycle or the bicycle with a treadmill. I'm a recovering outdoor cyclist, so it's also a little questionable if I can admit this, but that's why I admit that I love pelaton despite being ideally an outdoor cyclist, I have questions about your cycling, but Before that question, do you have a favorite life motto that you come back to often or that you share with friends or family that you find useful whether at work or in life? My mom told me something. That has stuck with me.

I don't know if I live it very well, but she said something good happens every day and the reason she said that was because she was encouraging me to be more aware of enjoying the little things in everyday life. . Instead of getting caught up in the hustle and bustle, beautiful final question, you are a great cyclist and triathlete. I'm curious what that sport and time has given you in your career or in life, what benefits you've found from putting so much time and energy into running biking being an athlete certainly for mental resilience, so while that sounds like physical strength, I found that endurance sports are a lot more mental and the way you go through the ups and downs and sustain yourself and then come back from the challenge, so those sports have had their ups and downs. and from the lows and the lows I've really learned how to bounce back and bounce back so they feel like universally applicable skills.

You have such an interesting mix of athletics and then Netflix, what a good balance for life. This is going to give me permission. to go watch a report of Netflix recording this on Friday afternoon Elizabeth, you are amazing, thank you so much for being here. Two final questions: where can people find you online if they want to communicate and maybe follow up on things, and how can listeners be helpful? um you can always find me on LinkedIn um so definitely reach out or message me if you have any questions or comments and I think the way that listeners can be helpful to me is maybe being curious about how they can show up even better in their lives. .

Now that we've done this reflection on the culture of Netflix and how we present ourselves to other people, I'd like to ask listeners to pay that tribute to the people they're working with and how they present themselves to them. I love that if you end up doing this and are listening, maybe leave a comment on YouTube or substack with something everyone covered about you Elizabeth,Thank you very much for being here, thank you Lenny, I hope you have a great weekend, same bye everyone, thank you. Thanks so much for listening, if you find this valuable you can subscribe to the show on Apple Podcast, Spotify, or your favorite podcast app.

Also consider giving us a rating or leaving a review as that really helps other listeners find the podcast you may find in the past. episodes or learn more about the show at Lenny's podcast.com. See you in the next episode.