The Lean Startup | Eric Ries | Talks at Google

Thomas Sharon: Hello, everyone. Thanks for coming. My name is Thomas Sharon. I'm a UX researcher here. And without further ado, I would like to introduce you to Eric Ries, the founder of the Lean Startup movement. And thanks for coming. Eric Ries: Look, I don't know. It's hard to know when you have an audience you don't know. It's hard to know what to say. So some of you may know the Startup Lessons Learned blog. Any? Quick show of hands. Alright. We have new and real people. So thanks. Thanks for coming to check it out. So my name is Eric Ries.

I really don't want all your attention. So keep your laptops on. Alright. And in fact, if everyone would do me the favor of taking their cell phones out of their pockets. I mean, I'm sorry. I know I'm not supposed to have this one, but I can turn them on. No, I'm not kidding. From your pockets, please. Because this is going to be a talk. You might get bored and want to tweet with each other. Or I don't know, whatever special internal tools they use. Whatever it is, feel free. The only thing I ask is that you use the Lean Startup hash tag, at least if you are on Twitter.

Well. Is that fair? I will not tell anybody. Don't worry. I'm in New York because I'm writing a book. So one of the things you do when you write a book is tell as many people as possible that this book is going to be published. It will be released in the fall. I thank you in advance for booking it. You can do it at

lean

.st. That's my life now as a professional expert selling books, which is a far cry from how I started, which was as a programmer writing code. I'm one of those people who grew up writing code.

I used to make a living writing code, which is a job I knew and understood very well. He was pretty good at it. And then I started doing

startup

s and I started having to manage people who wrote code for a living. I knew it a little less well. And then I was managing people who managed people who wrote code for a living. And now I'm this professional expert. That's why I advise people who manage people who manage people who write code. So I've stepped away a lot from actual product development work. But that journey has taken me from just writing code to doing this because I really believe that the way we organize new product development is fundamentally wrong.

And that most of the energy we're investing in what's called "entrepreneurship," when it's two guys in a garage, or "disruptive innovation" or some other buzzword when it's done within a large company, It is a waste of time for many people. . And I think we can do something about it. That's what I wanted to talk to you about. Thanks for coming. Anyway, and this book will of course be available in stores around the world in the fall. So if you read the book, you will learn five Lean Startup principles. And we'll go over them today. And I want to invite you to ask questions on Twitter, if you're not feeling so brave, or interrupt me at any time or we'll have time at the end.

Alright? So, entrepreneurs are everywhere. The first thing is, especially in front of an audience like this, I want to make it clear that entrepreneurship is not just about two guys in a garage eating ramen noodles. In fact, what makes you an entrepreneur is not the type of noodles you eat, but the context in which you operate. And as I traveled around talking about Lean Startup, what I learned is that there are entrepreneurs in all kinds of places that you wouldn't necessarily expect. And we have a lot more in common than people think, because entrepreneurship is management.

But not the kind of general management that we teach in MBAs and that we have studied for the last hundred years, something fundamentally different. It is the management of a type of work that is measured by validated learning, rather than simply doing things. We accelerate that learning through something called the “build-measure-learn feedback loop.” And then we measure and hold entrepreneurs accountable using a new accounting system called "innovation accounting." Now I apologize to you. I'm sure you've come to a place where you've seen signs saying, "Ooh, an exciting talk about entrepreneurship and

startup

s and that's going to be great." And what did you get?

You have management and accounting, which are perhaps the two most boring subjects in the world. So if anyone wants to leave now, I won't be offended. Alright. Because the truth is, what do people know about entrepreneurship? I feel like... who saw 'The Social Network'? OK. Good. I feel like that's probably the best modern example of business history that we're all used to. And you see this in magazines and you see it on 'The Social Network'. Last night I realized that the story, 'Ghostbusters'... do you remember 'Ghostbusters,' the movie? It is a story of entrepreneurship. They start a business.

They, Dave, everything remains the same. It's like the same plot structure as 'The Social Network', believe it or not, except it has a Stay Puft Marshmallow Man, which is awesome. In these entrepreneurship sto

ries

, what happens? It is a story in three parts. First act: The brave protagonist, his character, his flaws and how he came up with the incredible idea of ​​him. Second Act: What I call the photomontage. It usually lasts about two minutes. It comes from "they finally get the thing to work." Then they write on whiteboards, drink beer and bang on some keyboards. And then they get their first client.

And that's it. There is no dialogue or anything in the photo montage. And then act three: now that we're on the cover of the magazines, how do we divide the spoils? And who is in charge? And who is in Who's Who? And how do we deal with the EPA and all that? For 'Ghostbusters' fans. What I think is really interesting about these sto

ries

about entrepreneurship is that 95 percent of the film's time takes place in acts one and three, although in real life, all the important work of entrepreneurship happens during the photo montage. But the problem is that, from a narrative point of view, the photomontage, although it has no dialogue and only lasts two minutes, is indescribably boring.

What do we do as entrepreneurs that really makes a difference? We spend our time trying to figure out which customers to listen to and who to ignore, how to prioritize product features. I mean you guys, how many product prioritization meetings do you attend? It's not exactly a movie thing. It's incredibly boring. And how do we hold people accountable? How do we measure to know if we are really making progress or building something that no one wants? Look, watching someone pretend they don't have anxiety because their vision is wrong doesn't make for very good movies. But that's what most entrepreneurs do.

So, we're going to have to talk about things like management and accounting, because it's time to get into the photo montage and try to figure out what we can do to make the actual work of entrepreneurship more effective. So, entrepreneurs are everywhere. My goal, my mission in doing this whole Lean Startup thing has been to try to put the practice of entrepreneurship on a more rigorous footing. And so I started with a definition. Here's mine: What is a startup? "A human institution designed to create something new under conditions of extreme uncertainty." So I think the most important part of this definition, and for our purposes today, a very important part of our discussion is what it excludes.

It doesn't say anything about what the size of your company is. It could be five people, 5,000 or 50,000. Really do not care. It doesn't matter what sector of the economy you work in. It really doesn't even matter what industry you're in. If you fundamentally operate with extreme uncertainty about who your customer is, what product they really want, and how do we build a sustainable business? Then you are an entrepreneur. And when I work with large companies, one of the things I've tried to do is get them to adopt entrepreneurship as a job title. Entrepreneurship is a career.

When you become an entrepreneur, you are no longer an engineer. You are no longer a marketer. You are no longer a UX designer. Whatever you used to do, suddenly you now have a different job and have entered a new career path. But unfortunately we didn't get the memo that says that. So, it can be a little confusing. That's a fancy way of saying that a startup is an experiment. What I mean by "experiment" is not just submitting it and seeing what happens, okay? That's not science. If you just put some compounds in a beaker and heat it up, it might seem like you're doing science.

But unless you have a hypothesis that you're trying to test, you have a theory that suggests what experiments will help you, and then you make specific predictions, then, fundamentally, you're not conducting an experiment. And we mean, in a Lean Startup model, to "experiment" in the scientific sense. We're trying to create a science of entrepreneurship that helps us stop wasting people's time, because that's what we're doing on an industrial scale. And you know it. Anyone who has worked on new products knows that most of them are doomed to fail. And when you get to the end of that product, I mean, as an engineer, I kept having the experience over and over again of working on amazing technology that today is sitting on a shelf or worse, that fundamentally no one is using.

And I kept looking for more and more technical solutions to that problem. I thought, "If we could get the development methodology right, if we had the right number of unit tests or this or that right, then we could prevent that from happening." But the biggest waste facing product development today is not building things inefficiently, but building things very efficiently that no one wants. And I brought a demonstration. We all know that most startups fail. Who remembers Web 2.0? Remember Web 2.0 when it was really cool? At the height of Web 2.0, 2006, a graphic designer created this graphic. Have you seen this before?

This was like the logo of all the amazing Web 2.0 companies that were going to change the world. And in just three years, in 2009, a different graphic designer was feeling a slightly different set of emotions when he created this graphic. Our three-year Web 2.0 report card. I mean, the blood red Xs, these are all companies that are just dead. However, I think for our purposes today, a much more important part of that graph is the green circles. I won't single out any of them in particular, but some of those green circles are supposed to be Web 2.0 success stories.

But for this chart, what that means is that there are companies that were acquired by a larger company, including this one. And listen, I'm all for people making money. So when one company is acquired by another, usually the investors and founders make some money and that's okay. But my question is which of these companies are really success stories? Success stories by the highest definition, not "did anyone make money?" Rather, which of these companies managed to live up to the aspirations, dreams, time, talent and energy of the founders and their investors? And most importantly, its employees. Look, look, we all know that when big companies buy startups, at least half the time, they die afterwards.

So we bought something for hundreds of millions of dollars. And then we ended up selling it three years later for tens of millions of dollars. That's not supposed to happen. In general management, that doesn't happen. When you buy an asset, it depreciates predictably. But when big companies buy startups, it doesn't happen exactly how it's supposed to happen. And I think the problems that corporate development departments have in deciding how to buy startups and which startups to buy and then how to integrate them into the parent company, are exactly the same problems that internal innovators have trying to create new startups within. big companies have it.

And they are exactly the same problems that venture capital-backed entrepreneurs have with their investors. All of us lack a theory of entrepreneurship to guide our behavior and that is why we once again resort to tools and methods that are not appropriate for entrepreneurship. That is my belief. So I don't think it has to be that way. Look, it's one thing for startups to fail because they were taking too many risks. If one of these companies was working on teleportation and then it turned out to be too difficult (we couldn't get the technology for quantum entanglement like we thought), I'd accept that kind of failure; that happen.

But I chose Web 2.0 for my demo, especially for you. Know. There isn't a single company on this chart where you would say, "Wow, I wonder if that can be built." Good? "Geez, I wonder if that new social network... is it possible to build it?" We all know. Software companies, we can build anything we can imagine. Think about that for a second. The dominant question of our time is not whether it can be built, but whether it should be built. And theThe question is can we build a sustainable business around a particular product? So the future of our society, our economic growth in the future, the GDP growth of industrialized countries in the future will depend on the quality and character of our collective imagination, which I think is a very strange place.

This is actually different from the types of economic problems that general administration was designed to solve in the 20th century. Now, most of my startups have failed. So I know that's not how you're supposed to start one of these

talks

, like, "Hi. I'm a professional expert and I've had more failures than successes. So you can be like me if you follow my advice." So, I'm sorry. But those of you who spend any time in entrepreneurship know the truth that where there are startups, there are many failures. And it has to be someone's fault in a talk like this and obviously it shouldn't be my fault because I'm the expert.

And preferably, it should be the fault of someone who is dead so they can't argue. I chose this guy. This is Fred

eric

k Winslow Taylor. He died in 1915, which is very useful for the purpose of this talk because it means he can't respond. So, sorry, Fred Taylor invented something called "scientific management." at the beginning of the 20th century, what we today call "administration". Look, people don't really remember Fred Taylor and those who have studied scientific management in school probably remember him for some of the outdated and really ridiculous ideas of his, like. time and motion studies or the idea that a worker is basically an automaton and must be told what to do.

The reason we don't give Fred Taylor the credit he deserves is because he invented things that are so obvious to us that we can't imagine they were ever invented. Has no sense. Everyone knows that work must be done as efficiently as possible, right? And that we should treat work as a system and that we should have managers organize that system. That's just common sense. And my favorite, Fred Taylor, invented something called "The Task and Bonus System," which we simply call "tasks." The idea was that if you want to do a big project, the best thing you can do is break that project down into a series of individual tasks and assign those tasks to functional specialists.

And everyone just does their part knowing that if everyone does their part well and everyone else does their part, everything will work out just like the manager said. And here's the best part. If he does his task particularly well, better than expected, he should be paid a bonus rather than penalized. What could be more obvious? Except in the 19th century, the way work was organized was that if you did your task better than expected, you were penalized. Because? Because that showed a lack of integrity. Obviously you could have done better before. But you did not. That proves you're a liar.

It gets worse. Not only are you a liar, but what about all your compatriots, your coworkers, who do the same task the old, slow way? They are liars too. You all have been lying this whole time and you should all be penalized. Imagine working in a factory where, if you came up with a better way to do your repetitive job, not only would you be penalized, but all your coworkers would be penalized. Can you imagine the culture that would grow around something like that? That everyone is working very hard to make sure that no one does anything better because then we'll all be in trouble.

This phenomenon was so widespread in the 19th century that they had a name for it. They called him "soldier." That all the workers kept going intentionally, trying to do the work as slowly as possible so that no one would get into trouble. We laugh now when we think about that kind of thing that happens in a factory. Because for us, the way we manage physical products, and all the usual general management, is light years from what was possible in Fred Taylor's time. But they also believed in something else that I think may sound a little familiar to you.

They had the idea that it was basically the great man's theory of work. Basically, the managers' job was to select the best person possible. Of course, this is the beginning of the 20th century. So a great man of upright character, with good integrity and the right attributes, puts them to work and, fundamentally, leaves them alone. Because if you trust a great man to figure out what needs to be done, he will figure it out. Does it sound familiar to you? We continue to manage knowledge work and innovation work in exactly the same way. We still believe in the great man theory of entrepreneurship and we especially believe it in great software developers.

And yet, when we look back decades from now, I think we'll laugh at the kinds of things we do when we need to develop new products. It will seem as old to ourselves in the future as Fred Taylor seems to us. Because I believe that entrepreneurship is management. It is simply a different type of management than the general management that has been practiced since the days of Fred Taylor. Therefore, we need to create a different management paradigm that is no better than general management. It's not worse than. It is simply a parallel discipline specifically for entrepreneurship.

And so, here is my attempt. The first concept in the business management toolbox is what we call "pivot." Who is tired of hearing about pivots? Any? Okay, I apologize. By the way, in Silicon Valley everyone is raising their hands. This word has become ridiculously overused. Believe it or not, I saw this the other day in the New Yorker magazine. Can you read this title? "I won't leave you. I'm going to change to another man." So this word started in Lean Startup and then became this overused and overrated monster of a concept. Typical of Silicon Valley. We went from not knowing what the concept was to being tired of hearing it and claiming it's overrated, without having gone through the middle stage of learning how to use it correctly.

That's how we operate the hype cycle in Silicon Valley. So, I'm sorry. It really wasn't my intention. But you can't understand anything about entrepreneurship unless you have a word for it, because it's the universal constant of all successful startups. If you can learn the real story of what really happened in the early stages of a company, you'll discover that successful startup founders don't have better ideas than those who failed. Contrary to what you see in the movies, most founders of successful companies had ridiculously bad ideas at the beginning. And the amazing thing about successful startup founders is not that they just persevered indefinitely, but that they had this curious combination that when they run into difficulties, they don't just abandon ship and go home.

They also did not persevere so that the plane fell directly to the ground. They do this thing we call "pivot." By analogy with basketball, one foot firmly planted in what we have learned, changing one more thing about the business at a time. And the premise of the Lean Startup is this: if we can reduce the time between pivots, we can increase our chances of success before we run out of money. Look, your way of thinking about the starting track isn't how many months of work do I have left? Basically, how many opportunities to pivot do I have left?

And of course, we can expand the track by raising more money. But we can also extend the track by figuring out how to turn earlier. And every day we reduce that time is a magical extension of our track by a proportional amount. Makes sense? Well, I'm getting at least some nods. That's good. So to increase the odds of success, we need to figure out how we can pivot faster. We need to focus on validated learning. Does anyone know this? This is the waterfall methodology of software development. It is traditional in one of these

talks

when you are going to criticize methodologies to call a "waterfall methodology." So this is mine.

This is simply Fred Taylor's idea applied to software development. When I trained as an engineer in Silicon Valley, I was taught this as the manufacturing metaphor for software development. By the way, you can imagine how angry I was when I found out that they no longer use this in manufacturing. This is very outdated. So it's not clear to me what our excuse is in software development for copying an outdated manufacturing model. But I understand why it's so attractive. The idea is that since software is so intangible, we like to imagine that our work travels on an assembly line, a virtual assembly line, from department to department.

And if everyone does their part and trusts everyone else to do their part, everything will work out. It is entertaining to defeat the waterfall methodology because it has a very bad track record. But it's important to understand that waterfall works as long as the solution and problem are relatively well understood. So if we were building something that's fundamentally similar to what we've built in the past, this works great. If you're making a sequel to a video game, amen. If you're building the next version of a product that millions of people already use, that's fine. But entrepreneurship does not address those circumstances.

So it's basically a waste of time. Now, when you waterfall, you have this problem I call "achieving failure," where you successfully build the wrong thing and brag about how good you are at doing it. My question is, if you attend a startup board meeting or a milestone review meeting for most new products, what do we talk about? Milestones, right. We are on our way. We're building features that we said we were going to build. We talk about our raw numbers like, hey, we have as many clients as we said. I remember being at a company that was looking, they had a plan.

They said we were going to have this nice hockey stick shaped growth. And we're at the long, nice, flat part, and everything is going according to plan. Sounds a little familiar, perhaps. And you're going to go to court like the plan is great. Nobody is using our product as expected, as expected. But next week it should appear. And how do you know if you were on the long, flat part and are going to slide indefinitely? I think we can really answer those questions quantitatively. We'll get to that. So to me, we're bragging about how we're building a product that no one wants.

But we are doing it on time and on budget. Like I have this image of a general manager driving a car off a cliff and, as they're driving, saying, "But I'm getting great gas mileage," right off the cliff. This is what startup failures mostly look like. And I think that's true in large companies as well. Not that I dare talk about you. But I've seen it in other companies, for sure. So, in manufacturing, they abandoned that linear way of working. That's why we call it Lean Startup by analogy with Lean Manufacturing. These are two other sadly deceased men who made this possible.

Deming is famous for saying, "The customer is the most important part of the production line." By which he meant that everything we do should be evaluated and decided based on whether the customer cares that we do it. So if the product is of high quality in the eyes of the customer, that matters. But if we do a lot of internal transportation, with the meetings we have, the specifications, the client doesn't care about any of that internal stuff. So let's try to remove it. When these ideas were conveyed to me as a Silicon Valley engineer, they looked like this.

There's our own extreme programming guru, Kent Beck. You guys know Agile very well, so I won't bother going into it. Suffice it to say that all agile methodologies have their origins in the IT departments of large companies. Each. And there is a reason for that. They are designed for situations where the problem is known but the solution is unknown. And so, by building something that is well understood iteratively, we can increase the odds that the project will be successful. So, the classic: Chrysler Corporation needs a new payroll system. Agile to the rescue. But this is not the world we live in as startups either.

If the customer is the most important part of the assembly line, what do we do if we don't know who the customer is? Through whose eyes should we judge our work? In extreme programming, which client should we sit next to engineers to tell them what to do? The assumption of Agile and all previous management approaches is that there is someone who can give us an authoritative and definitive answer to design questions. And in business, that assumption breaks down. We are working on products where no one knows what the customer wants. At best, we have a theory, a hypothesis, a plan, a hope.

And this is what Lean Startup looks like. Now, at Lean Startup we have our own guru, Steve Blank. He's still alive, but I put it in sepia tones just to be consistent. Steve invented something called "customer development," which is an iterative process towas to focus on cycle time. And what it looks like is this. Very simple heuristic. This is the Lean Startup flux capacitor. Everything we are as a software startup is a catalyst that turns ideas into code. When customers interact with that code, they create data that we can choose to measure quantitatively and qualitatively. And then, if we want, we can learn to impact our next set of ideas.

We can use this to put the pivot concept on a more rigorous basis. A pivot is an important turn in this feedback loop. And the heuristic for any kind of startup advice that someone wants to give you is really simple. Minimize the total time through this loop? So I don't know about you, I attend a lot of startup talks, I read a lot of startup blogs. All the advice is like that. "You know, it's very important to have a great design. Design always wins. Except Craigslist didn't have a very good design and neither did EBay. So sometimes it's okay to not have a design.

But make sure it's very scalable, because it's not have". I don't want to be the next Friendster. Except Facebook wasn't very scalable and that was fine, so make sure you have a good design and the design doesn't matter. It's scalable, but not too scalable. It's not very useful advice and if you go through the list, it's like making sure you raise a lot of money, but not too much. Make sure you have the right kind of people, but not those other kinds of people.people, but actually, sometimes those other kinds of people are okay. And we focus on all these contradictory things because for any particular advice, I can find someone who has followed that advice and then made a lot of money.

I can also find you someone who didn't follow that advice and made a lot of money. I can find people who followed that advice and didn't make any money and people who didn't follow that advice. I can find you the four quadrants of a logical possibilities graph. So how do we know which advice to follow and which not to? I think this is the heuristic you want to use. If it gets us through this feedback loop sustainably faster, it's a good idea. And if not, no. Of course, there is much more to Lean Startup. There are millions of things in this graph.

You can read them all on my blog, Startup Lessons Learned. Of course, you can buy a certain book. I heard it's coming out in the fall. Is very good. All of these techniques, like continuous deployment, where we put software into production, like 50 times a day on average. So 20 minutes from main trunk to production, no branches. Things like Net Promoter Score, where we can evaluate in real time through a tracking survey what customers really think about our product. Everything you know about usability testing, five whys, extracted from the Toyota production system. Each of these techniques operates at one stage of the feedback loop, but has the net effect of minimizing total time.

That's what the Lean Startup is all about. But I want to mention one more really boring topic called “innovation accounting.” Look, we've forgotten what accounting was designed to do. I mean, we think of accounting as that thing really boring people do to keep track of where money goes, right? That's pretty much what it is. It's just a ledger that says, "Where did all the money go?" But accounting was invented for a very different reason. It was invented to drive accountability between departments. Because if you want to have a large company with many different divisions, you need to be able to hold the managers of those divisions accountable for a few things so that you know they did a good job.

General Motors, which invented most of our modern management paraphernalia, had this concept. When I first read this concept, I literally laughed out loud; I could not believe it. It was called Standard Volume. It was the ideal number of cars that General Motors should sell, division by division, in an ideal year. And they really had the calculations, and the staff, the macroeconomic staff to figure out, given all the macroeconomic data available, how to translate the standard year into our next real year. So they could go division by division and say to each manager, "Since we're in a recession, or the economy is booming, you should sell this many Oldsmobiles.

And therefore, if you sell more than the standard quantity, you'll get an advantage. And if not, you failed. And it's not fair if you didn't have that concept, so if it's a good year, all the managers seem to be doing well. And in a recession, it seems like everyone is doing poorly. You can't know which manager really made the difference. Now, when I read that concept, I laughed out loud because I thought, "Wait a minute. Are you telling me that there was a time when people could make predictions about what was going to happen in the future, and then it actually happened?" I don't know about you.

Never in my entire life have I seen a forecast for something that turned out to be remotely accurate. No startup I've ever worked for has had a roadmap that turned out to be even remotely true. I have never seen a company say how many customers they would have in the future and then follow through. I had never seen it. So to me the idea of ​​standard volume is ridiculous. But I understand that when you have a big enough company and a long enough operating history, you can do this. Maybe this sounds a little familiar to you.

So if we use accounting to drive accountability, but all of our accounting depends on having a long operating history and many customers, how do we drive accountability if there are no customers yet? If the CFO of a company, hypothetically speaking, gives a certain team a bunch of money and sends them somewhere remote to do their job, like Australia or something. And then they hang out in Australia or whatever. And then a year later they ask, "What are your results?" And the team says: "They are not very good, but we are on the verge of success." How is the manager who gave them all that money supposed to know if A: They're actually on the brink of success or just on the flat end of the hockey stick, or if they've just been wasting their time for, like, a year?

Or more likely, if they are simply executing a bad plan. And what kind of milestone should we hold them accountable for if we can't hold them accountable for the gross number of customers? Because that's fundamentally not fair. If we focus on the raw numbers, by the way, we might decide that we're going to do a lot of advertising and PR and say, "This is going to be amazing" to raise customer awareness. But we all know that if you find yourself on a team like that, that early awareness is fundamentally lethal. But it's not fair to just say, "Well, let them do whatever and hope for the best." You know exactly how that turns out.

So what is the solution? I think we can now answer that question with something I call “innovation accounting.” Here it is. Instead of focusing on product milestones and raw numbers, we have three learning milestones we can focus on. We have to divert our attention from vanity metrics. Vanity metrics are the numbers you include in a press release to make your competitors feel bad. As the total number of Internet pages you have indexed. Turns out I really like that one. There was a time when we had a large index, battle for the number of pages indexed. And it's like, "We have four billion and you only have two billion." But what does that really tell you about the quality of someone's business?

Absolutely nothing. That could be four billion really dumb pages. It could be the website of a guy who's really excited about the number four billion. Or it could be four billion people, each of whom has a crappy website. If you read "TechCrunch," you'll see millions of stories about "This company sent 400 messages through their platform." But is it 400 million people about to attend or one very excited customer? We do not know. We used to have a competitor on IMVU that reported the gross GDP of the value of their entire economy from user to user. And my CEO would sometimes come to me and say, "These guys have a GDP of four hundred million dollars.

What's our GDP?" I thought, "What does that mean in our context? If two users exchange some virtual currency, is that part of the GDP? I don't know." It's a completely meaningless number, but it sure made us feel bad. I'm all for vanity metrics and press releases. Go to the city. That's fine to make your competitors feel bad. But what happens when we use those numbers to guide our own business? Is that "when the numbers go up, it's always because of what I was doing"? Everyone thinks I did this feature last week and now the numbers are up.

So it's obviously because of me. Of course, the marketing people feel it's because of their new marketing campaign, etc. What happens when the numbers go down? Has anyone ever been to that meeting? Oh, it's seasonal effects. Has anyone ever heard of seasonal effects being used to describe increasing numbers? Never in my career. It's always like it goes up, those are the characteristics. If it's down, it's seasonal effects, or worse, those marketing idiots. And over time, each of us lives in our own private reality where the things I do make the numbers go up and the things those guys do make the numbers go down.

So is it any wonder we think the rest of us are idiots? Now expanding that organization further and further as people find themselves in ever more permanent silos, speaking their own language, and living in their own private reality. Is it any wonder they have trouble working together? Maybe that sounds a little familiar to you. Well. Just checking. So instead, we'll use actionable metrics, which refer to behaviors per customer, things that can be measured on a micro-scale. And the first thing we're going to do is establish the baseline. So now we can put the purpose of the minimum viable product into a much more rigorous framework.

Somewhere in our business plan, there's a model that says, "Hey, if customers behave this way, over time we'll have millions of them." And we can't go into all the details of how to build those models. Of course, a great book is about to be published. You can learn everything about it. In the meantime, what we want to do is figure out what the real numbers are for each of those microscale inputs. That's what the minimum viable product is for. So if there's some number, some spreadsheet somewhere, that says, "Hey, ten percent of the customers who visit our website should sign up for our product." So we should have a big sign somewhere in our office that says, "We must have ten percent conversion or we will die." And then each of us will have a minimum viable product as soon as possible so we know what that number is today.

And chances are, when you do that experiment, the baseline will be horrible. For example, it will be only one percent and it is supposed to be ten percent. And like, oh my God. In general management, that causes a crisis because now we fail and uh-oh. There's something called "zero audacity," which is how easy it is to raise money and get people excited when you're not getting results. Or having zero dollars of revenue in a startup is a good time to raise money. Having one dollar of income is a disaster. Because with zero it's like, "Well, why is it zero?" "Because we haven't launched." So obviously it should be zero.

Everyone says, "Oh, that makes sense." God forbid you have a dollar of income, because then they'd say, "Why is it only a dollar? I thought this was going to be an overnight success and now you're showing me it's not." So with zero you can always be successful overnight. With any other number you are screwed. But we need to change that. We need to say, "Discovering the truth about where we are now is progress. It's a milestone we should celebrate." And then we do step two, which is tuning the motor. We make changes in product development that are not designed to generate huge top-line numbers, but rather to move those conversion numbers from the horrible baseline to the ideal in our business plan.

And whenever I've done this with teams, I've only seen two cases. In the first case, it is assumed to be one percent. It's one percent but it has to be ten percent. So after a few iterations, it's one percent, three percent, six percent, six and a half percent, seven and a half percent. Now, it's still not ten percent. So the model doesn't exactly work, but you can say, "Are we going to get there?" Yes probably. Every thing we do seems to increase the number a little. It seems we are going in the right direction. We're split testing to make sure the changes we're making are actually driving change.

Everything is alright. Here is situation number two. It's one percent, three percent, three and a half percent, 3.75 percent, 3.8 percent.cent, 3.81 percent. Now, the numbers are increasing every time. So it's not that the numbers are going down. It's not like it's zero. But a question might be asked four, five, or six months after reaching that asymptote. Are we ever going to get above ten percent? I think it's safe to conclude that the answer is no. Of course, theoretically it is possible. The next iteration will be that one more magical feature that takes you to ten percent. But in reality, that is not the case.

When the team gets to the point where they achieve those diminishing returns, everyone knows you won't make it and you enter the land of death march. So, I recommend we do three. We schedule the meeting in advance. That in three months, six, whatever, we will have a meeting to decide whether to pivot or persevere. And by that meeting, we will have data on whether our efforts to tune the engine are working or whether they are seeing diminishing returns. So we have all these concepts in business, like product-market fit, that are very vague. This system allows us to put those concepts on a much more quantitative basis.

We can't turn the decision to pivot into a formula. I can't tell you what to do. I still trust human judgment, just as science does. But if we make specific predictions, if we use innovation accounting as an accountability model, then we can train our judgment to improve over time, as in science. So, don't do product development astrology. Do science of product development. I left a lot of questions unanswered because we've only been together for a short time. How do you know specifically when to turn? What is the relationship between our vision, our strategy and our product?

What exactly should we measure in each of the growth drivers? How do products grow? How do we know if we'll be on that hockey stick or on the long, flat end forever? How do we check if we are creating value? What specific features should the MVP have? Can we go faster? The answers to these questions and many more are found in the new book coming out in the fall, called "The Lean Startup." Of course, you can pre-order it at

lean

.st. Thank you very, very much for doing so. I will only give you my contact information.

Please get in touch if I can be helpful in any way. We have a new website, which in itself is a minimum viable product on theleanstartup.com. Please, watch it. We would love to receive your comments. And everyone is officially invited to the Startup Lessons Learned conference, which will be held on May 23 in San Francisco, but we will also be doing a simulcast. Last year we were in 50 cities. So we'll probably be in New York too. I hope that if you can do it, you will write to me. And if this is helpful in any way, I hope you'll email me and let me know.

Thank you very much to all. Do we have time for some questions? Okay, let it pop. If I stumped all of Google, I'll be very proud of myself. That goes well on Twitter. Sweet. Female Audience Member #1: So, I just wanted to touch on something you mentioned that gets revisited too many times; the pivot. Eric Ries: Uh-huh. Female Audience Member #1: I'm having trouble understanding exactly how much work I need to do for the first turn. I think the second and third ones might be a little more, okay, maybe not. But just starting out, how much work should you really put into that first turn?

Eric Ries: Honestly, there's no way to answer that question in general. You have to put yourself in a position where the team knows if it is working or not. And the problem is that most teams have a plan, which is basically just send it out and see what happens. And the problem with sending it out and seeing what happens is that you may feel like you're being very agile, but you're guaranteed to be successful by seeing what happens. Therefore, you will always feel like it was worth doing and feel like you are on the right path no matter what.

The only way to get into a position where you have to pivot is to make specific, concrete predictions beforehand that, if they turn out to be wrong, will call your theory into question. And the problem is that we all know that most new product projections are complete nonsense. You have to tell the CFO, or whoever, that you will have a zillion customers in year five. Otherwise you will not get the money to carry out your project. But we know we just made up those projections. So when they don't turn out to be correct, we think, "Well, that doesn't prove that our vision is wrong.

It just proves that it took longer than we expected." So yes, the hockey stick will still happen, but it will take longer. If we do innovation accounting and we make very specific predictions of customer behavior, something we will often get people to do is sell the magic version of their product on a landing page somewhere. And it's like, don't even say what the product is, how it works. Simply state what benefit it gives you and see if you can get people to sign up. If people don't subscribe to magic, they certainly won't subscribe to your product, because magic is always better.

And if the magic isn't even good enough, if the magic conversion rate is too low, then you know you have a problem. Not that that means you therefore give up and go home. It simply means that there is not enough latent demand for what you are doing. And then you'll be in a different type of market than maybe you expected. That helps? So the minimum viable product is really the minimum, the smallest amount required, to obtain that first information. It's not, "Oh, if it's not an overnight success, we'll give up." And if you release that pressure to get it right the first time, I feel like a lot of us feel like we have to do this circus act to make it look like we can predict the future.

Like one of those brilliant visionaries, the next Steve Jobs. Not even Steve Jobs is as good as Steve Jobs. That's a story we've all been told. And in every company I've worked with internally, there are genius heroes who always seem to be able to get it right on the first try and everyone else is trying to emulate them. But when you meet the hero, you're like, "How do you do that?" If they're honest with you, they'll say, "Actually, I don't know." Or: "Actually, that's not how it happened and it wasn't as easy as everyone thinks. Now I feel incredible pressure to replicate that success again." And so, you can imagine the negotiation that occurs with the superstar about his next project.

They never want to be in a position where they do something and it doesn't work, or they will have to quit and go convince another company that they are a superstar. I heard that happened recently. Anyway. Is that useful? Female Audience Member #1: Yes. Eric Ries: Okay. Any other question? Male Audience Member #4: So, most of the quick feedback you're talking about seems to be very applicable to consumer applications where the cost of testing something in the amount of time it takes to test something is pretty low. . I will or won't sign up for Twitter or Facebook or whatever happens next.

Eric Ries: Yes. Male Audience Member #4: How does it apply if it's something bigger? If it's an enterprise sale or if it's a larger commitment, is it basically the same process taken, but the timeline is longer because the commitment process is longer for each step? Eric Ries: Yes. I mean, that's the short answer. I mean, the long answer is my experience on consumer internet. So I talk that way, naturally, about large sample sizes and split testing. Just like me, I can't help it. That's my entire career. The funny thing is that Steve Blank, who I mentioned before, has a great book called "The Four Steps to the Epiphany," which I hope everyone reads.

When he speaks, his background is in enterprise software. He also gets this question all the time. Except when asked, people say, "Well, of course, this will work on enterprise software. But how will it work on consumer Internet?" Because we simply assume that tactics that have worked in one industry don't work in another. So the answer truly is: it's principles that matter, not tactics. And that's why the principle of the build-measure-learn feedback loop is cross-sector. So, for example, in consumer Internet, because we are used to having a large number of customers, we do all this analytical work as a crutch.

In reality, it's actually because we have it worse than corporate people. When you have a lot of clients, you can't get to know any of them particularly well. In fact, you simply have to rely on archetypes, summaries and assumptions. When you have a small number of customers, it's a big advantage because you can get to know each one of them extremely well and the experiments you do can have much higher fidelity. So physicists can't ask electrons what they're thinking about doing. They are not available for comment. But when you're studying something that can actually interact with you, it's a completely different ball game, completely different.

Ask around here? Yes. Male Audience Member #5: What product should Google focus on right now? Eric Ries: What? Which product should Google focus on right now? Listen, I would never dare say anything to Google about that. I mean, look. Male audience member #5: You were invited. Eric Ries: What is that? Male audience member #5: You were invited. Eric Ries: They invited me. But look, but seriously. The outside expert knows nothing. You know your business much better than I do. And you know which products need to be changed. Know. You don't need me to tell you. The better question is, how the hell are we going to get them turned?

Because we are trapped in a management and accountability framework in which revolving around failure is a problem. For example, I won't go into too much detail about a certain Google product that I know especially well because it competed with something I was working on at the time. One of our IMVU co-founders went to work on this product for Google. So you are Google people. You can talk about this. So they had a lot of insider information at their disposal about what we were doing. And at first we were very nervous because it meant that we were going to face competition from Google.

This is what happened. Google spent two years working on that product. Spent, I can't imagine how much money will be spent developing it. And when it finally came out, they launched it with a bang, they put the Google brand on it. And then when some things didn't go as expected and it turned out to be an embarrassment, they took him out and killed him. And the whole time we were like, "What's going on there?" Because we were constantly iterating and changing throughout that entire two-year period. When they launched the product, we felt like they had launched a product that did the same thing as ours two years ago.

And by giving the launch a lot of publicity, putting the Google stamp on it, the inevitable problems, the same problems we had had when we launched it, but we were able to pivot because we had a pathetically small number of customers and no one had ever done it. heard about us. That's a big advantage. It's actually a relief. Because it means you can make mistakes. Nobody knows. Nobody cares. Darkness is a benefit. By naming it after Google, by claiming it was the next big thing, I guess Google put that team in a position where there was no way to pivot.

It became an embarrassment to the company, or to someone, and the thing was withdrawn. Now, that product could have taken a turn. It was a really good product. It had a lot of really good things. There was no physical reason why it couldn't spin. But two million dollars, two years and millions of dollars, with all these expectations and the Google name, I can't imagine the pressure they were under. I felt bad. So the right question is: what could Google have done to create that same product and put it in a position where it could have pivoted?

In fact, I believe leadership in today's economy means creating experimentation platforms for teams. I borrow that phrase from Scott Cooks, founder of Intuit, who has been a great Lean Startup doctor. So if you want to be a GM at a big company like Google and you want Google to be more entrepreneurial, my view is that it's up to you, not your team, to give them a safe space to experiment. So for a new and risky product, don't do it, I would never put the Google brand on it. To shame. And I would never talk about it to the press, if it can be helped.

And I would try to have as few customers as humanly possible while still accounting for innovation. Then, the teams will rotate without problems. There is nothing wrong with Google employees. It is a management problem. In my humble opinion. Yes. Male Audience Member #6: Let me continue with that directly. Eric Ries: Okay. Male audience member#6: Let's say you write a new mobile app and publish it on-- Eric Ries: Hypothetically speaking. Male Audience Member #6: Yeah, yeah. The app store or the market. If you do that on Google, with Google's name on a blog post, you'll get a hundred thousand downloads, no matter what it is.

Eric Ries: Yes. Male Audience Member #6: It can be almost anything. But it is true. And by virtue of that, are you now, when people search for categories, shopping, personal finance? Eric Ries: You will be number one. Male Audience Member #6: You'll be up there. You will have that visibility and that will generate clicks and you will be featured. And you'll get started in a way that you couldn't if you're a nobody and publish an app. It's very possible that you're just in the middle of the noise and will never get out of it, no matter how good the product is.

Eric Ries: Right. Male Audience Member #6: So, I mean having done this twice-- Eric Ries: By the way, entrepreneurs never ask hypothetical questions. So, I understand. Male audience member #6: Yes. Yes. So I can tell you that there is real value. I mean, I know that the successes I've had are largely due to having that name associated with it. And that initial jump, the initial offset jump, was a market I couldn't have bought. Eric Ries: Yeah, look, you get free marketing, but so what? Free marketing is worth much less than we think. Yes, that sped up your success process, but only because you had the right product and it worked for your customers.

So putting rocket fuel into a rocket that's having problems isn't a good idea, right? You accelerate too fast. The thing explodes. Now, sometimes you get off the ground because you actually have the right product. But marketing money isn't as hard to come by as it once was. Today's truly great products have a growth engine that allows them to grow organically anyway. Then yes. Google has enormous advantages by putting its brand on it. You can get many downloads. But there's also a huge responsibility in that because-- Male Audience Member #6: Well, then that's a good question. So why isn't it okay for Google to fail?

Why not take a chance and just fall flat on your face? Eric Ries: Listen. Male Audience Member #6: And quickly admit it and move on? Eric Ries: If it were me, I'd be celebrating failures and making those people heroes. If it were up to me, but that's the culture I live in now. But look, failure is not what we want to celebrate. We want to celebrate successful pivots away from failure. And that's a challenge because it's easy. If you're charismatic, you can get resources for anything and you can send things out and have people use them and then you can engage in a lot of what I call "Success Theater," to try to make it look like you're very successful.

And so those people tend to welcome the celebrations, whether they actually add any value or not. In my opinion, they are like a cancer in most organizations. But what do we do instead? We don't want to celebrate those people. We don't want to just celebrate people who fail because they didn't achieve anything. So we have to have a system to evaluate which failures were truly instructive and then we have to celebrate the learning and what the learning became on the next attempt. So yeah, if the Google company is okay with people failing and the Google name being associated with unpleasant things, that would be fine.

But I think that's not realistic. I mean, I wouldn't be that comfortable. If I were Google, I'd launch those things under completely different branding. I wouldn't tell anyone they were made by Google until they are proven viable. How do you think Apple does it? The things we all consume and wait in line for, we are the first people: is the first person who buys an iPad at the Apple store the first person who has used the iPad? Come on. That is my humble opinion. Useful? Male Audience Member #6: I didn't follow that last line. Eric Ries: Then maybe I shouldn't have said it.

Cancel that. I think the answer is to give teams a platform for experimentation, have clear analytics to test whether they are making progress or not, and celebrate pivots. Whatever Apple does, who knows? Thomas Sharon: Very good. Last question. Eric Ries: Make it worth it. Thomas Sharon: Or this was the last question. Eric Ries: Too much pressure? It is an official Google brand question. It's happening on the Internet. Thomas Sharon: Very good then. Eric Ries: Ok, thank you very much everyone. I apreciate it.