Lizard Optimization

Understanding Lizard Optimization

Dave Farley: Hi, my name is Dave Farley. Welcome to the GOTO Book Club. Today, I'm talking to one of my good friends, and a serial author. I brought just some of the books of his that I own. And I'm missing two of my favorites, so, "Spec by Example," and "Humans Versus Computers." But Gojko Adzic is a prolific author. And he's just written a new book, which we're going to talk about today. Gojko Adzic, welcome. Your new book's called "Lizard Optimization," which I must admit, I'm kind of both intrigued and kind of doubtful about the topic. So what's lizard optimization and explain?

Gojko Adzic: Well, thanks very much for doing this with me. I always love chatting with you. Lizard optimization is a systematic way of figuring out how people are misusing a product, and then using those signals to make the product better. And by making a product better for people who are misusing it, we can make the product better for everybody. A lot of this is inspired by my recent work on a product I'm building, following kind of this thread of how are people misusing the product to figure out and augment user research, where and kind of figure out my blind spots in product management, has led the product to grow very explosively, and become significantly better for all the users. I've been so impressed with that. And I felt other people should hear about this and maybe try it out with their products as well.

Dave Farley: And from the book, some of the examples are based on two of your products, Narakeet and MindMup, and your experience with those and evolving those products over time. And Narakeet in particular, you've had explosive growth through using these techniques. Tell us some of the stories.

Gojko Adzic: Well, I mean, for example, one story that is really illustrative of this is, the tool started as a way to make videos from PowerPoint. And there were some people who were constantly making blank videos and paying me for that, which made no sense at all. And turns out that these people were actually really impressed with the text to speech capabilities, but they didn't need the visual aspect of the videos. They were going through hoops and making videos just to extract an audio track. So they can use the text to speech service. And once I figured this out, I simplified a way for those people to create just the audio file without having to build the video. From the user perspective, that was a better experience, because they did not have to create blank PowerPoints, upload PowerPoints, and wait for the video to build.

From the operations perspective, this was a lot cheaper, because it's actually a lot easier to build an audio than a video. So, kind of operational costs got down significantly. It was better for users. And then that kind of tiny minority of users that was, you know, less than 1%, over time actually became more than 99% of the usage, as the product grew. And the product is now a much more flexible text to speech production system than it is a video build system. And this is kind of the thing that came out as looking for these lizard behaviors and lizards.

Concept of "Lizards" and User Behavior

Dave Farley: So why lizard? So you do explain this a bit in the book, but for the benefit of our audience, why lizards?

Gojko Adzic: So sometimes when you look for these edge cases and exceptions, it looks like it's not done by humans. You know, why would somebody build a blank video? Why would somebody try to upload, you know, an APK file where you expect a Word document or something like that? It doesn't seem like human logic. And I read this wonderful article by Scott Alexander from the Star Slate Codex blog where he talks about this kind of noise in research. And he says that, you know, around 4% of people end up doing things that you wouldn't expect a rational human to do. And there's kind of some background why he names them lizards. And I thought, well, this is a wonderful thing. It's a really great way of providing comic relief when you're dealing with a head banger and you're totally confused, and you can think, well, this is not done by humans, it's done by lizards, you know. But then you start really looking at what these people are doing. And then understanding that there is a logic somewhere. It's not your logic. It's not my logic. But there is a logic there.

And one of the really interesting things that I took away from his article is that this is not an insignificant group of people. You know, he has this lizard man's constant of 4%. And if you're looking at reducing churn, if you're looking at, you know, improving the product, so making the product better for 4% more people can create significantly outsized returns. There is research from Amy Gallo, from "Harvard Business Review." And she has this number where improving retention by around 5% can actually improve profitability by 95%, because it has compound effects. And I think for me, that was the big lesson of how you don't just improve the product for those, you know, 4% of crazy people or people who are doing crazy things, you improve the product to make it better for everybody. One example from MindMup we had was this bug report that our software doesn't work well when it's running on a fridge. And I never imagined in my life, anybody would try to run my software on a fridge, why would you even do that?

But then, you know, it turns out that the person trying to use it on a fridge was a mother who was trying to work from home. This was before the whole, you know, COVID and work from home thing, because she was taking care of her two kids, cooking lunch. And there was an Android screen on her fridge, and she just tried to use it there without a keyboard, without anything. And we never even thought of that use case. But we improved the UX for this to work on devices that don't have a keyboard, that don't have...you know, and to work better with touch. And this thing significantly improved the product for everybody, it increased the usage. And I love this statement, there's a lovely book I found while kind of doing the research for this book, from Kat Holmes, on inclusive design. And she talks about this kind of principle, how by improving the product for people who are, you know, mismatched with the product, you don't just improve it for people who are disabled or have kind of impairments, you improve it for everybody. And I think that's kind of where we actually need to start looking at growth opportunities for our products.

Recommended talk: You're Testing WHAT? • Gojko Adzic • GOTO 2021

Importance of User Retention 

Dave Farley: One of the things that I really liked in the book, one of the ideas that I hadn't really thought about in terms of product evolution was the idea that, you know, if you want to grow your sales of a product, you can either go and acquire new users, or you can increase the retention of your existing users. And increasing the retention is the more profitable strategy if you're at a certain point in the life cycle. Could you just flush that thinking out a little bit?

Gojko Adzic: Absolutely. I mean, you know, and this goes back to... My horribly failed startup story from, what, now it's 15, 16 years ago, where we massively fumbled. You know, we built a decent product that did not have great retention. And then we spent our entire marketing budget to bring lots of people to the website, and very few of them stayed. If we had improved retention first, and then brought lots of people to the website, you know, I still hope the results would have been different. Who knows. And I think, you know, the company failed as a result, because we didn't have a lot of money to spend after that. And I think reducing churn, improving the product, so that people stick with it, improving stickiness is great, because then it compounds the effects of any future acquisition. If you have a way to bring a million people to your product, and 1% of them stay in average, then that's a totally different outcome from 5% staying on average, or 10% staying on coverage.

And I think, you know, early on in a product life cycle, you don't have anybody to retain, so you can only focus on acquisition. But I think focusing on retention... But I think, you know, retention is a result, but focusing on improving the product so that it's better for people, so that we get retention, is a winning strategy, because then people stay for longer, they recommend it to their friends. You know, in theory, yes, if you look at a funnel and how people design user funnels, you can add to user funnels or you can plug the holes, and that has the same effect. But I think getting people to stay longer and making the product better so that they stay longer and improving stickiness is much, much more effective than getting new people all the time and acquiring new... And, you know, look at your YouTube experience. You have an amazing channel where, you know, people come back and watch lots and lots of videos.

And I think it's much easier from a product strategy when somebody is already there. They know about you. They've seen your videos. They've seen your product. To kind of keep them engaged than it is to keep bringing new people all the time. And one other thing that kind of is interesting from a product perspective, for me, at least, outreach marketing and reaching out to new people is something that the product can rarely do on its own. You have to do it through external channels. But improving the product so that people are happier and they stick around for longer is something that the product can pretty much do on itself. And if you're not good at marketing, but you're reasonably good at product development, then retention is probably a better investment of your time and money.

Recommended talk: Requirement Specification vs User Stories • Dave Farley • GOTO 2023

Challenges and Rewards of Building Products

Dave Farley: Or keeping people away, as you described in one of your examples. There's lots that I liked about your book, as ever. I like your writing style as much as anything. But one of the things that made me laugh was when you say, look at how people are hacking your product. So you're going to look to try and find out how people are misusing it. And you had a good example of that where it was costing you.

Gojko Adzic: Yes, absolutely. I mean, at some point, like, the usage started doubling every two days, which was insane. And just for context, the product works on a free trial model. So you have to get a certain percentage of people to try it out and then upgrade. And the commercial users effectively end up paying for the free users as well. So when you have a balanced kind of growth with conversion, the money works out. But when lots of people kind of start joining the funnel, and they don't convert quickly enough... This really became critical for me about two years ago, where the usage doubled overnight, and then kind of doubled again two days later. It turns out there was this guy who figured out how to abuse the free tier, to constantly create free audios. And he published a popular YouTube video about it. And for some reason, you know, everybody with a computer in India started doing it at the same time.

And if you just look at the growth numbers, then it's amazing. But financials were completely... The company was going to go bankrupt in a week if I didn't stop it. But at the end, you know, figuring out what was going on, I realized that lots of people were using VPN tunnels to kind of cheat this free tier detection. And I actually implemented a restriction for VPNs. And then I started getting like hundreds of emails from people who've been using it for free. And actually, there was an email from an insurance company that said, oh, you know, this is critical for business for us. We insist you re enable it. And they were not paying, they were using it... You know, it's something critical for an insurance company, and they never thought about paying it. But I did get a few emails where people were actually commercial customers, they were just using VPNs to hide their tracks or details.

And then as a result of that, I realized, okay, accessing through a VPN could be a commercial feature. I can enable it for commercial uses only. And at the end, I ended that month with like a 25% increase in revenue, which was amazing, because, you know, some people needed it, they were going to pay for it. Some people, you know, obviously were never going to pay even though they were critical for the business, insurance company style. But yeah, figuring out how people are kind of hacking and misusing the product really helped me complement user research. I think you can discover only so much in research. And research has to stop at some point. And at the end of the day, if you make a product good for the people you've researched, it's going to be good for them, but you don't really know what the audience is going to be. And looking at how people are misusing or hacking the product is a great way to figure out what were the blind spots in that research.

Dave Farley: So it seems to me that there's a mix of things here, you know, at the heart of the advice that you're offering. So chasing the lizards in the long tails gives you the opportunity to potentially pivot into new areas of business for your products, but also just to enhance the software, so it's better for everyone. So if it's easier for the lizards to use, it's easy for everybody.

Gojko Adzic: Absolutely. And I think, you know, part of this is, I think, just trying to give a name to something that already happens, but it's embarrassing and people are pretending it doesn't happen. I mean, there's a great example I found from PayPal where early on, the idea was to transfer money over PalmPilot devices. And they really, like, built this amazing crypto library that works on low power devices. It wasn't a technical feat of excellence, it's something that, you know, you and I would admire. And they built, like, a website so they can test it cheaper. And a year later, they had 12,000 users on PalmPilots, because not enough people had PalmPilots. And they had a million and a half people using the website.

And the product management was fighting the people on the website. They were trying to prevent them from using this thing for eBay and for things like that. And I think there's something that I've seen happen with me and with other products I've been involved in, where, you know, when the audience wants to take a product somewhere that the product management didn't intend to, product management fights against that instead of listening in to the audience and thinking about, well, this is an opportunity for growth. Maybe we should be rethinking about our product a bit. And, you know, PayPal today is not running on PalmPilot devices. Nobody has PalmPilot devices anymore.

Dave Farley: It's not one of its main platforms.

Gojko Adzic: No, no, no, definitely. But it is an incredibly successful thing. And, you know, there's lots and lots of examples of things like that. I remember one of the things, you know, we realized a while ago on different things. We built this kind of multiplayer game that had a chat system, and people could chat there. And there was actually some really stupid bug in the game that ended up crashing the game, but people kept chatting. So we then invested a lot in improving these kinds of social capabilities. This was before social networks and stuff. And I think there's a bunch of examples like that. I think Slack started as a game that failed. But again, they realized that there's this kind of whole communication thing they built internally for themselves. Flickr also started as something totally different, and people kept uploading photos. And I think, you know, stuff like that happens, but it's usually considered like a happy accident, or it's considered as an edge case. And I think we should accept that this is a normal part of product management.

Dave Farley: I wonder how often we don't listen to the signals and just miss those opportunities. I can remember years and years ago going into a shop that I used to go into regularly, to buy some mustard that I particularly liked. And I asked them if they got some and they said, "No, we stopped selling it because we kept selling out." It's just that kind of idea.

Passion for Building and Solving Problems

Dave Farley:I want to move on a little bit, we've got a few more minutes. But you're very wide ranging. You know, you're an author, a software developer, and an entrepreneur I suppose, as well as a kind of well-known speaker. So when you're talking about, you know, doing these things, what things do you get most pleasure from? Are you still a technology nerd at heart? Or is it these kinds of product discovery things that really focus you?

Gojko Adzic: I like building things. I feel a lot of pleasure from being immersed in a problem and solving a problem on a keyboard. And I think software is the closest thing to magic we have today. If you look at, you know, all the movies and books about medieval wizards, is they say some enchantment and then something happens in the real world. But look, you and I can type something on a keyboard, and something does happen in the real world. And it's magic. We turn pros, we turn words, we turn commands into something that people can, you know, upload photos to, and share photos, or create videos, or play games. It's amazing. And I think for me, like, that was always my passion. And I think all this other stuff is really supporting that. And I think... You know, I write books... I heard Henrik Nyberg, I'm gonna steal an idea from him. And he said, you know, he writes books because he only has a limited capacity in the brain for ideas. So when it fills up with something, he has to take a dump on a thumb drive, or a copy of something. So I write books to free up the RAM capacity for something else.

Recommended talk: Agile Product Design From the Trenches • Henrik Kniberg • GOTO 2022

Dave Farley: There's a number of things that I would very strongly sympathize with in that. I have literally answered the question in the past of, what would you have done in a previous life if you had not been a software developer or before software development. And I've answered wizard. Because it feels like that sometimes. And it's a bit grandiose, but it does feel like that sometimes. But it's kind of that creative act... One of the things that I talk about a lot is, as software developers, software professionals, it's not our job to write code, our job is to solve problems with software. And it's the problem solving that I get great pleasure from, great joy from. And certainly, in my new guise as a YouTube host, part of my pleasure is solving the problem of how to communicate sometimes complicated ideas as simply as I can. And it's a different kind of challenge. But it's certainly aligned...it fits into the same part of my brain as writing code, which is trying to find nice, simple solutions to whatever the code problem is.

Gojko Adzic: Absolutely. You have lots of aspects to that. It's all, you know, solving puzzles, and figuring out all these things that people get pleasure from. Plus you have this whole arranging the world neatly into modules, that is very pleasurable for me, where when you start kind of thinking about, well, you know, how does the world work? And I think, that's... You know, there's a lovely paper by Peter Naur, he talks that programming is mostly theory building. You build a theory about the world, and then you challenge that theory, and you kind of figure this thing out. And I think, again, you know, going back to lizard optimization, this is where you realize your theory is wrong. You know, because some people do not fit neatly into that, they're kind of statistical exceptions. But if you can then build a better theory that encompasses those people, then your product becomes better, and you start addressing a wider group. And I think, yes, this is... Solving puzzles, and figuring out where the puzzles are, and what is a puzzle, is quite interesting.

Dave Farley: That's the joy of it, really. And whether that's a puzzle in product design, or a puzzle in scalability of software, or performance, or whatever, the nature of the puzzle doesn't change the joy in solving the puzzle very much.

Gojko Adzic: Absolutely. Absolutely. A lot of what people do today in software unfortunately is putting some buttons in some screen that copy some data from one place to another. And it's just self-defeating for me.

Dave Farley: There's this part of software development that feels a bit like it's turned into building Lego kits rather than solving those problems. And that's not the part that really fascinates me, I must confess.

Solo vs. Collaborative Work

Gojko Adzic: One really kind of controversial thing I've learned from this is how much enjoyment I get working on my own, like, totally solo. I've been, like, a longtime proponent of everything kind of agile before agile was called agile, and pair-programming is a big part of it. And, you know, continuous integration is a big part of it. And I think my experience kind of building MindMup, where David and I kind of constantly pair programming and this thing where I work on my own, is that pair-programming absolutely gives you better software design. No question at all. Pair-programming gives you two brains. Pair-programming gives you a fallback when somebody is tired or not focused. Gets you kind of to challenge and provides …. Absolutely. And you can't tell yourself, I'm gonna be tired now, I don't want to do this. You know, when I'm working on my own, I feel a lot less productive because I can always say to myself, you know, let's just go for a run or take another coffee. Or I don't want to wake up in the morning, or something like that. So pair-programming, better design, definitely better software product, no issue at all. Solo work, much more enjoyable.

Dave Farley: That's one of those balancing things in terms of the... I definitely want to be doing pair-programming as part of a team, that's how I'd like to organize the work. But I'm with you, I'm doing stuff on my own, I feel a bit freer and a little bit more... Less like a professional, but more like an artist, maybe.

Gojko Adzic: You know, there's something in, like, having a problem and just kind of deeply immersing yourself in a problem for a couple of hours, not speaking to anybody, not having to explain anything. Just, like, trying to solve it from lots of different perspectives until you figure it out. And, you know, there's a joy in the journey, there's a joy in figuring it out where... You know, with pair-programming, I find myself constantly having to talk, explain, and it's kind of giving me these stopping points, where it's technically better codesign, no contest at all. But is it more frustrating? Is the other thing more enjoyable? Absolutely. And I think there's an interesting thing to kind of think about there, if we're looking for joy at work, how to balance these things. And I don't really have an answer to that. But it is a trick that needs to be figured out. You know, going back to continuous delivery that you're famous for, and continuous integration, and things like that. I mean, there's lots and lots of value, we see that there. But, you know, maybe there's value in having somebody being immersed in a problem for a month, and not thinking about what everybody else is doing, and whether this is going to kind of integrate nicely with it or not. 

Dave Farley: I certainly recognize that in other things. In software development, the most fun I've ever had was working in teams that did pair-programming, though, not working on my own. So I like working on my own. And I recognize what you're saying. Certainly, there have been times when I've written code that was just purely for my own entertainment, really. So I spent a lot of time early on in my software development explorations, playing with computer graphics. And I built a 3D modeling and rendering engine. And I was doing ray tracing and all sorts of things. This was years ago on 286 and 386 computers and stuff like that. So, a very long time ago. And that was very pleasurable, playing with all sorts of different ideas. Nobody depended on what I was doing. Nobody was using what I was doing apart from me. I was just generating pictures.

And that was wonderful. But the most pleasure that I've had, I think, is working as part of a high functioning team in pairs. 

Dave Farley: I was interested in what you were saying, though, because I was wondering whether you thought that transitioned to writing as well. Because I know that you've done a lot of writing along with David Evans. Do you find the same thing? Do you prefer writing alone, or do you prefer writing in pairs?

Gojko Adzic: I think joint writing, we were kind of writing stuff individually, and then trying to kind of, you know, mash it up together. So I don't know. I've never actually tried to kind of pair right. Maybe that's an interesting experiment. But yeah, then again, you know, I do find a lot of pleasure in working on my own when I'm writing. You know, I can fully recognize that the product is better. And, you know, there is an enjoyment in kind of building a good product as well efficiently and things like that. But there's something that I figured... You know, I didn't even recognize how much I missed it until I started working on my own. I lost that part of it, you know, and I rediscovered the joy of...just pure joy of building something. Which is interesting. And I don't know how to reconcile that with my advocacy for kind of pair programming, and continuous integration, and things like that.

Dave Farley: It's complicated. As ever, it's been an absolute pleasure. And we must meet soon for steak and wine.

Gojko Adzic: It was lovely to chat to you. Let's do that. Thank you.

Gojko Adzic: Bye-bye.