Simplicity & Complexity: The Beauty & the Beast?

In most cases, there is more than one way to solve a problem. Based on our evolution, however, we are tempted to solve problems by adding features, code, and complexity. There is an alternative to that, though. Kevlin Henney and Sander Hoogendoorn discuss these attempts in this GOTO Unscripted episode recorded at GOTO Amsterdam 2022. A good starting point is to reevaluate the way in which we do things and try to eliminate the complexity that software developers are attracted to.

Intro

Kevlin Henney: Good morning, good afternoon, good evening wherever you are and whenever you are. My name is Kevlin Henney and we are recording this interview at GOTO Amsterdam 2022, which has been delayed from 2020. But here we are at last and I'm joined by Sander Hoogendoorn. What is it you're doing these days? Tell us about yourself.

Sander Hoogendoorn: Oh, yeah. Well, lots of stuff actually. I'm really happy that we can get back on stages, right? That's really part of the joy in this industry. I'm the CTO for an eCommerce company actually. I'm trying to, well, move my team in directions that I think they should move. That includes both the architecture and the structure and how they collaborate and autonomy and how to write code and how to test and how to do TDD and DDD and all this stuff. And...

Kevlin Henney: So software development.

Sander Hoogendoorn: Yes.

Kevlin Henney: Everything.

Sander Hoogendoorn: Yes. And I spend a lot of time writing code, actually, so it's...then I truly enjoy that.

Kevlin Henney: That really came out. So, you've just done a keynote here, the Zen of Programming, and there's a lot of talks which have the "Zen of," but yours actually does actually work for that.

Recommended talk: The Zen of Programming • Sander Hoogendoorn • GOTO 2022

Sander Hoogendoorn: Oh, thank you.

Kevlin Henney: Because it was very much a personal journey. It was very much the consistent theme of simplification. Rather than being a list of, here's a bunch of individual coherent things that are specific pieces of vice principles or practice, whatever, is that it mixed a bit of that but they were lessons and it was genuinely a story. Out of it, it included everything from "Hey, monads," to, "Take your mother out more," but actually there was a properly resonant theme of simplification. Really, that came through, which I kind of, like, I have to say, that was very nice. And there was code, most of which you said, "Yeah, this code, I touched it yesterday." And that was very, very clear. But that constant journey towards simplicity is...you made some really interesting points about complexity to get to that. So that whole idea is the Neil Ford quote, that developers are...

Sander Hoogendoorn: I love that quote actually.

Complexity

Kevlin Henney: It's a fantastic quote. Developers are drawn to complexity like moths to a flame and often with the same result. And that's something you see a lot of. Why do we do this? Why do we do this to ourselves?

Sander Hoogendoorn: That is a good question, I've been wondering the same question actually for years, I suppose, and it's... I don't know, actually. It's like we all do this, right? We are constantly saying, "Oh, let's pick that up too, and let's do that, and let's try this too." Then it becomes bigger and bigger and bigger. Then somebody comes up with a new framework or a new technology or whatever, and they're like, "Oh, yeah, that would be a much better solution to do this," before even having delivered anything, right?

So, we tend to add stuff to what we do before we actually push it out to business or to whoever, right, which makes it more complex all the time. The problem is with complexity, you can never get rid of it. So, it actually started this idea about talking about making it simpler when I was doing consultancy for a large bank in the Netherlands. Well, I probably shouldn't mention the name, but they're here anyway. And a couple of years ago, by the way, and they had an 11-layer architecture.

Kevlin Henney: Oh, wow. Okay. That's a winner.

Sander Hoogendoorn: Made up entirely by architects who never code, right? And there was no way they could ever deliver any software, so they took, like, a small project and it took 300 people for two and a half years to actually deliver a small part of it. So, that's the kind of culture that a lot of companies come from, right?

And the second thing that also changes when I stopped working for large consultancies, which is about eight years ago. So, I worked for Capgemini and for Ordina, which is a Dutch consultancy firm, and CMG, which is now CGI, I think, and they all make stuff more complex because well that's where the money is basically, right? So, it's better to sell 300 people than sell 2.

Also when I moved away from that, I stopped working for larger organizations because they all have this culture of making everything more complex, and I wanted to go in the opposite direction basically. So I started working for companies that became smaller and smaller and smaller. And that is...I like that a lot more these days.

Kevlin Henney: There's an interesting thing there. There's two themes that seem to emerge there. One is the additive nature of how we try and solve things. It's always by addition, it's very rarely by removal.

Sander Hoogendoorn: Yeah, we never take anything out.

Kevlin Henney: You made an interesting point when you talk about improvement of skill, which is that there may be jumps in your improvement of skill, but often if you look closely, there is a dip before that. That what happens is that true learning doesn't come about purely by addition of knowledge. It comes about by reformation of knowledge. It comes about by unlearning and rewiring and throwing stuff out.

Sander Hoogendoorn: Having epiphanies. I like that word actually.

Kevlin Henney: Exactly. So, there's this kind of moment, there's ka-chunk, something happens and it's not always up. There's a down, we become less effective for a period.

Sander Hoogendoorn: That is because you have to try out stuff, right? It's the experimental phase.

Kevlin Henney: That feels like if we want to get simplicity, then we are gonna have to get better at the recognition that this is a learning exercise, and that means throwing stuff away.

Sander Hoogendoorn: That is exactly what organizations need to do, right? And they need to start understanding that learning is the...basically the core concept of how we improve, of how we...well, that makes sense, but how we get better at delivering software.

Kevlin Henney: Yes.

Recommended talk: Expert Talk: Scaling Down Complexity in Software • James Lewis & Kevlin Henney • GOTO 2022

Sander Hoogendoorn: And not by repeating ourselves. There's lots of people who are in the industry that just repeat what they do all the time, right? And they deliver the same thing that they delivered all the time. But that doesn't make you better. I think it's like...a lot of people say, "Oh, this is what I can do." Whether that's React or they can just build a functional component and whether it's doing Java, right? Lots of people in Java are building the same code in exactly the same way as they did, like, five years ago.

Kevlin Henney: Yes.

Sander Hoogendoorn: SpringOne, basically, that's okay. Everything Spring and Spring Boot and that's the way you write code. No discussion. And they like that actually, which is really good. Those are the people like my mom, right, who went to the same campsite for 25 years, and never questioned whether to go somewhere else. And that's also good because you need lots of those people in this industry as well. It's just not what I do. It's like, I like to discover new stuff. That's not about learning a new language or learning a new framework. I couldn't care less. But it's about making code that is slightly more beautiful, more elegant than it was yesterday. And the funny thing is, in the 44 years that I've been doing this, there's always a step up.

Simple code, simple life

Kevlin Henney: That's a really interesting one. Because there's that idea of the beauty of the code. In other words, that idea of coming back to it and there's kind of like, have I left something here that somebody else will appreciate, myself included, but other people won't curse my name when they come to this code?

Sander Hoogendoorn: They will.

Kevlin Henney: But it's that idea of like, maybe they'll curse me less, you know? But our idea of beauty changes over time. We know more, we get older, we develop technologies.

Sander Hoogendoorn: That's the same with everything in life, right? So, when I was 19 I used to date 19-year-old girls, right? And those were the best at those times. If I now look at my kids or friends of my kids, I'm like, I would never be able to date people like this anymore, right? I'm much more happier with someone who, well, is more mature or well-aged or whatever you call it, but we change our perspective all the time.

And also there, there's big steps, right? Well, there's these holes, right? I went to a really deep hole in 2009, 2010 when I got this huge burnout. Then I realized, well, it's the same. You add more stuff to your life and it's too complex. You cannot do it anymore, so you have to move back to simplicity. And I took a deep plunge and I sort of, well, stripped out everything that wasn't really important or fun.

Kevlin Henney: You mentioned that in your talk as well, the whole kind of prioritization, it's just like, "Okay, gotta get back to... What's all this other stuff? What are the things that I don't enjoy, that, you know, don't bring me joy, don't add value?"

Sander Hoogendoorn: And I literally ask myself those questions, like, well, do I really enjoy doing this? If not, I stop doing it. And that's... And I'm not sure a lot of people do that actually. And that's okay by the way, right? It's not worse or better, it's just what I do.

Kevlin Henney: But I also think that that plays out, that's a very...I guess most people don't do that, most of the time, and when we do do it, it's normally as a result of an event. It's just like, "Ah, maybe I need to reevaluate things." But I think that also goes to the nature of how we work, more broadly, not just personally. And I think that that's kind of an interesting one.

We are looking for...again, it's the additive thing. We add stuff rather than throwing things out, rather than asking ourselves, could we do this with less? And that's that self-fulfilling prophecy. You talked about large organizations, how do you create a large system? Well, if you work for a large organization then that will create a large system. A large organization is incapable of creating a small system.

Sander Hoogendoorn: Yes, they are.

Kevlin Henney: And I think that that's the interesting thing, is that it becomes a self-fulfilling prophecy, is that somebody says, "We're gonna have 300 developers on this." And you go, "Okay."

Sander Hoogendoorn: It becomes a goal basically. Right?

Kevlin Henney: The headcount becomes the goal. And somebody might estimate and say, "Oh, for this architecture, we might need it." Well, the point is that if you chose half the developers or a 10th of the developers, you could probably also deliver something more effective. If you get 300 developers to deliver Hello World, it will be quite amazingly complex, probably with 11 layers.

Sander Hoogendoorn: Yeah. And it will probably not work.

Kevlin Henney: But the important thing is you couldn't develop a small version of Hello World with 300 developers. It's almost impossible. So you are necessarily anchored at that scale, which is kind of interesting because you also talked about scaling. You made a couple of comments about scaling. And you…

Recommended talk: Architecting For Scale • Lee Atchison & Ken Gavranovic • GOTO 2021

Sander Hoogendoorn: Did I? I always do that.

Kevlin Henney: You kind of quoted Abraham Wolf and the observation pretty much that scaling is the number one problem that people don't have.

Sander Hoogendoorn: Yes.

Kevlin Henney: And we are in an obsession phase at the moment, and we see it with things like SAFe.

Sander Hoogendoorn: Yes, that's the scaling method, I] was talking about scalability of microservices in that quote. But it's the same for organizations, right?

Kevlin Henney: Yes and that's the thing, is that we get it. Again, it's the number of people who go like, microservices. It's just like, and what are you looking to do here? And I had this exchange with a friend in Norway a while back, and he was just like, "They're rewriting this architecture we developed in PHP and now they're doing it in scalable microservices. The point is, our business is the same size. The industry is the same size. Nothing has changed. Why do we need that?" He couldn't understand it. He said, "Kevlin, help me understand it." I couldn't.

Sander Hoogendoorn: No, you have to do stuff for the right reasons, right? So, that's why...basically the question I ask my teams, and now they're asking me actually is, so do you really need this? Like, with every solution that... One of the guys on my team came up with, "Oh, we might introduce Kafka here." And I'm like, "Okay, so what problem does it solve?" And he said, "Well, it becomes more scalable." "So aren't we scalable enough then?" He said, "Well, yeah, we are, but..." When I develop, even this morning in the standup this morning, one of developers on my team said, "I'm doing the tree shaking on our framework." And I'm like, "Okay, why?" And there's nothing wrong with tree shaking, especially when you're doing node, but it wasn't solving a problem.

Because the problem he would have solved with it is everything would've gotten a little bit faster, but it is already fast. Nobody in the business will see the difference, right? So, why would we spend our time while we have so many other stuff that we really need to take care of? And he was like, "Yeah, you're right." But my teams do it to me as well. Right?

I'm thinking, "Oh, yeah, maybe I should remove this flex box and put in another one, which is slightly better." Then they say to me, "But the other one works." "Yes, it works. But this one, it would be slightly better." I'm still doing it, by the way. So that was like 2:00 at night and I'm starting to write a new flexbox or whatever. But we all do this, right? We all do stuff for no apparent reason, just because we think it's better or it's better in the future or later we will reuse it, which we never do. It's become an industry inside of an industry, basically.

Kevlin Henney: I think that's a really important observation, is that particularly for those of us who end up dishing out advice or, yeah, it's always to recognize actually half the reason we dish this advice out is that we know something about ourselves. We are as guilty of this as anybody else.

Sander Hoogendoorn: Oh, yes. Very much.

The most important question

Kevlin Henney: But I think that's... But you also asked the most important question there, which is what problems does this solve? And I don't think we ask that enough, that is actually...and again, the architects who created the 11-layer system if somebody had stopped and said, what problem does that solve?

Sander Hoogendoorn: Yes. And then they could... So that's basically...so that's why I said, well, you should always be able to ask the newcomer questions, right? It's like, always ask questions like this. So, okay, what are we gonna solve with this? Why do we do this? What does it do? And people forget to ask those questions all the time. It's because they're afraid to because they are expected to know all that stuff, right? And especially when you ask simple questions, because how simple can a question be? Like, what does this do? What problem does this solution if we do this? And asking yourself those questions all the time it saves a lot of time, actually, I think.

Kevlin Henney: I think that point about the...tying back into learning, what you're describing there is this idea...I guess I refer to it as the idea of, like, software development is a process of inquiry. It's not a process of production. And you're actually quite strong about the idea of, like, "Hey, we don't do the production." Production is a word that doesn't...you know, productivity, to be precise, is not a word we should really apply. We can use it in certain cases, but actually, we use it too broadly. Productivity is not a helpful term.

Here it seems that that's very much the same kind of things that we are trying to... It's a process of inquiry and we're trying to learn stuff. There's a whole bunch of stuff we don't know. We don't know exactly...

Sander Hoogendoorn: Yes, stuff needs to emerge, actually.

Kevlin Henney: It needs to emerge. We don't know everything about the tools that we are using. We don't know everything about the problem domain, because if we did, then that meant we'd already built it, so what are we doing now? So therefore it's a process of inquiry, and inquiry requires questions. And I don't know if we value that enough.

Sander Hoogendoorn: And it's hard to estimate.

Kevlin Henney: Oh, yes. Because it's a process of inquiry. How long is this gonna take? I'll tell you when I'm done.

Sander Hoogendoorn: I had to explain to the board of commissioners for the company I work for, like, why is it so hard for the tech to do estimates. And I took the Cynefin framework and so the CEO said, "Well, they're not gonna appreciate your model." Right? Because I think, well, maybe they do, right? So, I explained, okay. If you're in the obvious or the clear context, there is the best practice. You applied it, done. Right? That's the stuff we can all do in this industry. Then there's the stuff that is in the complicated context where we have a bunch of good practices and we need to choose one. So, you do some analysis and you make it work, and then you build it. That's the stuff that you can plan, right? You can even do it in a waterfall style if you want to basically.

Then you move to the other side. Where it's the complex and chaotic zones where, well, practices at best emerge or are not present at all, meaning nobody's ever done that before, at least in our company. And I said, well, we are now there on the left side where we have to question ourselves. So how are we gonna solve it? We don't know. We're in the unknown unknowns. That I shouldn't have said, by the way, because now the whole company every time they're like, "So how long is it gonna take?" "Yeah, we don't know. We're in the unknown unknowns," for everything, basically. Right? They don't like to distinguish. So the thing is, so what I explained to him, I said, well, we are now here, right? We're sort of in the middle between chaotic and complex. We sort of started to make some sense of what we were doing.

Then slowly, the whole thing will turn towards being in complicated. Meaning when we've automated enough of the stuff that we now don't know what we don't know, it will slowly move towards the complicated zone, in which case we can estimate, right? So once we've got all the pipelines automated, we figure out how to do the performance on our market service landscape, how to deploy our apps, all this stuff that we need to take care of, once it's taken care of, we can actually start making some more speed because it moves to the complicated zone. Meaning we will be able to estimate stuff at some point in time. And we are working our way towards being in the complicated zone.

Recommended talk: Small Is Beautiful • Kevlin Henney • GOTO 2016

The nice thing about the model, I love Cynefin by the way, is that it also turns...it doesn't stop there, right? You move from complicated into the clear context, meaning, "Oh, yeah, I know how to do this. It's da, da, da, done." But at some point in time, and that is, in my mind, when you reach the innovator's dilemma, is that you pass over from clear into complex again because well, you have to do something new because what you did doesn't work anymore. So you have to reinvent yourself and then you move back into the chaotic zone. And that's basically the circle of this industry, or actually of companies in this industry and probably in most industries.

That's an interesting cycle because if you realize where you are in that cycle then you finally have some arguments to talk to management and say, "Sorry, we cannot estimate and this is why. " And actually, they understood. And I was a bit surprised about it, not because they're dumb, they're really extremely smart, but because managers in our industry and outside of the industry are used to looking at software development as if it is production. And like, why can't you estimate this? Well, because of the Cynefin framework and this is why we're there. And they're like, the manager said, "How hard can it be?"

Kevlin Henney: And you give them the answer.

Sander Hoogendoorn: Yes. And well, "It's hard because..." And yes. And they slowly get that, right?

Software journey vs the project

Kevlin Henney: It's about mental models, you know, is that a point, as you said, there are a lot of smart people in this industry in all different roles. It's not the question of intelligence, it's the question of the mental model that they bring to it. And we all bring mental models to things and that's the problem, is where those mental models...and our language as well, it betrays the models that.. we talk a lot about projects. I still do it, even though I've known that that is not the right word for many years. It's just like, actually most of what we're building is products, it's not a project. A project is something with a well-defined end state. Software development is mostly about not ending. It's about keeping on going.

Sander Hoogendoorn: It's a journey. That's why a roadmap worked much better. I actually persuaded the company that I now worked for, and I did with previous companies as well, to use the word roadmap instead of planning because it has a very different meaning basically. Well, it still looks pretty much the same. Right? But the roadmap is much more flexible in the sense that you can always change it, which we do.

Kevlin Henney: And it may have more than one route.

Sander Hoogendoorn: Yes. That is the whole point. And yeah. Right. And also that it's hard to estimate.

Kevlin Henney: And I think that you're offering people enough information. You're saying, here's what we know at the moment. Given what we know and what we don't know, here's what's possible and these are the paths.

Sander Hoogendoorn: Sort your priorities, please.

Kevlin Henney: Whereas planning is a word with a very different suggestion, set of suggestions, and likewise, project, well-defined end states. And whereas product, the goal of developing a product is to keep on developing the product. In an ideal universe, I mean, it becomes an infinite game, and the idea is to always be developing. In one sense, you're never truly done.

Sander Hoogendoorn: Which is true.

Kevlin Henney: Done means you retire the system.

Sander Hoogendoorn: There's one point when you are done that is once you read... So, I like the analogy of the innovators' dilemma. It says, well, you're building a product, it gets successful, you add more stuff to it, and eventually you go to a point where, well, adoption doesn't grow anymore, independent of how many new features you put into it. Right? And that's where possibly another competitor or somebody from outside of your industry comes in, uses newer technology, and goes into the same cycle, but has a head start where you are. So at that point, you need to either jump or die basically. You need to reinvent yourself. So that is basically where the gap is. The funny thing is that I usually...usually when companies ask me to join in is when they get to this particular point.

They don't always realize it. And then when I showed them the model and they're like, "Oh, fuck, we're doomed." Actually, my previous client actually died. They added so much. They invented the smart thermostat, like, 15 years ago. That was a great device basically, but they added stuff to it like weather information, and traffic information on your thermostat on the wall, really useful. They had all sorts of features that eventually they got to a point where they could literally predict when your washing machine was on. Right? It was really cool that they could actually do that, but nobody cared. Now people slowly do, but it was too early basically. And at that point, well, Google came out with their Nest thermostat and Dial came out with stuff and they were like, "Oh, fuck, they have this too with cheaper technology. We can't survive."

Kevlin Henney: So that's kind of an interesting one, it's also...there's an idea there that, again, going back to the cycle, there's a kind of like a...there's a wave motion as it were. You're going through periods of consolidation. This actually kind of reminds me, based on what we've just been talking about, Henry Petroski is a civil engineer and a historian, and he wrote this book, "To Engineer Is Human" which was published in the 1990s. Really interesting. He talks about the role of failure in successful design. He talks about this idea of industries, particularly engineering. He focuses very much on that there's an experimentation period, and then something doesn't work out and everybody, you find a period of consolidation, things become very normalized, and so on. Then there's the next, it's cyclic. It's just like, then we move on to the next thing that's exciting and experimental.

Sander Hoogendoorn: Which is pretty much what Weinberg describes, right? It's like the learning curve with the...

Kevlin Henney: Yes. Yes.

Recommended talk: Journey from Monolith to Microservices & DevOps • Aviran Mordo • GOTO 2016

Sander Hoogendoorn: Cool.

Kevlin Henney: There's this idea of, in one sense, I think a lot of people are looking for stasis. They're looking for the one answer or the one thing when, actually, it's a very dynamic thing. And therefore the kind of map that you need needs to demonstrate these different domains of complexity. But I think that that's...but that also probably tells us why there might be a little instinct in some people when they're adding stuff. Part of our complexity comes from additive things, but there's also that idea, and you talked about it as well, that idea of future-proofing or, "Maybe one day we'll do this." And again, not solving a problem we have today, but also not necessarily solving what we have tomorrow. But in our mind that could be a tomorrow and then we make it a commitment.

Sander Hoogendoorn: Yeah, yeah. A lot of people think like that actually. Most of us, I would say. But it makes it really, really hard to... So the problem is if you build something now for something that might happen, what happens if it doesn't happen? So, what are you gonna do with all this added complexity in your code? You're never gonna take it out again, and that's why you end up with 11 layers of architecture. So not putting it in, in the first place, yeah, it's just way more effective. But it's counterintuitive I would say, that people find that really scary and especially developers that we want to make everything more generic. Right? That's what we do. It's actually what I love the most is writing frameworks. It's like, there's nothing more enjoyable than writing a good framework actually, so I've been doing it for most of my life actually.

But you always have to, even doing that, always limit yourself to what is now necessary. So the open-source framework that my team and I are now building, which is an open-source TypeScript framework for building microservices, is actually we only add stuff after we've used it. It's not before, so only after. So, oh, we now build whatever...what did I do? Oh, I added something on the gateway class to Mongo earlier this week, and it could have been there a year ago, but we hadn't needed it. And now we see the need for it because, well, we see stuff in our code. We say, okay, what if it would add that in the framework, then we wouldn't have to write this code anymore. We wouldn't have to write any tests for it because the framework would already cover that. Okay, let's do that and then we'll add it to it, and then somebody from the team will add it. And it grows organically from stuff that you actually do.

Kevlin Henney: So it's empirically driven. This is a real thing as opposed to, you know, maybe, what if?

Sander Hoogendoorn: Yeah, there's no, what-if. There are also no IF statements in the code, actually, but…

A "what-if" world vs the alternative

Kevlin Henney: Oh, we'll come to that in a moment. There's an interesting thing again about that approach that you're describing, which I think is quite important in terms of how we drive the development. If we have the what-if approach, the danger of what-ifing is that the better your imagination is, the more what-ifs you have, and therefore the more complex code. You know, if you are a person of limited imagination, then it turns out...

Sander Hoogendoorn: Your world is easy. Yes.

Kevlin Henney: The world is...you might still get the what-if wrong, but there's gonna be a lot less what-ifery

Sander Hoogendoorn: That's a very nice observation actually. So, that would suggest that people would... It's actually, I think this is a very true observation.

Kevlin Henney: So the more imagination, the more complex it could be, the more generic or whatever it might be. 

Sander Hoogendoorn: It's probably coming from long, long ago, being able to, when we still lived in trees or whatever we did, right? Being able to go through more what-if scenarios would help you survive basically. That's a very basic instinct.

Kevlin Henney: I think that explains where this comes from, but it doesn't tell us that we need to, but our temptation is not... There's no harm in the what-if questions, the problem is we then commit them into a piece of code rather than say, you know, "That's really interesting. Let's hold that one and see if it happens. You know, let's..."

Sander Hoogendoorn: It's an interesting realization to figure out that it probably comes from, well, survival mode basically. And the more possible dangers you can, well, you can meet is helping you survive. So it's probably the people...well, really early on, it's probably the people who had the most imagination who survived longer.

Kevlin Henney: So there's an interesting thing there in terms of when we think about things that make a good developer is actually being able to think through lots of different scenarios. I'd be not surprised. So therefore it's a positively selected for trait in the industry, but it clearly has...it has a potential failure mode if we are not aware of it in that sense. We overcommit, "What if? What if? Well, let's put it in the code." I and I know, it's great to ask the question, but maybe...

I think that that's the interesting thing, is this idea, again, coming back to being able to ask the question. And there's a question that's being asked there, but we put it in the code without ever asking it, whereas what you were valuing in your talk was actually asking the question out loud before, you know, actually being able to ask that question is probably more...is far more important in the long term.

Sander Hoogendoorn: Also to ourselves, right, being able to ask those questions to other developers on the team. It's not only just business, but it's also just the developers on my team. And the nice thing is that if I would've been a manager then asking those questions get a very different connotation to it, right? But because I write the code myself, I can actually ask those questions and it's easier to, like, if you develop...well, the situation this morning where they were doing tree shaking while we actually don't need it, we'll probably need it at some point, probably, but we don't know, right? It's fast enough now.

But because I could have done this myself, I could also fall into the same trap and have done stuff that I really...which I do all the time by the way, and it's writing the code yourself that allows...well, it makes it possible for me to ask those questions to other developers because they know that I do the same things. Right? I make the same mistakes and...

Kevlin Henney: You are real and you have skin in the game in that sense. 

Sander Hoogendoorn: Yes. I guess. 

Monads

Kevlin Henney: So, okay, you've taken us into the code. Let's talk about the code. Because you mentioned basically if-less code. Let's talk about that. So paradigms. So you're working primarily in TypeScript.

Sander Hoogendoorn: Yes. I love TypeScript by the way.

Kevlin Henney: You shared a whole lot of TypeScript earlier on the slides. And so how would you characterize the kind of the style and nature of the code, at the compositional level, you know, at statement to the function level? How are you thinking about the code?

Sander Hoogendoorn: Well, first of all, it's all asynchronous, right? So it's their microservices, but also absolutely use React as well. So, most of the code we write is in a promise, which basically means you then, then and a catch and etc, right? So, that means that, well, a promise is a monad and then it becomes tough, right? Because you have to explain to people what a monad is, which is according to Douglas Crockford, impossible to do once you get it.

So, I would probably say that my style of coding has evolved into a highly monadic, call it functional, but pure functional code is a little bit harder to write, also to read actually. But it's moved away from, well, let's say more procedural code. I started thinking about it differently and that took place over the last probably seven to eight years.

And it started when I was the CTO for an insurance company and we did Java and we were writing basically the first microservices architecture here in the Netherlands, so late 2014 or early 2015. There was a guy on my team who actually came with the trial class and he got it from Scala, and Scala has a trial class in it. And he like, "What if we would port this to Java?" So lots of the developers are on the team. I had a big team, like 40 people, and they were very traditional Java developers, right? If you look at the code, you open it up and it's like, if, if, if, four, if, if, four, if, like 10 levels deep, right? And I'm like, how do you test this stuff? And they're like, yeah, we do integration testing. Yes, okay.

But there are probably lots of exceptions that you probably missed along the way, right? So what if you don't do that? And then he came up with a trial class and we ported it to Java. And in the first couple of weeks that I looked at it, I had no idea what it was, right? It was so very much different from how I was used to writing code, and we started out by saying, okay, what if we put this in a try? Okay, we'll put it in the try, but there are no try-catch blocks anymore. Okay. And then what if we nested throughout the layers in the architecture? So meaning we pass the tries all the way up and down again, and then only in the end do we evaluate what happens. And the code became so much more, well, shorter basically, and so much more robust because you take away a lot of these side effects.

So for people who don't know monads, monads take away side effects. And the try-catch block is one of them. The code became very, very robust actually. So I took that along to other teams and other teams find it hard to grasp first, especially if you, well, if you're a newer developer basically, it's a harder concept to get. So the code is...the first time you see it, it's really different, and people find that really hard, but once you get it, you actually never go back. So I would never write code anymore without having all this monadic stuff in it and all the lambdas and stuff. And it's a very different way of programming.

I start all my functions by writing the return word first, right? I'm thinking, "Okay, what do I need to return and how do I get there?" And reasoning backward instead of reasoning, let's do all this stuff, and eventually this comes out. And it's a very, very different style of programming.

Kevlin Henney: So that was something also that you said in your talk, I can come back to that in a moment, the starting with the returns because I found that was a very useful way of integrating something I'd already understood, but I had not thought of making explicit. And now that's what I'm absolutely gonna tell people.

But that other idea of the kind of the monadic moment, monadic style, I think there's an interesting shift there, and one of them is to do with the thing that you're abstracting. You know, it's like if you learn a procedural language, one of the first things that you have to learn to move out, you learn to do procedural abstraction. You know, you just, like, I can pull out procedures, although not everybody does that. I do remember the time I had to teach a Fortran programmer, I was teaching them C and I said, "Oh, yeah, functions in C," and I said, you know, "the concept is like, you know, subroutines and functions in Fortran," which I knew, and I was using that as a kind of the point of engagement.

They said I don't think I've ever written one of those. It's like, they used to write everything in just one big routine, so they'd never really started. So for them, the big boundary was actually procedural abstraction was the first that they had not yet decomposed. Then we start hitting data approaches and start talking about our data and, you know, object orientation is a data abstraction technique. 

In other words, we have procedural abstraction, we have data abstraction, and then we start moving into other spaces, and you've got control flow abstraction, which is effectively what you're describing. Is full behavior, but not necessarily data-driven...it's not driven by the data, it's driven by something else. And perhaps that's a little less visible to people that, you know, that's why they find it disorienting.

Recommended talk: Functional Programming in 40 Minutes • Russ Olsen • GOTO 2018

Sander Hoogendoorn: Oh, yes. Likely.

Kevlin Henney: Yeah. It's just like you're actually offering a different...it's not just a different construct, it's actually a different abstraction universe.

Sander Hoogendoorn: It's a different way of thinking. It is.

Kevlin Henney: I can understand that's disorienting, but yeah, it's quite powerful that way.

Sander Hoogendoorn: It is. It is extremely powerful actually. But it is disorienting, that's true. And when I first met with it in like 2015 and I didn't realize actually the full possibilities of doing this. We only had to trial class, right? And despite the fact that Java has the optional thing in it, right, which is a, well, it's basically a poor man's monad, but it doesn't do much, and they could go so much further with it. And I did a talk at a Java conference in...when was it, November, and I'm like, you guys have lambda for, like, what is it, a decade now or something?

Kevlin Henney: 2014, Java it was 2014, yes.

Sander Hoogendoorn: You should be able to write much more monadic code. And people are like, yeah, yeah, but that's not how you write Java. And I'm like, yeah, well, you can. So yes it is. They're like, no, no, no, we write it like this. And I'm like, okay. Those are the people that go to the same campsite for 25 years, right? It's like, there's nothing wrong with it, it's just yeah, well, you can do so much more with it. And it's even like well-known Java developers had these discussions on, let's say, on Twitter or something, or like, they're like, "Yeah, why don't you solve this problem? Try some monadic stuff to it instead of writing all these stupid if-statements." And they're like, "No, no, no, but this is how we write Java." Okay.

Kevlin Henney: This is how we've always done it.

Sander Hoogendoorn: Yeah. Right. Exactly that. So that's why Java's becoming the new COBOL basically.

Kevlin Henney: Speaking of which, actually, the wonderful Grace Hopper who was responsible for the invention of COBOL, she did actually have this wonderful quote, which was, people hate change. People are fond of always saying, we've always done it this way. And she said that's why I have a clock on my wall that goes counterclockwise.

We don't always have to do it this way. This is the whole point. And I think that that is, again, it's that kind of incomplete knowledge. It's that notion of going around a cycle that rings that.

Return statements

Kevlin Henney: So in closing, just... Actually, I do wanna pick up on that point about the return statements. Because I think that's quite an interesting one because I've seen that with, as it were, I found it a helpful technique sometimes with test writing is writing, not the return, but writing your assertions and then working back. In other words, what's the journey that got me to this point? And the same actually applies to return statements and actually it makes it simpler. Something I've found, people have been, in the last decade or so, making their control flow far more complex, having gone through the structured programming revolution and understood why it was actually important, I'm finding people are doing a lot of unstructured programming. You're like, this is really imperative programming. It's very, very, it's not quite an assembler, but kind of object-oriented assembler.

They got a very, very messy control flow. And it's just like, actually, what do you wanna return from this? "Oh, we're gonna do this, do this, do this." Which you talked about, you mentioned it in your talk, it's a very procedural approach. "This is what I'm doing," as opposed to, "What do I wanna receive?" Let's turn the question around. And that gives you a much simpler thing. There's a flow there, it will come more naturally.

Sander Hoogendoorn: Yes. It's like you turn around a funnel, right?

Kevlin Henney: Yeah. Yeah, and I think that drawing out why this abstraction exists, therefore, what do I want from it? Let's start with that question rather than, what am I doing? Well, we can organize that into the flow, and if we can't, we've just learned something. You know, I think that's a very, you know, again, that comes from a questioning approach. It allows us to reshape based on questions.

Sander Hoogendoorn: I guess that's basically what I do, right? If you would summarize it, is I ask questions? Whatever I'm doing, also what I'm doing myself, right? Also, asking the question, is this what I want to do? Is this what I find important? Right? Like, yeah. And I think I do that with most of the stuff in my life, just wondering, why am I doing this? Am I doing this because I like to, or...and I actually narrow down the things I do, the way I spend my time because there's always more that you can do, not just with the cover but with everything that you have time for. Right? So what do you wanna spend your time on? And I don't have to plan that. I take that, like, in every day. Like, well, a simple example, there's a speakers' dinner tonight, I didn't register for it because on Mondays I normally have my kids.

And then for some reason, actually my mother-in-law died last week. So my kids are now with their mom as they should be, which all of a sudden opens the opportunity, do I want to go to the speakers' dinner? And I'm like, yeah, why not? And it's a matter of, how do you spend your time. So I ask myself, what would I do tonight if I wouldn't go to the speaker, I'd probably go to the cinema or have a beer somewhere on a terrace here in the city, but having a speaker's dinner is also fun, so let's do that. Right? And it's like...or I try to spend as much time as I can with my mother. She's 83, she's in a wheelchair, and she has one leg. So, she gave me a lot of attention throughout my life. So, I think she deserves to get my priority, right? So I do. Not always, but I try to do that. Right?

So, there's a very limited amount of things that I love to spend time on, but they're very deliberate choices. I'll give you one more example. So we were in Belfast, as you know, right? Like a couple of weeks ago.

Kevlin Henney: Yes, indeed, where you also learned...you learned something.

Sander Hoogendoorn: Yes. As a big Led Zeppelin fan, I should have known, right? But I didn't know that until you mentioned it, like the Giant's Causeway is the "House of the Holy" cover. So we figured out when we were in Belfast, there was a performance of the Holy Grail with, so I don't know, it's called a Folio, I think, right? Where you make the noises yourself. So it was in a church, and so I raised my kids with Monty Python, as you should, and coincidentally all three of them were there. And I said we are going to go to the show, right?

So we phoned up and they were fully booked. Then we said, well, we came specifically from the Netherlands to go to the show. And they were like, "Oh, yeah, we'll pull up three chairs and we'll...five chairs." And so we went there and there were people coming in with, what's it called? A habit in English as well as what monks wear and stuff like wooden boards to hit themselves in the head. So I had a tremendous laugh. And then I went out and, and the woman from the show said, "Well, this must have concluded your upbringing of your kids." I say, "Yes, I'm now fully." So we're traveling and we're seeing Monty Python in a church in Belfast, which is done. I'm complete.

Kevlin Henney: That's brilliant. So there is nothing that I can say that will follow that, except thank you very much. That's a perfect ending. Thank you very much, Sander.

Sander Hoogendoorn: You're welcome.