Expert Talk: What is Agile Sabotage?

Kevlin Henney, an independent consultant,and Fred George, an early adopter of OO & agile development, are exploring the ins and outs of agile transformation. They exchange views on what brings back the joy of programming while also offering companies a competitive advantage. They explore some of the frameworks for dealing with complex problems like Cynefin and agile development. They also talk about what went wrong with Microservices.

---

Intro

Kevlin Henney: Okay, hello. Good morning, good afternoon, good evening, wherever you are and whenever you are. My name's Kevlin Henney. I'm here at GOTO Amsterdam, 2022, which is a conference that has been postponed since 2020. Finally happening, and I am joined here by Fred George. Fred, tell us a bit about what you're up to.

Fred George: Well, I'm kind of doing my retirement work, which is, I go in there, make some troublesome places. A little bit of training, a little bit of development, make those transformations, and then move on to somebody else.

Kevlin Henney: I think that seems a fair summary. It's certainly how I've always remembered you, and it's nice to know that that's not changing. This morning, you actually gave a talk on sabotaging a transformation. Not with the intent of sabotaging, but understanding how these things happen. I guess you've seen a fair few and been a part of a fair few transformations. Transformation is a very nebulous word sometimes.

Fred George: Well, yes. Almost a quarter of a century now, just in the Agile space, which means a lot of opportunities to try things out, and a lot of opportunities to fail. What's been really disappointing, even surprisingly disappointing, is how many times the transformations haven't stuck.

Kevlin Henney: There's almost, like, an elastic band that kind of pulls a lot of organizations back to where they were. For some it sticks, for some, it doesn't. Now, how much of that is consultancy cynicism? How much of that is just that kind of, I don't know, organizational homeostasis?

Backslides of Agile transformations

Fred George: I'd like to think that... I always expected the backslide when you walk away because you're providing a lot of insight at many different levels to a client on how to get this thing done and what can possibly go wrong. It's just that you're not there in a constant day-to-day situation monitoring it, and basically interfering with the interference, to some degree, chauffeuring that interference back out. So, I expected, a bit of backsliding, but it's almost like I wanted to stretch the elastic band, but I didn't want it to be able to go back to the original. I wanted it to be stretched out a little bit so that some of that stuff sticks.

One of the things I'm finding enjoyable is that, particularly from a programmer's perspective, they're very happy in this style of working, that I'm leaving behind this joy of programming again. That's something that they've often lost. And to some degree, if the company tries to backslide all the way back to where they came from, these programmers will walk away. They've now tasted the forbidden fruit. They see what fun looks like, and they're not going back to sort of being told, "This is what you gotta do. This is where you do it. You sit in your cube and just get it done. Here's the schedule." They won't go back to that. Not the good ones.

Kevlin Henney: I think that's really interesting... Well, there are a couple of interesting points there, and I think particularly that idea of stretching beyond where they are now, with the understanding it'll probably relax back a bit, but not all the way. It's kind of like the tide coming in. Each wave takes it a little further up the beach. It rolls back a bit, but the next wave takes it further, and so on. But also that idea of showing people what good or fun looks like. I think that is something that we often undervalue. I think there are a lot of distractions that we have. I know that...this came out of a conversation with Steve Freeman a while back, where he sort of said, "A lot of people have never seen what a good project can look like or feel like. They've not felt that." So, it's very difficult when somebody gets up there and says, "You could do this." It feels abstract and remote, as opposed to something that's first-person. When you've had that, that's a very different story, isn't it?

Fred George: Yes. I sort of had that when I started working in Smalltalk and was hanging around with people that really understood how to use build objects. Basically, a very painful five years, as I was coming out of what was a couple of decades of waterfall thinking, trying to get into that way of thinking. But I was having more fun all the time. 

Then when Kent, Warren, and those guys unveiled this sort of Agile practices, it was just a back and front of a piece of paper. It wasn't even a book at that point. It was like, "Oh, my god. Everything makes sense that they're saying." And I go back and try it, and the fun level goes all the way to the top. Now I'm really enjoying what I'm doing, and why would everyone work differently?

The joy of programming

Kevlin Henney: I think that's an interesting point there about that idea of the fun. In other words, I think we get drowned or distracted by productivity and so on. Although that could be, you know, there are various metrics that tell us how we're doing, but being drawn towards them too strongly, or mistake them for the work, as it were, rather than the actual, the joy of the work. I think that's very similar to my understanding and my feeling when I look at Ward's work, and Kent's work. It's just like, "Oh. Well, this is completely different. This is quite enjoyable. That could be quite fun. That could be quite enjoyable." There's a different purpose to my work, and I think it's very easy to lose that, I think when people enter the industry.

Fred George: I sort of mentioned that programmers don't come to work today to write a bug. And yet it happens. Usually, it happens because of basic process breakdowns, not something they're intentionally doing. So, you want to think about it from that perspective. What gets them motivated? What makes them happy? And doing something productive every day. I mean, you're kind of after that psychological flow, you know. That sort of magic point, where they're focused on all these aspects. I think this process sort of brings that to the forefront, where you can sort of work in your favorite style at your pace, and you do not sort of dictating the pace of the rest of your colleagues or being held up by them either. You can work at your natural pace in these environments.

Kevlin Henney: It's a very personal way of looking at it, which I think is very interesting, very divorced from the very, I guess common perception of processes. But I guess there's another aspect to processes, which I think is interesting, and again comes back to transformations. If somebody says, "Hey, we need an Agile transformation," that's normally, sometimes it's a cover for something else. Then it's a lot of things that might actually mean. There are the words, Agile transformation, and then there's, "What's actually going on?" Maybe it's just a good old-fashioned reorganization for headcount reasons, or maybe there is some objective that has been not well articulated, but somebody says, "Well, they pinned their hopes on this idea of an Agile transformation." Maybe not everybody.

Fred George: I mean, I'm a little more cynical than that. I think you just can't get a job as a CTO unless you say you can do Agile. Now, whether it's true or not's irrelevant, but you're going to paint yourself as "I understand how to do Agile. Hire me to do this role." You can't say, "I don't believe in Agile." That closes the door completely. But by and large, they don't know how to do Agile. Not at the engineering practice level. I think at some of the management levels, a lot of this stuff clicks with them. The engineering practices are kind of foreign to them, but certainly, the management practices make a lot of sense, and executives can grab onto that. It's completely neglecting the engineering practices is what I find in most clients. They can hold the stand-ups, they can have the retrospectives, they can do the sort of ceremony associated with that. That's an important ceremony, but it doesn't really hit at the kernel of how to make the programmers more productive themselves.

Kevlin Henney: I guess it's kind of that idea that they see that and assume that that's all there is, as it were. It's very easy to perceive something, you know. "In the distance, there are the engineering practices, but here are the things I understand, so I focus on those," and we're drawn to those. But the actual…

Recommended talk: Expert Talk: Scaling Down Complexity in Software • James Lewis &; Kevlin Henney • GOTO 2022

Fred George: That creates the success model for things like Scrum Masters. They come in, people that are process experts, they can communicate that to executives, and they say, "Yes. I mean, everything you're saying I want to do, so let's bring you in." They don't realize they're neglecting the engineering practices in parallel.

Kevlin Henney: I think that there are a couple of things that come out of that, which I think is interesting. So, first of all, there's a point about job titles, and therefore employability, in that sense. There is a market for that. There's also another element there, and this morning you gave a kind of contrast between different stats. You know, I think it was, was it McKenzie? You know, basically an observation. Agile gives you kind of, like, a 13% edge. That's a very precise number, but it's still relatively small. 

Then you have somebody like Jeff Sutherland saying, "Yes, you can go hyperproductive. The factor of four." And there's clearly a disparity between these figures. 

What's happened is there's some kind of industry mean effect that is around the process expertise, as it were, where it stops there, and then there's something else. And it's that engineering practice, it's that team, it's that personal aspect that seems to be, you know, the magic sauce.

Fred George: Yes. You just can't sort of push that down from above. I mean, when you're a manager and you wanna make some of this stuff happen, you start putting rules in place, you start putting checkpoints in place. You're trying to manage the process to make it occur. Where you add an Agile coach, thinking you're gonna stir the pot with that, rather than almost walking down and talking to the programmers. I remember there was a lecture by one of the guys, the Ford executives, who basically was taught the sort of lean thinking back then. And the idea was, the executives should go down to the floor and listen to the people. Well, they go down to the floor and they start telling the people how to do it better. And the guy said, "No, no, no. You're supposed to listen, not talk. You know, it's the other way around." And I think we have the same phenomenon with our programmers. We're trying to tell them how to do it instead of turning around and listening to them, and sort of having that knowledge flow back, and then acting to their knowledge. Things we can do as executives that they can't do themselves, removing the barriers and the like.

Bottom-up transformations

Kevlin Henney: I think that there's a kind of recurring theme in your work and your talks, that certainly dates back to where I reckon I first saw you, was giving a talk at GOTO Aarhus in 2011, on programmer anarchy. And that idea of very much, it's just like, we're working from the ground up here. Here are the people that ultimately produce the software, and if our business is related to the software, then that's clearly a very important point. It's not about the layers that might come above it. It's grounded in that. And that seems to have kind of continued, and you've kind of expressed that through architecture. You've expressed that in a number of different ways.

Recommended talk: Expert Talk: Implementing Programmer Anarchy • Fred George • GOTO 2014

Fred George: Well, it's trying to find new and better ways to sort of get that focus. I mean, relabeling it sometimes, but also getting a little more clever about how do you overcome some of these barriers that people put up and in the way. Certainly getting the executives on board, what the effect of this change is gonna be. It's been one of the insights I saw, are you just really have to get the executives to understand the social change you're making to their organization. And most executives will welcome it. I mean, they're looking for people to make them more reactive, being able to move resources to where the problems are, and much more aggressively. These are the things that would broadly appeal to executives.

And they'll basically say, "Wow, I've been looking for somebody to tell me how to do this for years, and yet you say you can make this happen." And then you execute it. Now all of a sudden they're realizing, how to take advantage of this thing you're giving them. On the other hand, sometimes you have executives who are like, "Yeah, this is just another snake oil salesman. I've heard this story for the last 40 years. I'm not gonna believe it this time any more than I believed it the last 40 years." Those sorts of clients you sort of walk away from. They're not ready. You're basically gonna do something that's never gonna last. I wanna spend my time a little more productively.

Kevlin Henney: And there's another interesting thing there, is that you've also got executives, but in a large organization, there's a whole load of other roles that might act both as enthusiasts, but also people who are putting their feet on the brakes. And that's, I guess, where the idea of sabotage comes from. Sometimes it's explicit. Somebody's very much marking their territory, defending their territory, but sometimes it's a little passive-aggressive. It's not obvious.

Fred George: It's a little more obvious if you've seen it before.

Kevlin Henney: Yes, of course. I guess there's the experience element there, but yes.

Fred George: You can sort of see it, and they look like they think they're being subtle about it, but it's not subtle at all. I mean, they're clearly not buying into what we're trying to sell.

The Cynefin framework

Kevlin Henney: That's always an oversimplification, and then you throw people into the mix, which makes it inherently complex. And I noticed this morning you were also talking about the Cynefin model as a way of trying to understand the spaces in which the work happens, and this idea that a lot of what we're dealing with is kind of complex to chaotic. But many processes are actually in the, I guess, simple or obvious, and complicated space. Now, there's a mismatch there.

Fred George: Well, it's kind of a double mismatch. So, even if the problem you're trying to solve is a more complicated problem, more traditional, software development itself is complex. It's unpredictable. If it was already existing, we'd just use it off the shelf. So, every time you write some new code, you're innovating. Innovation just cannot be scheduled. And that's kind of the conflict between the idea of deadlines and processes that look, you know, nice on paper, but versus what can actually happen in reality. So, basically, you're continually managing a beast as you go through this process. You sort of need that explores, and sort of watch the results sort of capabilities. Typically I use metrics like continuous flow diagrams to reflect, "This is how fast this team is working today on this problem with these people."

And it could change tomorrow, but that's gonna be reflected in basically what we finished. So, rather than getting caught up in estimating when things are gonna happen, we just record what we've done so far. If you want to project a trend, you can go for it, but again, it's an unpredictable process. Fortunately, statistics kick in at a certain point, and once you get to a certain number of stories with a certain number of teams, you do hit sort of a very predictable flow. Now you begin to get comfortable that, "Here's when it can finish," or, "Here's where our problem's gonna be." You can discover that fairly quickly.

Kevlin Henney: So, it's kind of emergent, potentially, depending on the team and the nature of the work, you might find that something shifts from a complex to a more complicated kind of domain as things unfold. Or at least your perception of it shifts.

Fred George: At least the predictability of the overall project becomes sort of forefront because now we've done the key... Especially if you sort of do what Ken has always preached. You drag the risky stories to the beginning of the project, bleed the risk out of the project as soon as possible, then you get predictable. If you kind of do the easy stuff first and leave the hard stuff for later, hoping it will get easy, then you wind up being this green, green, green project that suddenly goes red and misses the date.

Kevlin Henney: That's an interesting way of looking at it. The idea of the volatile, risky elements. If you can address those, those will tell you much more about what you can do now, but also probably what will follow. This kind of interestingly suggests that there's a much more, I guess, dynamic approach to the kind of looking at how development will unfold. Probably more dynamic than many, many managers are prepared to take. In other words, people like a more static, predictable structure. Even if it's a work of fiction, it still gives a certain sense of comfort. Whereas what you're actually offering here is basically, "Oh, some of the predictability will emerge later." It's the idea of, like, "Let's keep doing this until such a point happens. When's that point gonna happen? I can't tell you yet. But we hit a critical mass," which is a very different kind of relationship. I don't know, it feels a little more... It is very grounded in innovation, but if we look at other disciplines, it feels more like sailing than it does like software development. Or rather, the product version, or production-oriented version of software development.

Fred George: I think that may be why I kind of enjoy the beginning of these projects like this. I kind of like living in the sort of fuzzy part of the world, the complex part of the world. That's what I enjoy. When it gets predictable, I get bored.

Kevlin Henney: But also, I guess when it gets predictable, is that the point at which potentially other, you know, the organizational, its older attitude might reassert itself? Is that where complacency might set in, or, you know, "We've done the changes..."

Fred George: Well, a little bit, because to a certain degree, if we've done a proper job of bleeding the risk out early, it gets a little more predictable. So, you can sort of see when things are probably gonna happen. But also, we've sort of overcome the initial inertia of working in a different style. The style has been established. And now my contribution becomes basically another good programmer. Not somebody who's, you know, basically building the right processes, and making sure we got the right team. Those sorts of aspects of my job is done. Now I'm just a valuable programmer.

A shift in leadership

Kevlin Henney: So, that also suggests there's a kind of, like, an interesting lifetime thing here. It's not just that processes are emergent and bound to their context, but they're also bound to their time, as indeed are particular roles.

Fred George: Oh, yes.

Kevlin Henney: And I think that's interesting.

Fred George: And I think leadership shifts accordingly as well. That's why I kind of always didn't like naming somebody as your tech lead. The person leading a project in the early part is gonna be more the application architect. The guy with the vision of what he wants to accomplish. These guys usually really suck at being able to get the code out the door at the end. You want to sort of have that shift in leadership accordingly. I'm not that good at getting the code out the door. I'm really good at getting the application architecture started, the process, getting the right people in the room, getting the right flow, get high satisfaction. And now it's a self-sustaining entity. Now the problem is I'm gonna be tempted to tweak it again, just to make it fun for me, and that's probably what I shouldn't be doing. I should be walking away at that point and letting the process go ahead and continue elegantly.

Kevlin Henney: So, that's, again, an interesting thing, is having affected particular change, then what is demanded of the people changes, yourself included. Therefore there's that idea of, like, "Okay, you know, maybe my work here is done, or I'm not the right person for this." But there's another aspect there about the emergence of roles, and this very much more dynamic kind of view. And I've had a personal experience where I was slightly surprised.  Years after a particular project, sitting in the pub, one guy said, "Oh, yeah. Yeah. You know, as our architect, Kevlin, you..." I'm, "No. Well, I was never an architect. I was an external consultant and I worked with you guys and I did a lot of code and stuff." And he said, "No, you were the architect." And it's just like, "Huh." If I'd been called that at the time, I might have walked out of the room. But it was rather interesting. That was clearly an emergent aspect. It's just like, "Oh, okay. With hindsight, perhaps I would accept that title," but it was a surprise to me. But perhaps that's a more normal thing. Titles emerge, or roles emerge, and then they subside later.

Fred George: And that's kind of what I wanna set up the environment. So, I've tried to make sure we don't designate that person ahead of time who's gonna be "the tech lead," because he's gonna be the guy that feels like he has to make the decision. I much rather try to train in a leadership role where the philosophy of a leader is to make sure that a responsible decision gets made at the right time, but feel bad if you had to make the decision.

How to handle impedance mismatch

Kevlin Henney: Yes. And back to that idea of mismatches. You talked about impedance mismatch during the talk, and I noted it down earlier as speed differential. In other words, different parts of an organization, but also how different roles might continue their previous practice. And then that collides. So, for example, somebody who's a traditional DBA might have a very different view of the relationship of an application to the database, and effectively they've become, without intending to be, a bottleneck. And likewise, UI and UX people, there's this idea of, I've always been surprised by it, I guess, you know, the UX process. And I've always wondered, what is this? And it's this thing that is very sequential, and very bottleneck-driven. And although filled with good intentions, that can actually upset, kind of, the rhythm, as it were, a different flow and rhythm that you're trying to get from development.

Fred George: Yes. And that transformation or that impedance mismatch is a source of high conflict in most organizations. I mean, the UX guy doesn't want to give you any input until he's had a chance to really make sure it's right. And of course, he's doing a fuzzy problem. "What is the sort right user interface for my users to do the following work activity?" It's naturally a very fuzzy, complex problem. He's not gonna come up with the right answer the first time, yet that's what he's been trained to think he's gonna do. And that, "When it comes up, it's right. And so you should do it this way." There's no question. You can't question it at this point.

Recommended talk: Understand the complexity of software systems and prevent over-engineering of the code • Hadi Hariri

Kevlin Henney: So also, we actually take a step back and actually go back to something you were talking about in terms of the kind of more personal approach. That's actually quite a lot to demand of somebody. I mean, I think we do it with a lot of our job titles, or highly specific roles. It's, regardless of whatever else somebody might bring to the table, when we tell somebody, "You are this person," and there's a kind of expectation, "You have a process for getting it right. Therefore that is the bit." It's a huge expectation and pressure on an individual to get right what is, as you say, a fuzzy problem. Something that is dynamic, and emerges probably out of the group, probably out of everybody not knowing the whole picture, trying to find the whole picture. But to ask one person to come up with the right architecture, the right UI model, the right...is a hell of a pressure to place on somebody.

Fred George: I think we couch it in various vague terms, like accountability. "We're gonna delegate responsibility. We're gonna make you accountable for these things." Which is just a fancy way of saying, "I wanna know who to blame when it goes wrong." And so, even that language has couched into it and creates fear. I mean, that's one of the things we try to avoid in these processes, which is creating fear. Because fear stifles innovation, it stifles the creativity of individuals. They're hesitant to do the right thing, because it may be wrong, and blame will flow from that. I mean, even GitHub calls it "Blame." It's gotta be the worst title associated with something that should be tagged something else. But, "Who do I blame for this frickin' mistake?" It's like, "Oh, this is not how you want to think about the organization."

Kevlin Henney: I think, particularly no matter what the irony or likeness that might have been intended by it, that gets washed away very, very quickly, and that word just stands for what it normally means.

Fred George: Yes.

Kevlin Henney: So, let's tie back to kind of the names and terms of things, and clarifications. I mean, one clarification I've already mentioned, is Cynefin, and mentioned the chaotic domain. I mean, it's worth clarifying, sometimes words have more than one meaning. By chaotic, it does not mean disordered. That's actually something separate. It's more mathematically chaotic. "Yes, it's deterministic. Yes, there's cause and effect, but no, we can't really tell what it is," is slightly different from "This is a living hell and it's disordered and everything." This also relates, interesting enough, to the term programmer anarchy. I mentioned it once. Some people might not be familiar with it. It's an idea, for some people, to use the term anarchy to represent something distinctly bad and negative, whereas, from a political perspective, anarchy has a much more precise definition. That is the one that, you know, you are talking about with programmer anarchy. And I think that's quite an important distinction and one that places us right back where we want to be.

Fred George: Yes. Basically, the first definition of anarchy is, basically a team who determines its own leadership. It organizes itself, and that's what we really meant by that. I mean, I was always sort of looking for a very controversial title to sort of convey the idea. Again, click-baiting, to some degree, with the term. And I took that from the heart from Kent Beck's Extreme Programming, which was, at the time, when I was trying to sell my consulting services, calling yourself an extreme programmer, it was kinda like, "Well, they're already scared of programmers. Extreme programmers are really scary. So why would you wanna hire an extreme programmer? Programmers are bad enough." And so it was a little hard to sell the concept, but it got the press, it got the clicks. I was looking for something that described our process. Now, we were working in strictly complex domains. Complex domains that had great feedback processes. You know, with Google Advertising, 20 minutes to get real feedback. And so you're basically in a complex domain with a great feedback cycle. I don't know what to try, but I wanna experiment like crazy. And that's what we were doing. We were working purely in those sorts of domains.

Kevlin Henney: So, it's decentralized.

Fred George: Because there are no experts. And if there are no experts, then there's nobody to tell you what to do and, "Did you do it well enough," and all these other things. The only real metric is success at the KPI level. "Did you get more sales? Did you get more clicks? Did you get more retention?" Whatever the magic metric is. So, I was fortunate to have that sort of domain to work with, and then watch the programmers do some very weird things. Like not rating unit tests, not pairing, doing some other things that were just kind of crazy except it was working, and trying to understand how that helped that. And certainly, the Cynefin model really helped explain why it was working for that sort of segment. But a lot of the things I try also work in complicated domains, but actually, the processes are radically different in that domain. So yes, we do still have stories in that, and we still have card walls, we have the stand-ups, we have some of the ceremonies around that stuff because it's useful for that sort of problem.

Responding to change

Kevlin Henney: Because that was the thing that I think I noticed initially with the term "programmer anarchy." I thought, "Hm. This is like XP." A polarizing term, very provocative, invites questions but also tells you that it's a particular model that's intentional. Although it is in one sense without structure, it is entirely intentional. You've just described the nature of the domain that that thrives in. And that idea of constant feedback also leads to another thing, which is the idea of the relationship of developers to the code. And the longevity of the code. I think that's another interesting one. How long does the code last? Now, a lot of developers find themselves on code that has outlived them so far. We actually see a generation of developers coming in now who are working on code and particularly we're talking Java code. The current wave of Java graduates, as it were, are now entering projects that are actually older than they are, ultimately, when we look back. But that's not the whole of the software landscape. The software landscape is hugely varied, and in other places, you've got code with incredibly short life cycles. And that's what, as I understood it, you were focusing on initially, is that idea of, like, "Actually we could throw this away."

Fred George: This was, again, for the complex domain, where experimentation is king. That competitive advantage accrues to the company that can experiment faster, and get faster feedback. Again, we were doing Google Advertising at the time, in London. Google was changing the rules all the time. This was constantly changing the rules, and they wouldn't tell you what the rules are. So it was kind of, again, you know, sort of a blind game you're playing with Google. And so, they would sort of make a change, "We're going from 3 levels of feedback to 10 levels of feedback." Well, some of our competitors will say, "Well, we'll fix that in our systems in the next quarter." We had it running the next week. Because we're having it run next week, we kill them for the next quarter. Of course, then Google changes the rules again, and again, we just dominate the market. It's all because of our reaction time. They were constantly... We could experiment way faster than they could.

Kevlin Henney: So, at the heart of that, there is a kind of a very strong sense of responding to change. And you've actually jettisoned everything that might stand in the way of that, at that kind of feedback level.

Fred George: Yes. You see that with day trading. I mean, the day trading houses always have had this. You take a couple of programmers, you put 'em next to a day trader, and they form their little club. They made...the programmer's not following the standards, they're not using their languages, they're not doing things according to the process of the institution, but they're making the trader happy, and he pays their bonuses. So, "I don't care what the IT guys say. I'm doing it this way." So you saw these clusters existing already. We're just saying that more domains are trading now than they used to be. All the really interesting problems you make real money at our complex problems. You get standard profit margins on complicated stuff, but there are extraordinary margins to be made if you solve complex problems.

Kevlin Henney: So, that's kind of a healthy kind of way of looking at the software landscape again, as to being filled with all these different kinds of software. But actually that the balance has shifted, that there was a, as it were, there's always been complex, but there was a predominant amount of complicated, but now actually where the action is, maybe the line has moved a little bit. More projects represent this highly dynamic kind of space.

Fred George: Yes. I would say we're still 90% in the more traditional spaces. But I would say at least half the profits are in the 10% at this point.

Kevlin Henney: Yes. So, I guess there are multiple metrics here. There's the number of developers, the number of lines of code, but then there's, "Where's the rate of change happening greatest, and where's the money to be made?" Again, looking at the space through different metrics will give you something very, very different, and therefore suggest very different processes.

So, that other aspect there, of small pieces of code, that idea of potentially throwing pieces of code away, because if you're working on this kind of life cycle, you're not really worried about the long-term future of amassing unmanaged technical debt, because actually all the debt is written off pretty much the moment that it's created. Which kind of leads to small things, and certainly from you that I first learned about microservices, and that world has gone in a very different direction over the last decade.

Recommended talk: It is not just Microservices • Fred George • GOTO 2016

Fred George: Well, certainly the term has gone in a different direction.

Kevlin Henney: Yes, yes.

Fred George: Because I think people have discovered that service-based architectures are the way to go. And I think people are developing services at appropriate levels, especially for traditional problems. They're not the really tiny ones you need in the experimentation world. They're the sort of much larger, more meaty pieces that you need for a business world, that's where the business is well-defined. So, when I was working for the Norwegian Welfare Association, we were implementing the law associated with unemployment benefits. Well, we wrote the law as a module. It's a big module, it's a complex module. And we built it with lots of small objects. You know, this sort of a Lego-style build, but it's still a monster service in terms of its complexity and its capabilities. But the good news is the law changes very slowly, and once you model the law correctly, this code is not gonna be under stress.

We wired that to an event bus, and then we wired another component very similar to that. It says, "Here's how the case workers wanna work on stuff." That has nothing to do with the law. It's the procedures that the case workers wanna follow. And again, a very stable business process. Built a service around that. Still hooked into an event bus. Then we had some microservices which are doing, you know, sort of adapter jobs. And I need to take your decisions and push 'em off to the data warehouse. I need to make sure a decision's happening in a timely fashion, so I have a little timer running, and this guy who's just looking at the timer and saying, "Oh, is it time to go tweak this again? Or something should be happening here." So we did some of those with microservices, but by and large, it was a service architecture.

Kevlin Henney: I guess what we're seeing is there's a lot of things that are being called microservices that are just good old-fashioned services, but because that's the branding that has kind of stuck, and they are definitely not micro, and it's a bit of an abuse of the term, but you're also tying it there to stability. That rate of change. The law is much more, kind of, you know, predictable, and well, I would say glacial, but glaciers are moving pretty fast these days. Geological in its timeframes, by comparison. But there's an interesting contrast there, because certainly, in kind of government-related work... Let's go back a few years. The historical approach to doing or addressing what you were just describing would be, "We need a unified identity entity model." Whereas what you've explicitly described is, "Well, there's this bit, there's this bit. These are kind of decoupled. Let's keep them that way, and let's organize it around the way things are done. What do they want to do with it?" Rather than the more challenging philosophical question, it really is a challenging philosophical question, what is the entity model for government and welfare? Which, that's almost an unsolvable problem if you try and make it consistent.

Fred George: Well, it's certainly a complex problem, and since it's a complex problem, it will defy definition. And so you're almost doomed to failure as soon as you start. You need to carve it into pieces that have that stability. Like the law, for example. I mean, one of the reasons we were rewriting this application is because they did want to change the law, but their software is unchangeable.

Kevlin Henney: Yeah. The software was actually less changeable than the law.

Fred George: Oh, very much so. I mean, it was  "I wanna change how this benefit works." "Well, I'm sorry. The system won't support it. You can't make the change." "But I'm the government. I should be able to make this change." And, "No, you can't make this change." So, that was one of our leading motivators for needing to rebuild it. But knowing that that was the stress point, we did build it in sort of this Lego style, where I have lots and lots of these little Legos running around, and if they wanna rearrange 'em a little bit, it's not a big deal. We can rearrange 'em quite easily for them.

Kevlin Henney: I think that we are at time, so thank you very much, Fred. I hope everybody's found that useful. I would also very strongly recommend watching Fred George's talk in its entirety. There are a lot of takeaway lessons that may be things you can apply immediately, or that you may find useful in the long term.