Building Evolutionary Architectures

The software development ecosystem is constantly changing, providing a constant stream of new tools, frameworks, techniques, and paradigms. Over the past few years, incremental developments in core engineering practices for software development have created the foundations for rethinking how architecture changes over time, along with ways to protect important architectural characteristics as it evolves. This practical guide ties those parts together with a new way to think about architecture and time. 

Book description by O'Reilly.

Join two of the authors Neal Ford, Rebecca Parsons as they are going through key topics from the book together with James Lewis.

Intro

James Lewis: Good morning, good afternoon, or good evening, wherever you are and whatever you are, or whoever you are, I guess around the world somewhere. Welcome to another edition of GOTO BookClub. This time, it's me hosting, James Lewis. I'm delighted to be joined this time around by Dr. Rebecca Parsons, and Neal Ford Ford, who are co-authors, along with Patrick Kua of "Building Evolutionary Architectures," of which there is a second edition of that's hot off the press, which is, it's actually a great read. So I guess we'll jump straight in. Rebecca Parsons, and Neal Ford, perhaps you could introduce yourselves, and tell us a bit about what you do and what interests you at that moment.

Rebecca Parsons: I'm the Chief Technology Officer for ThoughtWorks. And I actually joined in December 1999. So I'm one of the few people with this kind of color hair that got to party on December 31, 1999, because I wasn't mission-critical on anything. I've had a long-standing interest in evolutionary computation. And, in fact, I got involved with that when I was doing a postdoc at Los Alamos National Laboratory in Los Alamos in New Mexico in the U.S., studying genetic algorithms, specifically, at that point applied to the Human Genome Project of all things.

But when I started looking at architecture several years later after joining ThoughtWorks, I felt that this evolutionary metaphor was actually a quite powerful one when you think about this in the context of architecture. I remember listening to Neal Ford give one of his talks, and he called it "emergent architecture." And we had a very robust discussion about why that was a really bad name. And obviously, I won that particular debate, since we're calling it evolutionary architectures now. I also am definitely a programming languages geek, and so my primary research areas are either in programming languages, parallel and distributed systems, or in evolutionary computation in artificial intelligence.

James Lewis: That's all the easy stuff then, I guess. How about you Neal Ford? Welcome.

Neal Ford: Hi, everyone. My name is Neal Ford, I've been at ThoughtWorks not quite as long. I was actually a consultant in 1999 but did not have to worry about anything crashing. I just watched the news to make sure the world was still going to be all right. I joined ThoughtWorks in 2004.

Now, it's probably about 12 years ago, Rebecca first told me about this idea of applying evolutionary computing fitness functions as a way of doing software governance and in particular, automating software governance. And it really caught my attention because it was such an innovative approach. It's what I call a metaphorical mashup when you take two disjointed subject areas, but then you mash them up, and you get a new, interesting result. So I thought that was fascinating. That's when we decided to eventually write a book about it after we experimented with this on a bunch of different client projects and a bunch of different scenarios.

What is evolutionary architecture?

James Lewis: It's super interesting for me going back, like the historical aspect of it, because I remember emergent design being a huge thing back in the day, specifically, when it came to how you drive out the design of your software by like text. Was that ever an inspiration? Was that part of the inspiration behind this?

Neal Ford: Well, that was the inspiration for the talk that Rebecca set me straight on. Because it started out being emergent design in architecture. That was there were only a couple of iterations of that until Rebecca saw it, fortunately, because then it changed to evolutionary architecture and emergent design. And this is actually a question we get a lot in the architecture world. It's like, well, we're taught in the design world, that you shouldn't design a lorry, you should start with a roller skate, and then turn it into a bicycle, and then a car, and then a truck, and then finally a lorry.

But you can't do that for architecture because you can't take the same supporting elements for roller skate and just sort of incrementally turn it into something for a lorry. At some point, you have to start over because of stress, because of physics, and because of real-world constraints on the kinds of scale and other things that occur and that's why you can't evolve it over time. And that's exactly this idea of evolutionary architecture. You have to start with something but now the real trick is, how do you take that something you started with and then change it over time as you need to? That's really the mechanics part of what this fitness function idea got to.

James Lewis: I guess that sort of brings me to what is in evolutionary architecture. Because it feels like it could be many different things but maybe Rebecca can explain what you mean in the book by evolutionary architecture, maybe that's a good starting point.

Rebecca Parsons: Well, we define evolutionary architecture as it supports guided incremental change across multiple dimensions, and those three words, or three phrases, are important. Guided is where the fitness functions come in. Because if you think about design, good code is good code, bad code is bad code. It doesn't matter if it's retail if it's embedded, if it's healthcare, good code is good code and bad code is bad code. But there is no such rule about what constitutes a good architecture. And what we do through these fitness functions is say for this particular system, for my organization, these are the architectural characteristics that will drive this architecture to success. These are the ones that matter.

One example I often use is one of the clients I worked on early, we were in central London, and one of their perks was a sandwich ordering system. Well, they didn't take credit cards, there was no personal information, and there were only a few hundred people in the building. So having a highly secure, highly scalable architecture for the sandwich ordering system would be silly. And so the guided part is where we say, for the system, these are the architectural characteristics that I need. These are the ones that are important. And those are the ones that we use in our architectural decision-making.

The next is incremental change, where we are constantly looking at what is the current landscape. What are the current expectations? What's the technology landscape doing? And trying to say, are there ways that I can continue to either maintain this architectural level or enhance it? And then multiple dimensions is there's just a whole lot of abilities out there. You can't optimize for all of them because many of them are mutually exclusive. And so you're going to pick and choose. What we're trying to do is introduce evolvability as one of those first-class facilities. And it may be that you don't care about evolvability in this context. But from what we've seen, if you're in a large-scale enterprise with a complex IT estate, evolvability is something that does matter because your ability to respond to change, introduce new products and services, it's crucial that you be able to evolve, not just your code, but your architecture.

Neal Ford: That's actually one of the big changes from the first edition of the book to the second edition of the book because we wanted to focus... In both books, we focused on the mechanics of evolution, how you govern changes through fitness functions, and also this architectural structure that Rebecca was talking about. How do you build architectures where the structure supports this kind of incremental change over time? And in the first book, we did sort of a survey of different architectural styles and evaluated them, sort of as a head-to-head comparison. In the second edition, we felt like that wasn't really capturing because at the heart of the reason why something is evolving or not is how much you've let implementation coupling leak.

An extreme example of that is I worked with a client at one point that had exposed their relational tables to their integration architecture that anybody can just wire into the database directly through their architecture. Then they were shocked that a single change in a table somewhere can wreck their entire ecosystem. And it's like, well, that's the definition of brittle architecture. But that's not an evolvable architecture, because now any change breaks everything. So the more you can limit the blast radius of change, that's really that domain-driven design is about and bounded context. So in many ways, that translated into architectural terms, that's protecting the ability to evolve structurally because you're not coupling to implementation details, and you can create boundaries around things to allow you to evolve more gracefully.

Recommended talk: Is Domain-Driven Design Overrated? • Stefan Tilkov • GOTO 2021

James Lewis: It reminds me of a client some years ago, who shall remain nameless obviously. They were in the airline business and they had this wonderful problem that they suffered from where they procured that at the point a new shiny SQL Server database and set of stored procedures on top that made up their SI operations. I know they're developed by a third party, and then the internet happened and they thought, wow, wouldn't it be cool to sell tickets directly to the public? That would be an amazing thing to do. So why don't we just plug the internet directly into the SI database and pull it like it does everything now? It's really, really simple.

Of course, then something interesting, as a volcano happened in Iceland a number of years ago. What happened was they had a bunch of people who were frantically checking to see whether their flight was canceled, by pressing F4 and pressing F4 the database was getting slower and slower and slower until eventually none of the planes could take off because there were no passengers because they lost their SI infrastructure because of people pressing F4. And I think that's a great example of that kind of implementation coupling.

What is connascence?

James Lewis: Then there was one other thing. Maybe this is getting a bit too into the details because, if there is any such thing I guess, there was something that came up in the book, which I've not come across before, I'm pretty well-read and I've been around for a very long time, that really intrigued me, I really learned something about this idea of connascence. Where does this come from? This idea of being able to more deeply describe the different types of coupling within systems, feels like that's a missing piece that we've not been talking about and should have been maybe.

Neal Ford: It's fascinating because it's from a book that was published in 1993. I got a copy out of my bookshelf back here because it's a little bit misreported online, but I think it's a three-part name and I apologize to the author, I can't remember the author's name now. But the name of the book is "What Every Programmer Should Know about Object-oriented Design," which sounds really promising as a book title. But it turns out that about half the book is an object-oriented analysis design process. So if you remember, in the early '90s, everybody was building an object-oriented analysis and design process. In fact, that culminated in the Three Amigos joining and creating UML, because there were so many of them. So he was one of the competitors who did not become one of the Three Amigos. And so that part of the book is a big part of the book and nobody ever uses that anymore.

But the fascinating part of that book is his definition of connascence, which is really a vocabulary for describing coupling. And just like design patterns, you can go in and say, well, what we need here is an object that you can only ever create a single instance of. Well, it's a lot easier to come in and say, "We need a singleton there." The exact same is true for a description of coupling, you can come and say, "Oh, you should not have that magic consonant inside that method, you should extract that into a constant, or do the injection," or you could say, "Oh, you have connascence of meaning there, you should move that to connascence of name." And so it's just a more compact language for describing how coupling works.

But the other really fascinating thing about this is, so 1993, the very early days of distributed computing, one of the characteristics of connascence he talks about is the scope. And one of the observations he makes, the worst kind of connascence is dynamic connascence. It's the connascence by identity, which is basically transactional coupling, what we call it now. And one of the observations he made was, the more connascence you have, the tighter the scope it should be because the more you let it stretch, the more damage it does to your architecture. He said that in 1993, and nobody understood what he was talking about. And then Eric Evans came and introduced the idea of a bounded context and everybody got it at that point. But that's really a thread that has been through the software development world for now, a couple of almost three decades of realizing how to control coupling in a distributed architecture because that's the thing that prevents it from being able to change in an easy and effective manner.

James Lewis: What was the old quote about object orientation? It means three things. It's extremely late binding and all the things message passing and encapsulation. If you're doing that, it's object orientation. That was okay, was it? 

Recommended talk: Software Architecture: The Hard Parts • Neal Ford & Mark Richards • GOTO 2023

From genetic algorithms to evolutionary architectures

James Lewis: Rebecca going back a little bit to what we were talking about, I guess, a few minutes ago, about the history of this. You mentioned briefly, your inspiration behind some of the ideas and I've got this lovely picture in my head of kind of evolutionary trees where you have almost like this idea of a system, it can go in one way or another and reductio ad absurdum you're going to have an elephant and the other direction you're going to have some kind of sea anemone. How much was the actual sub-biological inspiration there for you? Or was it always to do with the genetic algorithms and that kind of stuff?

Rebecca Parsons: It was really more the genetic algorithms. One of the early things we talked about is we're not looking at breeding dogs. Okay, we're going to take this architecture and this architecture and try to get one that looks like this. But the key really was to find out using fitness functions, this concept as this unifying metaphor for talking about very different things, performance, security requirements, observability requirements, logging requirements, coupling, code quality, all of these different things.

There's been some real innovation in some of the tools, for example, that you can use to encode architectural constraints in something that looks very much like a unit test. That's a fitness function. Does my software architecture respect my layering? Or am I using the properly wrapped version of this library that has a vulnerability so that the wrapper actually solves the vulnerability? By using that same metaphor, it allows us to talk about what are the costs inherent in meeting a particular fitness function. And what are the benefits that we can get?

Rather than having security come in and say, "Here are all my security requirements, and here are the DevOps requirements, and here are these requirements and that requirement," it's like, okay, let's talk about this because we can't do everything. And so let's identify the things that are most important, but that fitness function, wouldn't... When you look at it from an optimization perspective, what you're trying to do is navigate a space, and you're using these fitness functions, and that to navigate this space to find a solution that best represents your competing objectives. And so it really did come more from the optimization realm, as opposed to the actual biology realm.

Neal Ford: Well, and trust us, we were tempted. We had a lot of conversations about how far can we push this metaphor. Because we're writers, we love to torture metaphors until they just scream, but we realized, and exactly Rebecca's point, the mechanisms we talk about are much more like the forces that cause evolution to happen. So tulips can't be infinitely scalable because, at some point, they'll run out of ground or air or water or something. So that's the force that is causing evolution to be constrained. That's kind of what we're addressing with fitness functions are more the evolutionary forces than the actual physical structural change generated by evolution.

Why should you use fitness functions?

James Lewis: One thing. Okay, maybe I'm being cynical playing devil's advocate to a certain extent here. But I mean, you could probably say that we've been doing this for a long time. This is just what I do. Is there an element of that to it or is that, or what are we getting extra from this idea of fitness functions that we're describing?

Rebecca Parsons: Well, I think it's a mindset shift. When I first started talking about evolutionary architecture, people would come up to the stage and whisper, "Don't you think you're being professionally irresponsible to talk about evolving an architecture?" It is the rock, the foundation. People really looked at architecture as being something that, I mean, it's even defined by some people as those things that are hard to change. But it's not practical. And so the mindset shift is accepting, much like in Agile software development, sure, if you could perfectly nail all of the requirements, you probably don't need Agile.

Someone asked me once, is there any application for which you would never use pure Agile? And I said, possibly a weather simulation because the laws of physics, don't change very often. Occasionally, we learn something new but basically, you know what those requirements are. But that's not the way it is in our technology landscape now. It is changing so rapidly and so you have to start with the mindset, this stuff is going to have to change and so what are the things that in the face of that change, I want to make sure I don't lose? I need this level of scalability, I need this level of encryption, I need this level of data privacy.

Okay, now, as I change things around, am I actually maintaining those characteristics that I say are crucial to the success of my system? And so it is that mindset. And although some people might have thought of it from that context, people used to put 5- and 10-year technology roadmaps on their walls. They weren't thinking about it changing next Tuesday. So it really is that mindset shift.

James Lewis: I find it funny. Sorry, Neal, please go ahead.

Neal Ford: I was going to say, I think Rebecca is exactly right in the unification of treating these seemingly disparate things because in every organization security is super important. But internal code quality might not be as important, but if they're both in the family of architectural governance because they are both things that can be governed by a fitness function, that gives you more of an apples-to-apples comparison between the seemingly disparate things. It's like, no, these are really aspects of our architectural governance that we need to prioritize appropriately.

How to avoid doing too much work upfront?

James Lewis: I mean, that actually brings me to another question I have. In the book, obviously, that's the mechanics of doing this as well, right? How to go about identifying when and that kind of stuff. I mean, is there a danger here that we end up in the same sort of situation, there's the Welsh coming out there, in the same sort of place where we do big up-front architecture planning? I remember going to one client and saying, "Have you thought about principles and architecture principles and those kinds of things?" They said, "Yeah, we've got loads of them, we've got all of them," and they have this amazing spreadsheet, which if he wrote it up on index cards, would cover all the walls of a room, completely incomprehensible that people wouldn't be able to actually make any decisions based on those principles, on those characteristics. Do you have any advice about where to start? How to avoid doing too much upfront, if you like, and sort of avoiding the big architectural framework problems that we've seen so many times over the years.

Neal Ford: Well, I'll start and let Rebecca add to my comment. So we realized that we've given architects a sharp stick here by which they could poke developers and we're strongly encouraging them not to use the sharp stick to poke developers. Because you could create an incomprehensible, we can make this impossible for any single developer to get to pass all these different constraints we've added. That's not our intent at all.

The example I give is, it's sort of like a checklist. There was a famous book that came out a few years ago, "Checklist Manifesto", that talks about the value of checklists. And that's really what architectural fitness functions are. Because there are a lot of detailed things that you need to think about as a developer, including things around governance. Most developers are professionally responsible but as soon as things like schedule pressure show up, it gets harder and harder to mind the individual details, and fitness functions help catch those things.

But part of this is the benefit of building these things as fitness functions versus the spreadsheet because you can keep adding to that spreadsheet forever. But if you have to start implementing something, well, now you have time and resource constraints, and that forces you to prioritize these things. Okay, which are the things we really need to have, and which things are kind of just because architects love to go to ivory towers and plan stuff? But actually having to implement those things and coming up with objective measures for them is trackable work, and there are real-world constraints that can apply to that.

Recommended talk: Evolutionary Architecture & Microservices • Rebecca Parsons • GOTO 2015

James Lewis: And for me potentially it's also a way of getting me to quote Gregor, I hope it's a way of getting people out of the ivory tower, riding the elevator down to the engine room, and actually seeing where the work is happening. And doing some of it themselves. You're actually seeing what the real problems and projects are, I guess, in some senses. Rebecca?

Rebecca Parsons: Yes, and I think that that last point of Neal's, you have to make these objectives and you can't just use fluffy words like scalable or maintainable is one of my personal favorites. You have to say what that means. And one of the benefits of that is the conversations that then happen. Because fundamentally, the objectives and the purpose of an architecture team and a software delivery team are in conflict. The software delivery team's job is to get some...get products and services out the door as quickly as possible to satisfy the needs of the business.

The job of the architect is to look out for the long-term asset value of the IT estate. Short-term and long-term are always in tension. And one of the things that frustrate architects is they say these developers, these cowboy developers, is normally how they're referred, they just don't care. No, it's not that they don't care but you haven't communicated to them in a way that can fit into their software development lifecycle, what things are keeping you up at night? And similarly, because architects in general are so far outnumbered by developers, they don't really understand what's going on down there.

By forcing that conversation, okay, this is the fitness function I need to achieve, how can I make that objective and realizable in the context of the software development lifecycle, you call a conversation to occur? And now, developers understand what's keeping the architects up at night. Architects understand how that cycle works, and how they can best influence it. And now you can have that trade-off decision. And sometimes the long-term should win. Sometimes, the short-term should win. But if you don't have an objective basis for that conversation, you're just gonna be butting heads.

Neal Ford: Well, that's a great example, too, of how this scales up the sort of Gregor Hohpe's architect elevator idea because enterprise architects too can define things objectively. The joke in our last team meeting is we would put it in the radar, everything was ultra-high performance. Everything was advertising itself. It was ultra-high performance because it was fast.

Rebecca Parsons: Blazingly fast.

Neal Ford: Blazingly fast, that's right. Because it means nothing. There's no actual measure there. Enterprise architects often fall into this as we need scalability. But putting a concrete number on that means that the strategizing they're doing can become reality, and tactics will just always win, which is what happens in a lot of organizations because the tactics are the ones who are actually producing code and the tactic often wins out over strategy. So a lot of organizations are frustrated that we can't get an enterprise architecture strategy because tactics always win. This gives you, I think in a perfect organization, the enterprise architects all the way down to the developers, everyone's equally unhappy. Because if one of those is too happy, then something's probably wrong with the other two, and they're too unhappy.

Fitness functions in different organizations

James Lewis: It's super interesting because obviously, there are so many different types of styles of organization. Now, when you look at the sort of digital native organizations, like Spotify, they still, for example, have a chief architect, there was a role there. But obviously, the activity of architecture is very much pushed down to the teams and people, maybe more experienced people working on teams, or maybe they've got other types of processes in place, rather than having the ivory tower. I mean, for me, as an observation, I really like, the idea of these fitness functions because they work in both styles of organization. They work for the modern digital organization, whereas you can be there writing fitness functions on your team as we talked to you, I think we had a thing on the ThoughtWorks radar, which was run cost as a fitness function. Teams should understand how much money they're consuming more energy, whatever it is they're consuming. Do you have any observations about those sorts of different styles of an organization? Sometimes you have these big architectural functions, sometimes you have the more modern digital natives, I guess.

Neal Ford: Well, that's one of the reasons we ended up not calling it Agile Architecture, or some other terrible term like that because you could have a terrible waterfall process but still have architectural fitness functions as you're governing some aspect of it. Of course, if you're in a waterfall process, you would in fact, I'm sure, write all of the fitness functions. Well, first, you'd create the design document for all the functions and then you pass those over to an implementation. But yeah, that would not work well. But you know, it does, in fact, because we're not describing a process so much as we're describing a set of mechanics. So it's really just a set of engineering practices and a perspective shift more than anything else. So I think it's applicable very broadly.

James Lewis: One thing I've noticed. Another observation, I did some work some time ago on a similar thing, which is this similar idea, hypothesis-driven software architecture, I call it, which is, I guess, similar in the book to the fitness function-driven development, which you call towards the back of the book, I guess, is in the later sections. And one of the things I sort of observed there was that actually, it's a comment from another colleague of ours, Erik Doernenburg, that people who are working in architecture, they tend to design for when they were writing code themselves, even if that was like 15 years ago. And one of the observations was that things move really quickly. Things that were hard to change infrastructure, like standing up VMs or any of these sorts of rolling out big caches and things, these things aren't hard to change anymore. But there's kind of a lag to the folks who are also designing these systems and thinking about these systems. So they don't necessarily know the art of the possible. I think the evolutionary architecture idea is also sort of pointed toward the art of the modern possible if you like.

Rebecca Parsons: I think a lot of what is enabling this innovation as well is the foundation of continuous delivery. And I think some of those things are going hand-in-hand. When you can start to do things like infrastructure as code and you can deploy to production by just hitting one button and then it all goes smoothly, it gives you a level of rigor and safety that allows for more experimentation, and then you can start to uncover, okay, well, this is still hard. What can I do to make this thing easier?

Recommended talk: Team Topologies, Software Architecture & Complexity • James Lewis • GOTO 2022

And I think, when you think about Docker, Kubernetes, and the cloud, infrastructure is code, all of these things are solutions to problems along the way of this thing is still hard. I want to make it easy. What do I have to do to make it easy? And we're continuing to check those things off. Okay. Now, we can do this, that part's easy. Where's the next hard thing I can look at?

I think this innovation that we're seeing is definitely expanding the scope of the possible far past what many enterprises have to deal with him. And I know when Neal and I were doing the first book tour, one of the things he said, was, even if you don't use Docker right now, you need to think about your architecture differently simply because Docker now exists. And that's where this starts getting into even those traditional enterprises where what skills you need and understanding you need to do the job of enterprise architecture at this mammoth organization is continuing to change.

The architecture quantum

James Lewis: Okay. It's slightly different. One other thing you mentioned in the email that we love, how far can we drive this metaphor? We drove it about as far as hitting another metaphor, which was an architectural concept that is from a very different field. Can you talk a little bit about that? Because I think that's a fascinating idea and it really sort of settled a lot of things in my head.

Neal Ford: The architecture quantum idea came about because connascence wasn't good enough. Actually, that's where we discovered connascence, we really were looking for something about how you analyze the coupling characteristics in architecture, because what we determined was one of the key drivers to how evolved our architecture was was how controlled the coupling is within the architecture, which we summarized, I think more concisely in the second edition. But the quantum idea was just a, we realized we needed some measure in distributed architectures like microservices that was basically the implementation concept of a bounded context. But you can't just call it a bounded context because that already has a very distinctive meaning in the domain-driven design realm.

And for the same reason, we did not want to, for the next 10 years say, well, bounded contexts, but not exactly bounded contexts because it's about architecture structure, not domain-driven design, that's a very cumbersome way to say something over and over again. Whereas architecture quantum, we stole that idea from the same reason that physicists did, it means it's an unbreakable thing. But you know, the idea of a quantum is really a scope of coupling. And that has a bunch of very interesting side effects because you can think of it as the smallest deployable unit within a distributed architecture.

But then people like Zhamak Dehghani, our colleague took that idea and kind of leverage it to create the data mesh concept because it was kind of defined around the smallest deployable unit within an architecture. It has turned out to be a really useful way of thinking about architecture. And in fact, in the "Architecture: The Hard Parts" book, which I wrote between the two editions of this one, we went even further and defined that kind of coupling, a static coupling, but then, once you have these quanta running and they need to call each other, there's a whole bunch of other forces you have to think about too and that's dynamic coupling. And that is a distinction that goes back to the 1993 book, developer story design. He also distinguished between static coupling and dynamic coupling.

Recommended talk: Software Architecture: The Hard Parts • Neal Ford & Mark Richards • GOTO 2023

And so it's been a very useful way of thinking about architecture as "a shorthand for a piece of my architecture that operates as standalone but needs to communicate with other pieces of my architecture." That's too much to say and so we just say architecture quantum instead. And it also is a great way to torture people with the whole quantum quanta thing, because nobody can get it right for a long time, the plural versus the singular. So it's a great challenge.

James Lewis: One other thing we talk about quite a lot is cohesion. Often we talk about cohesion and coupling as being almost completely related to one another. And one thing you talked about is how more or less by definition have like free, they might be acceptable from a single architectural quanta perspective but at the same time, they're doing all the things. So they've got this kind of poor functional cohesion. Is this an idea that you've been sort of thinking about a lot, the classifying of the different things?

Neal Ford: Part of that was if we realized, if we're only looking at coupling, then everything that is monolithic is a single architecture quantum. And that's what we're really trying to catch was the smallest scope possible for something like that, because, as you know, it's way too easy to wire everything in architecture together. So one good way to determine the static coupling in your architecture is to play the experiment of, okay, if that thing over there breaks, what else breaks? Well, there's your static coupling right there. And so a way to think about that, that's not all the things but a subset of all the things so I think we're getting with cohesion. And of course, we've realized that the more functional cohesion you have, the tighter the scope you can keep your coupling to. And so I think that's how they really go hand-in-hand, high cohesion allows you to limit coupling, and the more that you let cohesion spread out, the more coupling is required to restore the semantics of the cohesion.

James Lewis: It might be a bit of a leading question, sorry, because I had an experience with looking at the code base for a bank some years ago, the web frontend of a bank. At the end of that, I ran some static analysis here is Java code base. And I have like these dependency matrices and said that you've got some good news and some bad news. The good news is that you've got cohesive software, right? The bad news is it's all 75,000 lines of code. Sorry, Rebecca, you were going to say something.

Rebecca Parsons: I mean, because that is the problem. I've had people say, the only evolutionary architecture is microservices. That isn't true. But because there is this notion of a well-structured monolith and where people get into trouble is if you are working in a monolithic code base, it is so tempting to just reach out over there and get that thing. And now, you end up with the kind of codebase that you're describing. But the thing to realize is that from an architectural perspective, our ability to move pieces around, whether it be microservices or whether it be some small seed component in a well-structured monolith, is going to depend on how well we draw those boundaries. If you've got the right kind of boundaries, then you can move the pieces around. And if you don't, then it's harder and harder to move the pieces around.

I think we need to get away from this notion that you can't evolve any architecture that isn't purely microservices. I would assert that you cannot evolve a big ball of mud but there are a lot of places along that continuum. And there can be very good reasons for deploying something as a well-structured monolith. But I'm going to keep saying the phrase, well-structured, and it takes a lot of discipline and that's where some of these architectural unit test fitness functions can come in to help enforce that separation so you do keep those clear boundaries even though you are in a monolith.

James Lewis: So I guess we're going back to something we touched on, or maybe touched on earlier in this conversation, which is about, there's a whole range of things I could do, there's a whole set of things I could measure or I could create tests for. Could you give me as a jobbing senior software engineer some advice about where to start? Where should I be looking for? What should I be doing to sort of put this into practice from a standing start?

Rebecca Parsons: Well, I think the first thing is to identify what are those truly critical architectural characteristics that are going to drive your decision-making. For some, security is just paramount, or data protection is non-negotiable, and just start listing them until you get to the stage where it's like, I don't really care between these, that's probably a good place to stop. And then when you look at some of the ground up, I mean, Neal Ford and I have been asked many times, "Can you give me the fitness functions for a banking application?" It's like, no. But as a development organization, you can come up with a base of fitness functions from a static analysis perspective that reflects your coding standards, your preferred idioms, and things of that nature. There's both the top-down, what are the big things that I need to worry about I have to be able to recover in no less than x? And then you've got that base. But when you start to get to the point on that top-down, where you can't really make a compelling case for why this is important, that's where you stop.

Head First Software Architecture & next AI revolution

James Lewis: Thank you. I guess, maybe taking a step back from the discussion, which is brilliant. As I say, it's an amazing book, I learned an awful lot that I didn't know even after 25 years of doing what we do. So thank you for that. But I wonder whether we could talk a little bit about what's next. What are you working on next? If you've got any interesting projects coming up, what are you excited about for the future? Maybe, Neal, we'll start with you.

Neal Ford: Working on "Head First Software Architecture," which is an interesting book, because it's almost like writing a graphic novel about software architecture. So it's an interesting challenge. So myself and Mark Richards who wrote "The Fundamentals of Software Architecture," we're teaming up with Raju Gandhi who just wrote "Head First Git" which is the next generation of "Head First" books, which is really fantastic and a great subject for a "Head First" book. And so we're working heavily on "Head First Software Architecture" right now. That's the next book thing that I'm going after.

James Lewis: Which will probably take you about two weeks if I may know you at all.

Neal Ford: It's hard. You have to come up with a unique title for every page of the book. So that's kind of painful, but it's an interesting challenge.

James Lewis: Do they have any rules about allowing you to use ChatGPT for that or is that a problem?

Neal Ford: Well, you do have to audition even if you're an experienced author because it's a very specific idiom so I don't think ChatGPT would make it past the governors for that style.

James Lewis: Rebecca, how about you? What's new and exciting? What's coming next?

Rebecca Parsons: Well, as I said in my intro, I've been interested and involved in artificial intelligence for decades. And so I've seen a lot of AI winters. And so I'm following quite closely what looks to be perhaps another AI spring or maybe even summer. And trying to understand, particularly with a responsible technology lens on it, how do we use these things. What possibilities are these large language models opening up for us? And little things like how do we control hallucinations? But also, how do we exploit its power of it? So I'm in the cautiously optimistic camp, I would say, on this whole area but it's fascinating to see where Moore's Law has gotten us in terms of the power of these AI systems.

James Lewis: I read something, I can't remember where it was, but a phrase that I'd not come across before in relation to these large language models, which was people being worried about something that they called I think, capability overhang. I don't know if you've heard this phrase, it's this idea that the more they're being used, these tools, the more things that they appear to do that they shouldn't be capable of doing. And that there are these certain capabilities that these new tools have, and we just don't know what they are yet. I mean, it's a fascinating time to be in high school I suspect, yeah.

Rebecca Parsons: Yes, yes, yes.

James Lewis: Cool. I think we'll probably wrap it up at that point. So I'm just going to say thank you so much, Dr. Parsons. I'm being very formal. We've known each other for a very long time. Thank you very much, Rebecca. Thank you very much, Neal, for joining us to talk about all things evolutionary.

Rebecca Parsons: Always fun. Thank you, James. Thank you, Neal.

James Lewis: Goodbye.