Why Architectural Work Comes Before Coding

How much architectural work needs to be done before you actually start coding? Should you know what software architecture is and how to use it even if you don't have the "software architect" title? And finally, how can you use diagrams to shape your software architecture?

In this Book Club episode, Simon Brown, creator of the C4Model, together with Stefan Tilkov, co-founder and principal consultant at INNOQ, help you get the answers to all these questions while helping you understand why software architecture is important for anyone involved in software development.

Intro

Stefan Tilkov: This conversation actually serves a dual purpose. Actually, triple purposes, in fact. I'm getting a nice conversation, and I also get an audio track for a case podcast and we get a video track for the GOTO Book Club. This is just about perfect. So, Simon, why don't you start by telling everyone who you actually are and what you do for a living.

Simon Brown: I'm an independent consultant mostly specializing in software architecture. My background is as a software developer, building software either for or with customers. I used to work in a consulting environment. Now I get to do two things. I get to hopefully, one day again fly around the world and run architecture workshops, and I also have another company called Structurizr which is a set of tooling to help people create software architecture diagrams, essentially. So that's me.

Why is software architecture important for developers?

Stefan Tilkov: Very cool. So, as I'm a software architect myself, I will try to just ask questions but I'm very sure I'm going to have to agree or disagree, most likely agree with some of the things you say. Let's just start and discuss a few things. Maybe first of all the tools and the books, which we're gonna talk about as well, they all sort of seem to make a point of making their target audience developers. It's as if you want to draw attention to the fact that architecture matters to developers, and should matter to developers as well. Can you elaborate a bit on that? Why do you think that is or why do you think people could think that it's not important for developers to consider architecture?

Simon Brown: The whole software architecture for developers really came about when I worked for consulting companies. In order to scale and grow a consulting company, you need more teams to service more customers. For every extra team you need, you need more tech leads, more architects. So basically, a lot of this stuff came from me teaching our developers how to think about software architecture, to do software architecture, to communicate software architecture. When I was going through this myself earlier on in my career, I didn't really find these helpful in terms of, "I'm a developer now and I've been thrown into an architecture role, what do I need to do?"

There were lots of books out at the time. You've got all the SEIA and practice books and so on and so forth, but I just found they were very researched based and academic-focused, and it didn't really give me, if you're an architect, this is the sort of stuff you should do. So that's really where my focus came from and hence the software architecture for developers theme.

To answer the same question from another angle, I guess, when the Agile Manifesto came around in the early 2000s, we saw lots of people jumping on that, which is great and there's a lot of benefits that came out of the agile movement. However, a lot of people started dropping some of the more design-focused, documentation-focused, architecture-focused techniques and practices and processes. I wanted to figure out how we get this stuff back into the way that teams work without seeming like that horrible dictator-style architect that people don't like. And also, I want to reintroduce some of the existing ways of working but with a little bit more of a developer focus so that the developers potentially pay more attention to them, if that makes sense. So that's really the kind of developer focus.

Is “software architect” a role or a list of tasks?

Stefan Tilkov: So do you think “software architect” should be a role or is it simply a set of tasks that people... that somebody just has to do?

Simon Brown: I think both, I guess. If you look back like 15, 20-plus years, every team would probably have an architect on the team who would do all of that stuff, and they would rarely get involved in the code and they would tell people what to do and we learned that that's not a fantastic way to work.

Those tasks still need to be done but perhaps nowadays with our more modern agile approaches, they're much more collaborative. You know, trust is an inherent factor in many teams. Maybe we don't need that single person who's looking after all of those tasks, and that's why I like to think about it as a role. So it is a role that the collective team needs to do. It doesn't need to be one person. It could be many people. That set of tasks does still needs to be done in order to, you know, hopefully, get a better, more successful end result.

Stefan Tilkov: Yes. I think I completely agree with that assessment. I mean, after all, if you don't... you, sort of, have to do it because otherwise, what you end up with will be... will just be a matter of, you know, whatever happened to you, right. It'll be, sort of, accidental because some people made decisions and you just end up with some sort of architecture that may or may not be what you wanted.

Simon Brown: Yes. It's either the, kind of, accidental structure thing, so a big ball of mud or some of the quality attributes tends to get forgotten about because they, kind of, fall between the gaps between the people like performance and security and scaling. It's like, "Oh, I thought someone else is looking after that stuff but oh, no, it's our job, collectively."

Stefan Tilkov: Yes. I've also seen people build something that works perfectly well in some architectural dimension. The problem is just that that dimension was not one anybody really cared about. 

Nobody wanted that system to be that super scalable. You know, but it was a lot of fun to make it like that. I mean, can't say that I've seen too many systems that were too scalable. I mean, that would be a bit of an exaggeration as well. But right, of course, as you mentioned, the quality attributes are something that you need to take into account to make a decision, what actually matches those requirements.

Simon Brown: Yes.

How much architectural work needs to be done before you actually start coding?

Stefan Tilkov: But you have to know them first and then people often actually don't... yeah, completely agree. So, you mentioned the Agile Manifesto as, sort of, an influence, sort of something that let some people, I think we both agree, mistakenly to dismiss all of the architectural ideas and all of those things. But I think some of the reasons for that were that it was all very often at least associated with doing a lot of work upfront.

You know, like having somebody design the system, builds the architecture in the form of a lot of diagrams and prose and big documents and then just, hand it off to somebody else to actually build it according to those guidelines and decisions and rules that are in the big architectural master document. So, I think pretty obviously, that is not a good way to build things but maybe the other extreme isn't that great as well. So how much architectural work needs to be done before you actually start coding?

Simon Brown: There's a great quote by Dave Thomas. He says that "Big design up-front is dumb, and doing no design up front is even dumber." And it's that, kind of, flipflop from one extreme to the other which is exactly what I've seen teams do over the past, what, 20 years now. So, I completely agree. Doing too much kinda locks you down. It's too rigid. You spend lots of time getting there. You spend lots of time potentially solving problems you're never gonna have. Then you have to, kind of, order and make sure all of the people are doing all of the things according to the document you wrote four years ago, which, you know, and the world's moved on.

And I think a lot of people misinterpreted the Agile Manifesto. And because the Agile Manifesto doesn't explicitly talk about doing up-front design, a lot of people, I think, have interpreted that as the Agile Manifesto says don't do up-front design. If you read things like extreme programming and if you read some of the principles, it is easy to kinda get that view of what the Agile Manifesto is saying. But I don't think that's the case.

What I find amusing about all this is when I worked for consulting companies back in the early 2000s and the Agile Manifesto was coming out, I looked at it and I thought, "Well, that's kind of what we're doing anyway." You know, I was very hesitant in doing the big up-front design because I needed the... well, first of all, it's really boring. I did work on a few projects prior to that where we do like six months of just doing design and using rows and putting UML diagrams into rows. It was very interesting from a domain analysis perspective but we wrote zero code and I just got bored of doing that.

So that was never really my approach anyway. My approach was "Well, let's get the big building blocks in place. Let's understand the major driving factors, the major important quality attributes like security and performance, and so on. And then let's build a design around that and then we'll fill in the blanks as we go along." So when the Agile Manifesto came in, I was thinking, "Well, that's what I do anyway." So, from my perspective, nothing changed too much. But I guess depending on where you were coming from at the time, your perspective was very, very different and I think that's what led a lot of teams down the no design route.

Unfortunately, in some cases, fortunately, in others, of course, those same teams are now realizing that was potentially a bad idea and they're now starting to go back and think about, "Well, what sort of design should we do up-front? How much should we do and what, sort of, architecture process should we put around making sure people are doing the things we think they're doing to fit into the constraints of the environment and the guidelines and the principles and everything else?"

Stefan Tilkov: Yes. So basically, it seems the answer to that question is you should do just enough up-front design. Just the right amount. That's a very easy answer.

Simon Brown: Which sounds obvious, isn't it? But it's, like, super hard to quantify.

Stefan Tilkov: Yes. Indeed. But I think we agree on that as well. So, that was an interesting thing that you just mentioned, which is this impression that sometimes when you read a book or, you know, some sort of paper or some blog post or whatever, you look at it and say, "Well, so that all sounds totally obvious and I mean, that's what we're doing anyway so why is everybody making such a big fuss about the whole thing?" But I've come to think that, in some cases at least, is the sign that this is a very great contribution, a very valuable contribution because it actually articulates something that should be obvious and maybe is obvious to you as a reader but actually may not be.

So to be perfectly honest, I had some of those feelings when I first read your book, right, because I was like, "This is..." That's one of the reasons I said we'll probably agree, right, how... this all seems very reasonable. It seems so reasonable that I keep saying, "Well, yeah. Duh. I mean, sure. That's what you should be doing." But I still think there are excellent books and because they actually provide a lot of value to people who may not have that experience, who may not have arrived at those conclusions and they get it in the form that actually makes it tangible and gives them some actual guidelines of doing things. I think what I'm trying to say is that if you read a book or if you're looking at something and it seems kind of like what you're doing anyway, that's not a sufficient reason to dismiss it. It may be an excellent thing, just what you need to convince other people.

Simon Brown: Yes. And of course, there are lots of people now entering the industry and they didn't see the stuff that predated the Agile Manifesto. So they missed that whole journey and now they're just getting the end bit where it's, we move fast, we break things, we don't do design. And like, "Hang on a second. That's not what's happening here. We should maybe step back and plug some of the gaps that these people don't have in terms of knowledge and experience."

The importance of diagrams in software architecture

Stefan Tilkov: Yes. Good point. So, one of the things that you emphasize a lot is the role of visuals, right, the role of diagrams, visualizing architecture. So can you talk a little bit about that? Why is it not enough to just, you know, sketch something on a whiteboard somewhere and then wipe it away 30 seconds later?

Simon Brown: So that's funny. That is one of the big things that the Agile Manifesto and the agile movement has trended teams towards. It's the, "You don't need these big design documents, you don't need to do these big heavy up-front design processes and you don't need to use expensive tooling. Grab a few people, draw some sketches on a whiteboard, have that conversation, and then erase it." The emphasis here is the values in the conversation. And I completely agree, you know. If we've got a bunch of people around a whiteboard and we're talking about different ideas and different designs and different approaches and we're assessing tradeoffs and stuff, that's a super, super valuable conversation and we should definitely keep having those things. The problem I've seen is that once people erase these diagrams, a lot of that knowledge gets lost. Now sometimes people take a photo and they stick it on Confluence and that's all good but...

Stefan Tilkov: Nobody ever looks at that.

Simon Brown: Right. Nobody looks at it because nobody can find it, first of all, because the architect who drew it quit three years ago or something. If the diagram doesn't make sense from a visual perspective, it's very easy to interpret what you think the diagram means and that might be completely different from what everybody else thinks. So, I'm a fan of whiteboards and I definitely recommend teams use whiteboards for doing up-front design. I just want people to add a little bit more, and I have to be careful how I phrase this, formality and structure around what they're drawing because it's far too easy to draw two boxes on a whiteboard, stick an arrow between them and that could literally mean anything. I want the stuff that we're drawing to have a little bit more meaning.

Now, of course, you might ask, "Well, why don't we just use UML?" And that's a very good question. And up until, sort of, 2004, 2005, I was a big UML user. So all of the documentation I did, all the sketching I did was using UML. But it went out of fashion and people dropped it like a hot potato in, sort of, the mid-2000-ish to 2010 years and it's not really bounced back. 

When I go and see teams and I say  "What sort of notations are you using to draw your architecture?" They literally just say boxes and lines on a whiteboard. There's no mention of UML. Some of the people have now been through university and college and their apprenticeships. They're not being taught UML. So again, you have a whole bunch of people in the industry who have missed out on all that stuff and they just think, "Well, let's draw some random boxes and lines because that's what everybody else does."

And to get back to your question, I think, and I hope... I think visuals are super, super important. I think you can literally hang all of the other stuff related to the architecture role around a good set of visuals because that good set of visuals allows you to tell stories, it allows you to have design discussions, you can make design decisions, you can assess tradeoffs and it just opens up all of the information to a much, much wider audience. So that's why I place so much importance on the visuals.

Did remote work and the tools we are using add more structure?

Stefan Tilkov: So do you think, because everybody's now working remotely these days, do you think things have changed because tools are more important and the things that we create are more structured because the tools typically offer some structure as a default?

Simon Brown: I think, and especially if you look at remote working and especially with the whole pandemic thing, what teams have done... and I've definitely seen this. They've taken their whiteboards in the office because they're not allowed in the office anymore and they're using tools like Miro because they can basically fire these things up, you can get a bunch of people looking at them, you can all interact and collaborate at once. And Miro is a fantastic tool and I'll certainly use it myself for other things but it's not going to give you structure. 

So if you approach a whiteboard, it's not gonna provide you any assistance in drawing an architecture diagram. It's not gonna help you explain what types of abstractions you should be drawing and, you know, the semantics of the visual language you're using. Miro is the same. It's just a great way to draw boxes and arrows collaboratively. It's a collaborative online whiteboard.

I think we're missing a trick for tooling here. I think... and again, I have to be careful what I say here because it's very easy to interpret what I'm saying as, "We should go back to doing what we did 20 years ago with rows and have lots of rules and semantics and it was all very, very precise." And maybe that's too far. So maybe we need to reign that back a bit but have some tooling that actually provides some assistance. It's the same with, like, AutoCAD. Now if you want to sketch out a set of building blueprints, you don't fire Visio up. Visio's gonna offer you no assistance at all apart from you've got this box and this box and you can put them together and group them. You're gonna use some of that AutoCAD because it's going to provide you assistance and rules and different things like that. So yeah, I think there's still some more work to be done there.

Stefan Tilkov: Yes I think what people might be missing is that the assumption somehow seems to be that if you use something like UML or any sort of structured case tool, whatever, you know, whatever it happens to be, then, in the end, you might have to specify things at a level that is very close to programing, right. So people fear that they have to go into all of the details and provide all of the... fulfil all of the requirements for something that could then be automatically turned to code, which is not something that you have to do. I mean, you could possibly do that. Whether it's a good idea or not, probably beyond our scope today. But even though you could do that, you can just as well use a tool like that to model something at a very high level, right. Just draw the high-level structure of your system just with a little bit more meaning, more semantics as to this is what this kind of line means, right. If it's a dashed line, it means something different than a solid line because that's why we make it a dashed and not a solid line so...

Simon Brown: Right.

Stefan Tilkov: Whatever that... and you could invent your own notation...

Simon Brown: Yes totally.

Stefan Tilkov: ... and then stick to it. That would be perfectly fine. But you could also use one of the notations that are already there. For example, UML which, of course, gives you a ton of variation, lots of ways to customize it, and because it's a super-powerful, super-customizable tool, may end up being super-complex and way too powerful for what you want to do, as usual. There are alternatives, which I'm sure we're gonna talk about very soon now so...

Simon Brown: Yes.

About the authors

Simon is an independent software development consultant specializing in software architecture; specifically technical leadership, communication, and lightweight, pragmatic approaches to software architecture. Simon is the author of "Software Architecture for Developers", a developer-friendly guide to software architecture, technical leadership, the balance with agility and communicating software architecture with sketches, diagrams, and models. He is also the creator of the C4 model and the founder of Structurizr, a collection of tooling to help teams visualise, document and explore their software architecture.

Stefan is a co-founder and principal consultant at INNOQ, a technology consulting company with offices in Germany and Switzerland. He has been involved in the design of large-scale, distributed systems for more than two decades, using a variety of technologies and tools. He has authored numerous articles and a book (“REST und HTTP”, German), and is a frequent speaker at conferences around the world

Recommended talks

Five Things Every Developer Should Know about Software Architecture • Simon Brown

Software Architecture vs. Code • Simon Brown

Visualise, Document & Explore your Software Architecture • Simon Brown

"Good Enough" Architecture • Stefan Tilkov

REST: I don't Think it Means What You Think it Does • Stefan Tilkov