Architecture Modernization

Introduction

Eduardo da Silva: Hey, everyone. Welcome to this "GOTO Book Club" podcast series. My name is Eduardo da Silva. I am an independent consultant, helping organizations modernize their socio-technical systems, architecture, and leadership. And today, I'm really excited to talk with my friend and colleague, Nick Tune. I have followed Nick Tune's work for six, or seven years now. And I was fortunate to actually meet Nick Tune in 2020, at one of Nick Tune's workshops in DDD Europe. And since then we have been talking and collaborating on a lot of things. So it's a real pleasure to basically come today and talk about Nick Tune's new book, "Architecture Modernization." So welcome, Nick. And before we get into the details of the book, maybe you can just introduce yourself a bit.

Nick Tune: Hey, hey, Eduardo. Thank you. It's great to be here with you today. So, over the last 15 years, I've worked in IT. I had roles as a developer and principal engineer. I've worked as a consultant for five or six years. And now I'm back working with a single company full-time in a staff engineer role. So I've done a bit of everything. And the things that interest me the most are domain-driven design. How do we design software systems, by understanding the business we're working in better? Continuous delivery has always been a big passion of mine. How can we deploy software on a more regular basis? I was very lucky early in my career to work in a company where we deployed to production every day. So I had that reference point very early in my career. And I guess other things like good practices, TDD. I've always been into TDD. I like working collaboratively. I like getting on the whiteboard with my colleagues, working with different people, and talking to people who understand the business. So those are kind of...yeah, those are kind of my main areas of interest.

Eduardo da Silva: I think that really comes across in your book. But before we dig into the actual contents of the book, maybe you can just share with us, what was your main motivation to actually bring this great piece of the book together?

Nick Tune:  I think there were two motivations really. Why write a book? And why write a book about architectural modernization? So in terms of why write a book, any book, I think I've always found writing to be extremely useful. The old phrase, you know, writing is thinking, has been something that has lived with me for the last 15 or so years. I've always been writing blog posts. I try to write one blog post every month. I did that almost consistently for 10, to 12 years. And then I wrote a book and I stopped that habit because the book took over. But just the whole process of writing. You think you know about some topics, or you write a blog post, and you realize, well, I can't even write one blog post about this topic. There were some key things I didn't understand. So, I've always been a fan of writing.

I wrote a book about DDD about 10 years ago with Scott Millett. And I got a lot of value from that, from the writing process. It was good to meet other people. People read the book, and they found value in it. So it was kind of a nice feedback loop that people found useful. So, I was always thinking it would be great to write another book. But I never really felt like I had enough content. I never felt like I could... I could write a book about stuff, but would it be useful? Not really. So I was waiting for... I felt like I could write in a book and it would be useful. And it wasn't really feeling that way when I wrote this book. But the pandemic happened, and I suddenly had a lot of spare time. So I'm like, well, let's start writing a book and see what I can put together. Even if nothing happens, it's a good side project to keep me busy.

So I started the book during the pandemic. I'd done various modernization projects over the years before that. So, that felt like quite a relevant topic. As I was a consultant, I got to work with other companies, various companies at different stages of modernization. So it's like having a lot of knowledge on that topic. I thought, yeah, let's see if I can write a book about modernization. I wrote a blog post about modernization, and that was quite well received. I'm like, well, I wrote a blog post, now let's try and write a whole book. That was the kind of motivation really.

The 4 Major Themes of the Book

Eduardo da Silva: I can relate a lot to that. When I read your book... And well, we talked about it a bit before while you were writing, it really feels like you actually brought together a lot of pieces, right? A lot of things tend to not be into a coherent story, which I think that's the struggle of writing a book. That's why it sometimes takes years to bring it together. So, maybe you can share with us for the people that haven't yet read the book, just what does that story look like? So if you had to sort of give a short summary of the book, then we can go into details in some of the parts. But what would be sort of your summary of your book, if you had to do it in a few minutes? What are the high points? What's there?

Nick Tune: I would probably say, probably like four... I experimented with the book structure at various times. In the end, there were no sections in the book. It's just a flat list of 17 chapters. But there are kind of, I would say, four rough themes. So I think the first one is really around the business side of things. So what is your company trying to achieve, if modernization is just fixing some legacy systems with new tech, and it's hard to articulate the business benefits? Then those are the kinds of projects that typically don't work out that well. So the start of the book is really, how to have a strong connection between your modernization and what your business is trying to achieve. What are your products, North Stars, for example?

Are you in a game of growing more revenue, moving into new countries, targeting new customer segments, and becoming more profitable? Are you consolidating multiple acquisitions? Is it your operational costs that are too high? What are the key metrics in your business? So that when you talk about modernization, people understand, this is about moving the business forward, and not just technical. So there's a chapter on business objectives and North Stars. How can you identify what those might be in your company, using concepts like Ansoff Matrix to answer questions like, is your company building new products or developing existing products? Are you targeting new customers or focusing on the existing customer segments? There's also a chapter on Wardley Mapping, which I found very useful as a consultant over the past few years. Because even if you talk to people on the business side, you might get a lot of ideas, a lot of opinions about what the business is doing. But it might not be clear enough in terms of modernization.

Recommended talk: Crossing the River by Feeling the Stones • Simon Wardley • GOTO 2018

And when the business goals aren't clear, the system becomes very complex. You could spend a lot of time and effort modernizing things that aren't worth it, trying to support all these possible future use cases that just will never be needed. And then when you're trying to build things that do matter, your system is more complex, and so it's harder to do that. So with concepts like Wardley Mapping, you can map out parts of your business, and how they connect back to parts of your software. And you can ask questions like, what do we consider to be a commodity, things that are reaching that end of life in terms of innovation over the next three to five years? Things that you won't be investing in too much, in terms of building new features and developing those capabilities? Then we can also ask, what is at the other end? What are your Genesis or your custom-built capabilities? These are things that do give you a competitive advantage, where the ability to out-innovate your competitors will make a difference to your company.

So by mapping out the portfolio with a Wardley map, you can start to answer those kinds of questions. It's quite an amazing experience really, Wardley Mapping. You talk to a lot of people and they've got a lot of ideas, and they've got a lot of legacy they want to modernize. But when you build a Wardley map, and you've got like a bunch of stuffing commodities, people are like, you know what, we don't really need to modernize that too much. We just need to reduce the costs and actually put our efforts into the other bits in Genesis and custom build. So the first part of the book is really about the business objectives, how to talk to your stakeholders, how to go around your company and find out what's important to people, and set that narrative of modernization as a business journey, not just a technical one.

The book also talks about how you can start to define the actual requirements of what your modernized system needs to do. Sometimes people think that modernization is just take the old system and rebuilding it in new tech. But there are some surveys that are saying, up to 80% of your legacy systems are either features that aren't being used or are rarely being used. And, you know, there's all kinds of manual processes, stuff happening in spreadsheets. So, you really want to use techniques like event storming to figure out, how does the current system need to behave? What features do we keep? What do we remove? Where do we simplify? Where do we add new things? So there's part of a book that looks at this, how to map out the current states, explore the future states. The book also looks at how to design architecture, looks at things like how you can apply DDD, concepts like message flow modeling, event-driven architecture, how you can plan your migration from a current state to the future states, and terms of technical patterns.

A very common example here is, that when you're migrating parts of a system, you'll have the new and the old running in parallel. You might use things like the strangler pattern to do that incrementally. But even so, each piece of your architecture, you might have a particular domain, and that will be incremental by itself. You have to decide, do we migrate the reads or the writes first. And then once we do that, how do we synchronize the data and the legacy? Maybe the legacy handles the reads, and the new modernized piece handles the writes as a first step, and then you migrate the reads later. But in between, you have to keep them synchronized.

And then the last part of the book is really around setting the vision. It's around leadership. It's around supporting modernization. Because whilst the techniques, and the patterns, and the architecture, while all those things are crucial and important, what really counts is being there every day, turning up, having conversations every day. You know, teams have to go from, we're building product features, to, oh, now we have to do this modernization thing as well. It's very easy for teams to just think, let's just keep building features. And even if you convince the teams, you still have the middle layer, the managers who might say, well, you know, the new CTO is talking about modernization, but I'm gonna get more product features done because that's what I get rewarded for. So there's that whole project management, change management piece. And I try to touch on a lot of that in the book in the final sections. So that's kind of a quick overview, I would say, there are four main pieces.

Socio-Technical Alignment

Eduardo da Silva: Quite a lot of things. And in the subtitle of the book, you have this nice...I think you sort of have a nice summary when you say that this is about the social technical alignment of software, strategy, and structure. Right? So you are sort of...it's not just about technical modernization. And for example, when you're talking about the event storming and Wardley Mapping, I imagine that those are not sessions that typically... it just says the technical people doing it, but this goes further. Can you elaborate a bit on the why of that, Nick Tune? Why did you end up articulating that? And also, for example, there is a chapter on team topologies. There are all of these elements that go further than just looking at the technical system, but looking a bit broader.

Nick Tune: Good question. In the title of the book, there were many options explored and considered. Some were mine, some were from the publisher, and some were compromises. But I think this one was quite... It was okay. But I think it represents an important concept, the socio-technical side of things. And I would say that's really something that happened in my career a few times, both as a consultant and as a full-time employee, as a principal engineer at Salesforce. At the start of my career, I was very interested in DDD, event-driven architecture, and SOA. One of my big influences was Udi Dahan. He would come to Skills Mastery in London, and do some talks in the evenings at meetups and stuff. And that was my big area of interest, event-driven systems, DDD, SOA, as I was talking about.

So that was my area of focus. Those were the kinds of things I was blogging about, reading books about, and the kinds of projects I was getting into at work. Whenever I had a chance, I would say, yeah, let me do something with asynchronous workflows, or try and get myself on those projects. And as I became a bit more senior, that's when the socio-technical side of things came in. So I had a job in the UK government doing some modernization. And we had this issue where we owned half of a domain and another team owned half of the domain. And every feature was touching both sides. And we all found it very painful. And we were like, wouldn't it make sense if one team owned both parts of this, and we could build a whole feature ourselves? And it was happening to a few different teams. Basically, we were both in different departments. And we reported to different CTOs, a CTO of digital and a CTO of enterprise IT.

Recommended talk: Accelerating Event-driven Architecture with Domain-driven Design • Brian Zambrano • GOTO 2023

That's when I realized, yeah, the politics is shaping the architecture here. If you applied DDD by the book and you applied all these good practices, there would be one domain here with one team owning it, or a subdomain, as you might say in DDD. But now the politics was...your chart was shaping the architecture, very strong Conway's Law effects there. And I also went to work at Salesforce. They had an old monolith with 100-plus developers working in it. And I was working on one part of the marketing cloud. They wanted to modernize it, and have teams working more independently. So my job was to help them apply DDD, and slice up the architecture in a more loosely coupled way, with event-driven architecture. And again, I realized, if we start changing this architecture, and we move around these responsibilities, that's going to have an impact on the teams. Some people have skills that just wouldn't be relevant in the team they're working in now. Some things are more front-end based. Some things are more back-end and integration based. Some teams are too big. So it would end up making some significant changes to the teams to support the architecture.

We couldn't just change the architecture, and expect the current teams to keep working. They'd be tripping over each other in the same code bases and stuff. So team ownership had to change, and team composition had to change. And when I told the manager, he said, "I spent a long time building these teams, getting these people to work together effectively. Now you want to chop it all up and break it. I didn't really hire you for that, Nick." And then as a consultant, I started out as a DDD consultant. I like to do things like event storming to help companies shape their architecture, help them modernize their architecture, and design a future state architecture. But the clients I worked with, never asked me for just that. It was always, how do we organize the teams around this? In fact, a lot of the time, what a lot of the companies were asking for was, our teams are not working effectively. And it was actually architecture that was the enabler of the teams working more effectively, not the other way around. So, I think for the last 7, 8, almost 10 years, I would say, this architecture and team organization, what we call socio-technical architecture, has not really been able to separate those two things, to be honest. I see them as a package.

Role of Architecture Modernization Enabling Teams (AMET)

Eduardo da Silva: I think this is where we crossed paths somewhere in 2018 or so. I started reading some of your explorations. And some people tried to bring that together. And I was sort of in the same struggle. I think many of us started really noticing that it's not enough, right? It's not enough to just focus on the technical architecture and ignore what's going on around the teams owning that. Because they, as Conway's Law say, drive that. But you touched on a very interesting point there, which I think you also go into... It's really well articulated in the book, but I think we should highlight it here, which is, the execution of modernization.

So you go into those great workshops, and everyone has fun. But then what? Then the real modernization starts, and this is where a lot of the things tend to also die down because that's when, as you said earlier, the hard work that day in and day out work of actually modernizing happens. So maybe we can spend a moment there, Nick Tune. Because I think this is probably something that deserves quite a lot of time, but we can spend a little bit of our time there. So you have some chapters going into this part. Can you elaborate a bit on that, Nick?

Nick Tune: As a few chapters that go on this part, and I think as you touched on, modernization involves different activities, right? One activity is mapping out, what's the current state of the system? What could we modernize? Another activity around design, is, what does our future state architecture look like? And then another aspect of modernization is actually doing the work. And it's easy to have workshops and to draw diagrams, and have meetings, and get excited about PowerPoint decks from the CTO showing off his exciting new architecture that does all these amazing things like microservices and Kafka. And those things are important, those things are very important. If you don't do those things well, modernization probably won't be successful. But the real effort is sometimes just getting started, and then keeping things going.

So many times, over the last 15 years, we've been given these promises or assurances, yeah, we're going to modernize, all this technical debt that's holding us back and slowing us down, stopping us building new features, making it difficult for your developers to get your job done. We're going to fix it, do things properly. And we're going to try and innovate as a company. And then, you know, there's this real big excitement, promises hype up front, and then you start modernizing. And it's like, can we just build this one new feature? I know we said we wouldn't, but as a client who really needs it, we've promised it, or... And another thing comes in, and then it's like, we aren't really doing any modernization work here. We just come back to doing product work and features work, because the product managers can't just sit around doing nothing while you modernize, and they've got their own OKRs. So those are the kinds of challenges.

That's one example of a challenge, I would say, of the execution challenges. Another one is connecting the big picture, and the day-to-day details. You know, I've worked with a lot of teams, they'll go to some meeting or some presentation talking about modernization. There's a big strategy deck from the CTO that shows this new architecture. And they go back to their team, and they could be thinking a number of things. One of those things is, this is just extra work, we have to do our products work and this modernization work. Let's just go back to what we know, doing product work in a monolith. And let's not try to do the modernization work.

Or another feeling they might have is, we've heard this before, we've had these failed modernizations before, we have incomplete modernizations that weren't finished. And now we have three legacy systems that all have to talk to each other. Or in six months, this will all be gone, there won't be any modernization, it will never happen. So just kind of ignore it. And even if the developers and architects do want to modernize, you still have this problem of, they've been working in a certain way for a long time. They might have been working in a way that involved getting into the legacy, building features in the simplest way possible without breaking things. To go from BAU in a legacy to doing this big, amazing, crazy modernization, that is a huge mindset shift. And you can't just expect people to go from not doing modernization to doing modernization. And they might not have the skills to do modernization. You might be asking them to do things that they just haven't done before with technologies they haven't used before, applying patterns like microservices they haven't done before.

And that's where we can end up with tightly coupled systems, where teams just start building all these random microservices. And they are all tightly coupled, and you break one, and you end up breaking all of them, or you have to deploy them all together, a distributed monolith. So that execution piece that every day, supporting the teams, supporting leadership. That's one of the key areas, I think, the most difficult areas getting started and keeping modernization moving. So in the book, I would probably say the chapter I think touches on that the most is a concept we call architecture modernization enabling teams. An enabling team is a concept from Team Topologies. It's a team whose responsibility is to support a specific kind of outcome, which involves basically helping other teams to achieve that outcome.

So an architecture modernization enabling team is a team with architecture skills, modernization skills, but they don't own any parts of the system. They don't do their own modernization work. They might be engineers who spend time with the teams, implementing features with the teams. That's my current role. I do a lot of that. A couple of weeks ago, I worked on customer support, dealing with customer support tickets, just to understand the pains that the current system is creating. But I'm not part of an enabling team that owns that work. My job is to support teams however I can. Sometimes stepping forward and leading by example, sometimes stepping back and just letting them explore topics for themselves. Maybe make some mistakes, but that's okay. It's part of the learning process. So yeah, I would say the AMET chapter is probably one that touched on that topic quite a lot.

Eduardo da Silva: So AMET stands for architecture modernization enabling team. And so yeah, that's something we also did a bit of work together. And we touched on a few things there. But one of the things we see is that the modernization challenges vary a lot from organization to organization. But there is, like, this typical initial challenge around AMETs, which is, how do you sort of get buy-in to actually invest on... As you said, you need proper time to actually facilitate the modernization, because this typically is not something that a single team is doing on their own. Maybe this affects multiple teams. So, someone is working across these multiple teams that are affected or doing that extra work. And this is not, as you said earlier, that's probably not the typical BAU product, feature, work. So the investment there tends to be a bit more tricky. So what are some of the motivations you use to actually get leadership to support the investment on these people, basically, right? They are spending time doing this. It's not an immediate outcome. But the impact is huge.

Recommended talk: Adaptive Socio-Technical Systems with Architecture for Flow • Susanne Kaiser • YOW! 2023

Nick Tune: If we're at the point where leadership wants to modernize, then the risk at this point is that modernization won't happen or it won't happen at the speed they're hoping for. Because the organization is not set up for modernization. The organization is set up for building features, BAU work in the legacy system. So, an AMET is a team that is there to help that transition to the modernization, to help with modernization. So if you're an organization, and you want your modernization to be successful, you can't just expect the teams as they're working now, suddenly start modernizing. It's just too much of a mindset shift. They might not have the skills they need. So an AMET is gonna be there to support that journey. An AMET is there to identify what's slowing our modernization down. An AMET is fully focused on modernization. It's almost like an insurance policy. By having people fully focused on modernization, who are looking across the entire initiative, they can identify problems that are happening at any level in any team and provide the support needed.

Eduardo da Silva: Sort of an investment. I think that's a word I saw. So you're using an investment to break that initial inertia, right, that's there, that tends to want to go to the normal, to the usual way of doing things and all of that.

Practical Steps for AMET

Eduardo da Silva: Maybe Nick, you can share some...like, if we stay on the AMET, some practical notes on what would be some of the first activity? So say now the leadership is supporting investment in having some people doing this. But what would be some typical activities that you'd recommend that team to start getting into play so that the modernization can effectively be launched and sort of sustain some momentum to break through that initial inertia that we were talking about?

Nick Tune: Yeah, good question. To be honest, I think getting started as soon as possible is highly encouraged. Obviously, there are some risks here. But the point I want to make is, when you actually deliver something, you start learning these ideas we had, what's possible, how to break parts out of our legacy, how to build new features in this new world, using new tech. You start getting feedback on those ideas, which helps you to start learning. And also delivering early actually starts to change the mindset. All these people who feel like, is modernization going to happen? Is it just a buzzword? Well, they can actually see something being delivered. And suddenly, the more you do that, the more the mindset shifts to, hey, all those other teams are modernizing, and we're not modernizing. Now, suddenly, the scale has gone the other way, and teams want to modernize.

So I think one of the biggest motivators for modernization, one of the biggest incentives teams can have is to see other teams inside the same organization, who are facing the same legacy, the same constraints, actually doing great modernization work. So my advice is definitely try and deliver something early, in the first three to six months. You might not have a chance to map out your whole system before you do that. But in most companies, you can identify a shortlist of maybe five starting points that would be, even if not the highest priority, would still be worth doing. Or you can look at your roadmap and say, what's coming up in our product roadmap? It might take us a month or two extra to do this in a modernized way. But that could be worth it. And that's a way to modernize whilst delivering something of business value.

And to be honest, I do feel like in most companies, it is possible to find something you can do in three to six months. I will say the caveat can be, sometimes infrastructure, if you're moving from on-prem to the cloud, and you need networking from on-prem to the cloud, there's a lot of security involved. That can be difficult sometimes. But for me, that just reinforces the need to try and deliver a small proof of concept as early as possible, a small slice. So that would be my focus. If you're working in an AMET, how can you help to start delivering early, deliver something that's going to be valuable, help you to start learning as early as possible.

Recommended talk: Legacy Systems & Digital Transformations • Ian Cartwright, Rob Horn & Hannes Lowette • GOTO 2022

Eduardo da Silva: Definitely. Start practicing also those ideas, right, of modernizing and getting that in the organization. Let's shift gears, Nick, and touch into something... Because as you go through the book, and as we are talking now, it feels like this is a huge story that never ends. So we are always discovering new topics that are... Because this is...as you framed at the beginning, this is a social technical modernization. So you are touching on organizational aspects, technical aspects, business prioritization. There are so many things. So, is there anything that you feel that you could write a few more chapters or a few more things, things that you feel that you would be...or that you are excited about, continue exploring around this topic of architecture modernization?

Nick Tune: I think the most interesting topic is that whole migration piece, how to keep momentum high, kind of building on what's in the AMET chapter. Talking more about migration patterns. So, for example, you've got a new system and an old system. How do you start migrating functionality and data across? I would love to talk more about this. Like, do you start with your new customers first, your highest paying customers, all those migration options. I think there's a lot more in that topic I would like to talk about. And also again, how you can support that journey. All of those leadership and enabling qualities that are needed on a day to day basis, all the rituals you need to put in place, just all those small, tiny edge cases you face on a day to day basis. Below the surface level, like, all the 1000 cuts, you could say, or the small edge cases and things that can go wrong or that are difficulties you'll face on a day to day basis.

I would love to... I think if I could do again, maybe like write a book about a single modernization project, and what does this one person face every day, what are the things that they're facing on every given day, all these different problems that come up, these conflicts that arise, things that you can't always prepare for, but you need to have some heuristics and some guidelines to help you navigate them.

Eduardo da Silva: So this is sort of a complex problem, right? There are so many dimensions that it feels like we are looking for some heuristics, some ways of navigating options as we go through. And also when we talk about modernization, maybe this is something just to make it very clear, which is, there is no end state, right? So you are improving. But as you are modernizing, probably the goal is that the organization starts learning to making modernization as sort of a default or a part of their way of doing things, right?

Nick Tune: No real end states. Some things might never be modernized. You might keep some systems that are 40 years old. I was talking to the Norwegian government recently, they got systems that are 40 years old. You might keep things around, maybe not 40 years, but not everything will be modernized. But you'll get to a point where the legacy systems are no longer a major blocker to what your business wants to achieve. There might not be a specific day, it might be over time, this pipe around modernization goes down. So yeah, it's not an end, it's not a fixed date. And ideally, you'll be doing small bits of modernization continuously to avoid these big multiyear projects to fix a lot of the technical debt that's built up.

Crucial Tips for Modernizing an Architecture

Eduardo da Silva: Cool. Nick, we are reaching the end. But I wanted to sort of leave our listeners with some final tips from you. So if you had to pick sort of, say, two or three really important things to keep in mind when approaching architecture modernization, what would be your top two, top three tips?

Nick Tune: So, probably aligned with the different sections of the book, I would say, I think the first one is a really strong and compelling business narrative and business objectives. As I said before, if you don't have those business goals, there's a lot of questions. Why are we modernizing? What's the point of this? Where do we focus? Are we building a system that's too complex? If you don't have a reference point to answer those questions, it's very difficult to design an architecture, to start modernization, and to keep it moving. I think the second one will be around the AMET concept. Just think about upfront, how you will support the organization, how you will support the teams, how you'll connect leadership and the teams to make sure they're all aligned. And how you'll solve this problem of, we've got an organization that is not currently modernizing, it's not set up to modernize. How do we get people in the right mindset to modernize? How do we start that? How do we upskill those people? Very difficult to do that.

And then the third one, I would say, on the technical side of things, I would probably say, one of the most interesting technical challenges, one of the things I would recommend doing early, is to think about how you will keep your new system and your legacy integrated. So like I said before, you'll have them both running in parallel. And you might need to have your data replicated across both. You might have some rights coming into the new system. But some of your reads, some of your APIs or web pages still read data from the old system. How will you keep those things connected? In the companies that I've seen doing modernization well, they have the infrastructure, and they have patterns to make it easy to build things outside the legacy and still have it synchronized. They invest in doing that well. And they do that early. If you start building a bunch of custom integrations or teams are having deadlines to modernize something, but they don't have a reliable way to synchronize the data, they'll start building hacks, doing it however they can, and that will come back to bite you. And when companies have this problem solved, it makes life much, much easier.

Eduardo da Silva: That sounds like three great tips to start doing this, and continue, right? As we stressed a few times, this is probably a big effort to start, but then also as you highlight in the book, hopefully it's something that then stays. And people are approaching that more continuously. Nick, was a pleasure talking with you. And congratulations for the great book. And see you around.

Nick Tune: Thank you. Appreciate it. It's been fun.

Eduardo da Silva: Bye-bye.