Architecture for Flow

Introduction

James Lewis: Hello everyone, and welcome to this next edition of the GOTO Book Club. My name is James Lewis, and today I'm chatting with Susanne Kaiser about her brilliant new book "Architecture for Flow: Adaptive Systems with Domain-Driven Design, Wardley Maps and Team Topologies." The title is a bit of a mouthful, but the sheer amount of context and content covered in this book is amazing. It's riding high at the moment with lots of great comments on LinkedIn. I'll hand over to Susanne. Maybe you could introduce yourself and tell us a little bit about yourself. Congratulations, because it is a fabulous book.

Susanne Kaiser: Thank you so much, James, and thank you for having me on this podcast. My name is Susanne Kaiser. I am the author of "Architecture for Flow" and an independent tech consultant from Hamburg, North Germany. I have a background in computer sciences and decades of software engineering and software architecture experience. I used to be a former startup CTO. For the past couple of years, I've been supporting organizations in designing and evolving adaptive sociotechnical systems, optimizing them for fast flow of sustainable value and constant feedback. I try to apply holistic approaches in that style.

Bridging Multiple Business Strategies

James Lewis: There's a lot in this book. The sheer number of techniques, practices, principles, and actual practical workshop ideas that you describe is a tour de force. You've brought all the good things that have been happening in this understanding of sociotechnical systems over the last decade together in one place. How did that come about? How did you get to that point?

Susanne Kaiser: It did not evolve overnight. It evolved over a couple of years. At the beginning, when I was a startup CTO, I was diving into domain-driven design because we were dealing with a monolithic big ball of mud with messy models and unclear modularity boundaries. I came across domain-driven design, which helps you decompose your system into components called bounded contexts. I was also interested in microservices as an architecture style. I remember one of my first talks where you were sitting in the front row—that was my very first talk, by the way. All the authors of books and blog articles were sitting in the front row and I almost fainted.

That was my first journey into domain-driven design and microservices architecture. Then I became interested in serverless technologies. Whenever I attended a serverless technology conference, Simon Wardley was there giving a keynote, and that was the first time I encountered Wardley mapping. I thought it sounded really interesting and wondered how I could apply it to my own context. I realized it could go very well together with domain-driven design because Wardley mapping—specifically the visualization of a landscape with the value chain on the y-axis and evolution stages of genesis, custom build, product, and commodity on the x-axis—blends well with the subdomain categories of domain-driven design: core, supporting, and generic subdomains. One provides competitive advantage, another is non-differentiating but specialized, and the third is non-differentiating and non-specialized. I thought it was a good idea to complement them and visualize them on the Wardley map to drive the value chain and help make build, buy, and outsource decisions.

That was driving me until 2020. At the beginning of 2020, when I read the book "Team Topologies" that came out at the end of 2019, it was like a missing puzzle piece. We have a lot of discussion about Conway's Law—that an organization's communication structure is mirrored in system design. Team Topologies, with its well-defined team types and interaction modes, brought everything together as a holistic toolset to focus on building the right thing and building the thing right.

I thought bringing together business strategy, software design, and team organization from three different perspectives made sense to my own context and to my clients' contexts. But it didn't evolve overnight. It took a couple of years, and then trying to apply them depending on what current problems clients are facing, trying to solve them with this holistic toolset and identify blockers to flow and how to mitigate them. That's the gradual evolution in a nutshell.

Bringing Clarity to Value Streams and Team Alignment

James Lewis: You've covered so much. What I really liked about it, and one of the things I realized when we were reading it, is how you brought terrific clarity to a number of ideas that have been bubbling around in my head for years. You've described them in the book and brought them together such that they make more sense than they do individually—they create something greater than the sum of the parts. You touched on one thing with team topologies. We have these stream-aligned teams in Team Topologies, which sounds great, but what is a value stream? How do we identify what value stream the teams are aligned with? You've brought clarity to that through this mapping process where you identify these value streams as part of the value chain, delivering value to your customer. Is that what you were hoping to get to? Did that emerge?

Susanne Kaiser: That emerged because I was thinking about how when we talk about a Wardley map, it visualizes the evolution of a value chain where you have your users and user needs at the top of the value chain. It's reflected as the anchor of the map with components of the value chain connected, coupling user needs directly or indirectly. Whenever we want to optimize change, we need to identify blockers to flow. Then we come to value streams where you have the end-to-end flow. Customer-facing value streams could be aligned to user journeys, which might be reflected in user needs as well. But also, a component in your value chain could have a value stream involved as well.

I consider a value chain supported by a network of various value streams. You might have core-related value streams that support their outcome and then support the component of the value chain. I usually try to use Wardley maps as a high-level entry point for assessing your flow of change. For example, what happens in your value chain? What components are involved in that change? What teams are involved in this change of a specific component to make this change effective? That automatically brought me to value streams.

It's not only the external customer point of view but also the internal customer view where teams become the users of your platforms. You have external value chains and value streams, but also internal value chains and value streams, connecting them together to get an understanding of how change would flow through the system. What components are involved? What steps are necessary to make this change effective? Then, coming from Theory of Constraints, you identify bottlenecks—where constraints are slowing down your overall performance.

If you're not addressing your investments to improve or mitigate constraints, it doesn't matter if your teams before the constraint are performing rapidly. The backlog of the constraint team will pile up, or teams behind it waiting for the constraint team will be idling. All effort has to first address the constraint. That's how to prioritize where to start unlocking blockers to flow—to address those that require a lot of changes and are slowing down overall system performance.

James Lewis: You touched on something quite important: the concept of local versus global optimization. You might have a team that's powering along, super high-performing. I tend to use Daniel Terhorst-North's thought experiment on this: what would happen if that team was infinitely fast? If they weren't the constraint, where are the other constraints in the system? How do you identify those? How do you define the flow of changes upstream and downstream?

Understanding Team Dynamics Through Context Mapping

There's a section in the book where you talk about different relationships between teams and different types of change coupling between teams. I think there's a specific technique you espouse that passed me by in the domain-driven design world—is it context mapping?

Susanne Kaiser: Yes, context mapping. I was trying to blend it in. Context mapping comes from domain-driven design and gives you an idea of how your bounded contexts are integrating with each other from a domain model perspective—which domain model needs to integrate with another bounded context's domain model. They don't usually live in isolation. It also reflects the team dynamics underneath, where teams own these bounded contexts and take responsibility for specific bounded contexts in your system.

There's also a power dynamic involved that you don't usually see when you try to address change coupling. Change coupling defines what side effects occur when you change one part of your system—what other parts of the system you also need to change. Context map patterns visualize and make change coupling explicit. This isn't always technical integration like calling each other's APIs or consuming events on a message bus. Sometimes it's "separate ways" where you have two bounded contexts that aren't technically integrating, but they're duplicating functionality. There could be different reasons—maybe the teams don't communicate well, or the integration might be too expensive, so they prefer duplication of business behavior and go separate ways. This isn't reflected in an API integration at all.

You might see it when you analyze code commits. When you want to introduce a behavior change, different parts change simultaneously or in a short cadence after each other. Whenever I change this, this team is also aligned to that one, and they're not calling each other—instead, they've decided to go separate ways. This needs to be made explicit.

Another example is team dynamics or power dynamics in the customer-supplier pattern. When you offer an API to another team's bounded context, you're usually the supplier. But as a consumer of the API, you can gain additional influence on the supplier's backlog priorities. This isn't reflected in technical integration, but you could have a "veto customer." For example, you provide an API to multiple consuming customers. If one team would like a change but another team that consumes the API has a customer-supplier relationship and has a kind of veto, they can block it. That's not reflected in any technical API, but the dynamics of team influences are very relevant and must be considered when designing systems.

James Lewis: I really like the iconography that you introduced to talk about team dynamics. There were some penny-dropping moments for me—"Oh, this is what I've been trying to describe over the years." To have that common, shared language—this is why I think Team Topologies is very important as well, having a shared language—is quite brilliant. The iconography runs throughout the book. Gregor Hohpe always talks about the importance of having a consistent iconography, going back to "Enterprise Integration Patterns." You seem to have developed that. Did it take a long time to build up this very self-consistent system throughout the process?

Susanne Kaiser: It has evolved over time. I'm really a visual person. Whenever I read a book, I visualize it by illustrating it or creating sticky notes. That's the reason all my illustrations contain sticky notes—I guess my brain is organized into sticky notes. I recently made a little behind-the-scenes presentation. My book has 126 illustrations, and 125 of them are hand-drawn. These illustrations contain 1,599 sticky notes and 248 hearts, because everyone—at least the user notes—have little hearts. A lot of love goes into that one.

James Lewis: You're not a massive Swiftie, I don't know, because of these hearts?

Susanne Kaiser: No, I'm not really a Swiftie, I would say, but maybe it aligns with her. It has evolved over time because for me, it's better to transport something complex and complicated with visuals. I've also provided an icon library which is very simple. I don't want high-polished magazine-style presentation. You have to see that they have rough edges, not all linear and shiny. Instead, it's just like me—it's my illustration. It represents me. That's a lot of things that I value. Over the years, I've developed this presentation style. Usually when I give a presentation, you see the same style as well. It's my way of addressing complex topics.

Collaborative Modeling Techniques and Practical Workshops

James Lewis: It very much reflects that, and it's great. I can see myself using these next week. They've entered into my imagination. Changing the subject briefly, one other thing I really liked running through the book—it's common to have a worked example through the book, but you've got very specific techniques with worked examples that you apply. You've got event storming in there, you've got domain storytelling, a wide variety of practical workshops that people can run. You can just look at them, find out more, and then run them. That must have been a big challenge, trying to bring all that together.

Susanne Kaiser: That was a real challenge. I would have loved to include even more, but at some point I had to cut it down. It's a challenging balance—briefly touching on them but giving enough information that allows you to hop on and dive deeper into that topic. I wanted to keep the three pillars—Wardley mapping, domain-driven design, and Team Topologies—as the core principles of the book, but also express that they're not mutually exclusive to others and that they're complemented with other collaborative modeling techniques that you can apply.

I didn't want to have them as a side note like "Okay, these also exist, but if you want to dive into that, it's beyond the scope of this book." Sometimes I guess it's my thought process—we have to at least address, for example, Theory of Constraints or value stream mapping, because otherwise I would miss something. But on the other hand, I have to deliberately leave out some areas because otherwise it gets too widespread. I didn't want to have an 800-page book. First of all, it's hard to carry around, and second, it's really hard to read in one go. I wanted to give you enough appetite that you can move forward and get your own insights and experience, to spark ideas. One of my goals in writing this was to have a cohesive view but also give you new ideas where you can maybe start tomorrow to apply these or to learn or dive deeper into other areas as well.

James Lewis: It very much reminds me—if you'd written about everything in detail, it would have changed the name of the book to "A Theory of Everything for Sociotechnical Systems: All the Things." I think you did just enough with that.

The Architecture for Flow Canvas: A Practical Framework

James Lewis:  The other thing I like with the examples is how you transition into how you can do this yourself, which is the Architecture for Flow Canvas. You walk through all the things that you do when you use these techniques. Can you talk briefly through that? I find that super interesting.

Susanne Kaiser: The canvas—the Architecture for Flow Canvas—is an opinionated, structured approach for how to bring those three together. It starts with assessing your current situation and then propagates designing your ideal future as an initial idea of how to move forward. It's not a rigid framework. It gives you as little structure as possible and as much direction as needed to move forward to the next step.

Assessing the current situation starts with the teams of today, and then envisions the future—the teams of tomorrow. Assessing the current status quo starts with the teams, trying to reflect the current team structure, current team responsibilities, team dependencies to other teams, and the way of working today. Then, going team by team into what blocks the flow of value and flow of change, identifying blockers to flow that we want to mitigate, and meeting teams where they are. Usually teams know what doesn't work on a daily basis, but we also collect elements that support the current flow of change to identify areas we'd like to keep in the future as well.

Then we get an understanding of our business landscape, visualizing the current business landscape the organization is operating in and competing in. We start from the user perspective with users and user needs and the value chain, mapping them to evolution stages just to get an idea of our current business landscape and where we want to move to.

From there, we blend in gradually into the ideal future—the to-be state—by categorizing our problem space, identifying what user needs belong to core domain-related specifics, supporting subdomains, or generic subdomains. Where are the areas where fulfilling those user needs generates competitive advantage—what's differentiating and specialized versus non-differentiating and specialized?

Then we switch to the solution space. Now we're going into the ideal future and trying to modularize our solution space into smaller, modular components. We bring in one of the architecture principles from continuous architecture: architect for change and leverage the power of small. If we're dealing with a legacy system—usually a monolithic big ball of mud with messy models and no clear modularity boundaries—how can we split it apart into modular components from a business behavior point of view? That's where I bring in the bounded context from domain-driven design, having event storming sessions and so on.

Then I bridge those together. I come from the subdomain categories—core, supporting, and generic—and blend them with the bounded contexts we've identified in the solution space, bridging them together and visualizing the future landscape. Which parts do we want to build in-house? Where would we like to use off-the-shelf products or outsource to utility suppliers? Then we align teams around it, blending in the Team Topologies perspective. Bounded contexts are usually good candidates for stream-aligned team ownership. We also identify services that support the stream-aligned teams delivering their work, so platform teams can come in. They're usually part of the infrastructure components of the Wardley map.

You can switch from an external customer view to the internal team view and also provide Wardley maps where the teams become the users. What services are needed that can be provided as self-service capabilities from platform teams or platform groupings to support the stream-aligned teams delivering the flow of value?

This sequence might be different depending on the context, but this is the canvas I evolved and developed during my client engagements. I want to highlight that the ideal future we try to establish with this canvas is not set in stone. It's continuous architecture—continuous evolution. The moment we gain more insights, we need to readjust our understanding and course-correct our ideal future as well. It's adjusted due to feedback loops and evolves in this continuous, iterative approach.

James Lewis: This is about giving the teams the tools to do this themselves going forward?

Susanne Kaiser: Exactly. One of the success factors is when the teams make the decisions themselves. When they have ownership of the decisions—not imposed on them from others, like from the management level—when the teams are making decisions about their ideal future, how they see it or how they'd like to envision it, that enables them to take full ownership of the solution space as well.

I usually detach the decision of who's going to be in what team—the team membership assignment—until the very last, because then you have emotional attachment to that solution. I try to defer the decision of team membership assignment to the very, very last. First, we'd like to focus on what are our problems and what could be a potential solution. Then, how do we get there? Then we consider whether we need to reorganize our teams or maybe it's just a shift of responsibilities—the teams stay the same, but the responsibilities shift. That depends on the team's context as well.

James Lewis: What I like about it is it's a clear, repeatable, opinionated framework that anyone can use. You can pick it up by reading the book. I know there are resources as well that people can access to explore and do this themselves.

The Most Important Takeaway from the bookI've got one final question for you before we go. If you could take one thing away from this book, what should I take away? What would you like me to have taken away the most?

Susanne Kaiser: One thing is, instead of diving directly into the solution space, first try to understand the problem space. Also, apply a holistic view—not only from software architecture but also from team organization and business strategy points of view. To build the right thing right, we need to have a holistic approach to design, build, and evolve adaptive sociotechnical systems. Start with the problem space. That's what I would suggest.

James Lewis: I think it always comes back to starting with user needs, doesn't it?

Susanne Kaiser: It is. Clayton Christensen, the inventor of the Jobs to Be Done theory, stated that people buy products in order to get a job done. If we don't help users get the job done—whether they're external customers or, later on when you provide platforms, internal customers consuming the platform—if it doesn't accomplish their job of delivering their work, then it's not beneficial at all. We always have to have this in focus: we have to fulfill user needs, either from external customers or internal customers.

James Lewis: That's a perfect way to finish. Thank you so much for joining us for this edition of The Book Club, talking about and celebrating your new book, "Architecture for Flow." I heartily recommend it to anyone out there who's interested in these topics. It's been a brilliant read. Congratulations again. It's fantastic. Thank you very much for joining us.

Susanne Kaiser: Thank you so much for having me.