Principles of Web API Design: Delivering Value with APIs and Microservices

 James Higginbotham, author of “Principles of Web API Design: Delivering Value with APIs and Microservices”, outlines the key points of creating and using APIs in today’s world. In the conversation with Mike Amundsen, author of “Restful Patterns and Best Practices for API's Cookbook” you discover the principles of James’ ADDR process and how job stories and event storming contribute to a successful API launch. Furthermore, they touch upon key terms such as minimum viable portal and why API boundaries are so hot at the moment.

Intro

Mike Amundsen: Hi. Mike Amundsen here with the GOTO Book Club. I'm here to talk with James Higginbotham, a fantastic author that I've had the opportunity to work with a few times. And known for several years. James is an executive API consultant at LaunchAny, experienced with API strategy, software architecture, training teams, and API and Microservice Design. James guides enterprises through their digital transformation journey and training alignment between business and technology through product-based thinking to deliver a great customer experience.

James, it's so good to be talking with you again. How are you doing?

James Higginbotham: Great. Mike, thanks for being here. Appreciate it. Looking forward to this.

Mike Amundsen: Absolutely. And we're here to talk about your book, Principles of Web API Design. I love the book. You know, buyer beware. I read early copies and early drafts, and I was happy to write a forward for the book.

So I really, really enjoyed it very much.

The first question I'm just going to ask on this on this is why write this book? Books are huge. Books are big investments. What prompted you or motivated to write this book?

Why a book on WebAPI and Microservices?

James Higginbotham: The reason I wrote the book, is I've been helping organizations for nearly a decade now to understand how to approach their API programs and to design APIs that are designed to deliver a great developer experience. So through the workshops, I've delivered and the time I've spent helping people with designing APIs and designing their API programs, I wanted to be able to capture all of that that I've learned and share that with others so that everybody can benefit from the insights that I've had an opportunity to gain over the last decade.

Mike Amundsen: One of the things that I really love about the book is, while it's focused on web design and design in the real sense, I love how you started out early on talking about digital capabilities and business alignment, that's one of the key elements in your process. It goes all through the experience, including documentation, portals, and change management over time. So this is a thing that I really love, all these experiences, sort of, add up to what design is like, design in the bigger sense. I think that makes the book a bit unusual in that sense, so I really love that.

What I also think is really, really cool is that this book is part of the Vaughn Vernon signature series, this "Addison-Wesley Series," and this is heady stuff. How did that work out, because Vaughn is known for his enterprise architecture stuff? How did you two meet, and how did this arrangement work out?

James Higginbotham: Vaughn and I have known each other for a while but really this whole journey started when he sent me a direct message on Twitter and said, "You know what? We should work together on something. We're working on similar things." He was working and has been developing a framework for some time that's a reactive framework to help people design software by applying domain-driven design principles, and also to help support this growth in microservices that had been out for a while. He has this framework called Vlingo and he said, "Look, I've been working on that for a while. I've been teaching on DDD for a while but you're focused on APIs. I'd love to sit down and talk about, how can Vlingo support APIs? How does that framework need to think about APIs?"

So I started sharing a bit about what I coach teams on, the perspective I came in from, and he had mentioned to me that he had the "Signature Series" starting and I thought, "Wow, that's gonna be great for DDD fanatics." And getting deeper and deeper, he has the two books already that he's written, or three now, I think, just focused on DDD purely. But over time, what we ended up talking about was that his "Signature Series" was going to go beyond DDD, and that he really wanted to have a book that would talk about API design in there. So I shared with him some of the work I'd done before and the perspective, and a table of contents of what I would see going into it, so it just took off from there.

It's not unlike when I first authored my first book for Que Publishing back in '96, '97 with Jeff Schneider. It was called "Using Enterprise Java," and it was one of the first Enterprise Java books that came out before Enterprise Java, or J2EE had even been announced in '99 at JavaOne. So I stumbled into it the same way, where Jeff sat down and said, "Hey, well, have you ever thought about writing a book?" And here it was years and years later, Vaughn comes to me and says, "Have you ever thought about writing a book about web API design? It would fit great in the series."

So it's just been fantastic. Vaughn's a great series editor. He has a heart for helping developers and giving the author the freedom to tell the story to get that narrative out. And Addison-Wesley has been great, too, to give us that flexibility to write the right number of pages, not write a lot of filler, get right to the meat of the problem, and deliver something that's a really just high value to developers and non-developers alike.

ADDR process: Align, Define, Design & Refine

Mike Amundsen: That's one of the things I noticed in the book as well, and we talked about this when I was doing the review. It's a dense book. I mean, it's a good read. I really enjoyed it very much and I keep it handy quite often. In fact, folks should know that I've ended up helping teach some of this material from this book with you a couple of times, and just I love working with it. You've got this real keen sense, and it's not just enterprise-level but all sorts of levels about, sort of, drilling into the things that really, really matter. That's one of the things that I wanted to make sure we talked about when we got together was your key organizing principle around the book which you call ADDR. I hesitate to ask it, but can you, in a nutshell, or in a short way explain what ADDR is and why you use that as your guiding principle?

James Higginbotham: Let me see if I can summarize this quickly. So ADDR stands for align, define, design, refine. It's the four phases that anyone really goes through when they're designing an API. They may not be consciously thinking about it, but they're starting, or should be starting by going from the requirements, what is it that the API needs to do, and aligning on that, not just on technical requirements but also the business requirements, product requirements. And then we evolve that into a definition of what the API needs to do and then we apply the right API style or styles during the design phase. And then we use refine to spend a lot of time getting feedback.

We recognize that we do need to write a lot of code and that sometimes we can do this work in parallel or we can do it iteratively, but we also have to recognize that APIs are forever. The fact is that once we establish a contract, this isn't like code, an API for some code. It's in a shared library in the organization, and if we factor to make a change and we coordinate with a few internal developers, we can make a breaking change, or we can uproot something completely and swap things out. It's not something where we can just run some unit tests and integration tests and determine where we broke things. We have, most often consumers of our APIs that are outside our organization are outside our control, so our contracts are gonna be fixed. We can add to them. We can enhance them but we can't break them, or at least we don't want to do that. We don't want to deliver that kind of poor experience.

So the ADDR process is really designed to help people go from requirements to an API design they can feel confident finalizing the code for. It doesn't mean you can't code in parallel or do some prototypes to mitigate risk or explore some aspect of the technical solution. But we do want to be careful not to just start writing code and delivering an API that we haven't thought about the design for and then create a lot of breaking changes because we have to go back and make last-minute changes, or quick, fast, follow-on changes.

What are job stories and why do they matter?

Mike Amundsen: Sure. But again, that's one of the things that really stands out in the book itself and when I've seen you teach the material as well. This sort of business alignment is the starting place. I think you do a fantastic job with that, and one of the things that come out, is one of the devices, and we've talked about this a couple of times throughout our careers. We all have some kind of way of approaching this. I love your idea of job stories, of the way that you, sort of...this is one of the key assets that you put together, right, which is a little bit like a user story, a little bit like an API story, but you call it a job story. Can you, sort of, expound on where did that come from? How does that work out, and how does that be a part of this ADDR process?

James Higginbotham: Yes, job stories, I didn't invent job stories but I definitely took advantage of them, because, for me, it's the right tool to apply at the right time. Job stories are rooted in the job-to-be-done concept that Clayton Christensen introduced years ago, and he introduced and built upon a few concepts from the '80s, one of them was the voice of the customer. How do we design a product that represents the needs of the customer, not just the needs of the business building or delivering the product or service? And he framed that in the position of jobs to be done, what is the job to be done? And he did this from the idea of we're helping, coaching startups.

If I'm a startup and I'm trying to figure out, how do I go to market and start to get traction quickly, it's not that I'm bringing some new whiz-bang feature, or technology, or anything to the market? It's that there's a job to be done because of a problem that exists. A problem exists, I want the solution or the outcome, a certain outcome to be the case, solve the problem, get into this positive position or address this issue, move on. So in that gap in the middle is the job to be done, and that's where an organization, startup, individual, or large enterprise can start to think about, what is that job and how am I gonna go offer a product or service that's gonna solve that?

When I started seeing that and I bumped across Alan Klement's job stories, which is a way to frame the job to be one in a repeatable way, it creates this very thoughtful, easy-to-use framework that frames things a bit different than the way our industry has used user stories traditionally, where we're, as I want to, you know, so I can, or something like that, some sort of template like that, where it tends to be more feature-centric. This really helps us to step above that, get in there beforehand and start empathizing with those that are, in this case gonna be using our APIs. So we write this, when, what's the triggering event? I want to, what's the job to be one, so I can, what's the outcome?

If we center everything on that outcome, then everything that we do for our API design, for the solutions that we built, and the UIs we build, anything else, they'll center on that outcome. And so it's a great way to align also our APIs with teams that are performing UX. So anybody doing UX work, they're thinking about jobs to be done.

So now we're starting to align with other team members that may not be necessarily involved directly in the API but are going to be involved in the solution and start to think about how our API is gonna help move that forward, deliver that job to be one so that it can be automated or integrated in a mobile web app, whatever we have, and then to accomplish or produce that outcome. So that's where we like to start from and it's a great way to talk about things without talking about HTDP and all those technical details when we have the stakeholders in the room. So we can have a business, we can have a product, and we can have tech all talking about the same thing.

Mike Amundsen: I think that's the thing that really, really struck me when I saw the way you were using it in the process, and that is, before we talk about solutionizing, I think I've heard you say this phrase before. Before we talk about solving the problem, let's make sure we're defining it really, really well. You've got a line in the book, I actually had to pull it out because I really love it. Teams want to have the confidence that the API they plan to deliver meets the stakeholders' needs. I think that's a wonderful way to just hone right in on...that's another kind of empathy, right? It's like, let's make sure that the teams know this is what's really needed, this is what's really wanted, and the job stories can really help that. I love that line.

I think the other thing that comes to mind when you talk about Klement and you talk about Christensen is the Poppendiecks, Tom and Mary Poppendieck. She's got this great thing about the biggest mistake is not building it wrong but building the wrong thing, building the thing that's not really needed. 

Event storming

Mike Amundsen: So I love that you're really able to use job stories as part of this. Now job stories start to sound a little like another thing that you talk about in the book, event storming, but they're really a different approach, right? They're really, sort of, different things but they have, kind of, the same goal, is that right? I mean, do I get that right? Are they in the same league or something?

James Higginbotham: They do. So in the line phase, I encourage several things. One is to have that customer empathy so that we can start by mapping out our job stories and we can start thinking about our API from those outcomes, not just moving data around or connecting system A to system B. There are two other tools that I recommend, one is to take job stories and break them down into activities and steps. So what is the sequence of steps required to get us to that outcome? So now we know the outcome, what it is, and we know the job to be done, how do we break that job into smaller increments? So I recommend that and that's built upon business process engineering, or BPE, that's been around for years. So it's just simply taking those steps and breaking them down.

There are times when sometimes things are uncertain or we're not quite sure where things need to go, that's where event storming comes in. So event storming is not really like a brainstorming activity as much as it's an exploration, communication, and alignment technique. Traditionally it's done in person on a big wall with a big sheet of paper, and lots of sticky notes, and pens, and everybody's contributing and putting things up, and moving things around. But it's really a framework that lets us think on different levels. It lets us think of a big idea or big picture level, what's the big idea of the whole solution or product that we're trying to do, as well as, what are the smaller ideas? What are the steps? What are the events we're producing? What are the commands we're sending? So it blends some of the ideas of DDD, but in a way that we can have non-technical people in the room, our subject matter experts, our domain experts to all talk about this.

So when things aren't clear, and we either can't figure out what the job stories are, or we figured out the job stories but we're not quite sure what those activity steps are that our API is gonna need to power digitally, then I recommend using event storming. There's a section in the book that talks about it. It's not meant to replace Brandolini's book by any means, but meant to kind of summarize, how would you use it in the context of ADDR to surface up those things that your API is gonna need to do, and also to find other aspects and elements to fill in the gaps of your API?

So it's used, I've used it a lot around the world. I've trained other people on how to use it in my way just like Brandolini has done and others have done in their way. It's a lighter-weight version of what Brandolini offers specifically for helping to narrow down the scope of the API and what it's gonna need to do, get everyone aligned. It's always really fun, a great activity, and I talk in the book about how to facilitate those kinds of discussions. If you need to facilitate and you've never done it before, it gives you some hints there, and also a bit about how to manage it virtually since we're more and more moving to a distributed virtual world where we don't have everyone in the same room anymore like we used to. So it's a great technique to be able to expand or help to surface those job stories and those activities and steps.

Mike Amundsen: You know, especially listening to you now talk about it, it really...one of the other things that Brandolini talks about, and I think you mention this in the book but I know we've discussed this idea before is, Brandolini points out that often as systems architects, software architects, developers, tech leads, and so on, and so forth, we're actually operating in a domain that we don't know really well. I might not know insurance really well when I'm trying to help a company solve their problems, and come up with job stories. So event storming and these other things are a way to kind of get us into the domain to learn about the domain and learn the language, which is another key element of domain-driven design as well, right?

I love the way you use event storming and the way you use these job stories is really a way to elicit parts of the domain or the activities that sometimes I have a real challenge for when I'm trying to help someone. So I think that's really, really, really handy. I think that probably I'm guessing here, I know you've worked with lots of different organizations to help them with their software, their enterprise, and design. That must be a challenge for you, right, in the sense that you're coming to a domain that maybe you haven't had a lot of experience in, or a set of particulars that are new to you.

James Higginbotham: It is. My background is fairly diverse. I've worked in a lot of different industries. I've worked in commercial insurance, banking, even helped a startup airline get off the ground literally, helping a new startup airline, while they were getting their pilot certified and their first plane certified to fly from JFK to Stanstead, we were working on building all their infrastructure and tying in the mainframe systems, and other things in the airline industry. So as a result of working in a lot of these different domains, it's forced me to have to learn a domain pretty quickly. So my career has been based on that, and a lot of the software analysis and design techniques that were taught years ago that we've since lost or got mixed in with other techniques have really come to help me figure that out, and I've tried to bring that into the ADDR process as a result.

But the other thing to keep in mind is, that just because you've been in an industry for a while doesn't mean you necessarily know that particular part of the organization. I was talking to someone who was working in commercial insurance for 18 years and they have never been on the claims side of things when someone needs to make a claim because of an auto accident or whatever else, a flood or whatever else, and they need to make that claim. They never worked in that part of the organization, so they really didn't understand how it worked.

So even as someone who's been in the same organization or in that same vertical domain for almost two decades, they still need those techniques to understand how to break down a domain and think about it in new ways and get that knowledge transfer from other domain experts. Using those kinds of techniques like event storming, job stories, and activity steps really helps to ensure that everybody is on the same level playing field, using the same vocabulary, the same ubiquitous languages we would say in DDD terms. We all mean the same things when we're saying the same things, and that's really, really important, and absolutely important for API design.

Mike Amundsen: Yes, and it reminds me of this old saying attributed to Confucius. Someone asked Confucius, "If you were the king of the world, what's the first thing you would do?" And he said, "I would make names correspond to things," which is the hardest part, right? When I say "widget" is that the same widget you mean, and so on, and so forth? So it's a huge part of this detailed process, right, and I think that's one of the things. We've talked about the alignment with business and now we're really, kind of, talking about the “define” part of your ADDR, this idea of finding all the details, the activities, and the steps. One of the things I really love about your approach is still at this definition process for API design, you're not talking about HTDP post, or get, or put, or GraphQL, or anything. You're really still focused on all of the details, all of the language, ubiquitous language you would say in domain design of all these elements. I think that's so, so critical in the process.

The difference between versioning and revisions

Mike Amundsen: I think one of the other ones, sort of, jumping ahead, there are so many things we could talk about. One of the other ones that really stood out to me in the book was when you actually start talking about the life cycle of the API, you have this really nice detailed life cycle process, one of the change management issues we deal with all the time is versioning, right? I always say that if you wanna start a nice fight in a room of tech geeks just mention the word versioning and see what happens. You describe in the book a really interesting difference between versioning and revisions, versions and revisions. I wanna know if you could, sort of, explain what that is, where that comes from, and why that's so important?

James Higginbotham: Sure. So credit where credit is due, this really stems from the work of Microsoft. Daryl Miller shared this some time ago. Microsoft was trying to figure out, "How do we manage versioning for our APIs that we onboard onto our own gateway inside of Azure?" And they wanted some guidance and they wanted to define some terminology that would really be clear and understandable for people publishing an API and managing it with a gateway, and determining, am I routing to the same set of instances of an API, or am I routing to a whole new version? And what does that mean?

So the way that they've defined it and the way that I capture that in the book is that a revision is like a minor change. If we think of semantic versioning, or SemVer, it's like a minor version. We're making a change but it shouldn't be breaking. We should have the same functionality, we're just adding an enhancement, adding a new operation, or perhaps sending back something on our response that we weren't sending before, but clients that are integrated with the previous revision could safely ignore. It shouldn't break anyone.

The revision gives us the freedom to make improvements to our API iteratively as long as we don't break what's already there. So part of the ADDR process is aligning and defining our API initially to understand what we need to do, but then as we move forward and start to design and deliver the API, we could chunk it into smaller chunks and iteratively deliver it, and we could take advantage of that revision semantic to support that. Now compare that to a version where those are like major versions in SemVer.

Those are major versions that we're probably gonna introduce breaking changes and they should be treated as separate products, meaning if I have a customer on version one and I introduce version two, I really have to sell them and convince them to do the work necessary to migrate to version two. That's a much bigger ask than the revision changing behind the scenes and maybe they're not even aware of it, that we've added a new operation, we've improved something slightly, but we haven't broken that contract. Their code will continue to run and they don't need to ask someone to go fix something.

So version and revision gives us two different elements of the life cycle, allowing us to be agile in how we approach our design, but also recognizing that there are times when we need to make a fundamental change, and we have to then start thinking about how do we own that change, how do we communicate that change, and how do we encourage people to migrate to that new version, spending the necessary money, and time, and effort to jump to that new version? So we have to think about, what are the incentives, and what's the value? Not just the value it's providing to me because I got to change some backend framework, I got to clean up some code, delete a bunch of lines of code that were unnecessary because I was able to create a slightly more efficient and effective API design.

Anyone that's using our API probably doesn't care about that and they don't wanna spend the money to migrate to a new major version to gather that up. So that's what I like to recommend for that life cycle aspect of API design, and it gives us flexibility. It helps us from having to do a big-bang design upfront. But we wanna do the investment to align and define where we're headed and then break that up, and then learn and incorporate our feedback just like we normally would, and our little agile processes, no matter what framework or methodology you might be using. It allows us to do that while still recognizing that APIs are forever, and so we do have to be very careful about how we approach our designs and make breaking changes.

Mike Amundsen: The things that I take away when I was reading that material and in hearing you talk today, too, is this notion that revisions are these minor changes that the service can choose, that the API producer or the API server can choose to make as long as they're backward compatible. But the major, the sem major, the sort of, breaking change things, version one, version two, version three, these are like you say, they're different products and consumers make the choice, right? I'm going to choose when I go to version two, not you. So this idea, I can continue to make minor changes. If I make a major change, it's a new product. That's a great way, I really love the way that you talk about that in the sense that that's a new thing on the shelf. That's a new product that people need to deal with, and they can choose classic Coke or they can choose new Coke. You have to sell them on the idea of why it's valuable to move on. I really, really love that idea.

API Boundaries

Because I think one of the things, and you touch on this later in the book, one of the things that we deal with as designers, everybody who's a developer for APIs, is it's not a monoculture, right? There are APIs that have been here for years. There are APIs that are in experiments. There are APIs that just got released. There are APIs that are gonna be changed, and as you grow that ecosystem, if you have hundreds and hundreds of end points, and you have dozens and dozens of API packages, there's always gonna be somebody at some new revision, somebody that might be thinking about a new version, somebody that's experimenting. It's a complex ecosystem, right?

Mike Amundsen: You've used Werner Vogels, I think, is the person I attribute to the quote, "APIs are forever," right? You've used that a handful of times. That's a real challenge. How do you help people get a handle on the notion of that ecosystem, of that life cycle, that multi, varied world that really everyone's dealing with?

James Higginbotham: There are a few techniques that I recommend. One is making sure to communicate to your consumers what the expectations are. So if I'm standing up an API to experiment with something new, then marking it as experimental and making sure that people understand that this API might go away, as opposed to something that maybe I'm pre-release and may have some design changes that may be forthcoming. It could break their integrations but for the most part we're getting closer and it will be supported. And then a supporting element or promise that says, "This API is in production and supported, and we're gonna follow revisions and versions, and we're gonna be very careful about how we change things."

It's also recognizing that sometimes we have to step back and take an inventory of what we have, and make sure that we understand, what APIs do we have today and how do they fit in the ecosystem? And then also just the idea of how we bundle our APIs. Sometimes we might have APIs that are very specific, have a specific use, and others that are more generic. So we have flexibility in how we bundle those things together so that we can really think about the productization of our APIs. We have lots of internal APIs in a large organization, how are we gonna bundle them and offer them as a cohesive product so that they can solve problems and work together to accomplish bigger and bigger things?

There are techniques and ways that you can do that so that you can approach using ADDR or just approach any kind of API initiative from the perspective of recognizing that not everything is gonna be greenfield. There are going to be existing APIs and we need to work within those bounds, and we need to recognize where they're at, what they're doing, how they fit into the bigger ecosystem, and then, what kind of promise are we giving to our consumers when we push it out the door, or make it available to other developers inside our organization as well? How is it gonna be supported, and how are we gonna be able to continue to leverage that, or will we? Could it go away tomorrow because we're just trying something out? So it gives us that flexibility that we need.

Mike Amundsen: I mean, it sounds to me, this is another topic that is sort of an evergreen topic whenever I get designers, and architects together, and that is the topic of boundaries, of the bounded context of where you draw the line. This endpoint belongs here. These endpoints belong over here. You talk about this in the book, how do we approach this notion of API boundaries and why is it important to do that?

James Higginbotham: The boundaries allow us to really focus on a couple of things. One is the scope of the API. What is it designed to do, or what are its responsibilities? And the other is ownership of the API. So if we know what the scope and responsibilities are, do we know who's gonna own that API? A lot of organizations are producing a tremendous amount of APIs today, but a lot of them are being pushed out and then forgotten. It's unfortunate because there are opportunities to take advantage of a product ownership life cycle of, how do we not only just get that first version out, but how do we make it better?

So being able to find our boundaries, determine who's gonna own something, what's inside that bounded area of that API? Not a bounded context from the DDD perspective, because that's a bit of a smaller boundary sometimes, but more of the boundary of what the API is responsible for. That may reflect the way the organization operates, the way the product is designed, groups of features, operations, or particular areas of it. So there's a bit of nuance to it, but we use things like the activity steps and the event storming early on to help find those boundaries and it gives us a starting point. Those may move. They may shift as we learn more, but it's really, really important.

We don't want to have our APIs exhibiting those anti-patterns of either being one operation per API and having to hop around 20 different sets of documentation to figure out how to get something done, or I have an API with 300 operations and I gotta dig through this massive documentation to find that one operation that I need to add to what I'm doing today. So we wanna avoid a lot of those anti-patterns, use those boundaries to tackle and avoid those things, and to really, kind of, reason about how our API lives, and how does it fit in the overall ecosystem of APIs? How is gonna be owned? How is it gonna be supported? And what other APIs in other bounded areas might complement it without overwhelming that consumers? And it also kinda goes to that bundling idea I mentioned before of we might be bundling different portions of an API together into different product offerings to meet the needs of different personas, or market segments, or whatever our target audience is.

Mike Amundsen: Yes, I think that's another thing that comes to mind just in general in my experience as well. We're seeing more and more of this composability happen, right, where people wanna use, "Well, I wanna use this user management API, and this payment API, and this shipping API, and our product API." We're composing all those APIs together into something, and all of a sudden that composability, now boundaries are really important, right? Because I have to, sort of, shift. I have to pick everything up and hoist it over the boundary if I'm gonna send it to payments, right? And then I have to take certain information and hoist it into the customer experience, the user management side, authorize accounts.

So we become really focused on boundaries. In fact, I don't know if this has been your experience, but I think I'm seeing more and more questions about boundaries over the last couple of years than I have ever before. I used to hear lots of questions about, what is an API, and why, and how do I design one, so on, so forth? Now I'm getting a lot more of this, where do the lines cross? Where do they meet? How do I compose well? I wonder if you had that experience at all, or if that's shaped anything about the way you talk about boundaries now, this idea of composability?

James Higginbotham: Yes, it's shaping a lot of where I came from in this book and just generally when I'm working with organizations. The idea of microservices has shifted and brought this to light now because we're not building multiple big monolithic systems that we can just slap one API onto that has everything. We're building a lot of smaller services. We're using techniques like lambda, serverless functions, and other kinds of techniques to break things into smaller composable components that may have APIs. And those component APIs are oftentimes a bit different than a productized API that we would share outside the organization.

So these productized APIs are the things that represent the front door to your organization, the front door to a particular aspect of your organization, a particular line of business, business unit, and domain area. So composable organizations are starting to leverage microservices but there's a bit in the middle that we have to think about. And it's that packaging and bundling as I mentioned earlier of how we take these more component-oriented APIs or these backend APIs that are decomposed because of the way we're designing a system, or the way that we have, as Conway's law would say, bringing in some of the organizational structure and the way that we have our people organized, and the way that our processes are organized and are conducted.

We have to really separate those two concerns out, and the ADDR process really wants to focus in on the product, APIs first, or the bundling of those. Some big organizations might think of them as platform APIs, those APIs that are representative of different areas of the organization that could be combined and recombined together to produce different outcomes. So if we think about the boundaries and what we're looking at, what we often have is we have these little bounded areas that someone's responsible for, for a microservice, a coarse-grained service, or kind of a semi-coarse-grained service from the old SOA days, or whatever it else it is. You're gonna have a mixture of those levels of granularity, but they're oftentimes owned by one team.

And that team doesn't really understand how to bundle that in a product or what the external needs are. They just know that they need some function as a service to extract some data out, event on something, react to an event, something like that, that's in a different level of granularity. So boundaries are really, really important, and we're having more discussions because of the more distributed nature of today's software architecture, and it's requiring us to think about APIs at different levels, to re-think what APIs really are, and the different levels or types of APIs and how they might be used, and combined, and packaged, and bundled.

Differences between modeling & designing an API

Mike Amundsen: I totally agree. I sort of think of it almost...I don't know if I think of it as maturity or somewhere along the path, where we're sort of beyond the basics. Now we're really thinking about the composable elements, the units, the interactions between them, so it's very exciting. It's a very exciting time. One of the other things, talking about this notion of beyond the basics and composability, another thing that's through the book which I really love is you talk about the difference between modeling an API and designing an API, this idea that modeling is still this non-specific tech, but coming up with these basic pieces, and then actually selecting technology to actually design an API that implements this thing. Can you sort of expand on that? Do I have that idea right, the difference between modeling and designing, and how that plays out in your experience?

James Higginbotham: Yes definitely. There's definitely a difference between modeling and design. Modeling helps us to focus on what the API needs to do independent of the API style that's used. So we want to use the API profile to capture the digital capabilities and the operations that API will provide, then we can take our model and use that to design our API. So we may start with a REST-based API, or GraphQL, or gRPC. And we can use that model then to translate the operations that we've identified that help us with the jobs we've done and the desired outcomes into that API style of choice. So it gives us the opportunity to separate out the implementation details from the idea of how we're gonna design our API away from what the API really just needs to do. The accomplishments or the outcomes that it needs to offer. So there is definitely some value there in making sure that we have a profile that we can then leverage and use to translate into the API design.

Mike Amundsen: Yes. I think so this really, kind of, comes back to the things you were talking about earlier how you say a team wants to know that the API they're defining is really the one that's needed. Like, by focusing on this model, I love that you give that, sort of, kind of, a North Star. However, you wanna think about it. A North Star or a foundational, pillar, which is the modeling, and then you can solve specific problems. You know, we're gonna do it like this with GraphQL, or we're gonna do it like this with gRPC, or you can go back to the well previously. And I loved the way you talk about it in the book also is this idea that it is still iterative. Each one of these, you can iterate on the model, you can iterate on the design, you can iterate on steps and activities, which I find really, really liberating for that sort of, like you said, you know, small, agile kind of approach to things.

There's a quote that I've used in relation to some of this before. It's from Frank Lloyd Wright. It's from the natural, physical architect. You know, and he says, "You can use an eraser around the drawing table or a sledge hammer on the construction site," right? And I love that you have all of these elements before I commit to a working version. I have all of these places where I can use my eraser, where I can improve, and iterate, and compare before I get there, so that's a really, really, really, really cool feature.

Now, you talked about styles. This is another thing that I find makes your book very unique. You actually talk about the same model in various styles, don't you, throughout the book?

James Higginbotham: Yes. I like to use the same model in multiple ways. Once we have the model defined with the capabilities that the API needs to deliver, we can then start to look at opportunities for delivering multiple API styles for the same API. So I found that very common to have APIs start off with REST. That's the most common style and the most comfortable for most developers. But they may expand and integrate in a GraphQL query to support ad hoc queries, or response shaping, or other things. We may also look at AsyncAPIs as well. So how can we offer WebSockets, or webhooks, or other mechanisms to incorporate as well? So we start with that API profile, then we allow those designs to emerge as they're needed. So spend time listening and understanding to your API developers, the consumers who are gonna be using your API, and listen and see if there's new styles you need to incorporate. You can always go back to the API profile, use that as a foundation, and start to design using different API styles.

Minimum Viable Portal (MVP)

Mike Amundsen: Yes. And I love that section of the book. It's so empowering. It's so enlightening to see that sort of design approach for each of these styles with the same sort of like, backing material. I love that idea. One of the other things I'm just gonna mention so many things, but I wanna talk about this one other item that you talk about in the book that really struck with me. That is when you talk about, sort of, refining the design, like, past the implementation details. You talk about testing, and you talk about documentation and all that. You spend a good deal of time on this idea of a portal, of an API portal. You don't just talk about designing the API and implementing the API, you talk about the collection of APIs. I don't even know how to explain it. But, I mean, you spend a lot of time on this idea. You've even designed, kind of, a minimum portal frame, right?

James Higginbotham: Yes. I worked with a technical writer, Hilary. She and I sat down, and started to ask ourselves, "What parts of the portal are absolutely essential when you're launching a new API? Whether it's one API as part of a whole, part of a bigger bundle, and we just need to think about the documentations needed to support that particular API as part of the whole or if you're launching a full API product. It can be overwhelming to deliver documentation in that way, you know, like a full portal from day one, so we wanted about to think about what's needed.

It's important to realize that things like OpenAPI's spec for rest APIs as well as IDL and SDL for gRPC and GraphQL, they only provide referenced documentation. How to use each operation. But they don't tell you how to complete the entire workflow. So when you think about an API, and what it's trying to do, and the outcome it's delivering, which we, again, talked about during the aligned phase with job stories and the jobs to be done. There's oftentimes more than one operation that needs to be used, and they need to be coordinated or orchestrated in some way.

We need things like getting-started documentation and other kinds of documentation as part of the overall API offering. So the minimum viable portal starts to look at how do we tackle that documentation effectively in an iterative fashion so that we can continue to deliver on that little agile type of approach when we deliver our API, and not do a Big Bang delivery of everything all at once. That's what the minimum viable portal is about, and it's part of that refined phase. How do we go from just the API design, to the documentation, to really delivering robust documentation for the API consumer that's gonna be integrating it into their solution?

If we started to look at the MVP and start to think about what our consumers really need to get things done, it's also helpful during that refined phase to capture, say, like, a static mock or a read-me mock that details how to go through step by step. And that allows us to make sure that, one, we can validate that we have all the operations we need in our API as well as it starts to contribute to that getting-started guide that we mentioned earlier that should be part of that initial MVP of your portal. So being able to make sure that you have sufficient documentation, and that you validated that design that you have to meet the needs and deliver those outcomes is really important. So using read-me-style documentation where we iterate step by step or demonstrate step by step how to use the API with multiple API operation calls, requests, response interactions will go a really long way. 

Mike Amundsen: Right. You know, that's the way I use sequence diagrams actually. The same way you just described. Which is, yes, I know you have a series of endpoints, and I can pass this in and get that. Why would I do that? How do I use this collection to compose a solution? And if I'm missing something. If I can do the sequence, and I realized, "Well, and magic happens here." Well, I guess, I need another endpoint or I need something else here. So I love that. I love that aspect of it as well. And there are just so many things we can really talk about, but I know we're running short on time.

API journey: This is just the beginning

Mike Amundsen: I mean, what are your final thoughts? You've worked with this material for several years. The book has been out for quite a while. What would you leave listeners with to make sure they go forward? Like, you say, you know, towards the end of the book, "This is only the beginning," right? What does that really mean?

James Higginbotham: One thing to keep in mind is that when we design an API, we don't just design it once and then walk away from it. It's a living breathing product just like anything else we need to manage. So, first off, we need to recognize that we're gonna use the ADDR process over and over again as we discover new needs or new opportunities for our API. So that's the first thing. The other is to keep in mind that, and I wanna leave the audience with this. That our APIs they're about communication. The documentation communicates with the developer to tell you how to use it to get things done.

The API underneath the coverage is really a communication mechanism for the end-user through whatever interface that they're using to talk to some backend system, or service, or a combination of systems and services. So remember that our APIs are really products. They need to be nurtured, and developed, and expanded over time. If we keep that in mind, then we'll be able to go back and leverage our ADDR expertise, and use that over and over again as we learn and refine our API over time. Gain more feedback, and want to add more to it.

Mike Amundsen: You know, I don't think I can improve on that. Leave it at that. James, and the book is "Principles of Web API Design." I love the book. I highly recommend it to everyone. And now people can pick up the book. You also have the newsletter, isn't that right, James?

James Higginbotham: I do. I have a newsletter where I hand curate API-related content each week, so go to apideveloperweekly.com. You can sign up for that. It just comes out once a week, and there's always some interesting articles whether they're technical in nature or more business-oriented. Sometimes I have things that are tangentially related in some ways, and it's there for everybody. REST, GraphQL, gRPC, API, security API design, consuming APIs, code examples, all kinds of different things that I link to. And if you're listening, and you might have an article you wanna submit. Feel free to sign up for the newsletter, and there's details about how you can contribute as well.

And then you can also go to launchany.com to learn a little bit more about me.

And, you know what? Cut that last little bit, end with submitting content. Cut out my website, I think it's on the next one.

Mike Amundsen: Yes. And we can find you. There'll be links. And we can find you on Twitter, right, as LaunchAny, is that right?

James Higginbotham: Yes, LaunchAny on Twitter, and launchany.com is the website for my business for consulting and training.

Mike Amundsen: Excellent. Thank you very much, James. It's great to talk with you, and I look forward to the next time we get together.

James Higginbotham:  Thanks, Mike. Great talking with you as well. Look forward to the next time.

Mike Amundsen: Bye-bye.

James Higginbotham:  Bye.