Restful Web API Patterns & Practices Cookbook

The API ecosystem is evolving and the focus is changing from “how to design them” to “how to implement them”. Mike Amundsen, author of the Restful Web API Patterns & Practices Cookbook, and Alianna Inzana, VP of Product at WeTransfer, cover the recent changes while also touching on broader subjects like evolvable architectures. Join them in this conversation around restful web API.

---

Intro

Alianna Inzana: Welcome to GOTO Book Club. Today, I have with me, Mike Amundsen. Mike is an internationally known author and speaker and a consultant with organizations around the world on network architecture, web development, and the intersection of technology and society.

He's also authored numerous books and papers including such great works as "Design and Build Great APIs," which I personally am a very big fan of. And also "Continuous API Management," "RESTful Web Clients," and "Microservices Architecture."

So today, we're going to talk about his new book, which is called "RESTful Patterns and Best Practices for API's Cookbook," connecting and orchestrating microservices and distributed data. Welcome, Mike.

Mike Amundsen: Good to see you, Alli, great to be here.

The shift from Design to Implementation 

Alianna Inzana: All right. So I wanna start with a question that was the first in my mind when I read the book, which is, any intro of you sort of starts with the foremost expert on API design, so why write a book for consumers?

Mike Amundsen: Well, there are a handful of things. As you said, I've spent a lot of time on API design on the producer side, services side, design and functionality, and patterns and practices. But over the last couple of years, I've really sensed a shift in sort of the leverage of the market. I think we're finally getting to the point where we're getting more and more consumers and the balance is starting to shift. And consuming APIs is just as much of a challenge as it is producing them.

And in fact, often to help designers, to help authors, I tell them to put themselves in the chair of the consumer. What's it like to discover your API? What's it like to try to use your API? What's it like to try to survive changes that are made to your API? So I think taking a sort of a consumer point of view is really, really valuable for everyone. It's sort of the voice of the customer, if you go back to product marketing, in the 80s, it's job to be done kind of idea. People use your API to solve their problems. What is their problem and how do they do it? So I think that's really important.

I will say, at the same time, this particular recipe book, this cookbook, I think actually shifts perspective several times. There's a whole chapter on design, there's a whole chapter on client patterns, there's a whole chapter on provider patterns, there's a whole chapter on data, and then a chapter on workflow. So I think whether you're a consumer or a producer primarily, you can still get a lot out of what I'm putting together here.

Recommended talk: Practical API Design • Ronnie Mitra • GOTO 2019

Alianna Inzana: I absolutely agree. And as a product manager myself, the perspective of jobs to be done of what is the point of building that great API or designing that great API, it's always the consumer. So for me, I think it was a perspective that resonated really well. I don't think that there's much of this out there. But I could definitely see the advantage that you would find for both sorts of the consumer and the producer angle, because it's hard to consider, especially when you're sort of very focused on the technical aspect.

Mike Amundsen: Yeah, I think that's an important point. I'm sorry, you go right ahead.

Alianna Inzana: I'm sorry go ahead. Well, I mean, that's really what I wanted to talk about sort of the shift of perspective. And I think this is not just something in the book, I think this is something I've personally been seeing in the industry over the past few years, which is, you know, the shift of perspective, from endpoint to behavior, to a recipe, to workflow.

Recipes and HTTP: from endpoints to behavior

Alianna Inzana:  Thinking about the kind of cookbooks you might find in your kitchen, some of those provide readers with detailed instructions for creating each recipe. But then there are others that focus on teaching you transferable principles and techniques. Where does your book land?

Mike Amundsen: So this book definitely leans towards the concepts and principles idea, but I try to make it tangible with concrete examples. One of the challenges of working in network software APIs, the space that I typically work in, is you don't have a homogenous technology. So you could be using different programming languages internally, you could be using different orchestration tools, different gateways, and different frameworks for doing the HTTP. So it's hard to say, "This is the exact recipe," or, "This is the exact step to do."

So I decided not to do a book on Java or JavaScript, or go, or C#. But what I do is a lot of HTTP. So often the examples, while they're narrated, they often say your HTTP conversation will look like this, the request and the response. And that's about as direct as you can get in this kind of network software space. That's a bit of a challenge.

Then, towards the end of the book, I speak a lot about interactive patterns. So interactive between resources, interactive between services, that's even more difficult to come up with a direct recipe for. There are lots of diagrams, and flowcharts, in the latter part of the book as well. It leans towards that common practice but I try to come up with at least solid examples that you could recreate on your own.

Alianna Inzana: I think you do a wonderful job of that in the book and really sort of taking people through the thinking behind how you implement some of these different recipes, which is important because not everything is going to yield itself to easy, sort of, step-by-step diagramming.

As you said, the languages also I think can provide a barrier to entry if you don't understand how to think through the problem, as opposed to, you know, just applying the technology. And I think you know, for me, that's the conversation, workflow is the conversation that we need to be having much more, the streetlight I want turn on in the API space.

So can you talk a little bit about workflow just as a sort of general guiding principle through the book, because it's interesting there's a bit of duality there, I think? On one hand, you're writing a recipe that implies a certain set of steps, a certain set of things that you're doing. On the other hand, one of my absolute favorite Mike Amundsen-isms is you're solving problems you never thought of for people you never met. And that's absolutely true as well.

So that's not the thing that the product manager, the API designer intended it's, I got your Legos and now I'm gonna build something different with them. So can you talk a little bit about that duality because I think it's very cool?

Mike Amundsen: So that phrase, solving problems you never thought of for people you've never met has always been a kind of a guiding principle for me for decades. That leads to building platforms and frameworks often, instead of building one-off solutions.

So if I understand the job to be done, I can build a one-off solution that solves your problem today, the challenge will be your problem will morph over time, and your problem will begin to change in ways that I don't anticipate. So what I want to do is build something that solves your problem today and solves those new problems tomorrow. And that's much harder, that's more abstract. But what I find so fascinating is I think the HTTP protocol, the way it's designed, the way it's built, is so good for that kind of work. So I do kind of vacillate back and forth between those two.

Recommended talk: HTTP/3 Is Next Generation HTTP. Is It QUIC Enough? • Daniel Stenberg • GOTO 2020

I think the other thing that's so evident, in the last 30 years of the web itself is that the value of the web is in connection, it's in the whitespace, between things rather than just the things themselves and that's workflow. So, yes, it's important that I know how to handle a payment, it's important that I know how to schedule and manage shipping, and it's important that I know how to do a shopping cart, and pick from stock.

But the real value is when I can get all of those connected, all of those working together. And often, that starts to look like that problem that we hadn't thought of, the way it's connected, who connects it, and how it interacts with other things, are the things that I can't think of.

So I want to empower people, that's sort of the message of the book, to make sure that they remember, there are people you've never met they're gonna use this material. That's sort of why I talk the way I talk about the examples in the book.

Alianna Inzana: HTTP, I think for me, it's sort of fascinating to think about the fact that it is still getting the job done. It was designed for such a different world that we operate in. On one hand, it shows that it can be done but I thought that was actually a really interesting aspect of the kind of what you were calling out in some of these recipes and the reliance on HTTP.

Mike Amundsen: One of the lines of conversation that I've had with lots of people over the last five or eight years or so is challenging to think about a time when we don't have HTTP when we have something else, when HTTP is no longer powerful. Well, I keep getting reminded of how flexible, powerful, and resilient the HTTP protocol continues to be because I agree with you 100%. It was designed for static documents, linking documents back and forth, it wasn't designed for application hosting, and it wasn't designed for the component building.

And in fact, early component building really went badly, you know, we had these plugins, ActiveX, and all these other kinds of applets, and things that really didn't work out really well. Yet, HTTP continues and continues to grow and morph. So I'm really, really fascinated by the resilience of this particular protocol. There's a series of them, email, chat, and several other things that all kind of come out of those late 80s, early 90s. I think we're very lucky to be working on a platform as powerful, resilient and as flexible as HTTP and that's one of the reasons I love it.

The biggest challenge in building and evolving REST services

Alianna Inzana: I have no arguments from you there. As we're sort of talking about this, so we're talking about jobs to be done, we're talking about APIs, and, you know, the concept of workflow, both creating the components to build workflows, even workflows that you don't as a creator anticipate, but also, you know, engaging with the workflows that you intend.

I guess what I start to come to sort of think of, especially as a product person, is we don't always get it right the first time, right? So there are a lot of challenges not only in building these services the first time but also in evolving them. What do you see as some of the biggest ones?

Mike Amundsen: From sort of the evolutionary kind of standpoint?

Alianna Inzana: Yes.

Mike Amundsen: The biggest challenge...the thing that I run up when I talk to people and it sort of gets reflected in the book is often we don't think in the same timescales that a lot of these protocols really lead us into thinking. We can think in the timescales of decades of years, 5 years, 10 years, 15 years, and 20 years for our designs, and that's a challenge because so many things change.

So I think one of the things that come in this sort of evolutionary nature is building systems that can be changed over time without disrupting the rest of the participants, the rest of the connectors. In real life, this happens all the time, I change the furniture in my room, I move to a new location, I change my outfits, I change the way I look, and I buy new glasses. We have all sorts of changes that we do with every single day, change the way I get to work in the morning.

But often when we build computer systems, we don't build in that ability to take a different route, to dress things up a little bit differently, to do things in a different order. I think that's a big failing. So I think we need to... If there's anything that I'd love to be known for is helping people sort of level up to that notion where change is a feature and not a bug, right?

So I'm constantly trying to help people figure out how to think about this in a way, like, what happens if it was rearranged? So a lot of the recipes throughout all of the clients' servers, data, and workflow, a lot of them talk about making sure that if you make a change later, things don't break, and I think that's probably the biggest challenge.

Alianna Inzana: It is really interesting to me, too, because I think most people have heard of the sort of, yeah, YAGNI, you're not gonna need it. And sometimes that becomes a proxy for, "I'm just gonna build for today," which it shouldn't be in my opinion, because it means that you're not necessarily dealing with the future which hopefully will have, for example, issues about scaling, or issues about, you know, exposing potentially aspects of your data model.

That can be a really big problem I think, too, because you then wind up building and rebuilding systems and incurring technical debt when if you kind of thought about like the reality of your situation when you first started, it might not have come to that. And it's not something that necessarily can be tested at the level of the spec either. It's something that...you know, it's the interaction between these things, it's delivering the capability that you really start to see some of those challenges become problems writ large.

Recommended talk: REST beyond the Obvious - API Design for ever Evolving Systems • Oliver Drotbohm • GOTO 2019

Mike Amundsen: This part of our discussion is kind of a perfect example of the challenge I think. We wanna think ahead, but we don't wanna overbuild, right, that's what the YAGNI principle really tries to remind us, you ain't gonna need this, at least not now, right? So I think we have to be able to build systems that we can go back and rebuild later.

Alan Kay had this great line that what you wanna be able to do is build something and then go back after you've learned and rebuild some other part that you had built earlier but you wanna do it safely, right? So you wanna go back, you wanna learn from the future and then make changes.

So that's what I'm trying to encourage people to do is build what you need for today, but leave yourself an out, leave yourself a doorway, leave yourself a chance to be able to tack something on later. And I think that's a big part of it adding on is a lot of the way resilient systems work, they don't replace, they add-on.

What happens is you sort of...like if you think of biological systems, we have all sorts, we have the little toe, with the appendix, and the tonsils, we have all these extra things that used to have a role, but they're really just kind of add-ons now they're really not too important. But we don't get rid of them because taking them out is painful and complicated.

And the same thing with large-scale systems, I think you have to think of the fact that this will add-on, this will be changed over time. So trying to encourage people to continually accept the YAGNI, but also prepare for the fact that circumstances will change in the future.

Alianna Inzana: It's almost like you should over architect, but under build like follow the YAGNI on the build.

Mike Amundsen: That's actually very good. That's a very good idea. It's a version of hope for the best, but plan for the worst, right? It's kind of the same thing. I like this idea a lot over architect and underbuild or something, right? Is that what you said?

Alianna Inzana: Just the idea of like thinking about those future states and future possibilities, without necessarily building the thing to deliver them yet, but making sure that, again, and Mike wrote that I love, "Affordances are there within the system," which I think you know, for me again, as a product manager, you have to think in those terms because you know, whatever you deliver the first time absolutely will not be the right thing.

Mike Amundsen: Let me lean on your product experience, right? I mean, a lot of times, you do the research, you figure out what that product could be, but often the very first product you put on the shelf maybe sort of limited, like it's targeting a market, or it just has a certain set of features, right? And then you sort of see how that goes. And if it goes well, you can add to the family, right, you can expand the product, you can add more products. Is that right? Am I kind of...maybe it's oversimplified?

Alianna Inzana: No, I mean, I think that is, sort of, definitionally in the era of agile how we try to do it. And, you know, it's more about trying to understand from our research what that core use case is. Like, what are the things you absolutely must have, but also, what is the thing that could provide the most value?

I think being humble enough and unbiased enough to see, okay, I've provided value, do the people like it? Are people using it? And are people using it in the way that in my vision and my team's...

Mike Amundsen: That's right.

Alianna Inzana: ...vision, I thought that they would? You know, that's the building things for people, either they're gonna use it a different way. 

Mike Amundsen: Right because that's the pivot, right? Suddenly, I realize oh, I'm not building a game engine, I'm actually building a communication network, right? Okay, so let's embrace that, let's do that. So I think that's right. And I think that really is part of the way I'm talking about these and all these recipes are being prepared for serving the needs that you haven't quite figured out yet.

Collaboration and Risk

Alianna Inzana: And recognizing how those needs change as the user's engagement with your system, your tool, and your interface change because that is also an inherent proposal. So I wanna shift a little bit because I think we're actually kind of touching a little bit on the next topic that I was thinking of covering with you, which is collaboration and risk.

So collaboration and risk, I love pairing these two concepts because collaboration is something that everybody loves and is tremendously excited about. And risk, is like, clearly what we all wanna avoid. But definitionally collaboration means something is going to exist outside of your individual or your team's control. And that is actually an inherently risky proposition.

One of the things I was looking at in the book is does this book, do the recipes in the book, help organizations, and help developers mitigate that inherent risk around collaboration?

Recommended talk: How Microteams Change the Way We Collaborate. Again • Sander Hoogendoorn • GOTO 2021

Mike Amundsen: I love this idea, this sort of counterbalance, or the fact that the two come together. I talk a lot in the book about the notion that you can't eliminate bugs but you can survive them, you can't eliminate the possibility of something going wrong because you don't control everything, right?

So part of this, I think part of the theme we're kind of talking about here is this notion of people you've never met and problems you've never thought of has a lot to do with the fact that you're not in control of things, you're not in control of when service A changes their interface that you're consuming. You're not in control of when the consumers no longer think this is an interesting thing or they start using your tool in unexpected ways. So that becomes part of the collaboration.

What you have to do is, there's a level of trust that has to go on to make sure that everyone can participate. But there's also a level of self-defense that has to go on to mitigate unexpected things. Unexpected things are not just people, they're also technologies, the server goes down, the service is overloaded, and we ran out of stock. I mean, you have to prepare for all of these kinds of things.

The example I...I think this is in the book. But the one that I often use is this notion that somebody wants to order a product, and you send a message saying how many are in stock? And you send a message to the service, the service figures out how many are in stock, and the service creates a message and sends you the answer. Well, by the time you get the answer back, that's probably not the right number anymore, right?

There's time and things change, so you have to sort of protect yourself against even minor little things like, yeah, well, they were in stock, but you know, now all the seats are sold, and there's no front row seat for you at the concert. We have to build all of these things into the system and that's where that risk I think comes in.

So when you collaborate, when you build the collaboration, and I love this notion of thinking of it as something outside your own sphere, you have to be prepared for what risks might come up and even be prepared for risks you haven't thought of, there's a kind of a self-defense part of that. And that's a big part of the way I describe, especially the data chapter where you're really pretty worried about what's happening to data you're in charge of and that's all part of the process I think.

API Security

Alianna Inzana: Data is a place where it becomes very apparent, like your book example or whatever, like, this is the place where things can go wrong, this is the place where things may become inconsistent in the process of transmitting and receiving messages. And this is also a place where we can expose ourselves.

I think some of the big challenges we're seeing now with Graph QL, for example, is because people were like, "Well, we all understand our data model so we're just gonna basically back end for front end, just like throw it out there." When you do that internally, there's still risk involved. But when you start externalizing it, there can be a lot of potential security risks as well to the data.

So I think maybe again, a little bit of my background peeking through security is a topic that I think is finally for API is getting some way overdue attention. In my opinion, a lot of the conversations that I've been hearing about it are shifting the responsibility onto the producers, like building a secure thing.

But I love this analogy of you could have a house that has a lock, that has a security monitoring system, and all these other things. But if I leave the house and I don't lock the doors, and I don't turn on the security system, none of those things are gonna help. So is there an aspect that you're addressing in the book about security, or is that something that kind of comes outside and that is something we should be thinking about?

Mike Amundsen: You know, it's the classic challenge. There are some security elements in the book there, but I didn't do a chapter on security. And part of the reason is some of what you were just saying here because I think it's we're sort of getting to the point where security is embedded in things rather than a standalone unit. Security isn't some gateway that we have to pass. Security is our behavior, just like you said about, I have to lock the doors and take the key out of the car and all these other kinds of things. I have to behave responsibly.

So, I toe the line a little bit, but really not enough. Honestly, if I had more time, I would write more but I just haven't been able to do that. But I do think, I really liked the point you're making which is, that it isn't just a producer problem, it's an everyone's kind of problem. And I think when we think about other kinds of transactional systems, especially if you move to the financial and fintech internet transactional systems, we're constantly trying to figure out how to design systems that have security built into them. Or that take advantage of other kinds of information that we know so that the secure behavior is kind of widely distributed.

A great example is most credit card systems...this is a great messaging model where there's lots of collaboration and risk. Most credit card systems have a backup system, they will take a certain amount of float, a certain amount of errors, a certain amount of...like if you lost your card, they're not gonna put you in jail if somebody buys a boat on your card, right?

But at the same time, they also had these patterns where they make sure that, you know, that was kind of unusual that you move to this...you're in the city that you've never been in before and you bought a very expensive suit, is that actually you? You know, we have these ways of thinking about it. All the way to blockchain, where we wanna actually infuse the message itself with the validation information.

So I think APIs, we're getting to that spot where it is everyone's responsibility to behave in a secure manner and to mitigate against their own kinds of risks while they collaborate with other people.

Recommended talk: Container Security • Liz Rice & Eoin Woods • GOTO 2020

Alianna Inzana: And I think a lot of that is around data. It's interesting, a recent article that I read was talking about some sort of digital transformation. So one of those things that, you know, the execs all love to hear about. But the digital transformation in banking and one of the observations was that not a lot is happening internally.

I spent many years as a tooling vendor so I've talked to a lot of banks in the course of those years. I actually think it's not entirely true, I think that they are using, as you said, machine learning models against your transaction history, so that they can understand what looks atypical for Mike, as opposed to what looks atypical for myself. But at the same time, I think it's not necessarily translating outwards, and again, it could be that they're afraid of how to expose that data externally.

So it's an interesting kind of duality because it really is about that relationship, not just the relationship between, you know, two endpoints or something like that. But the relationship between the producer and the consumer, when the producer is a human perhaps and the consumer is a human that is trying to transact a piece of business.

Mike Amundsen: I really agree with that. Again, it goes back to this idea of relationships. You know, here's a not-so-popular point of view. I talk about this a little bit in the book, but not a lot. And that is that often people think data is an asset, I think of data as a liability.

Data is the evidence of activity, data is the left behind remnants of something you did. I bought a product, I wrote an article, all these other kinds of things. As soon as that data is sitting somewhere, that's a thing that can deteriorate, that can break, that can get lost, that can get stolen, that can get misinterpreted, it's a liability.

So designing systems where you think of data as the way to solve a particular problem, I think kind of levels us up where data becomes, it's the message, right? It's the message we're passing so that we can get something done so that we can affect some outcome. It's the outcome I'm interested in. And if I can ignore the data after that, or deep-six the data somewhere where I can analyze it later or offline, that's just fine.

I want to actually limit the amount of reliance I have on a particular single source of truth or these other things because, in real life, we don't have those kinds of things. We don't have a single source of truth for the news, or for any other aspect of our lives. And I wanna kind of build systems that operate in that same way. So I think of data very often as evidence of something that we did, and now that evidence is something I'm liable for, that I need to take care of or protect in some way.

Alianna Inzana: I think that's a really great point vis a vis data and also about the consistency of systems. I mean, if you sort of talk to sort of the web 3.0 of crowd, you know, this is a distributed system, the inherent difficulty in distributed systems, stuff is not consistent, you know. When you talk about event systems and the concept of eventual consistency, like, eventual consistency can be kind of like a hopeful term, like, we're really hoping that we get to that point.

Mike Amundsen: I agree, it's eventually consistent if at some point you stop.

The need for evolvable architecture

Alianna Inzana: Right, exactly. And these systems do not. In fact, as they constantly evolve, the data in them evolve, and the interactions are at some point millions and millions per second. So how do we talk about this high intensity, high velocity, data flying everywhere, data in motion system, and evolvable architecture? How do we bring those concepts together and how does the book...?

Mike Amundsen: The way I try to do this in the book, I actually struggled as to whether or not I would have a separate chapter on data. I had a chapter on services, a chapter on clients, a chapter on workflow, and I added a chapter on data because of this thing we talked about, about how I see it as a liability as a challenge.

But if you look at the book, there are recipes in the producer chapter, recipes in the data chapter, and recipes in the workflow chapter that are actually very, very similar. In other words, it's sort of all this thing being infused. If things change over time, there are services that need to be able to adapt to that. If the data changes over time, there are clients that need to be able to adapt to that. And there's a workflow that affects the change of data.

So I think in some ways, I wanna get us to think about these as sort of the circle of life that like, these are all related anyway, and not think of them as to how do we get the two to interact, but just the fact that they are already kind of stuck together. So some of the recipes actually point ahead and point back to each other. The recipes about how to make sure that...like how to deal with a flood of data is often using cues in implementation detail.

My producer chapter has ideas about how to deal with lots and lots of transactions and that's to use cues. If we think of them in sort of a holistic way, they kind of help each other out. I don't know if that makes sense but that's kind of the way it works in my head.

Alianna Inzana: I think the problem with it is, that it is an inherently interrelated problem. And if you don't deal with it, no matter what end of the stick you're tackling at a particular moment, you have to because the other part is not gonna go away. So I like that.

I'm not sure if we'll have time overall, but maybe we can talk a little bit about how the data that's in motion, and how the services that are in motion, interact with each other. And I always think...and I'm totally dating myself here. I always think of the movie "White Christmas," Danny Kaye doing the dance to...they're doing choreography, whenever we start talking about the differences between orchestration and choreography. You know, made it sound like choreography was like something special, something different.

So when we're thinking about like, the recipes and the workflows, there are definitely trade-offs between deciding on something a little bit more stable and orchestrated or a centrally organized and choreographic approach. So what for you is the difference or is there like something that kind of calls out?

Mike Amundsen: To me, there's definitely a difference. I've sort of adopted that same notion of thinking choreography is this collaborative activity of people on the stage, right? By the way, my favorite song from that movie is "Sisters." I don't know if you remember that song. But there's a great story behind why there are two versions of that song in the movie. But anyway, we carry on.

The other idea of this is rather than collaborators on the stage is some central person or central source that kind of manages the flow and the pace and the organization of all the pieces, that's typically thought of as orchestration, right?

So I like both of those and there are times when each of those is really important. There are times when it's a tight enough integration or tight enough collaboration, where I don't need somebody telling me what to do we can work this out together. And that's where I think this idea of choreography works really, really well.

Recommended talk: "Good Enough" Architecture • Stefan Tilkov • GOTO 2019

But there are also times when they're lots of like, really complicated, not necessarily complex, but complicated interactions or various interactions, or lots and lots of choices, or it may take a lot of time like days maybe even to complete a workflow. Then I think you need some kind of central source that is kind of coordinating, or at least managing, or nudging in some kind of way. And that's where I think orchestration helps a lot more.

I take one more step to it, and this really dates or stretches as well. In my previous life, I spent a lot of time playing jazz music, and improvisational music. And in improvisational music, everyone has the same chart or graph, and everyone has the same bits of information, but we each play our own part, the bass, the drums, the keyboard, the guitar, the saxophone, the trumpet, so on and so forth. We all have our own versions of this and every time we do that song, it's slightly different, right?

That's more of what I think the HTTP universe is really about. We each have the same basic plan but we each contribute in unique ways. And doing it one way doesn't mean it's gonna be done the same way over and over again. So that's a sort of a different level to it and that's where I think HTTP is actually really good at this.

I think we need a case where there's some tight collaboration where we practice, practice, practice, and make a performance. You know, that's Fred Astaire, Ginger Rogers, all those things like that. I think there are also cases where we each practice our own part, we do our own thing, and when it's my turn, I will go ahead and process the payment. That's the orchestration idea.

Then I think there are these cases where there's sort of loose collaboration, where I could...what I talk about in the book are these ideas of being able to find and bind, to discover in real-time, the services that are available in order for you to solve a problem. So it's not even planned ahead of time. It's like when it comes a time, if somebody says I need a payment, I'll go find a service that does payment, interact with her, and then that's all I need. So I think we need all of those, not just one of those, but all of those  to create a successful ecosystem.

Alianna Inzana: Well, I think you've heard it here folks in both jazz and APIs, we never play anything the same way once.

Mike Amundsen: That's exactly right. At least that's the way it works for me. I don't know.

Is API tooling lagging behind developers’ needs?

Alianna Inzana: So this is fantastic. I think one of the other aspects though that as a former tooling vendor...makes it sound like I'm in a special meeting here. But as a former tooling vendor, one thing that sort of struck me I think was you created a lot of these recipes and everything. And I know very well what's on the side for people who are exploring APIs, for people who are implementing them, designing them, and even for testers. But did you notice any place where the tooling had not caught up with developers' needs, and, you know, kind of, how did that influence while you were writing the book?

Mike Amundsen: Yes, there's a combination of things. The stuff that I write about, I tend to write often about the next horizon. I write about things that we could be doing but aren't. And there are lots of good reasons why we are not doing them cost, expense, technical details, not all the tools that are available.

So I found when I was working on the book, that, oh, this is a really important principle, there is not a good process or tool that is widely used right now for this. In some points of the book, I say, you know, there are about four or five different ways that you can actually record the definition of an API or describe the workflow of an API, you'll have to decide on one.

So what happens to me, and if people have read some of my other books they probably realize this, I tend to go off, wander off and write some other kind of tool or some other kind of thing to fill in a gap or something like that. Because I often am running experiments I need to kind of build these things.

So it turns out there are a handful of tools, actually, just a couple, tools that I consistently use while I was working on the book and had to either kind of build myself or cobble together from existing bits and pieces. I think the challenges that I see have to do with describing APIs and API interactions at a more abstract level than an implementation level, like OpenAPI or AsyncAPI or Protobuf, or Schema definition languages. These are really powerful implementation details that I must have in order to complete work.

But often, when I'm working on my projects, I'm not sure if I want this service to be implemented as GraphQL or implemented as GRPC, or implemented as a simple REST CRUD. So I use sort of a generic language to describe all of those and then generate those implementation details, that's a tool that I think we don't have a really good version of, we got a few.

So I end up working on a thing called ALPS, which is a profile semantics language that I think some people may have heard of. At the same time, often I'm testing interactions between services. It's kind of fiddly for me to keep writing in a language like C#, or Java, or Node, or something like that. So what I want is a sort of a higher level almost like curl, but for workflow.

So I ended up creating a ripple tool, a command-line tool that has a memory that allows me to interact, sort of like a browser, but through just APIs. So I can actually script API interactions in script interactions with other services, find services that match and then use them in real-time. And that's turned out to be another project that I worked on last year, I think it's called the Hyper Client. And so I ended up building this language.

I think none of these are production-ready or product-ready. As a product manager, you would probably say, "Well, these are quite far from being stable," but they helped me solve my problems. And hopefully, as people read through the material, even in a year or two, they'll be like, "Oh, no, no, no. We already have a tool for that." And that would be great, right?

So I think the ones that really touched on...you know, hit me as I was working this high-level abstraction of how to describe a workflow, and describe interactions, and the actual running of them in a sort of an abstract kind of way. So those are the things that I actually built. I refer to them in the book and I think I pointed the repos on them as well.

Recommended talk: Introduction to OpenAPI • Lorna Jane Mitchell • GOTO 2019

Alianna Inzana: I mean, I love the idea that tooling evolves from a developer's needs. And, you know, we've had the Lego blocks that are provided to us by specifications for so long. But in, another case of sort of looking for your keys under the streetlight instead of looking for your keys where you dropped them, you know?

Mike Amundsen: But the light is so much better.

Alianna Inzana: This is where the light is. But this is sort of the challenge of specifications is, that specifications have allowed us to describe in great detail these aspects of an API. But, you know, something that could even be used to approximate in REST, in AI, like, links, we don't use. I would love to do a study and see the number of API specifications that are in production, that use links, for example, just as a sort of things you can do with the telephone numbers.

So we don't really deal with that next level of abstraction, but the next level of abstraction is also where the value lies. Because in the end, each individual endpoint provides a piece of the story, but not the entire narrative of the thing you wanna do.

Mike Amundsen: I think you hit on a really important point. And that is the last point that you made about each individual endpoint. I think one of the big challenges we've had over the last decade or so of doing API things is, that we really focus on encapsulated APIs. This is the API for my service, this is the API for my company. We don't actually have a lot of examples of successes of integrated APIs or interoperable APIs, this is the service that uses all these different companies' APIs.

So designing a workflow where we're not in charge of all the services, designing a client that isn't just bound to one domain, but actually uses lots of domains to create their solution, that's sort of that next level of abstraction that I think we're starting to come close to. So I think a lot of workflow focus, a lot of choreography language, and things like these are starting to get us toward that place. That's really I think one of the key messages of the book is to think about HTTP as a world that you can interact with, not just the one thing that you built in the world, and then focus just on that.

Alianna Inzana: I actually think that was one of the most...for me, at least, one of the most compelling sorts of dualities of the book is that you, kind of, inherently you're separating the meaning, and the message, and the message format, you know, like, that's what you do. And you can reuse vehicles of conveyance like messages and stuff like that, but then it's the compilation of all of them that actually results in, you know, the thing that you wanted to do.

And that is a really powerful aspect I think of the recipe approach and the sort of teaching technique approach of the book, which I personally really enjoyed as a sort of, more non-technical reader. So, absolutely love that.

What has the book taught you?

Alianna Inzana: So we've gone through a lot of different sort of aspects of what you cover in the book, and who the book is for. But I wanted to, sort of, bring it back to you as an author because as somebody who's done some teaching myself, whenever you teach you to learn something, too. So what is something that writing this book taught you?

Mike Amundsen: A lot which is why it's taking me so long. I think probably one of the things that I kind of suspected, but I really didn't like, tangibly, understand until I actually started working on the book is, over the years, I've sort of collected all these habits, these, "Oh, you should design this thing this way." Well, I don't really remember why. I don't know why I do the things I do. That part of my brain is locked away somewhere. All I know is that you do it this way, and it's working really well. It's worked really well before.

So as you mentioned, as a teacher, when I start to, you know, answer the question why when a student asks me, "Why is that important?" Suddenly, I have to kind of rethink, and I have to go back and I have to go to, not necessarily first principles, but at least replay some of the alternatives. And not surprisingly, there are certainly times that I'm going through, and I kind of digging back on my history, and I realize, ah, actually, that's not a good reason. That might have been true 10 years ago, but that's not true today. So I've had to learn. I've had to relearn and you know, put on that student's cap.

So one of the things that have happened to me in this book is, as I start to explain why I do things the way I do, I am also inspecting the reasoning behind it, and I end up learning a new way to do something. I ask some questions, I look at some evidence, I talk to some people and suddenly realize, oh, yeah, this is actually a much smarter way to tackle this problem.

So I think one of the biggest things throughout the book is I've had to learn myself, why I do the things I do and how I can improve them along the way, which is kind of what I want all the readers to experience as well. Oh, I've seen these names or I've seen these things, why is this so? And there are certainly cases I mention this in the book, especially early in the design area, and a couple of other areas, where there's no science here, you can make choices yourself on, you know, what fits best in your environment.

So giving people the tooling and the opportunity to make their own choices, to enable their own teams in their own ways is also really, really important to me. So I think, you know, writing is always a learning experience but this, in particular, has been a really valuable experience for me.

Alianna Inzana: Thank you so much, Mike Amundsen.

Mike Amundsen: Absolutely, great to talk with you, Alli.