Building Multi-Tenant SaaS Architectures

Intro

Bill Tarr: All right. Hey, everybody. My name is Bill Tarr, and I'm a SaaS evangelist with the AWS SaaS Factory. And today, I have the privilege of being joined by Tod Golding to discuss his new book. So, Tod, why don't you take a second and introduce yourself? Tell us about the book that you've just written. Give us a title and maybe where people can come find it right out of the gate so we can leave them with the takeaways right in the beginning. What do you think?

Tod Golding: Sure. That's a great place to start. So, thanks, Bill, for doing this, by the way. And basically, as Bill said, my name is Tod Golding, and I've been working on SaaS problems at AWS for the last eight-plus years in this domain. I guess I don't know how long ago, it's been maybe a year and a half or so ago, I decided we've been publishing in this space long enough, we've been in this space long enough that it was...and we've written about a bunch of it that it sort of seemed like it was a moment where we're like, "We should put this in a book. Like, this belongs in a book." And I'd written a book before, but I swore I'd never write another one again. And yet here, I sort of dove into this. With O'Reilly, we published "Building Multi-Tenant SaaS Architectures" earlier this year. And the reception has been fairly positive to that. And I'm sort of just happy that we're sort of beyond the moment of publishing and now talking more about what's in the book and what value I hope it brings.

Bill Tarr: So, Tod, I heard you said you'd never write another book. So, I imagine as you were thinking about writing this book, because you've written white papers, you've given speeches, you've done everything you could possibly do on SaaS, who did you have in mind that you thought really needed this book that we weren't really speaking to yet with all the other media that we're producing?

Tod Golding: That's a great question. For me, the challenge of having published so many things and done so many talks was that it felt like it was sort of scattered all around the universe and that people could find bits and pieces of the thing they wanted in a very targeted area. And they could go into any one topic and sort of get that targeted guidance for that one subject. But there was nothing that sort of cohesively sort of connected the whole story and the end and tried to tell it end to end. It was kind of piecemeal before that. So, for me, when I wrote the book, I was thinking there's all these builders, developers, architects, and to some degree, even business people, product owners, I think, who have heard elements of this story along the way, but not heard it told in an end-to-end fashion.

I felt like being able to sort of tell that story in that manner let me target these builders and these developers, a lot of them who might be at the very front of sort of figuring out SaaS, others who are, "Already have a SaaS solution, but I don't know where to start with thinking about making it a better SaaS solution. And I could come at this problem more comprehensively and really walk the entire service of the problem and find the bits of it that would help me apply that to my solution." So, really, the target here is anyone who's building SaaS with clearly a heavy leaning towards architects and developers and that universe. But there are chapters in there that would absolutely say, "If I'm a business leader, I'm a product owner in a SaaS space, I probably should be thinking about these things as part of looking at how I connect what I do to what the tech side of the house is doing."

Bill Tarr: That absolutely makes sense, Tod. Just sort of the unwinding all of these different individual pieces. Now, as you think about how the flow of this evolved over time, was there some piece of the conversation that actually came way later than you thought it would? As I was flipping through the chapters, I saw Tenant Isolation was, like, Chapter 9, and I was like, "Wait, that's the first thing I would usually talk about when we come with the customers." And I see that you changed some of the language, or the things like that surprised you about how the flow evolved as you wrote a book about it.

Tod Golding: The flow is really challenging for this book for me because I certainly want to get to the tech topics. I want to get into the weeds of the interesting technical bits of this problem. At the same time, and I know you and I have talked offline about this, there's this foundation of knowledge and sort of this mental model and this sort of vision for what it really means to be SaaS. And being unified around that, having this base that is, like, this common value system for what it means to be SaaS is was I thought essential to telling the rest of the story. So, when I get to Tenant Isolation or I get to Data Partitioning and I get to these other subjects, I really wanted to be able to say, like, "These build on that broader foundation." So, for me, those things did come later in the book than I thought they would come, but I was still happy that they...where they landed and where they ended up.

Obviously, if you're somebody who just wants to jump right into those super meaty topics right away, you're probably going to jump ahead in the book probably, and that's all good. But I even say in parts of the book at the beginning, like, if you don't have this common foundation, like, because there are many people with many different definitions of what SaaS is and using different terminology, all these other challenges that we've faced as we've engaged different companies who are building SaaS, and if you don't have that common base, then when you wander into these depth topics, there's confusion because they're like, "Well, what do you mean when you say silo, or what do you mean pool, or what do you mean when you're talking about tenant isolation? Is that just a deployment model, or what is it?" So, that was sort of core to me to sort of setting the table, and is why so many of those things came later in the chapters of the book. But it's probably a surprise, especially to somebody like yourself who also wanders into this topic.

Recommended talk: JVM Performance Engineering • Monica Beckwith & Kirk Pepperdine • GOTO 2024

The SaaS Mindset and Control Plane Essentials

Bill Tarr: Absolutely, Tod. And it's always refreshing to see how people reset when they think about a topic holistically. You did start off, though, with the SaaS mindset. I feel like this is an important part of the conversation. Like, if you're speaking to the people who are listening right now and thinking about reading the book, where should they be starting in their thought process when they're thinking about building SaaS? Why did you start there to try to connect them with the real spirit of SaaS?

Tod Golding: It's almost been like a little bit of religion for me when I talk about the SaaS mindset because I go into so many companies where there is this huge disconnect between the technical side of the house and the business side of the house about what it means to be SaaS. And they're struggling with those boundaries. Some businesses are just saying, "Go tech, go make a SaaS. We don't know what it is, but we want the economies of scale, and we want the agility. So, go do it." And that's a mistake. But then there are tech sides of the house who think, "Well, this is a tech problem only and just we'll figure out a price and sell it, but we don't need much else from the business, which is also a mistake." Right? The other fundamental problem is people often think of SaaS through lots of different lenses. And we end up blurring what SaaS really is. We'll see MSP find its way into the discussion. We'll see lots of people say they're SaaS, but are doing one-off deployments and offering customizations, doing lots of things that sort of undermine what it means to be SaaS. And so, for me, I wanted to put everybody in the same frame of mind about the fact that Saas is, A, a business model, and then SaaS is also, B, it's about driving growth and agility and getting all the goodness of having one deployment for all customers that's managed and operated collectively through a single pane of glass. And if you don't get that sort of base as part of your understanding, then it sort of erodes everything else going forward.

Bill Tarr: It's a great thought. It also leads me into sort of the next thought of, as we've talked about SaaS over the years, Tod, I think one of the concepts that really has emerged and risen to the top is control planes. So, having a control plane and how you manage that, I noticed that you jump right into that after the SaaS mindset. I don't know if we always talked about it in that way, but can you sort of explain your thinking about the control plane and what it means for SaaS providers and how they should be thinking about it?

Tod Golding: The control plane was something that came to life over multiple years, right? I had people who when we first started talking in this space would talk about a control plane for SaaS, and I would be, "Aah, I'm not sure. Is there really a control plane? Is there not a control plane?" Then a few years ago, we just finally had enough pieces of the puzzle come together. We've seen so many patterns out there that it became very clear that there was this need to be able to separate what's happening inside the application side of the house where we're doing tenant isolation and data partitioning and all the things people mostly think about from what it means to just collectively manage and operate all the moving parts of your SaaS environment. We definitely at that point made a big shift to start talking about the control plane and the app plane as the two halves of SaaS of every SaaS solution.

That model, that distinction, brought a lot of clarity to the problem for teams and also illustrated for teams the fact that they probably needed a control plane. They had pieces of a control plane, but hadn't openly acknowledged that they really actually had a control plane. So, it drove this value system of saying, "Well, what really is in your control plane? What is in your app plane? And what role are these two things playing? And what services belong where? Where do you provision tenants, for example? Does that all happen in the control plane, or does some of that happen in the app plane?" Like, it drove all these great questions. And so, for me now, like, fundamentally, anytime we talk about SaaS environments, we're always talking about, "Well, what's in your control plane? What's in your app plane?" And as we build sample solutions and reference architectures now, we draw this line between the control and the application planes. And customers and partners and other people we've worked with sit in rooms now when you start talking about it, and they're like, "Oh, we have a control plane, and we don't realize it. But we've never really called it a control plane. And because we didn't formalize that, we missed some of the nuance and the value we could have gotten out of that."

Recommended talk: SaaS Deep Dive: Designing and Building Multi-Tenant Solutions • Tod Golding • GOTO 2020

Evolving SaaS Terminology: Multi-Tenancy, Onboarding, and Identity

Bill Tarr: That's a great point. Now, one of the things you said, so, we're talking about control plane and application plane. This is some of the terminology we've evolved. Another one of those pieces, and I saw along the way, somewhere in the book, I saw the blurb that we always say, which is the question about multi-tenancy and how we describe tenancy. Do you want to sort of dive into some of...? Maybe not even just multi-tenancy, what are some of those keywords that we've evolved in our terminology that you would want people to look for and connect to in this book?

Tod Golding: I feel like multi-tenancy, that phrase has so many roots in the history of technology. And it got naturally attached to SaaS. But the problem is it had a history attached to it as well, right? Because so many people talked about multi-tenancy as just sharing infrastructure. And then, yes, SaaS was about sharing infrastructure, and they just said, "Oh, then they're the same thing." But the problem was SaaS itself isn't always about shared infrastructure. Sometimes tenants have dedicated infrastructure. Sometimes they have shared infrastructure. Yes, we want the economies of scale of shared infrastructure, but not every SaaS solution shares infrastructure. So, not every SaaS solution is, in the classic sense of multi-tenant, really multi-tenant. We had to rename and rethink the brand of multi-tenancy when we started talking about it in the SaaS context. Yes, we'll let everybody keep the old definition we had. But going forward, we said multi-tenancy has to be a bigger term. It has to have a broader scope. It has to say, "We are operating and running a business in a multi-tenant fashion, but allowing for the fact that some tenants will share resources and some will not. And we'll have a variety of deployment models inside of our environment, all under the umbrella of multi-tenancy."

This is where we then introduced new terms to try to overcome some of the challenges of multi-tenancy. So, you'll see in the book all over the word silo and pool use left and right. We intentionally introduced those terms because we needed to talk about dedicated infrastructure versus shared infrastructure separate from the notion of multi-tenancy. Any resource that's shared is called pooled, any resource that's dedicated is called siloed. Then we just allow for any granularity there and allow for the fact that in your architecture, some tenants may be running siloed and pooled, some resources may be siloed and pooled. Once we sort of adopted that terminology and detached from multi-tenancy, it made the rest of the discussion much easier. Even as I talk to customers about this, as soon as you do that, it simplifies the whole discussion because they had something in their mind about it, but they were just saying multi-tenant for everything, but we were trying to parse it out, and this made it easier to parse all that out.

Bill Tarr: It continues as the conversation goes on. I feel like the language has evolved piece by piece. One of the next things that I always hear for terminology, when we talk about onboarding, we talk about identity, I always hear us say frictionless onboarding and SaaS identity. And this sort of, again, continues to evolve from the control plane toward what we're building. Do you want to talk a little bit about those onboarding and identity stories and what they mean to you and what the reader should be looking for?

Tod Golding: Yeah, onboarding is one of those interesting things, which, to me, is at the epicenter of starting a SaaS, the construction of a SaaS environment. It's often the thing people put last or later in the process, which are like, "Yes, eventually, we'll have to onboard multiple tenants. But right now, we've got this working with one tenant, and we have these ways of creating another tenant. There are these manual things that will let us get the system up and running." When in reality, to me, how a tenant gets introduced into your environment, how you automate all the moving parts of getting a tenant onboarded, how does it provision the unique sort of infrastructure elements that are needed, how does it support tiering, how does it deal with setting up the billing? There are lots of moving parts to this, including creating the tenant, creating their identity, and creating all the things that set the stage for tenant isolation potential in the future.

If you get the automation of that right, and you think about what it means to automate all of that early in your process, it drives all these sorts of boundaries into the depth of your architecture. It makes you make choices in your architecture. And then going forward, it makes you think about, "Now, I've got tenant context injected into everything I did because I've introduced this tenant, and I can introduce as many as I want. Now, all the rest of my system has to think about, 'Well, what does it mean to truly operate in a multi-tenant context now? How are logs going to work? How are operations going to work? How are all these sort of moving parts of multi-tenancy going to be here?'" And onboarding ends up being that forcing function of bringing all those moving parts to life, including the control plane and the app plane, who owns what parts of onboarding in the control plane, and who owns what parts of onboarding in the app plane. Lots of good bits of that. So, I always, in fact, if you see talks I did and they say, "Where would I start?" I would always start with how are we going to get onboarding to work? Even if the app barely does anything, at least we can start building that foundation.

Evolution of Tenant Routing in SaaS

Bill Tarr: Let me pull back a minute from some of the implementation details here because I love the detail, but I also...I'm kind of curious as a writer, was there something that you did in this book that you're particularly proud of, something where you're like, "Wow, I never quite phrased this the right way, and I feel like I really said it right this time, and the reader's really going to feel...even if they see me on every re:Invent, even if they read my white papers, they're going to feel like they got something new out of this book"?

Tod Golding: I think it's not necessarily any one chapter or any one section of the book that I wrote that I would say, "I really nailed that." I do feel like the luxury of taking a little more space to say things in a few places gave me the freedom to tell more stories and to make more analogies and to do things to make an idea go from concept to a little more concrete for people. So, I think the freedom of having more space to write did that for me across multiple topics. But it was also more the connective tissue of you set it up well in this chapter and then now had, like...so, if you look at tenant isolation, tenant isolation has its tentacles into so many parts of the system. And before when I had to write about tenant isolation, I had to drag all those little pieces in with me to try to tell the story, but it sometimes convoluted it. So, for me, being able to pull these concepts apart and tell them in sequence and then connect them to one another was the thing that felt like I was finally giving a better version of that story than I might have told before. I would love to say that the bit I wrote on some particular topic made me feel better. I will say that the last chapter of the book, actually, the guiding principles piece of that, I did feel like that was one of the first times I tried in one place to sit down and just say, like, "Here are the things I would tell to a business at the highest level or to a technologist or an operations team that are like, 'You've got to have these things on your radar.'" And I think those have been laced all over other things before. So, putting them in one place maybe felt good.

Bill Tarr: Let me ask another sort of high-level question about this because I've actually showed up in several customer meetings now. People have held up your book in the meeting, and they're like, "Yeah, I came prepared. I actually read the book, I've got the Bible already." So, this is great. But, you know, it's exactly what you'd hope for, right? That people are arriving at the table with a lot of this information you provided in the book. But perhaps, is there somebody you've gotten feedback from in particular that really, like, it's shocked you, it's surprised you like, "Wow, I can't believe I actually...this landed in this person's lap. I can't believe that I'm hearing this feedback about the book"? Is there something that's really jumped out at you since you've written it?

Tod Golding: Probably, some of the LinkedIn sort of private DMs I've got from people who picked up the book, people I've never talked to before, who are picking up the book and they're at some stage in their business of trying to adopt SaaS. I always knew the guidance, I guess, was helpful and it would help people get over hurdles and it would help people sort of solidify their view of what it meant to be SaaS. I mean, obviously, this is not just me doing anything fantastic, it's just connecting them to the knowledge. But it felt like people were using the book to shape their whole new sort of vision and strategy of where they were going. And then it was more invasive into the broader strategy of companies in ways that I didn't expect. I thought some technologists in the corner would get it and then go use it and sort of try to help the business to understand, "Hey, we got to be doing more of this," or, "Now, we know how to do data partitioning." And instead, I saw people were like, "We're rethinking our whole approach to this based on the book." And I don't think I expected that would happen with people. I knew maybe you'd get one or two of those, but I got more of those than I expected.

Bill Tarr: That's awesome. It's really good to hear. And I think we do hear this from customers sometimes that this is really helping to evolve their business. But kind of going back into some of the technical weeds, as I was thinking about the topics that struck me as maybe evolutionary in terms of what you've talked about before, tenant routing is something that I feel like we've always mentioned along the way, but you had a lot more detail in the book about the different options for tenant routing and how tenants flow into a multi-tenant system, whether it's siloed or pooled. How did you feel about the evolution of that topic? Is that something that's been in your mind that you've been trying to get out in some way?

Tod Golding: It's a great question because tenant routing was, like, we'd go build a serverless SaaS reference architecture, we'd go build an EKS one. And you'd figure out kind of routing in there and it would be a nuance of, "What are we going to do for routing for this particular architecture?" But we never sort of pulled it out on its own and just said, "Wait a minute, this is a big decision, actually, as part of your multi-tenant architecture." And it has more complexity to it than I think we gave it credit for at the beginning. How are people coming in the front door of your application? Are they using subdomains? Are they not using subdomains? Or how are they identifying tenants as they come in the front door? And then which stack you're in, and how you're scaling, and what deployment models you're using mean you may need all kinds of different techniques to successfully route, and how does that routing scale successfully? There are so many choices.

If you just stay in the EKS space alone, there's probably, like, 15 choices, just about how to deal with tenant context and how to route it. And are you doing namespace per tenant? Which sort of partitioning and deployment model are you using? All those things, including back to identity even. Like, how do you route potentially and go force somebody to off and then off and get back in with the right tenant context and have all that happen seamlessly? I will say that is an area that I....now that you call it out, is one that I certainly, I think, wrote in more depth about than I had written about before. We've done talks about it, but I don't think we've put anything on paper. I definitely have a higher appreciation for what's involved there. And I do feel like the tech is moving so fast in that space. Like, if you said to me, "Come back two years from now and do another edition of this book," I would bet that chapter would be heavily influenced by new things that had evolved and emerged in space.

Bill Tarr: Even from my own experience recently, Tod, you and I have been talking a lot about cell-based architectures lately, and the routing layer of that is another new innovation. So, do you want to speak a little bit about how you...? You know, we've sometimes called it pod-based, sometimes called it cell-based. As readers go into the book, I think this is another new topic that a lot of people are going to be sort of surprised about. How should people be thinking about cell-based architectures even without the routing? Just the plain theory of it, how would you position them?

Tod Golding: It's funny that you bring that up because very late in the book, I had this moment where I covered it. I called it pod-based architecture because we had all this history of calling it cell-based that had this baggage attached to it. I think I'd go back and call that cell-based architecture now, but it's too late. I'd probably make a whole separate chapter on cell-based architecture if I could, just because I feel like it's emerging as a really popular model for organizations, especially people who have a big global footprint. I feel like cell-based architecture is a new way to think about scale and resilience for SaaS. It's not new for SaaS as much as it's a model we haven't emphasized a lot for SaaS.

I think for many, it's a way to sort of compartmentalize tenants into these groups, right? So, if I take cells and I say, "I'm going to take a set of tenants, I'm going to put one in a cell, and it's some logical grouping of tenants, and I do another cell, and then cells become a unit of scale for my business," I deal with, "Well, now deployment only has to think about the blast radius affecting deployment to individual cells." The scaling policies are constrained to whatever scale is within that cell. And I can then build more creative ways to figure out, "How big should a cell be? How many tenants should be in it? What kind of tenants should it be? Will I put my higher tier tenants in separate cells than my pool tenants?" So, I feel like it just opens up a huge range of possibilities, and then it also opens up the ability to say, "I want to be multi-region, or I want to span the globe," and the cell just becomes another unit of deployment to extend into those geographies.

Recommended talk: Enabling Developers in a Multi-Cloud World • Mauricio Salatino • GOTO 2023

Business and Technical Integration in SaaS

Bill Tarr: That all makes a lot of sense, Tod. Let me jump back up again a level into the business conversation, though. So, you said in the very beginning, SaaS is a business model that heavily involves the business. We started with a SaaS mindset. Now, we've talked to a bunch of technical topics that get pretty deep in terms of technology, cell-based architectures, routing. For anyone from the business side who happens to be listening, if you're coming from finance, if you're coming from product, if you're coming from operations, and they're thinking, "Man, I don't connect with my technical teams. I want to have this type of relationship with the technical teams. I understand it's a business model, and we should be involved," how would you encourage them to transform their organization to connect to some of these points? Are there some specific connection points for specific roles you would drive those users toward?

Tod Golding: In the last chapter of the book, I talk about the fact that these different guiding principles, and one of them is having a top-down vision for what success really looks like. And that's not new. That's a great business thing. But in a SaaS setting, it becomes especially significant because what I really want to be able to say is I want these product people and those marketing and these other roles that might have somehow, and operations even, view themselves outside the development or downstream of the development of the solution to be more directly embedded in sort of thinking about how their role is now influenced by the fact that they're going to be running a SaaS business, that they're selling a service. This is one of the points I make in the book quite a bit: if you're selling a service, you're not selling a product.

And part of the top-down vision is, "How do I get these different roles, both the tech roles and the non-tech roles, to think about the fact that we're building and selling a service, and how does that affect what goes in the backlog? How does that affect how the tech team thinks about, 'Well, what does it mean to build scale and resilience?' Well, I have to have a sense of what kind of service experience I'm trying to deliver, and I need product people and operations people to be my right-hand people in this view. And I want management value from both of us. Like, time to value for customers and these operational metrics and our ability to scale and achieve agility, I want that to be something the business is holding us collectively accountable for in a way that makes product, ops, tech, all accountable to this broader sense of we're delivering to a service mindset."

I'm actually pushing tech in another direction as much as I'm pushing fraud and the others, right? But to me, as an example I always give is, like, if I'm a product person who traditionally worked with features and functions as my primary deliverable, now I'm thinking about time to value and, like, "How could a noisy neighbor be potentially affecting our tiering strategy? And what kind of tiering strategy should we have? And what are the cost efficiencies we get out of supporting these different models?" And the answers to those things are answered by technical directions as well. So, the tech team has to be like, "Well, what kind of tiering strategy do you want? What would be a good model? What kind of margins are we really trying to get?" So, both sides should be a little uncomfortable, I guess, in this process. And I want them to be surrounded by a vision or direction from the management that is saying, like, "We have to succeed as a service, not just as a product."

Bill Tarr: That's great. Now, I think a lot of the technologists on the call, speaking to the other side of this table, would be surprised when they read the book that they're not going to see EKS and serverless or ECS or any of the technology-specific decisions until fairly late in the book. How would you help them? Because often, we show up at a customer like, "Hey, we're an EKS shop. We want to build this in EKS." How would you sort of justify or explain to them why those technology decisions are so far downstream from these other decisions you want them to think about first?

Tod Golding: That's a great question because I think a lot of people do start this like, "Well, we're going to SaaS. We're going to be containers. We're done, right? Like, we're done. Are we chosen serverless? That's it. We're all good." And for me, it's like, "Well, why? Why is EKS the right fit? Why is serverless? Why is it not a mix of those things? Where would you use which technology?" Right? And for me, this drives back to being much more fine-grained in the questions you ask of your architecture, right? It's back to the silo pool question even, like, which resource to be dedicated? Which resource should be shared? Well, it's not like a global decision for your whole product. This microservice might be siloed, but its data might be pooled. And those same sort of questions apply as you get later in the technology stage. Like, what are we really trying to achieve? What deployment models do we need to support? What tiering strategies do we need to support? What kind of economies of scale are we going to get? What's the nature of the workloads? And, like, the way that tenants are going to push the system, will some of those pieces be better supported by serverless, or will some of them be better supported by EKS? So, for me, I don't ever want the question to start with, like, "We picked the tech, now let's figure out what our SaaS strategy is." Like, that's sort of inverted for me. And I don't think people do that... I'm sort of going to the extreme there. I think people are weighing the pros and cons, but they do tip a direction from the beginning without necessarily asking all the hard questions.

Recommended talk: Building Distributed Applications with Event-driven Architecture • Eric Johnson • GOTO 2023

The Future of SaaS: Integrating GenAI and Operational Insights

Bill Tarr: That's a great point. One, I don't think we can get through a talk without mentioning the word GenAI. I know it comes in the book toward the end, and it's a newer topic that's evolving, but do you want to quickly sort of talk about how you think about Gen.AI and how you would position it for anyone who's in their mind like, "Hey, we're doing SaaS. We're trying to figure out the angle here"? How would you position it with them?

Tod Golding: Well, GenAI was the last chapter added to the book. It was not in the original outline for the book. And I definitely asked myself and the people around me and O'Reilly who were helping, like, "I don't want to put a GenAI chapter in the book just because everybody's talking about GenAI." But we had the foundations of some really interesting ideas about, like, what does GenAI mean to SaaS providers? And there was enough meat on the bone that it went from sort of, "This is a whole chapter, or is it a part of another chapter?" It clearly became its own chapter, and there was a lot more meat on the bone as I dug into the topic. And so, really, the idea here is not to look at GenAI, like, generically for SaaS applications. It's, what is...? And this is the story with almost everything in the SaaS space and multi-tenant space . What is the way that multi-tenancy changes the way I design and develop my solution around this particular technology? And in this case with GenAI, how do I offer targeted experiences to my tenants like we always do? Like, we use silo, pool, all those other things to offer targeted experiences. What's the GenAI version of that story? So, where do I use RAG, or where do I use fine-tuning? Where do I use knowledge bases? Where do I use all these other mechanisms?

And even silo, pool, and tiering, and noisy neighbors, all those things have a representation and a place that have to be addressed. Start looking at consuming these different technologies. And in reality, I feel like at the end of it all, I was thinking, "GenAI could have a huge impact on some domains and it gives you a way to offer a distinct experience on a shared platform to individual tenants without having to..." it's another flavor of economies of scale. If I'm in the e-commerce business and I sell to golf stores and tool companies and clothing stores, what it means to search for a product or, like, interact with individual stores could be different across each one of those spaces. But under the hood of that, all could be served by one GenAI experience. That's a fantasy version of it. I think the real world still has to prove out that those things are going to happen, but that's the scope of what's kind of in that chapter is, like, what's the overlay of multi-tenancy across these different GenAI concepts? And it ends up being the case that there's lots of them, including, like, how do you do pricing for a SaaS product that is using GenAI? Like, what is the size of the prompts and, like, how are you dealing with that variability, and how are you going to calculate that and pass that cost along? Like, there's lots of nuance in there.

Bill Tarr: So, anyone who's listening and thinking about reading the book is probably like, "Well, a lot of the stuff you're talking about is how I would build SaaS. Now, people who've already built SaaS and are running and operating a SaaS product today also have probably questions about the operations of SaaS. And in the past, we didn't talk about it as much, but I noticed you got two or three chapters, I think, dedicated just to the operational side of SaaS, how you operate a SaaS solution. Do you want to sort of talk about what that experience is and how we think about that to anyone who's already built a SaaS solution?

Tod Golding: One of the common themes is that we talk to lots of SaaS companies, we engage lots of SaaS companies, they're successful SaaS companies, they're doing well. But if you look at the bar we set for metrics, and analytics, and operational insights, and proactive policies, and all these things we want to be able to do to say, "Hey, the more we share infrastructure, the more we have one unified environment for our customers, the more robust our operational insights have to be." And our operational insights can't just be, "Is it healthy, or is it not healthy?" Our operational insights have to be, "Hey, what's this one tenant doing that's affecting other tenants?" Or, "What tenants of this tier are doing that's affecting the SLA of the other tenants in the system?"

And so, the scope of what you look at, the granularity of what you look at in a multi-tenant environment on that operational side is wildly different, and that's just through the tech view. Then you layer on top of that, the business now wants to know lots more about what are tenants doing? How are they consuming the system? And what patterns are they consuming the system? How is this new feature going? Even down to, like, how might we release new features and functions and how might we think differently about how we stage and roll out waves of canary releases or A/B...whatever you want to sort of layer into that.

So, for me, A, the bar is way higher for just the risk to the business if you don't have robust operations. But, B, the richness of information you can have in that environment can tell the business, the tech, the ops teams, "We need to go add this stuff to our backlog of stuff that needs to get done because we want to see problems before they're a problem for customers, and we want to be able to identify them." And we build all these cool scaling and resilience and all these other policies, but then new tenants are coming in, new features are coming in, and you want to know how the architecture and the environment is reacting and responding to all these different stimuli and be sure that you have the tools you need to sort of proactively see what's happening or just make good decisions about how to make it better. Like, maybe we're over-provisioned on all kinds of things because the policies we adopted are wrong, but we don't know it until we see it in the wild. Well, I want to know all that stuff. I want to have my finger on the pulse of all that. And I want the business people to want to have their fingers on the pulse of that.

Bill Tarr: Great answer. One final question, Tod. So, you've already mentioned a couple of things. I heard you talk about cell-based architecture, how that might evolve, and GenAI, how that might evolve. If you think we're going to write the sequel to this in five years, what are some of the topics that you think are really going to change over that time? And are there any new topics that are just, like, on the edge of your radar, you're like, "I think it's not ready to talk about, but we're going to be talking about it later"?

Tod Golding: Well, there's a fantasy version of that to me, and there's a reality version of that, right? Because in the fantasy version of this, I'd like to think that these concepts, like, if you take a concept like tenant isolation, for example, and you look at the way we have across the cloud vendors, lots of sort of managed services which hide all the detail of their implementation to us, but they haven't totally infused all of these multi-tenant concepts into the services directly. Even the notion of a control plane could be something that becomes more standardized. We already see people, in fact, building control planes. We see partners building control planes. So, for me, the fantasy version is that the scope and the nature of what a multi-tenant developer will have to do in the future and the complexity of what they'll have to tackle, the scale, the resilience of it will be a smaller footprint than it is today. You'll have less things you have to think about. And how to get some of these complex problems to work won't require all the nuanced things that we have to talk about and all the questions we have to ask.

So, I feel like a longer-term version of this would be, like, "Well, then how are we now extending the capabilities of what SaaS brings both to the business and to customers that it couldn't achieve before? Like, are we able to release new features even faster than before? Are we innovating faster?" And so, the capabilities of all these solutions are suddenly growing at a much faster rate before... And is the price model, the approachability, the ability to sort of get that value at a reduced price point going to come down? I feel like SaaS is also going to find its way even deeper into domains where people think it wouldn't go. So, I feel like more...all kinds of industries said they would not be SaaS that are now SaaS now. I think that wave will just continue and it will be accepted. So, compliance and security and tenant isolation, all these things people will work about will get resolved to the point where even more people will come.

Bill Tarr: Great. I think we're all looking forward to it. Simpler, faster, more broadly applicable SaaS is, I think, where we all like to go. Tod, thank you very much. This has been a great interview. Thank you for taking the time. And I hope everyone who hasn't read it yet will reach out and read your book.

Tod Golding: Thanks. I really appreciate it.