Serverless Apps on Cloudflare

From CDN to Platform: Cloudflare’s Serverless Story

Ricky Robinett: What's up everybody? Welcome to GOTO Book Club. I am Ricky Robinette. I lead up developer relations at Cloudflare and I'm super stoked to get to chat to Ashley Peacock today, who wrote Serverless Apps on Cloudflare. Ashley, you want to share a little bit about yourself?

Ashley Peacock: Hi everyone. I'm Ashley as Ricky said, and the author of Serverless Apps on Cloudflare book all about Cloudflare, and I am a staff engineer here in London. And that's me.

Ricky Robinett: Awesome. You said a book all about Cloudflare. One of the things you and I have talked about is sometimes when we say to a developer, like, have you heard of Cloudflare? They say yes, but they maybe haven't heard about what you're talking about in the book. Do you want to talk a little bit about that?

Ashley Peacock: Yeah, I think so. Cloudflare is effectively like a competitor if you like, to the likes of AWS, Azure, GCP and all those. When people hear about Cloudflare, they think like a security company. They just do like firewalls and that kind of security side, which obviously they absolutely do. But the side I'm more interested in and the side I talk about today is the developer portal side of the platform where you can deploy your apps. There's databases, caches, basically everything you need to build your applications. And from my point of view, I wrote the book because I felt like it was kind of under utilized. Not many people knew about it. I think it's gaining a bit more traction now. That was my kind of driving force behind writing the book.

Ricky Robinett: Awesome. I love telling the story a little bit of how the platform happened. If it's not too much of a divergence, which is to build the Cloudflare network. Often people think of us as a CDN or security company. We had to put data centers all over the world, and as we did that, something interesting happened, which is customers said, hey, you have these data centers everywhere, close to our users. Can we run our code on those servers? And it was a thing that actually we didn't set out to do, but customers kept asking for it and originally to customize their CDN and do different custom rules. We put this workers layer that let them customize things with JavaScript. And then it just kept growing from there. So, I always love that side of the story just because it's such a customer driven part of our platform.

Ashley Peacock: I remember I think the first time I did anything on Cloudflare was editing JavaScript in the browser to modify HTTP headers. At that point, I was like, oh, this is terrifying. I'm writing JavaScript in my browser, and I have no idea if it's going to work. But it's come a long way now. The first app I built was like a discord bot. And I remember even then, this was probably 3 or 4 years ago, and I think there were only like workers and maybe KVS was the only storage. So even then, it was still a little too green to build full apps. But nowadays, I feel like it has everything you need.

Ricky Robinett: I love hearing everyone's first apps and I do think the best kind of coding is a little terrifying, where you're doing it and you're like, what is going to happen here? And so I like that you had that feeling a little bit in the early days.

Ricky Robinett: The book - talk about what is in there, the things you show people, what they'd learn if they went through it.

Ashley Peacock: I think it tries to cover all the key parts of the developer platform. So it kind of takes a series of mini projects and breaks them down into sizable chunks, where each chapter effectively focuses on a different product. So we've talked about workers here, for example. And you can think of workers - the nearest parallel I can draw is AWS Lambda. It's very different in terms of runtime and how you build on it, which the book will cover. But that's the nearest parallel I can draw - most people have heard of Lambda, so it's the easiest one to draw.

Moving on from workers, you learn how you can use databases. D1 is Cloudflare's SQLite database. There is KV in there for caching. There is R2, which is effectively like AWS S3. It's compatible with the S3 API. There are queues, so you can build asynchronous applications on Cloudflare queues. Of course it includes some AI because it wouldn't be a book in this day and age without some AI. Cloudflare has their own inference platform called Workers AI, where you can run a bunch of models.

Using the suite of products, you have a bunch of projects that will teach you how to use each product. And I think each of them has some unique things to Cloudflare that's important to learn, especially like I talked about workers. It's similar to Lambda, but it's not the same. And it's important to highlight that part of the platform. And you just gradually level up on the platform throughout, ending with how you deploy them as well, because I think it's important. In the book it uses GitHub actions, for example.

Recommended talk: Creating Software with Modern Diagramming Techniques • Ashley Peacock & Stefan Hofer • GOTO 2024

Diving into Cloudflare Technologies

Ricky Robinett: What's your favorite thing you've built on the platform?

Ashley Peacock: I think one of the things I've built lately is an email-based AI support agent that can also do live chat and also do human-in-the-loop. It uses this cool piece of functionality for workers, where you can effectively bind email addresses to workers so that when an email is received to a specific email address, you can have a worker that's invoked, some code is executed, and then you could do whatever you like with it. In this case, you could send an email to this AI support agent and then get a response back via email. But then similarly, you could flip over to a live chat and then live talk to the AI as well. And then there's also a mechanism where a human could join that live chat.

The thing that brought this all together is Cloudflare's party piece of durable objects, which I feel like I describe slightly differently every time. But I think I've settled roughly on that it's like many little servers that you can deploy pretty much anywhere in the world. And it's durable and it has state built into it. In this case, going back to the email agent, a support case is one durable object. It kind of maps 1-to-1 to your key entities. And then it has a bunch of cool pieces of technology on top of it. In this case it uses WebSockets, so it has sockets built into it, which really enables those real-time applications, like a live chat.

Ricky Robinett: I love hearing you explain things. Also if folks listening don't follow you on social media, you do these great posts where you do a detailed breakdown of different concepts. So not just the book. You can get it online. Thinking of agents, can we zoom out just for one second? Can you give how you think about agents in general and explain them?

Ashley Peacock: I feel like everyone has their own definition of what an agent is. I guess the kind of leading one that I think Anthropic has is it's kind of an autonomous thing that can do a job on its own. That's kind of the high-level explanation of what I would say an agent is, with simpler things just being prompt engineering. But an agent has a more involved life cycle to it, a bit of autonomy to it. It can do things like, you can give it a task and it can go off and do it.

Ricky Robinett: I like that a lot. I had someone describe it recently as an agent that can both read and write from the internet, which was a very easy way for me to grok it - like it can go out and consume the internet, but it also can take action and leave an impression on the internet. Was an interesting way to describe it.

Ashley Peacock: That makes sense to me. I think Cloudflare has its own agents package. So I think that's one of the newer things that it's not in the book, because Cloudflare produces a lot of stuff at a rapid rate, but it's actually a framework to wrap a bunch of common things that you need to do with agents. Like I just said, it uses durable objects under the hood, I believe. So it gets you WebSocket connections so you can talk to your agent via WebSockets. You can send them HTTP requests. You can schedule tasks for your agent. And it just wraps in a nice package that makes it easy for you to use.

Ricky Robinett: Next edition of the book, it'll make it in. I don't want this to be too rose-tinted view on things. What are some of the things that you think surprise people when they're using the platform that you cover in the book that people have to think about?

Ashley Peacock: I think the main thing is, if you're used to, let's just say AWS since it's the easiest one and the biggest one, and you're deploying a lambda to a specific region, right? Because by default lambdas are deployed to a given region. With Cloudflare, when you're deploying a worker, it's deployed globally. There is no region. Even if you want to, you cannot deploy it to a single region. It's going to go everywhere, which has benefits.

So if I'm in London, you are over in New York. If I deploy something to Cloudflare and we both access that website, you're going to get served from the data center that's on the East Coast. And I'm gonna be served from the data center in London. That's obviously going to reduce the latency, and the application is going to be snappier and just quicker, which is nice. But when you throw in other resources, let's say you've got a Postgres database that's hosted in AWS, like a legacy database, and you don't configure things properly in your worker, you're going to suddenly be feeling like, oh, my app is really slow because you in New York, your calls are going from New York to Europe every single time. If you're doing like ten database queries, it's got to get back and forth every single time.

That's where things like smart placement come in, where you can effectively enable smart placement. Then Cloudflare will automatically place your worker in the place that's going to reduce the latency the most. So in this example, if the database is in Europe, it makes sense that the worker is in Europe and the round-trip time from the worker to the database is minimal. You do increase the latency from the end user to the worker, but overall it's going to decrease your latency. So that's one thing where you have to think in global terms. I think it's one of the trickier things.

Recommended talk: The Art of Embracing Failures with Serverless Architectures • Anahit Pogosova • GOTO 2024

Developer Experience and AI Integration

Ricky Robinett: I agree and I want to talk a little bit about DevEx, but I'm going to keep it on this thread of things that surprise people. One of the things you have to walk people through in the book is bindings. Can you talk a little bit about bindings? I have found it always takes people just a minute to wrap their head around it. And then once they get it, they love them. But it's a new concept for a lot of folks.

Ashley Peacock: If you're used to other platforms, say you spin up a database and you want to connect to it, you probably reach for some sort of SDK, some sort of npm package or whatever it might be. And you probably go and get some username or a password or a connection string, and you put that in your secrets and then you spin up the SDK and off you go.

With bindings, what you actually do is you create the resource in the same way you would in the other example. You create a database or create a cache, and then you just get given an ID from Cloudflare. You put it in a configuration file in your worker, which is called a Wrangler file (recently you can use JSON for that Wrangler file), and it basically configures or binds your worker to that resource, let's say a database.

Then what happens at runtime - and there's no need to use any secrets, you use namespaces just with the ID - what happens at runtime is Cloudflare will inject the correct binding. So if it's staging or production, it'll inject the different binding at runtime into the environment. And then you can access that environment throughout your codebase.

Effectively it's an automated kind of platform-level dependency injection. You don't have to worry about it yourself. You just know that when the worker runs, you're going to get an object injected. And what I should add is it has methods, like it's a proper client object with methods you can call. So for a cache it has to put and get and set and so forth. And it's just really elegant because you don't have to worry about secrets. There's no configuring IAM roles because Cloudflare handles all that for you. You can talk to your own resources and it just feels kind of effortless.

Ricky Robinett: Yes. I did a demo last week and I showed the AI binding, and people really had faces of shock when they realized, wait, I don't have to get API keys. I don't have to do that whole song and dance. I hope more platforms embrace this model selfishly, just because as a developer, it makes my life so easy.

Ashley Peacock: I think if you extend it, one of the other common things you need to do is call between services. So if service A needs service B to do something - that's really common in a microservices architecture - with service bindings in Cloudflare, you can effectively bind two workers together and you get the same sort of thing where it's injected at runtime.

But rather than you having to fetch via HTTP going over the internet, the actual networking is internal at Cloudflare. You don't worry about it. Quite commonly they'll even run on the same device, same data center, same machine. So there's no real latency. And it removes a lot of the problems of the fallacies of cloud computing - the network's not reliable and so on.

You can even go as far as rather than having to serialize JSON between your two workers, it supports RPC. So you can literally just call methods between two services, and then you don't have to worry about serializing/deserializing data. And again, all those security concerns around having to have API keys between your two services just goes away. So it just simplifies things a lot.

Ricky Robinett: I agree. More broadly on developer experience, how do you assess developer experience when you're using a new tool? It doesn't even have to be Cloudflare, but you know, you're sitting down, you're thinking about what you're going to build. How do you decide, hey, I really like this DX or hey, this is leaving a bit to be desired?

Ashley Peacock: I think for me, the way I like to think about it and the reason why I like Cloudflare and other things as well - recently Cursor is one that I really like as well - is things that empower me to focus on the things that I care about. So I like to write code and I like to deploy applications simply and easily. If the tool or the platform is working in my favor and working for me rather than me having to fight the platform - like I get my IAM role slightly wrong and suddenly my lambda can't talk to my database - that's not good.

So for me, the real focus is on empowering the developer to do what they enjoy and offloading as much responsibility as possible to the platform. To steal one of AWS's things, they do the "undifferentiated heavy lifting." Don't get me wrong, I use AWS in my day job all day long. It's a fantastic platform, but it's also humongous and has so many years of trying to add more and more, and it just builds up. Whereas challengers have the opportunity to do things simpler. They didn't have all those years of condensed legacy systems to try and plug in. So that's what I think about when it comes to DX.

Ricky Robinett: Also a huge Cursor fan. Walk me through how you're building with the assistance of AI now. What's your process? You're sitting down to write a new application or update something you've already built. How does that all go with you?

Ashley Peacock: I think it depends largely on whether I'm doing my day job where I have the luxury of using Cursor, or my side projects. Recently I compared Cursor and Copilot because Copilot brought out their own kind of agent thing. Cursor with the tab completion kind of reads my mind quite often, whereas I don't find Copilot does. And I kind of lean into the Cursor a lot because I find it just better.

I think the thing that I focus on with AI is getting it to do 80% of the work, like the boring work - the loops and if statements and things that I have written like 10 million times that I don't want to type again. And then allow me to focus on really honing in on the final 20%. What I find, particularly in my day job where it's a codebase with millions of lines of code in a huge monolith, is that there's lots of talk about "is AI going to replace engineers?" I think it's focusing on the people with domain knowledge and better prompting skills - they're going to get more out of it.

Ricky Robinett: There's something about knowing how to talk to the AI and knowing what to ask for, that you can't just tell it anything and get the right answer that I think people miss out on.

Ashley Peacock: Bringing it back to Cloudflare in particular, sometimes if you prompt it, it might use some old SDK stuff. I've been using the Cursor Rules that they shipped (the mega prompt), and I find that helps a lot because it basically keeps it up to date with the latest platform developments, rather than relying on their training, which is not always updated, especially with a platform that moves quite quickly.

Ricky Robinett: The prompts, the mega prompts Cursor Rules are such a huge unlock. It took me way too long to start using one, and once I did, life got a lot better.

Recommended talk: Beyond the Cloud: The Local-First Software Revolution • Brooklyn Zelenka & Julian Wood • GOTO 2025

Local Development

Ricky Robinett: All right. Let's talk about local development.

Ashley Peacock: I think what we talked about is super cool when you go into production, and we've talked about bindings. And one of the tricky things when building applications locally is you need to emulate databases, caches, queues, whatever it might be. Maybe you reach for Docker containers or maybe use LocalStack or real resources in AWS sometimes, but all these things have drawbacks.

Whereas with Cloudflare they've done something very different where they effectively emulate the same runtime that runs in the data centers locally. It's called Workers, and that's the exact same runtime. So you don't have the problem that it behaves differently locally versus in production. You can catch errors earlier.

But the other really nice thing is when we were talking about bindings and creating databases, when you run that worker locally, because you can run everything locally as you would in production, it will emulate that database or that cache or the queue or whatever it might be, so that you don't have to spin up anything. You can literally just run "npm run dev" and you have your worker running locally on localhost and it will just emulate those databases. You don't have to worry about anything.

If someone's new to Cloudflare, they could have a worker with a database, a cache, a queue, a durable object running locally on their Mac in like five minutes and everything is just going to work out of the box and be emulated. And this is probably one of the things that people enjoy most - they can just get started locally. You don't even need to deploy to the cloud. You can run everything locally. And again, this is that platform doing the heavy lifting for you. You don't have to worry about that local dev experience. Everything is just out of the box and it's beautiful.

Ricky Robinett: Yeah, this is a bit of a tangent, but my daughter is eight years old. She's been learning to code. She's been using Cloudflare. And to that point she doesn't have to think about deployment or locales. And I remember the time I set her up with her first D1 database because she needed to store some stuff, and she's still grasping what a database is. But she's not having to learn it through setting up a server with a database. She's learning it through the concepts that she's using the database for. And so, just so powerful.

Data and Storage

Ricky Robinett: We talked a lot about workers and touched on some of the other components. In the book, you take each product by chapter. Can you walk me through the data and storage side? So KV, D1, R2, everything in that part of the world.

Ashley Peacock: I think KV is a good one to start with. Some people might look at it and say, "Oh, it's like Redis," which the interface is very similar to. But if you didn't read the docs, you might be confused by your reads and writes taking longer than expected, or sometimes getting something back different from what you wrote two seconds ago.

While the interface is like Redis, it's eventually consistent under the hood. The writes are a little slower because it's designed to be read-heavy. It's perfect for caching things like configuration and auth tokens. What happens when you write is it goes to a central data store, and then it'll take up to 60 seconds to propagate around the entire network. So if you write in New York, it might not be immediately available if someone reads it two seconds later in Europe. Understanding this behavior is where reading documentation really helps.

Durable Objects are also worth highlighting. I think as they become more understood, they unlock a different way of building applications. You can effectively represent your domain entities - like agents in an email system or orders in an e-commerce website - as durable objects. They're defined as JavaScript classes in your code, so you can add methods like "add items to an order" or enable a customer to query their order directly. It feels like a completely different approach to building applications.

Ricky Robinett: It's such a powerful concept.

Ashley Peacock: Another recent addition worth mentioning is Workflows, which I really enjoy. There's a growing trend around durable execution - being able to recover from failure. Let's say you have a process with ten steps, and on step five you make an API call that fails. What do you do? Do you roll back your last four steps? Do you retry? How do you ensure the state from previous steps is still available?

Workflows allows you to define your processes in normal JavaScript, and if a specific step fails, all the state from the previous steps can be retrieved, and it will retry from where it failed and then continue. I see it used a lot, especially in the agent space where you're often going through multiple steps and need to handle errors gracefully.

Ricky Robinett: It's such a common problem developers face. I've been using it in the agent space because agents are so workflow-oriented and often go through these steps. When they're browsing the internet and things go haywire, I want that step to abort and try again. There are lots of benefits to those retries when you're doing something non-deterministic.

Exploring Cloudflare's AI Ecosystem: Vectorize, Workers AI, and AI Gateway

Ricky Robinett: We touched on the data and storage side. Let's talk a little bit about the AI side. So vectorize, AI gateway, Workers AI. How are you thinking about all of that and using it?

Ashley Peacock: To take them one by one, let's vectorize first. Vectorize as the name suggests is Cloudflare's vector database. This effectively is key in things like RAG systems where you need to be able to insert your knowledge base for your agent such that when you want to do your RAG, when you pull the relevant context for a user query, you can query vectorize and say, hey, here's your input. Give me the five or ten most relevant pieces of information based on, let's just say cosine similarity for example. So that's vectorized in a nutshell. Super useful if you're doing RAG.

Workers AI, as I mentioned before, is Cloudflare's global inference network. So again, like I said, everything pretty much on Cloudflare is global by design, and Workers AI is no different. They have GPUs around the globe so that when you're running your AI workloads there's not increased latency. Your worker and the GPU are probably very close to one another. That's going to reduce the amount of time that it takes to generate the response. They have a plethora of open source models. So the flagship ones are probably the Llama models and the Mistral models, those are all in there. There are Whisper models and all sorts of things. So they have a pretty strong catalog, at least in terms of open source models. You're not going to find Anthropic or OpenAI models, but you will find those open source models.

Then if you are using OpenAI or Anthropic, there is AI Gateway, which I think if you're on the Workers plan, which is $5 a month, you get that for free. And as the name suggests, it's just you can funnel all your requests through AI Gateway and it will allow you to do things like rate limiting. There is caching in there so you don't actually have to go downstream. You'll just cache it at the gateway. You can configure that so you don't have to cache all the time. And they're adding new features like guardrails I believe are coming so that you can detect things at the gateway level like abuse and personal information. So to me, because it's free and if I'm using a downstream provider, it makes complete sense to put it through the gateway, because I just get all these features for free that can speed up my app and make it safer.

Ricky Robinett: I am personally super excited for guardrails. I've been on the internet long enough to maybe not fully trust the internet, and I remember the first AI application I launched, people can just say and do anything with this. The guardrails concept of being able to make sure people aren't doing bad stuff with any AI application I launch is super exciting to me.

Ashley Peacock: It's not there yet, but it's coming. I think as a judge - like eval. You can evolve at the gateway level using an LLM where you can almost like get the AI to second-guess itself, where it's going to say, "Hey, was the question, does this actually answer the question based on the context?" I don't think there's information specifically on how it's going to work yet, but I know it's been talked about, and I think that'll be super useful. I've seen a bit out there already, but it certainly doesn't seem like something that's a commodity yet. It will be hugely beneficial when it's there.

Ricky Robinett: Agreed. What are you most excited about right now?

Ashley Peacock: I think for me, I would take it from two different angles. First, next month there's normally a Cloudflare developer week. If you want to see what is coming to the platform, that is the perfect week because there'll be lots of announcements, product developments, new products and stuff. And it's just a really cool week.

For me, I think it's just kind of nice because I've been around Cloudflare for a while, and it's just nice to see the adoption of the platform and it getting more popular, more people appreciating it. I'd really like to see it used more in an enterprise context. I think organizations are coming, which is effectively the ability to have different accounts within an organization. I know that's something we struggle with where I work in my day job, because you have one account, everything has to be namespaced just by name, which the security folks aren't too happy about because your staging and your production are on the same account and there's nothing stopping them talking to each other.

I think the role-based access control has got better - it's more granular now, but I think some improvements to that will be really nice. Where I work, I'm the only person that can log in because if you log in, you can do lots of things - you can delete the worker or edit the worker. And I'm like, I don't know if I want everyone on the team to have that access. I trust my colleagues, but it's more about accidentally pressing a button. So I think there are a couple of areas where some improvements would be nice, but it's more from the enterprise perspective. I just love to see it more widely adopted because I think it's powerful and you can move super quick.

Ricky Robinett: I can neither confirm or deny any rumors, but I would agree. Those are exciting things. And I even had to delete a GitHub fork the other day of our Cloudflare docs, and I checked like 20 times that I was on the fork because if I accidentally delete our entire Cloudflare docs repo, it's going to be a long day. And so plus one to the idea that it's not that people intentionally do something bad, it's just sometimes you get lost in the interface. And luckily I deleted the right fork.

Closing Thoughts

Ricky Robinett: So in your book, let's close out. What is the main thing you want people to know as they're looking at the book and thinking about, is this for me?

Ashley Peacock: I think from my point of view, if you've been using other cloud providers for a long time and you want to try something different, I think this is the perfect chance to try something different, because I think it reshapes how a lot of things work. As I said, the platform is not faultless. There are obviously problems with every platform, but it's rare in my experience to introduce someone to the platform and for them to think it's terrible.

I think if you were to track retention, it would be pretty good because the platform works for me and I think lots of other people will have that same feeling. And I think it's just fun to learn a new way of doing things and just seeing a completely different approach. So deploying your workers, the bindings, the durable objects are completely different to anything you've ever seen before. So it's really just if you want to try a new platform, you want to learn some new technologies and just try something different, this is definitely a good thing to try.

Ricky Robinett: Well, Ashley, this has been great. I always love hearing you talk and explain how you're thinking about things and helping others understand it. The book is Serverless Apps on Cloudflare, so hopefully folks get a chance to check it out. Any closing words?

Ashley Peacock: I think you did it justice. So thank you. I really enjoyed our conversation. I hope everyone enjoyed the conversation too. If you have any questions, you can reach out. We're both on X/Twitter. Just reach out. I think we're both happy to always chat about Cloudflare.

Ricky Robinett: All right. Thanks everybody.