Embrace Step Functions for Efficient Workflow Management

Introduction

Eric Johnson: Hey. It's another GOTO Unscripted. Welcome. I'm Eric Johnson. I'm a Principal Developer Advocate from AWS. Ben Smith, who are you?

Ben Smith: Hello, I'm Ben Smith. I'm also a Principal Developer Advocate at AWS.

Eric Johnson: Ben's a real Principal Developer Advocate.. He knows his stuff. So, how's your day going, Ben Smith?

Ben Smith: Pretty good. Sunny here in England, 4 P.M..

Eric Johnson: Oh, that's kind of a rarity. Right? Okay. So where are you? Tell them where you are?

Ben Smith: I'm in Brighton on the south coast of England, about 50 miles south of London.

Eric Johnson: Nice. So, now, you say Brighton, so there's no T there. But there's a T in the spelling.

Ben Smith: Yes. And if you're posh, you say, actually.

Eric Johnson: I don't know if I'm posh enough for that. I can go to Brighton. And that way, you know I'm an American. 

Step Functions: What They Are and How They Help

Eric Johnson: Anyway, so we're gonna get started here. Today we're going to be talking about... We're really GOTO Unscripted, which goes anywhere. So we talk about all kinds of things, like where Ben Smith lives, and how to say Brighton the posh way. But we're also going to be talking about Step Functions. Now, for folks who have never heard of Step Functions, they may also be called state machines. Ben, can you kind of fill them in, what is it?

Ben Smith: Sure. I also like to refer to them as workflows, because I think it maps more easily to day-to-day stuff. Step Functions... Sure. It's a service on AWS that allows you to create workflows that consist of different steps or different states, that orchestrate pretty much all the other AWS services and actions that are available. It's serverless. That's why we're talking about it, right, because we're serverless. It's fully managed, it's pay per use, and it's a really great way of building out serverless applications that orchestrate other AWS services.

Eric Johnson: Okay, so let's kind of get into that for a second. So this idea of orchestrating any other service on AWS, how does that work?

Ben Smith: So it works by using the SDK of that service. So let's say you want to invoke a lambda function, right? There's an action or a state that you can use to invoke a lambda function. Let's say you want to get the result or the response of that lambda function, and you want to persist that to a DynamoDB table. Well, then you can use the put object for the DynamoDB action. And you can add decisions or logic. So you can create branches, you can use parallel statements, so you can run these different branches concurrently. You can have dynamic map states. So maybe you don't know how many inputs you're going to have for a particular array, you can make that dynamic. So you can run things in parallel dynamically. And you can add fail states, you can catch errors, you can retry errors, and you start to get really robust workflows for building out your serverless applications.

Eric Johnson: So what's really cool about workflow Step Functions... And I do the same thing, I kind of change what I'm talking about. What's really cool about this workflow... And you hit on a lot of that. I think you call it spaghetti string code. Spaghetti code, spaghetti string, spaghetti code. And I totally see it because I've had code that, you know, it's 30 lines long going, if it's this then do this, if it's this then do this, if it's this then do this. And then when I get it all working, then I have to go back and write all the error code. If it's this then do this, but if it's not, or if it failed, or if this... And then my code just gets loaded, and long, and...you know, like that. And Step Functions kind of takes that all out for us, right?

Ben Smith: What you've just described there is orchestration as code, right? You're building out a bunch of select, or switch cases, or if else statements that will do different things based on different inputs and outputs. Imagine if you could visualize that as you're building it, right? Because what you're building really is an orchestration workflow. So why not use a service that lets you understand that cognitive model visually. That's what Step Functions is.

Eric Johnson: That's exactly right. It's interesting to me... I almost fell over there. So keep an eye on me. I could fall over. It's interesting when we talk to folks, some pushback that I've seen in Step Functions is like, I don't want to do all that logic. It's like, but you already are. You're already doing it, you know, in a way that... And it's making you do more than that because you have to handle retries, you have to handle errors, you have to... Code is logic, right? And so, people ask me, when should I use Step Functions? And we're going to get to that because there's a pretty good answer on that, and it gets further along as we've added more to Step Functions. But my answer is, if you find a lot of the spaghetti code in your functions, or in your lambda functions, if you find yourself writing a lot of logic, I call it routing logic, routing logic versus business logic.

So I wanna step back to something you said earlier. So several years ago... And I remember that it's all kind of jumbled, it all runs together. Several years ago, it used to be in a state machine, you had about nine integrations, right? We integrated with DynamoDB, things like that. If you wanted to integrate with anything else, then you had to use a lambda function. So you would invoke a lambda function. It would use the SDK to talk to something and get that information back, then kick it back into the workflow. But we changed that. Kind of go into the SDK. I know you talked about it a little bit earlier. But how does that help?

Ben Smith: Sure. So I think it's only been about three years that we've had this feature, right? So before this feature, you would use something called...we now call it optimized integration. So this could work with things like S3, DynamoDB, lambda, I forget which other ones there are. What this does is it kind of obfuscates a little bit for you, the parameters that you need to set for the action, right? It's sort of a more tidy, neat way of doing it. But what we found is that developers needed to do way more than use these nine services. They needed to, I don't know, send things to IoT Core, right, spin up the ECS tasks. Pretty much anything that you want to do on AWS, you need to be familiar with the SDK. And so what we found is that developers were creating a lambda function, and in that lambda function, they would pull in the requirement for the SDK, maybe as a layer or in a package dependency, depending on your runtime.

They would pull in that SDK, they would write their 10, 20 lines of code to run that SDK action, and handle the response. And they would send that response back from their lambda function. They would call that lambda function from Step Functions, right? So what this means is a few things. The first is that you're getting a cold start if you're triggering that lambda function. You're getting an invocation cost. And you have to wait for that lambda function to return its response. And you have all this code, this extra code to manage, which really, all you're doing is calling an SDK on another service. You're using the lambda function as a kind of glue code. You've got to manage the security permissions for your function to be able to do this thing. And you're adding complexity to your application. You're adding latency, complexity, and cost, right? And then you're orchestrating that inside a workflow.

So what we did in the Step Functions team, is they made a way for you to call many, many of these other actions and these services directly from within the Step Functions substrate. So the Step Function service itself is responsible for performing that SDK action. Whether it's spinning up an ECS task or sending a message to an IoT Core topic, you still have to configure your workflow with the action permissions required to do that thing. So still nice and secure, but you will eliminate the cold start times required from lambda, you will eliminate the invocation costs that might be required from lambda. And you eliminate all those lines of code that you would need to run if you would manage that or handle that using a lambda function. Right?

Recommended talk: Building Low-Code Applications with Serverless Workflows • Ben Smith • GOTO 2023

Eric Johnson: I remember sitting... I don't know if I've told the story or not, but I was sitting with Justin Callison, who's in charge of a lot of Step Functions. And when they explained this to me... my jaw physically dropped open. It was like, first of all, that's brilliant. Whoever went, you know what, let's just use the existing SDK. I was like, oh, my gosh, that's brilliant. Because inside Amazon, when we add features, things like that, the SDKs are automatically updated. One of the first things updated, right? We have a lot of automated processes wrapped around that. We have Smithy, and we have, you know, things that are going on there. So that allows or enables Step Functions to just automatically take that.

And the interesting thing here is, a lot of times we think we're just passing... Look, so when I use a transition, here's the JSON. So that configuration you were talking about, when you were talking about the state, but also inside. You still have to tell it, hey, here's the DynamoDB table, either some JSON you'll have to do, just like you would do with a lambda function. But then it handles the signed request to the service, it handles all that, you know, behind the scenes. And it was brilliant. I was like, oh, my gosh, this is incredible. I was so excited about it. And it's good to see him do that. And so Step Functions went from seven, nine direct connections to... And you said the number before, what's the number now?

Ben Smith: I think it's 220 services plus, and I think that gives us about 11,000 actions at this point.

Eric Johnson: And so if you've never seen the workflow studio, and hopefully we'll get a picture of that up, or we'll get a link to it, it's inside of your AWS account, or...and I'm just gonna throw this out here, because I've actually been using quite a bit. If you use App Composer locally in your IDE, you can open a workflow locally and edit it locally. It's pretty cool. So I've been doing that quite a bit. So check that out. So you could drag those in. And you could just search through all this. You could go, oh, what am I trying to do? I'm trying to put an item in DynamoDB., you know, and there it is.

AppComposer

Ben Smith: Well, hang on, we just jumped over a bunch of stuff there, some big stuff. Right? Big launches from back in November. So, going back a year, I would imagine there's many people that are not using or don't know about the capabilities of App Composer. It might be worth explaining that a little bit.

Eric Johnson: So App Composer is... It's just that. It's an Application Composer. And they're doing some really cool stuff. It's going really fast. So they came out... It's very limited. And we knew that. Hey, we're gonna put it out, see what people think. And what it was is, inside your AWS console, you could open App Composer, and you have this blank canvas. And you would have a selection, mostly serverless selection of services that you could drag and drop. And here's what's really cool is, when you would drag on... Let's say I drag on an API gateway. And then I drag on a lambda function. I want to invoke that lambda function from the API gateway. One of the really cool things is that the App Composer will deal... The thing that Ben was just talking about. It will deal with the creating the IAM roles, it will deal with the lambda permissions too.

Because an API gateway cannot invoke a lambda function out of the box. You have to give it permission, because everything's locked down. We give privileges. So if you drag it on there, the minute you connect them with a little connector, I'm doing that if you see it, if you connect with a little connector, App Composer creates the... It's very verbose, so you can read it. It creates these connectors for you, that gives you these privileges, hey, they can invoke lambda functions. So they handle that. It also sets up your logging. I mean, it does a lot behind the scenes.

Ben Smith: It's like a best practices way of connecting one service to another. And all it's really doing is building up the infrastructure as code templates that you can copy off and, you know, place into your Git repositories, right? And then deploy the application. It's building up that template for you.

Eric Johnson: Right. But now, you don't even have to copy it. It connects locally to your hard drive from the console. So it builds SAM. So if you're not familiar... If you use CDK, we don't support that at the moment. But, you know, keep your ears open. But it builds a SAM template. And now, this year, and tell me if I'm missing something, but this year, we released a IDE plugin for VS code. So in VS code, if you have the AWS toolbox, and you have a SAM template, up in the top right corner, you'll see a little...it's a little iconic, I can't even describe it. There's a little icon at the top right, that allows you to open that in App Composer. And then if you have a Step Function in there, and this is what drove this whole point, if you have a Step Function in there, you can say Edit Step Function, and it brings the workflow studio to your desktop. I love it. It's really cool.

Ben Smith: I've not used this much yet. Only way, like, early times, just before it was launched. You know, we get some early access to things sometimes. What that allows you to do is to have workflow studio in your IDE, with all the same actions that you can search for and browse and drag in, build out your workflow, right?

Eric Johnson: That's right. That's right. So the interesting thing here is, they're not recreating the workflow editor in the local, they're actually pulling it down. So if it changes up here in the console, it gets updated locally. So yes, it's an exact replica.

Ben Smith: You know, you were saying how App Composer kind of rearranges your file system for you in real time as you're dragging things in. This scared the heck out of me the first time I used it. I opened up a real project that was already in production and thought, oh, I wonder what happens if I drag this over here? My entire template just transforms.

Eric Johnson: There's a couple of things that go on... To be honest, that we're working on. One, the other thing it does, like you said, you drop a lambda function on what creates the function for you, but it never deletes a function. So let's say, you know what, I don't want that to be node, I want it to be Python. It's going to create a new lambda function, change it to Python. Because we don't want to destroy the work you've already done.

Ben Smith: Let's say I create a new node lambda function, it creates an app.js file and the handler is ready for me to type into and all of that stuff. That's pretty amazing. I need to check back into, try that out.

Eric Johnson: Even over the last...since re:Invent this year, they've come a long way with it. And I have constant conversations... Because I use SAM all the time. I'm not smart enough to use some of the other ones. But I use SAM all the time.They take feedback constantly. So if you have feedback, or something's going on, let me know. Because they automatically start working on that stuff. So it's a good team, and they're working fast to take care of it. 

Recommended talk: Journey to EDA: Patterns, Best Practices & Practical Tips • David Boyne • GOTO 2023

Evolution and Features of Step Functions

Eric Johnson: Let's get back to step functions for a minute. I'm gonna ask you a question here, do you think every single workload should start with step functions?

Ben Smith: I think you need to consider why you wouldn't start with step functions, right?

Eric Johnson: Oh, okay.

Ben Smith: I always, when I approach a new project or a new build, I think what's a good reason not to use step functions? And it's quite difficult to come up with that sometimes. Sometimes you're building something so simple or something that really does require a lot of custom code. Maybe it doesn't lean into this service-driven approach as heavily. That's a good reason perhaps not to use step functions, but I think anything that is consisting of multiple AWS services that need to pass data across each other, I would almost always go with step functions first.

Eric Johnson: Okay.

Ben Smith: How about you?

Eric Johnson: Yeah. So, I used to have this conversation. There's this guy who used to work and his name Rob Sutter? Do you remember Rob?

Ben Smith: I know that guy..

Eric Johnson: Rob's a good guy. And he used to always say, "You should..." I'm making myself, there we go. "You should always, always use step functions regardless." And I pushed back on it because when he was saying this, we'll kind of dig into some of the features of step functions, but when he was saying the step functions, there were only standard step functions or standard workflows, and you could only invoke them asynchronously.

Ben Smith: Sure.

Eric Johnson: So, at that point, it didn't make sense to start with step functions all the time.

Ben Smith: Yep. That's a good pushback. But, as you know, those two things, there's different options now, right? So, you are talking about invoking asynchronously, which is the standard workflow type, which makes sense why you would invoke them only asynchronously, because they can run for up to one year. So, you don't want service blocking hanging around for that response.

Eric Johnson: No. We'll...yeah.

Ben Smith: Exactly. But we've since introduced something called express workflows, right? Which transitions through the stage far more quickly. They only run for up to five minutes. They have a totally different billing model, so you're billed based on how long your workflow takes to complete at a different memory allocation. Whereas.standard workflows are about how long or how many state transitions your workflow has. So, that's good. So, if you have a workload that does take years but doesn't have loads and loads of state transitions, then that's a really effective cost model. But back to express workflows, I think the majority of step functions that I build are very quick, very short duration workflows, right? That just transitions through a bunch of AWS services really quickly. And so having a cost model that's based on how long your workflow takes to complete makes a lot of sense for that. And these are the workflow types that you can invoke synchronously. So, for example, you can trigger that from an API gateway call to an endpoint that can run your workflow, hang around for 29 seconds, which is the API gateway timeout limit, and return that response back to the thing that called API gateway, right? It's a really nice little pattern.

Recommended talk: Bringing EDA to the Containers World • Jessica Deen • GOTO 2023

Patterns and Workflows

Eric Johnson: In fact, you have a pattern. If you're not familiar with servers, I know you are but for those watching, if you're not familiar with Serverless Land, we have all these patterns and workflows out there. You have a pattern out there. I think I do too, but you got it first, and yours is better. You have a pattern on using an express workflow for a CRUD API, right? Yeah.

Ben Smith: I like this idea, right? So, it's a bit against some principles that we speak about sometimes because what you're doing with this concept is you're routing any valid API call, so based on the path and the domain and the authentication, anything that's valid, you route on to your workflow. And then the first state in your workflow is a choice state that says, "Based on the inbound requests, based on the parameters or the body or the method, which one of these branches should my workload take?" And so what you're doing really is you're moving that routing mechanism into step functions...

Eric Johnson: Well done.

Ben Smith: ...and taking it away from API gateway. I know you don't like me saying the word routing.

Eric Johnson: I don't mind it, but I make fun of you every time, and I was ready, but you've...okay. What is routing? What is beta? It's the same as beta. For you Americans or for those here, it's routing and beta and all those things, but yeah. I agree with what you're saying, but, again, this pattern of... You know, I do API gateway a little bit, right? And I love API. This pattern of API gateway being the front door, and it's gonna...your validations there, all kinds of stuff is happening there, and then you pass to this step function, and there's some advantages here. It's kind of interesting, is, one, there's no cold start with this express state machine. It's just there. It's a running memory. It's fast. Two, you have this drag-and-drop designer that we were talking about earlier. And the cool thing is, you do need a little VTL in API gateway to connect API gateway to step functions in just a tiny bit. But both SAM and App Composer, because the App Composers work on top of SAM, will do that for you.

So, you can actually drag API gateway on, drag your step function in, set it to express, tell it and connect it, and off you go. So, if you haven't checked that out, it's worth checking out. But, yes, this idea of now that you can do synchronous, because if you ever hear us talk, we're always saying asynchronous whenever possible. I still believe that. We are big fans of running asynchronously. That gives you a lot of grace when you're building an application. If something goes wrong, you can recover and then get the customer back the answer. It saves a lot of things. But there are times when you just have to have a synchronous invocation. And so, when they open that up, to me that was kind of like, "Okay, that seals the deal." I go to step functions first all the time as well. That's the first thing I do, I open my app composer, I drag my step function in, and then how am I gonna invoke it? Well, an EventBridge rule or an API gateway or something like that. And so, Rob Sutter, you were ahead of your time. You were wrong then, you're right now, sir. I hope he sees this. I'll have to make sure he sees this.That's what I love about, again, going directly to step functions.

Ben Smith: You mentioned the synchronous, asynchronous point there when you invoke from API gateway, and, like, often you'll have something that will be triggered from a webhook, for example, where you wanna do some stuff in your AWS account. And you might wanna use step functions for that, but you wanna get the acknowledgement that your workflow is running straight away, right? So, maybe it's triggering something via Slack or some other tool that you use day-to-day. You can still do that, right? You can still go API gateway to, I don't know, SQS or API gateway direct to step function and return that acknowledgement back to the caller. So, I think it's worth pointing out you still have that option as an express workflow as well, it's just you get to pick now if you wanna run that synchronously or asynchronously.

Favorite Patterns in Step Functions

Eric Johnson: It sends the acknowledgement, all of our asynchronous services send it, and you can do a step function, or, yeah, you could do. That's a really good call out. All right. So, Ben Smith, let me ask you this. What's the coolest pattern you like in step functions? I'm gonna tell you one in a minute that you did that I love, but go ahead.

Ben Smith: There's a couple...

Eric Johnson: And you're probably gonna take it now.

Ben Smith: I think the most powerful one is this concept of...this is what I think the cloud is all about and what Serverless is all about, is breaking down large workloads into smaller and smaller pieces that you can run concurrently, right?

Eric Johnson: You can wrap your head around too..

Ben Smith: So, AWS has the economies of scale and the resources to be able to process things in parallel in a way that most businesses can't on their own, right? And with step functions, things like the dynamic map state, which allows you to concurrently branch out to 10,000 other workflows at the same time, right? That's out of the box. That's ready to go. So, if you can break down a large task on a video processing, file processing into multiple smaller tasks, run them concurrently and then fan back in, so you have the fan out, the parallel execution, fan back in, I think this is the fundamental point of building applications using cloud services, using AWS, and using Serverless. So, this is where I think step functions are really the key to the whole thing, right? It's orchestrating all of that in a workflow. That's my favorite pattern.

Eric Johnson: No, and I agree with you. That's super powerful, the distributed map. So, we have an internal process that I run. It runs every six hours to do features, things like that. And we track stuff, and I have to go through probably, you know, not a lot, really about 1,000 records each time. But I can do it in just seconds because I don't go one then do this, then do this. It's just... So, with that...and I'm gonna tell you my favorite. I do wanna go back to this. One of the patterns that I recommend...and I wanna see if you agree, and it's okay if you don't, it can be wrong. One of the patterns that I tell folks, going back to the express and standard, I always start with express, and I think you were kind of leaning for that. And then when I realize I need standard, I swap it over to standard. Because it's just literally just as easy as, say, express type equals standard in ISC. But one of the cool patterns that I like is I'll use an express, and then I'll invoke standards from that express as subs. Oh, I'll invoke express ones as well, but this idea of kind of having...you're really getting crazy in this, you have this master controller or main controller, and then you invoke these little ones. And I think you do that on the plugins manager that we're doing with the video.

Ben Smith: That's interesting. It's usually the other way around that we see people doing it, is having, like, a master standard workflow that can run for one year, and then that calls out to lots of synchronous express workflows. Maybe it wants to wait for them to return the response, or maybe not. Maybe they're asynchronous. But I think this is another really, really interesting pattern, right? Where you are embedding workflows within workflows, so you're using the durability, the state, length of processing for up to one year of standard workflows, and then you're using the speed and the different cost model of the duration-based cost model of express workflows. And you can really start to grind out a really cost-effective model when you start using these two things together.

Eric Johnson: That's what I recommend, is start with express, and then when you need standard...and sometimes that standard, and especially if you're doing an express, you know, asynchronously, that standard...like, so one of the features that are in standard but not express is the wait for token, right? So, maybe I kick that off asynchronously and move on, and it completes that up. So, I only have one state, one or two transitions in that standard, so I'm saving some money there, and my express is moving on.

Ben Smith: That's my other favorite pattern, is this wait for task token. So, what that allows you to do is to get to a step, and that step will do whatever it needs to do. So, let's say that it's putting something onto an event bus. Now, if you also choose to use this wait for task token pattern, then along with whatever payload it puts onto your event bus, it will also generate a unique task token, and what that means is that your workflow will now pause at that state until it's told to resume. And the way you resume it is whenever that payload is triggered, whatever it needs to trigger and that thing is finished, you grab that task token that hopefully you've stored somewhere, and you use an SDK to go back to the workflow and say, "Okay, you can move on to the next step," or throw an error and move on to a different step, right? And so this way we're able to use human approval steps, even multiple human approval steps, right? Maybe you have a process with lots of gates, with lots of different people who need to approve. Maybe one of them is a master approver that supersedes the others, and you can build all of this in using this simple...I think it's fairly simple to wait for a task token pattern.

Eric Johnson: Once you wrap your head around it, and when you first look at it, it's like, "Oh," but then when it clicks, it's like, "That's the coolest thing ever." I used it...we built something for reinvent for the video processing we're talking about, and I built a plugin, and my plugin was using AI through Adobe. It actually is calling Adobe endpoint and doing some stuff, and so I had to wait for it. And this is probably the first time I'd ever actually used a task token in, like, a true...I use obviously conceptually, and so it was just really cool. So I saved the token, and I saved the Adobe job in DynamoDB, so when the job came back, then I could map to the thing, and then I just kicked it off again. It was pretty slick. I love it.

Ben Smith: So, that thing you've built that was using Adobe to generate thumbnails using all sorts of interesting filters as a JSON file, right? So, you were posting out to Adobe from your workflow, waiting for that response. I think you gave them a signed URL on S3 for them, right?

Eric Johnson: I did. 

Ben Smith: So, that thumbnail gets uploaded to S3. The task token is then used to resume the workflow. I think it's a really neat pattern and building...

Eric Johnson: It is. And it is super powerful for those pauses or for the waiting, you know, to...especially, like, the manual interruption like you mentioned earlier. So, I'll tell you the one I like, and this is much simpler.

Ben Smith: Oh, it's none of them.

Eric Johnson: One of the things was... What's that?

Ben Smith: It's none of them. We haven't said it yet, your fav.

Eric Johnson: I know. That's what I'm gonna tell you. This is it. My favorite one...and I saw Ben do this once. I remember looking at it going, "What's he doing? I'll have to teach you how this works." And then I realized what he was doing. I was like, "Oh, my gosh, this is brilliant." Basically, the idea is...so step functions, when you do a state, you can add...you can make the state...you know, you can add it to the state, kind of gets bigger. You say, "Okay, I'm gonna get some results from a Lambda function. I'm gonna get some results from DynamoDB," and then kind of putting all that together...

Ben Smith: You add to the payload as it moves down.

Eric Johnson: Right, you add to the payload. But sometimes you get something that comes in right at the beginning that you need right at the end. Now, you can pass it down, and it's not that hard, but it's a little cognitive load of, like, where, you know? Ben Smith does a parallel. So, when something comes in, he does a parallel state, he runs all...put this thing, see in the thing, he runs everything he needs to do on one side, and the parameter is saved all the way to the end, and then I just have an array with all my results from this one and the parameter, and I put that in. And, yes, that's the simplest. It may be a lazy move, but I think it's a brilliant move, and I use it. I always...first thing I do is I drop a parallel, and every time I build a step function now.

Ben Smith: I like that too, because what you've done there is you've got, like, a global variable almost, right?

Eric Johnson: That's right. That's right.

Ben Smith: And it's ready for everything until you exit that parallel state. So, you can use it at any point.

Eric Johnson: Because more often than not, you get data on the initial call that you need to send back somewhere, right? You have a job ID much like with Adobe or something like that. And I've done it at little spots where I've had to come together, and then I break it back out in the parallel again to continue to save. So, it's really cool.

Ben Smith: What we really need is to get rid of that pattern, and we need some sort of global variable parameters concept, right? This is something that customers are asking for. AWS said they'll likely shoot it to us.

Eric Johnson: We know the folks.

Ben Smith: So let's hope that comes soon.

Recommended talk: Building Distributed Applications with Event Driven Architecture • Eric Johnson • GOTO 2023

Step Functions and AI

Eric Johnson: Let's push for that. Anything you wanna throw out last minute before we head out in the shameless plugs?

Ben Smith: Well, the only other thing I would say is worth mentioning is the potential for step functions with AI, things like training models, invoke models, and then chaining prompts and responses together. I think there's so much potential that you can use your workflows for kind of similar to what we were just talking about, right? Saving some kind of global state of your conversation, either in DynamoDB or in the workflow itself in the parallel state, and chaining together these responses, I think, is a really interesting use case that I'm looking forward to seeing more and more how customers are doing that with step functions.

Eric Johnson: So, I've done some work around this, and I've talked about this, and I was shocked because it could do LangChain. But replacing that with step functions was fairly easy and just quick do your drag, grab this context. And so, basically, what it is, you know, the question comes in, let's just say a chat, but the question comes in, save the question to DynamoDB. Actually, I would go ahead and hit the LLM, build the prompt, so, like, hit the LLM, and then I would save the question and the answer to DynamoDB. So the next time it came through, you could say, "Look, I asked this, and here was your answer. Now, ask this."

You can also use it to grab...like, do the RAG stuff. So you can call it, like, Kendra can do indexes for you and embeddings and things like that. And so I had Kendra go and index all of Serverless Land, and now I can ask a question and just have Serverless Land be the scope of info. So, all through step functions. The only time I rolled out, and this is interesting, the only time I rolled out to Lambda function was to create the actual prompt. I could have done step functions, but it would've just been a chain of passes to kind of get that to be all the right things, so it was just easier. And I think this is an important call out. It's common sense. People ask if I am anti-Lambda. No, I'm not anti-Lambda. I just think you use it for the right job, and that's one of 'em. So, it's common sense to orchestrate business logic. There you go. So, really good call up, and yeah, that gen AI, it's huge. So, anything else?

Ben Smith: I would say if anyone's interested in seeing more things you can do with step functions, there's this awesome website called serverlessland.com. You can go there to serverlessland.com/workflows, and we've got about 160 different patterns all built with step functions uploaded by customers and AWS experts. That's a great place to kind of jump off from and try things out, right?

Eric Johnson: That's been Ben Smith's baby. Ben kind of created that and has done a lot of those, and you've done a great job around that. Super impressive. If you're looking, and I don't think you have your Twitter or LinkedIn up there, but look for Ben Smith on Twitter if you've got questions. He's... Or me, I'm glad to help, but Ben Smith is literally a genius in step functions, so he's kind of our resident expert. Also, if you are in the London area, and you're seeing this before May 14th of 2024, we are having EDA day, which is an event driven architecture day. And as we've said quite a few times, step functions are at the heart of EDA in orchestrating, you know, and so come check it out. We'll be talking quite a bit about that. In fact, and I haven't even announced it yet, so I'll just say it here, we've got someone who will be doing what we just talked about, talking about doing prompt chaining and that kind of stuff using step functions. So, it's really cool. I just read the abstract, and today I was like, "Yes." So, definitely one of our essays, Uma, will be talking about that. So, with that, Ben, thanks so much for joining me for Unscripted here. I have my GOTO crew, by the way, I don't know if you... I'm just saying. I have like nine of these shirts, but whatever.

Ben Smith: Thanks, Eric.