Expert Talk: Functional Programming

Learn why you should look into functional programming in this Unscripted interview with Russ Olsen and Christian Romney. They explore the nature of the languages as well as advantages and misconceptions. Russ Olsen also shares how he started with functional programming and how it's influenced his code.

Intro

Christian Romney: Hello, everyone. My name is Christian Romney, and I am a director of engineering at Nubank in the flexible engineering alliance. And today, it's my distinct pleasure to bring to you my friend and mentor of many years, Russ Olsen. Russ Olsen, say a few words to introduce yourself to our viewers.

Russ Olsen: Hey. Well, thanks, Christian. I'm Russ Olsen. I am also a director of software engineering at Nubank, and in my spare time, I've written a few programming books, most recently "Getting Clojure," I also wrote "Eloquent Ruby" many years ago. I do a fair bit of conference speaking when the world is a little more normal, or at least I did when it was, and it is great to be here.

What is functional programming?

Christian Romney: Yeah, it is. It's going to be fun today. All right. So, Russ Olsen, we're here to talk about functional programming today. I guess that would be the perfect icebreaker. Why don't you tell us? What exactly is functional programming?

Russ Olsen: Yes! Functional programming, it's this... it's funny, if you look on the internet and you Google or something functional programming, you get all of these sort of mysterious, almost Zen-like statements about how it changes everything. I think it has this perception of being this, like, mysterious, almost philosophical approach to programming. Fundamentally, functional programming is a different way of looking at programming that makes building programs a bit simpler. It doesn't solve all of your problems — it makes some of your problems easier to deal with, it makes your programs a little easier to understand. And if I sound like I'm trying to undersell it, I'm really not. What I'm trying to do is, sort of, take away the mystery and the darkness of it.

Recommended talk: Functional Programming in 40 Minutes • Russ Olsen • GOTO 2018

Functional programming is fundamentally this idea: if you are an object-oriented programmer, you tend to take for granted things like classes and inheritance and methods and fields and all that kind of stuff. Functional programming is the idea that you can essentially replace all of that with a few simple tools, a few simple programming tools. So, it doesn't change everything about the way you code. It changes a lot of things but, fundamentally, it is the idea that one way to look at it is that it is the idea that you can take a lot of the complicated machinery that you might be using if you're an object-oriented programmer and replace it with some simpler things, which I think every programmer knows. You make it simpler and the code is easier to understand and all of that kind of stuff.

What are the advantages of functional programming?

Christian Romney: I think it's a good segway into the next question I have for you. I think you're beginning to highlight some of the advantages of functional programming but maybe explore that a little bit. What do you think the advantages of programming and the functional styles are for you?

Russ Olsen: Well, so, back up and talk about what the tools of functional programming are, okay?

Christian Romney: Sure.

Russ Olsen: You know, the first tool of functional programming are these things we call pure functions which always, it sounds like it's sort of a good/bad, kind of evil, dark side, light side kind of thing, my functions are pure. A pure function is a function that just computes something from its arguments and returns a result. That's it, right? So, every programmer, since day one, has been dealing with pure functions. Things like all of the math functions, the absolute value is a pure function, you can think of addition as a pure function, right? Takes two arguments, the two numbers you're adding together, and produce a result. And the key thing with pure functions is that they don't perpetuate any side effects. A pure function doesn't read from a file, it doesn't write to a file, it doesn't update a database. It just takes its arguments and computes a result and returns that result. And, critically, for any given set of arguments, it always returns the same result.

It's like one of those reliable, easy-to-understand functions that every programmer knows about. Oh, you know, nobody sort of worries, writes extensive unit tests for the cosine function, right? Someone wrote that thing, tested it once, and was done. And, I think, if you could imagine a world where most of your program was written in pure functions, I think that any programmer who's ever wrestled with what state is my database in, do I have the right information in my files, is that global variable or somehow that global state in my program the right thing, would be able to see that, you know!  If I had a program full of pure functions, it would be easier to test and easier to understand, right?

I send it some arguments, it gives me a result. If I build a test around that, I know that for any set of arguments, it's always gonna give me the same result. So, programs full of pure functions are just, I think they're sort of mathematical proofs that they tend to be simpler. But just, intuitively, if I don't have to worry about the global state and I don't have to worry about the state of the database and the state of anything outside of the code inside this function, that'll make the program easier to understand. And the whole rest of functional programming is, how do I build interesting programs that do useful things with more or less only, not quite only, but mostly pure functions? And that, fundamentally, is it.

I don't think most programmers need a lot of convincing that if they could write their programs with mostly pure functions, the programs would be easier to understand. The question is, how the heck do I do that? And that's what the rest of functional programming is about.

Recommended talk: Functional Programming for Pragmatists • Richard Feldman • GOTO 2021

Christian Romney: So, let's go there. You talked about pure logic, pure functions and said, "Well, if I don't have to worry about a file or a database."  This is a common thing that most interesting programs have to do. So, what's the next step in the process? How do I begin to incorporate some of those things?

Russ Olsen: It's like another one of those sorts of misconceptions about functional programming that floats around is that functional programming is all about getting rid of side effects and that's true, right? Like, a pure function, I've said it, it doesn't but it's also not true because, as you say, all of the useful things a real program does, they're all side effects right? In the end, I need to write the file, update the screen, hit the web service, put a row in the database, whatever it is you're trying to do. Chances are they're 100% that it's a side effect because nobody can see what's going on inside the program. The idea of functional programming is,  we're in the business of producing side effects but we should not use side effects as the tools to do our work. It's sort of the end result.

And the reason you do that is that side effects sort of are like something sticky on the road or the table or something, it makes things more difficult, it makes everything messy. So, yes, perpetrate the side effects that are necessary, but leave, don't use them as a tool. So, a typical functional program has some side effect at the beginning, maybe it reads something from a file or inputs a request and that's a side effect, sure. In the end, it produces some result, maybe it updates a database or updates a screen or something. Then, in between, we try not to have, well we try to have as few side effects as we possibly can in the middle of the program because we're trying to avoid, we're trying to get the side effects down to the absolute minimum, which, ideally, would be the thing that came in and the thing that's going out.

Misconceptions about functional programming

Christian Romney: Interesting. So, you mentioned a moment ago also that there's this misconception. What other misconceptions do you feel are out there about functional programming that you'd like to dispel?

Russ Olsen: The main one is that it's this Zen, you going to go to the functional programming monastery and study for half a lifetime before you can really get it. Yes, exactly. I think there are two sources of misconceptions about functional programming and one is completely understandable. It is that people when you get the idea of functional programming, it's really exciting. I can remember the moment, I was like sitting in front of my screen with my keyboard under my fingers and it hit me, I don't need all of this stuff. I don't need classes and methods and friend functions and private and protected methods and all the rest of it, I just need functions and Clojure's and first-class functions, just a small set of tools and I can read all the programs that I need.

That is really exciting. I think of it as one of the high points of my programming career, except that there's been a lot of high points in my programming career, right? Programming is all about looking at the world in a different way. You have some application and you're struggling,  how can I make this thing make sense? How can I put this together in some reasonable way? And then, at some point, it hits you, oh, I need a message bus or something like that and suddenly,  all the pieces fall into place. And you're like, "Yes, this is a high point of building this application." Functional programming is like that, right? It's like, suddenly, you see the world in a different way and that is the way I should do this!

And it is exciting, it is different, it does make you look at the world differently but that's what we as programmers do, right? Like I'm looking at the world differently, this must be Tuesday, right? So, there's that misconception. I think there are a lot of terminologies. A lot of the ideas of functional programming were borrowed from mathematics. The idea of the pure function is, what do they call New York pizza in New York? They call it pizza and in math, a pure function is just called a function right? We bring it over and it's a different kind of thing but there are a lot of terminologies that come from mathematics. So, functionals and monads and things like that are kind of foreign to most programmers. And I think this is a case where the terminology, when you're a beginner or when you're just trying to absorb functional programming, the terminology does not really help because, fundamentally, the ideas are pretty simple.

What do you need to build functional programming? What are these simple tools you need? Pure functions, that's not terribly exotic, you need to be able to treat your functions as values. You need to be able to pass functions in and out of other functions as arguments or as the result, you need to be able to cook up new functions on the fly. So, if you've created inline functions in your code and with Clojure's where the function you're cooking up grabs hold of the data, kind of, around it in this version of the function, x is 4. In this other version of the function, x is 7, that kind of thing. And you really need immutable data structures, things that look like arrays or lists or maps or what have you that are like their nonfunctional cousins but which are immutable.

And that really just puts the teeth in the idea that your functions are pure functions because if you can't change your data structures in place, I kind of know my functions are pure functions because they can't perpetrate at least data structure side effects. You need that. You need some way to make all of that efficient, which is kind of a solved problem, and off you go and that is the blinding insight of functional programming. Like, it's Tuesday morning and I'm a programmer, so I'm going to have an insight. The Tuesday morning insight of functional programming is all you need, right? Functional programs are full of endless elaborations and there are libraries and there are things that help you but, at the core, you only need functions, pure functions, you need to be able to treat them as values and you need some efficient, immutable data structures and you're done-ish.

Do you need to understand category theory for functional programming?

Christian Romney: Just to be just to be sure we stir up some controversy, can I get you on the record saying that you don't need to understand category theory to understand functional programming?

Russ Olsen: You don't need to understand category theory. I will say this, so you and I are both Clojure programmers and Clojure is a dynamically-typed language. One of the advantages of dynamic typing in functional programming is there's no, you do not have a complex type system, by definition, interacting with your functional programs in a dynamically-typed language. So, because you don't have that, there's a whole category of conversations you don't have to have. A lot of the really complex terminology in functional programming comes from the interaction of strong typing with functional programming. I'm not saying that strong typing is bad, what I am saying is that if you have a strongly-typed language and it's got really strong opinions about which types can mix with which other types and you marry that to functional programming, that is necessarily something you're gonna have to have a conversation about, you're gonna need words with.

So, my friends who are Haskell programmers, do talk about functionals and things like that and monads. My friends, like you, who are Clojure programmers tend not to just because there's all,  there's no strong typing there to have a conversation about. Doesn't mean the Haskell people are wrong, it just means they have something else to talk about and they need words to talk about it with.

How did Russ Olsen start with functional programming?

Christian Romney: Surely. So all this talk of programming languages reminds me of something. You're a self-described, programming language geek. So, tell me a little bit about what first got you interested in programming generally and maybe afterward, functional programming in particular.

Russ Olsen: I started out my career as a mechanical engineer back in the late middle ages, just before the Renaissance, I guess. And it was right around the time that people were starting to use computers for mechanical engineering applications for design. So, CAD/CAM was a new thing when I was recently graduated from college and I rapidly discovered that I was a much better programmer than I was a mechanical engineer. I was way better as a programmer than a mechanical engineer. It's remarkable that all my fingers are still attached, you know!  And I was lucky enough that I got a job with a long-defunct computer company, that manufactured mainframes company called "Control Data." My particular job, I got to frequently be the only user of computers that, at the time, cost $2 million or $3 million.

Just the nature of my work, I was porting applications and things like that and it was just so much fun. As sort of a mechanical engineer, I kind of took programming languages as something that existed in the world, like a rock, right? You find a rock on the ground, it's just there. And, at some point, I started wondering how do these things work? And I started going down the rabbit hole of the compiler producing assembly code, which is actually a whole different language, which produces machine code, which is similar but still a different language. And the computers that I was using were microcode, so I started digging into the microcode and I don't know, programming languages just seemed like if computers were magic, programming languages were the magic that made the magic go.

And I've just been endlessly fascinated by programming languages and then the relationship between programming languages and human languages. I think there's some parallels there and there's real parallels, I think between programming language communities and regular communities in  the wider human world. It's the interaction of people and technology that is just wonderful.

Christian Romney: And what made you drawn to functional programming? Was it just when a few years back now that it was all the rage and you're like, all right, let's go check this out? Or does your interest predate sort of?

Russ Olsen: Yes, so when I started programming, I made myself very unpopular in my first programming project because I indented my code, okay? Sort of put yourself back in that world, if you can imagine, and as I went through my career, every step of the way, there always seemed to be some better idea. Like, "Hey, let's indent our code," was the first one that made programming easier, made the programs easier to understand. And so I went from arguing with people about whether we should indent our code to arguing with people about whether this object-oriented thing was ever going anywhere. And I will say, there were lots of blind alleys, I wasn't always right, you know! I was always like the person looking at the next thing for better or for worse.

Recommended talk: Functional Programming Through the Lens of a Philosopher & Linguist • Anjana Vakil • GOTO 2021

And, at some point, functional programming just seemed to be there, were a lot of smart people that I knew who were saying that this was something we should be looking at. I will say functional programming was one of those things that took me a long time to get and I felt like part of it was because I was reading all of these sort of,  it's this Zen-like experience that I've been going on about. And when I finally figured it out, it was both like this is wonderful and this is way simpler than the internet would have you believe. This is just very straightforward.

Christian Romney: I remember one of the difficulties I had making the transition from object-oriented programming to functional programming was like,  but it was just a function I didn’t  understand. Where was the design patterns and all the other things that I "need" in order to make a good program? It was very liberating to realize that you don't need all of this stuff.

Russ Olsen: I think that's the big leap if you're an object-oriented programmer or programmer who...and we really shouldn't even say if you're  describing people as object-oriented programmers, right? We're all software professionals here, we shouldn't be defined by our tools but if you are used to using object-oriented programming, there's a level of complexity that you expect from your tools. You just described the experience that I was trying to describe earlier. It's like this moment where you say, "I don't need all of this. I just don't." You know? And it's a liberating kind of thing. There is,  there is work in learning the program in a functional style. You constantly, in the beginning, find yourself thinking, "Oh, I'll just jam 44 in the middle of this array." And then you think, "Oh no, I've gotta figure out a way to return something."

But, again, those are the kind of problems that programmers deal with all of the time. Maybe not that exact problem but it's that, "Oh, I need to look at this a different way," is kind of the definition of what it is to be a programmer.

Russ Olsen’s war stories

Christian Romney: Yes, so, let's contextualize this a little bit? We're talking a little bit abstractly but let's talk about something concrete. So, relate, for me, maybe an experience, think back to your most disastrous programming project and tell me, if you had only had functional programming available to you, you know, what would've been different?

Russ Olsen: There are so many to choose from. You know, in the long view... I'll answer your question in a second but you just made me realize, I think in the longer view of going from those days when people argued about indenting code to today, I think one of the things that gets lost is that we are a lot better at this than we used to be right? Programming projects still fail at an alarming rate but nowhere near as badly as they did back in the day, even ten years, 15 years ago. I think we are getting better at it. My most disastrous project, I worked on this document management system that was kind of an OEM product and it was….we took code from another company that was aimed at a different market and we would adjust it for the U.S. market and for the industries that we were selling to.

And one release, our partners had completely rewritten this big C program into a C++ program. And they had a great idea, they had great ambitions that that was going to make the system easier to deal with and, all the promises of object-oriented programming. And we got the thing and we probably spent eight months and probably $3 million, back when $3 million was real money, trying to change this thing just ever so slightly to make it work for our customers at our market. I just remember the experience was that we were just in the middle of this maze of objects, trying to make sense of the slight differences between the classes. I think of it as object dust!  Like a badly designed object-oriented program, all of the objects do something but you're never quite sure what they do or if you have the right object.

I mean, I think even at the time, I remember maybe I'm projecting backwards, but I remember thinking, "Gee, I'm just trying to move this document from this place to this place and put some attributes on it and store it all in a database. Why is this so hard?" Well, could you write that system in a functional style? Sure. Could the Russ Olsen of the 21st century, 2nd decade, 3rd decade of the 21st century do a better job? Sure. I'm a lot more experienced. But I think it is harder with functional programming to just get lost in the tools, get lost in the infrastructure of your program. Functional programs, fundamentally,  the idea of the code is you take some data, you transform it, you've got some different data now.

And I think it's harder to get lost, not impossible, I think it is still possible to write bad code in whatever style and language, if you try hard enough, you absolutely can. But yeah, so I look back on that and I think, if we had been using functional programming, that might have been better, but thanks for reminding me of that! 

Christian Romney: That's what we're here for. For me, looking back, there was this one project that you and I had worked on briefly, many years ago at a particular cruise line and they had a reservation system and I remember this was non-functional programming but they had some objects that they passed in as parameters. Then you farm out work to other subroutines in your code but somebody got the idea to start changing the values that were in the object which was supposed to be just a parameter which had things like your customer's preferences. It just changed out from under you, like five levels down the call stack and made bugs that we just don't have a certain class of bug in functional programs. We have others to be sure, right? But certainly not that kind.

Russ Olsen: I think people talk about this a lot and it's true. It's certainly not the only advantage of functional programming but if you're working with immutable data structures, there's a whole class of multithreaded problems that just go away. Basically, you can never get in a situation where I have a data structure, which is, kind of, in mid flight, I'm in the middle of making a multi-part change and I've done two-thirds of it and then somebody looks at this data structure and it's complete garbage. You tend to just automatically not have those problems. In Clojure, and I think in most other functional programming languages, you tend not to have synchronized blocks and things like that, maybe in very special cases. You just do your thing and, sort of, by default, it's thread-safe and when was the last time you talked about thread safety in your program? We don't talk about it. It just, you know, happens.

Christian Romney: Yes. Much of the time, we get it for free. That's for sure.

Russ Olsen: Yes.

The influence of functional programming on Russ’ code

Christian Romney: So, how would you say functional programming influences...how else would you say it influences the shape or the character of the code that you write?

Russ Olsen: So, on a larger scale, as I say, you tend to have the side effects at the beginning and at the end for whatever the definition of beginning and end is.  Like when you take an input and when you send out an output and not in the middle. I think that's one thing. I think the other thing that it does is it tends to make code changes local, right? If all your data structures are immutable and you're dealing with pure functions, then if I make a change to this function, it can affect  the behavior of the function changes, it can certainly affect the callers, the users of that function.  Maybe I'm returning something different but it's very narrow, well-defined... I'll put it to you this way, think about what a good interface is in object-oriented terms.  Like, I'm gonna build an object and it's gonna have a great interface. That means that the object is completely opaque inside. I'm not really concerned about what's going on inside the object.  That's a good interface.

And then it has really well-defined inputs and outputs. That is the description of a pure function. Unless I look at the code, I have no idea what the details are going on inside this function. The only thing I can see is the interface, which is what arguments this will take and what does it send out? And because of that, the experience of working on a functional program is, I'm looking at this function and I'm confident that if it really is a pure function and there's nothing untoward going on, nothing unusual going on, then the code that I'm looking at inside that function is pretty well isolated from everything else. And that really is sort of the key to why functional programming makes life simpler for the programmer because,  most of the time, I'm looking at some code and the code does exactly what it says it's gonna do and there's very little hidden behind the scenes.

You know, if it's a pure function, I can't call something and my argument isn't gonna change or, you know, nothing's going on behind the scenes. It's all, kind of, right there in front of me. You know, you just reminded me of a war story from my mainframe days, which is you were talking about passing in an argument and then somewhere, maybe deep in the call stack, changing something in that argument and then you're surprised. In my mainframe days, it was possible the constants were stored in writeable memory in those, like, more primitive and so it was possible to, you know, pass five into a function and have somebody change the value of five so that anytime you had the numeral five someplace else, you now had it different. And we would look at that today, right? Any programmer today would say, "That's insane. Why is five different from a three-element array or a hash table or any other object you're dealing with?" Right? Why is one surprise and the other one, not? In functional programming, they're both surprises, which I think is maybe a more sensible way of looking at it.

Christian Romney: The value of values, certainly.

Advice on how to get started with functional programming

Christian Romney: All right. So, I guess maybe it's time for a parting question, Russ. What advice might you have for programmers out there that are just starting to get into functional programming and want to explore it?

Russ Olsen: So, I think my advice is the same advice I give people who are trying to  learn a new programming language, which is... and, in a way, functional programming is kind of,  well, you probably are going to be learning a new programming language but it's a similar, I'm changing worldview and it is, find some problem that is interesting to you but that you have solved before, right? We all have these programs we've written half a dozen times in our career, maybe more and solve that, write functional code to solve that problem, okay? And it wants to be something interesting, bigger than hello world, less than  a full-blown database or something like that. Find the problem that's sort of medium-sized, that you understand really well and write a program, a functional program to solve that. And that way, you understand the details of the problem, you know exactly where you're going and you are just trying to look at the problem in  a different way. I think that is a great way to  learn a new programming style and learn a new programming language.

Christian Romney: Yeah. That's great advice. All right. Well, this was a blast for me, Russ Olsen. Thanks.

Russ Olsen: Well, it's been a pleasure, Christian Romney.

Christian Romney: Excellent. And, I guess, to all our friends out there on the internet, pick up a functional programming language. Hey, pick up a copy of Russ Olsen's book.

Russ Olsen: There you go.

Christian Romney: He didn't actually say that but... Awesome.

Russ Olsen: All right. Well, thank you, Christian Romney.

Christian Romney: Thanks, Russ Olsen. Cheers.