Code, Immutability, and the Future of Development: Insights from Dave Thomas

Early Days of Software Development Chaos

Hannes Lowette: Hi, my name is Hannes Lowette. I work as the head of learning and development for Axxes in Belgium, and I'm here at GOTO Amsterdam with Dave Thomas. Dave, can you please introduce yourself to the audience for the people who do not know who you are?

Dave Thomas: Absolutely. So, I am not the GOTO Dave  Thomas who was the small talk guy and everything else. I'm Pragmatic Dave Thomas. I've been programming for more than 50 years now, and I'm the author of "The Pragmatic Programmer," a few Ruby books, Elixir books, and one of the creators of the Manifesto Gradual Software Development.

Hannes Lowette: That's something you undersigned in the past and that has changed the software industry for the better, I would argue.

Dave Thomas: Well, that's nice.

Hannes Lowette: What was that like? Because I grew up in a world where that manifesto was already there, right? I started programming when that already existed. What sparked you guys to think about this and to try to formalize the ideas in the manifesto?

Dave Thomas: Survival, largely. Back then in the '90s, software development was a total mess. I mean, the vast majority of projects never came in. And when they did, they were buggy, they didn't do what people wanted. It was just a total disconnect between the process of wanting software and acquiring software. It just didn't work. And so people were exploring different ways of doing it. Back then, the idea was you came up with some kind of requirements and you got those signed off, and then you came up with some kind of architecture, got that signed off with the design. And then you would hand it over to the coders, and the theory was that they would go away, do their thing, come back with exactly what you specified, and miraculously that would then get delivered and work.

Hannes Lowette: Right. And in the meanwhile, obviously, the business could not change because the requirements were signed off months ago.

Dave Thomas: Well, you tell that to the business. No, ultimately, the business did change. It wasn't just that. The thing that surprised people. I don't even think it surprised them because they didn't notice it, but people don't know what they want. And when they came up with requirements, it was a best guess. So people would invest tens of millions, hundreds of millions producing software that was based on really just like a wet finger in the air….

Hannes Lowette: Bad assumptions.

Dave Thomas: Bad assumptions, bad understanding of where the value was and the bills and all this kind of stuff. So, the thing that a lot of people were looking at was, how can we shorten that gap between what we're doing and what's wanted? And there are many ways of doing that. I mean, XP was probably the most famous back then in terms of enforcing a short, short cycle and lots of feedback. And that is pivotal when it comes to writing software. Right?

Hannes Lowette: Especially if you approach software from the angle, like, we can still change things. When you're building a big skyscraper, that's a lot harder to do. But the fact that we are...It's in the works. It's software. The fact that we can still change it means that we need to adopt different processes to.

Recommended talk: Agile is Dead • Pragmatic Dave Thomas • GOTO 2015

Dave Thomas: Oh. It is both a blessing and a curse, right? It's what makes software exciting to write, but it also is what makes it a pain in the butt that you do have to change it all the time. If you look at it in a slightly different way, I would say that one of the problems of agility is that back then, we didn't have this kind of explosion of frameworks and techniques and tools and everything else because we couldn't. I mean, there's no way we could keep up if people were throwing a new framework at us every week. And now that people are looking at shorter life cycles and shorter iterations, then they can start playing with different frameworks to see if it works for them. I think to some extent, the success of that way of writing software is also now becoming a bane of everybody's life, trying to keep up with all of these changes that we're allowing.

Hannes Lowette: For me as a .NET developer, I started at a time when Microsoft would release a new Visual Studio every couple of years, and it was shipped on DVDs. I had time to learn all the new things before the next one came out. Those times are long gone. I think it's also very exciting to see that we get, on that side, shorter feedback loops. You see the things that are happening in the industry making it back into the language design, which I would argue is a good thing, but it makes it a lot harder for people to keep up with everything that's happening.

Dave Thomas: Not just that. I've been working on something and I had the freedom to mess around. It was a toy project. So, I was playing with React and then I tried Next and then I switched across to try Solid, just it looked kind of cool. And in the end, I bumped into the lack of a library. So, I went back to React. And in the time it had taken me to go around that loop, my application no longer worked with that new version of React. So, yeah, I mean, it's good, but it's a pain in the butt too. Unless you lock down the version of absolutely everything, then you're guaranteed to be out of date within a couple of months, which is scary.

The Next Shiny Thing vs The Coding Catalyst

Hannes Lowette: I had a discussion with a customer a few weeks ago when I was writing a proposal for them. They asked me, "Why do you want to use .NET 6? .NET 7 is out already." And I was like, "6 is going to be supported for longer and we will swap it to 8 and we'll skip the intermediary ones because it's not going to be big revolutions in between." So let's, like, keep some stability for a team and focus on delivering something that you can do something with because your customers are not gonna care whether it's .NET 6 or 7 or whatever.

Dave Thomas: Right. And I think that's a really wise move because we are obsessed with bright, shiny things in this industry.

Hannes Lowette: I call it Mac by developer.

Dave Thomas: Exactly right.

Hannes Lowette: So, it's like, "Ooh, new shiny thing. I wanna put that on my resume, so I'm going to use it in my project."

Dave Thomas: Exactly right. And invariably that makes projects take twice as long as they should.

Hannes Lowette: Yes. It takes away a lot of the focus from what we're there to do. And what I have always felt that the manifesto was about is putting value in the hands of the users. Right? And I would argue that that is our entire reason for being, software developers is to deliver something that works and that people can do their jobs with.

Dave Thomas: I think that's certainly the primary value that we have and the primary reason we have to exist. I think we also have a secondary reason, which is to make ourselves better.

Hannes Lowette: Sure.

Dave Thomas: And that is the hard one.

Hannes Lowette: But that's the one that brings back to the whole pragmatic programmer and doing coding catalyst to improve yourself, practicing inside production, so not in your day job, that sort of stuff, right?

Dave Thomas: Yes. I think that's still valuable advice. I still do it. In the old days, I used to try and keep up with everything. Now that's just a recipe for mental breakdown. But I still try to keep up. I still try to try new things because...

Hannes Lowette: Expand your toolbox.

Dave Thomas: It's not so much expand it as refresh it now and then.

Hannes Lowette: Okay.

Dave Thomas: So, if you've been using the same tool so long that it's comfortable and you kind of like worn your fingers into the handle, maybe it's time to think maybe it's a different one.

Hannes Lowette: I love the fact that you mentioned Elixir as well. I'm rooted in the .NET ecosystem. But one of the things that made me think completely differently was learning Akka.NET and the actor model, which we have in Elixir as well, made me think about the whole threading problem from a completely different angle. And I always love it when something takes me out of my comfort zone. So, learning Java will not change that much because they're very similar languages, but something like an actor model or functional programming, that makes you think about certain problems from a different angle. Then you bring them back even to C#, like, you start applying different patterns in the language that you're using because you saw something somewhere else.

Dave Thomas: Yeah. My first object-oriented programming was in Simula in, I've got to get this right, 1974.

Hannes Lowette: Right. I was minus 8 then.

Immutability and its Impact on Software

Dave Thomas: That makes me feel so good. And I have been doing OO programming. There was a spell when I was doing assembler in C and then other languages. But from the mid-'80s onwards, almost all the programming I was doing was object-oriented. I was like, "This is the way it has to be. It's just so natural." And I wanted to get into functional and to try it. And I could not find a language that I felt comfortable with. And then I came across Elixir. And it's not a pure functional language, but it has...a lot of the things that I thought were wrong about functional languages were in Elixir. The immutability, for example, lack of control statements. And I discovered that I was wrong. And having spent so many years writing this code, it took me a while to come to that realization, but the realization, actually, yeah, OO has its place. But now, when I write, for example, JavaScript, I rarely write a class. I'm writing my JavaScript as functions and closures and doing it in a very, very much more functional style.

Hannes Lowette: And that came from learning Elixir.

Dave Thomas: That initially came from learning Elixir, and then I finally convinced myself to do some Haskell and Reason and a whole bunch of other functional languages. And now, even in JavaScript, I will write most of my code using mutations on immutable data structures because it's easier to reason about.

Hannes Lowette: Yes. It takes away a lot of side effects that you would have from traditional OO programming as well.

Dave Thomas: And particularly in JavaScript, where it's such a mess. It's...

Hannes Lowette: We have a song about that.

Dave Thomas: It's a good idea, I think, to put some discipline and constraints on yourself. Yes.

Hannes Lowette: Obviously. I think that teams struggle with finding a way for the whole team to work within the same boundaries when you're producing code, especially in modern languages. If you look at, like, the big mastodons in the programming language world, you can use them in fairly different ways. I would argue if you apply that functional programming style in a language like C#, it should be a team decision because you wanna have the entire code base behave similarly, otherwise, it's gonna be quite impossible to onboard new people on products like that.

Dave Thomas: Absolutely. Nothing is worse than a team with one evangelist who says, "We have to do it this way," and they all go, "No, no."

Hannes Lowette: I've used the analogy of that particular problem is you build a tree trunk and you can date a certain feature in the codebase on the practices that were being applied to build that feature. So, we have this frontend framework and this way of structuring the code, it must've been built in 2018.

Recommended talk: Functional Programming for Pragmatists • Richard Feldman • GOTO 2021

Dave Thomas: Right. Right.

Hannes Lowette: And what teams forget is it's okay to change your practices, but you have to haul out the trunk, like the old practices need to eventually go so that you have a manageable stack that you can onboard new team members on, otherwise, that becomes something too painful.

Dave Thomas: I think the mistake people try to do is to migrate existing codebases. And I think the idea...I mean, you were talking about actors and Akka and things. Once you start looking at a more distributed, that's not even the right word, decouple architectures like that, then you can be writing your actors in C#. Some of us will be using F#. Some of them will be using Visual Basic for all I know. Right?

Hannes Lowette: And they'll be deployed in the same cluster and talking to each other through messaging.

Dave Thomas: Right.

Hannes Lowette: Which is what we see on a bigger scale with microservice architecture as well, being made...also that has become easier to do with the modern stuff.

The Future of Software Development: Immutable Code

Hannes Lowette: What future do you see there? As you already mentioned things are moving fast and they're changing quickly. In the current state of the industry, what are the things that you're excited about to see where they're gonna go?

Dave Thomas: Immutability is, I think, the next frontier. And by immutability, I mean way more than just data structures that you can't change in a functional language. There are so many problems that go away if once you've done something, you can't change it. It's a stunning...For example, if your databases are immutable, then you can think of every single change you make as creating a brand-new database. Right? And, therefore, if I have the ID of that database, some unique number that identifies that particular database, and pass that around, then that will always be the same. It's an immutable database, right? Which means I don't have to worry about transactions anymore. Right? I do have to worry about merging now and then, but that's okay. It's constrained by the database. There are so many things that you can do better if you have immutable databases, even down to the fact that you can do queries backward in time because each database is there, and if you have some directory that says, well, yesterday's database was called 123, then I can query 123 and get yesterday's data. It's amazing.

Hannes Lowette: Okay. That's one of the things that I've been doing with event sourcing, where you have an appends-only log of the things that are happening, and you can go back to a certain point in time. But what you seem to be proposing goes even a step further than that. You can get, like, yesterday's snapshot of the entire...

Dave Thomas: But it's no longer a snapshot. It's the reality. It is the database. It just happens to be a different database than the one that someone else is using in today's.

Hannes Lowette: Is that one of the things that's already brought into practice?

Dave Thomas: Oh, yes. Datomic is a database in the closure world, JVM, that is immutable. There are quite a few. If you look around, there are quite a few immutable databases available to you. But I think that that will start to spread a lot more, particularly, it's good at having local databases that you then have to resolve off into more general global databases, so for edge stuff and things like that. It's just a great technology.

But here's the one I think is gonna freak people out, immutable code. So, you write code and the second you make it shareable, either by publishing it or whatever else you do, it becomes immutable. It cannot be changed ever. And what happens is that your code becomes known, not by its name, but by some unique identifier for the code in that particular state. And all the other code that wants to use that is gonna bring it in by that identifier, which means that you can change your code 50 times a day, totally breaking any interfaces that you created, and nobody else's code will break because they're all using the immutable code. Now, you think, "Okay. That's gonna be impossible to do."

Recommended talk: Unison: A Friendly Programming Language from the Future Pt 1 • Runar Bjarnason • YOW! 2021

And yes, if you're doing it by hand, but there are languages now, the poster child for them is a language called Unison, where they have...it takes it so far that you do not have a definitive copy of source code on your local machine. You're editing on your local machine, and it's like an environment in which your code is sitting in your local machine. You can make changes and it picks them up and runs them and runs the tests and everything else. But at some time, you do effectively a push on your code, at which point it assigns a version number to it. And the cool thing is, the version number is a 512-bit hash of the AST of your code.

Hannes Lowette: Oh, nice.

Dave Thomas: Right.

Hannes Lowette: So, the checksum is built in as well.

Dave Thomas: The checksum is built from the AST. But the cool thing is, that the symbol table is not in the AST, right? It's just referenced by number effectively. So, if you were to write a function that said, function add A, B returns A plus B, and I was to write a function called sum, that takes number one, number two, and returns number one plus number two, they have the same AST.

Recommended talk: Unison: Expert Talk: DevOps & Software Architecture • Simon Brown, Dave Farley & Hannes Lowette • GOTO 2021

Hannes Lowette: The ASTs are going to be identical. Right.

Dave Thomas: Yes.

Hannes Lowette: This means that even if I write a function that you already wrote, in the end, we will end up using the same function.

Dave Thomas: Until one of us makes a change.

Hannes Lowette: And then there are two versions of it.

Dave Thomas: In which case, then there are two versions of it. Right?

Hannes Lowette: Right.

Dave Thomas: Even better, you say to yourself, "Somebody must have written a function that adds two numbers." And you can query it to say, "Is there an existing function with this signature?" And it comes back with the functions and you say, "No, they're not right," or, "Yeah, I'll use that one." And from your code's point of...

Hannes Lowette: That's never going to change or break your behavior because you're using a specific version.

Dave Thomas: Right. So, what happens is that when you write new code, and if your new code uses sum, then that little runtime environment thing on your machine says sum, okay, I'm going to go look that up. Right? It knows that the sum corresponds to this particular hash so that you see the sum, but inside the AST, it's the hash. And then when it saves it up, it's the hash.

Hannes Lowette: So, effectively, if you build a product thinking this way, a lot less of the code is going to be unique to your product because you're dragging in things that already exist for...

Dave Thomas: Maybe. Maybe.

Hannes Lowette: ...whatever matches.

Dave Thomas: I think, ultimately, that will be the case. I mean, you can imagine a situation where...

Hannes Lowette: This is DRY (don’t repeat yourself) on steroids, isn't it?

Dave Thomas: It is. It is. But I think the main thing for me is it stops all my React hell, right, where things break because I'm always going to...Here's the cool thing about this. Think about deploying your program. Right? So, you have a piece of code and it maybe uses...Well, I don't know, the average JavaScript program probably uses 2,000 to 5,000, right, separate libraries. So, you've got those 2,000 to 5,000 separate or NPM locked somewhere. Right?

Hannes Lowette: Yes.

Dave Thomas: Not anymore. You deploy your main program. That's all you deploy. And your main...

Hannes Lowette: And all the other hashes get pulled in.

Dave Thomas: All the other hashes get pulled in automatically, and they automatically work because they're automatically compatible with the code that you wrote. So, deployment is literally, here's my file.

Hannes Lowette: So, left path issues would no longer occur.

Dave Thomas:  I mean, whole industries will go out of business. It'll be great.

The Role of Developers in the Age of Advanced AI

Hannes Lowette: Well, that's the thing that a lot of people are scared of for the future is the training language models that are going to reduce the value of some of the skills that we have today. How do you look at that? Let's end on that.

Dave Thomas: We have always worked on making our tools more efficient. And ChatGPT, sorry, is Stack Overflow with a better interface. Right? Developers have always copied other people's code, and they've always gone in there to work out, "Okay. I have no idea how to do this, so I'll go through GitHub and I'll find someone else that's done it and work it out." So, it's a more efficient way of doing that. I use Copilot a lot for repetitive stuff. So, if I'm filling in some test data and I need, like, name, Dave, after about two lines, it works out what I'm doing and fills the rest in for me, I love you.

Hannes Lowette: That is the code that you don't care about crafting or writing or whatever anyway because it's a repetitive task. You want to move on in your head to the next task that you're gonna need to do.

Dave Thomas: Which is exactly why I like it. Yeah. But I think in reality...

Hannes Lowette: So, you're not scared for jobs just, like, yet.

Dave Thomas: I think that there is a class of developers who could easily be replaced because I think there's a class of developers that has never learned how to code and survives with copy and paste and, you know, good excuses. I think that the time of Chat...I don't think you will go to ChatGPT and say, "Write me a payroll system." That's not gonna happen anytime soon. But I think that you couldn't go to those copy-paste developers and say that either. Right?

Hannes Lowette: Right.

Dave Thomas: So, I think what you're gonna do is say, "What differentiates me from something good at searching for existing answers to solutions?" And the answer is finding better answers. Right? If all we do is have ChatGPT writing code for us, and if that ChatGPT then trains itself on the code that it wrote, well, I mean, that's just aggressive. I mean, we're gonna end up back with, you know, 10 print hello if we're not careful. So, I think there's always going to be room for developers to add humanity and creativity. Let's put it that way. But I think to do that, you've got to be consciously making yourself better.

Hannes Lowette: I think that's a great message to end this talk on.

Dave Thomas: Well, there you go.

Hannes Lowette: So, thank you very much for being here. I hope you enjoyed the talk. Look up the stuff that Dave Thomas has been working on. It's amazing. Thanks for taking the time to talk to us.

Dave Thomas: Always my pleasure.