Teaching Kids to Code with Sonic Pi

Another social perspective on programming

James Lewis: Hello, and welcome back to "GOTO Unscripted." I'm your host today, James Lewis. Thank you for joining us. Good morning, good evening, good afternoon, wherever you are in the world and whatever time zone you're in.

Today, I'm at GOTO Aarhus, and I'm joined by none other than the great Sam Aaron, creator of Sonic Pi and, well, originally Overtone. We're gonna be talking a bit about music, a bit about his love of teaching, that sort of thing. So, Sam, maybe you could...the great Sam Aaron, maybe you can introduce yourself.

Sam Aaron: I wouldn't say that at all in the slightest. I'm Sam, yeah, hi. I'm a programmer, now turned to a musician. I love programming. I love music. And I love finding ways to explore the creativity of code, and using code as a form of expression, really.

I use that obviously, personally because I really enjoy just using code to express myself and particularly through music. But also, predominantly, to use code as a way to...to use the expressivity of code as a way to teach computer science in schools, and to try and get the next generation of programmers not just to be able to program, but hopefully try and change a sort of social perspective of programming that it's not just a tool for business, but it's really a powerful, expressive tool for humanity.

James Lewis: As like a socioeconomic, socio-technical kind of construct programming for the greater good, if you like?

Sam Aaron: Absolutely. I think the problem is that most people in society don't really know what programming is at all. When you see it on TV films and movies, it's just like "The Matrix."

James Lewis: Ping.

Sam Aaron: Coding...oh, yes a typewriter, right? But yeah, it's, like, totally not the reality. I don't think people know what it is. I usually explain this issue through analogy of writing, where I invite people to imagine a world where writing is only ever used for business/legal usages. And would that be a world you'd like to live in?

James Lewis: I can honestly say that, no, I wouldn't like to live in that world.

Sam Aaron: I think that the problem is programming in that world right now because it's so young and so nascent. We really don't...we're still banging sticks together. It looks so futuristic and science fictiony. But it really is very, very young compared to many other disciplines that humans have created and explored and researched and worked on.

So I still think there's a huge amount of exciting things to happen through programming. I'm hoping that we're gonna see more exploration of things not necessarily related to business and business needs.

Recommended talk: Sonic Pi - BEAM Up The VJ! • Sam Aaron • GOTO 2023

James Lewis: It's super interesting. So there's 50 things to unpack just in that opening statement, which is brilliant. Sorry. I mean, because you talk about programming being a nascent sort of thing, computer sciences is new. But you also, as you said, you're interested in music and things which is amongst the oldest of things that humanity does, right? And you're banging rocks on other rocks, I guess is music, right?

Sam Aaron: I remember watching a video of some person talking about K-person instruments, which were literally little rocks that were tuned. They found the rocks add different melodic constructs and they bang them to make melody. I also read somewhere whether it's true or not, we don't know obviously, that there's a possibility that music came before speech. So even that level of ancient music is deeply ingrained in our humanity.

What came first: music or code?

James Lewis: And so what made you bring the two things together? What was the...I mean because you describe yourself as a computer scientist, and then a musician later on. Were you a musician before? I mean, did you play instruments?

Sam Aaron: I'll answer that in sort of snarky way and then I'll answer it in a more appropriate way. So the snarky way is that I hear this question a lot when I go to conferences and people meet me and say, "Oh, did you learn music before you did computer science and music stuff?" And it's like, my response always is, "Well, what domain do you currently program in?" And they'll say, banking or they'll say whatever it is. Then I'll say "Were you a banker before you were a programmer?" They're like "Of course not." And they don't see that's the same as asking me the same question.

And, actually, the beautiful thing about programming is you get to explore the domain you're in through the programming and learn about that domain through the programming. And so, yes, I mean, to answer your question correctly, I did have a background in music, and in school, I played in a jazz band, clarinet, and saxophone and played in a county wind band. But, yeah, they're very much the same kind of thing to me.

But also really early on my first experience of programming was on a Spectrum 48k, and the programs I wrote on that...it was amazing these things. You could turn them on and the little programming environment popped up, no complicated thing to download or install it was just there, and there was a manual. You read the manual and it will tell you how to use it. Brilliant. Like, where's that today?

One of the first things you could do is you can learn the programs too...they had a very simple basic in the Spectrum, but the basic had some cool functions for drawing and for call functions for making beeps and sounds. So I made it draw a little house and play a little melody. That was one of the first things I ever did. So even from day one, I was programming music and coding at the same time. So I don't think they were separate.

James Lewis: It's interesting because you mentioned that you were in the wind band and jazz band. I mean, interestingly, I was as well. I was in a similar band.

Sam Aaron: What did you play?

James Lewis: Well, I was a percussionist, right?

Sam Aaron: Oh, nice.

James Lewis: So...

Sam Aaron: You got to do the really...

James Lewis: Yes, I did a gong. I played the gong at the Royal Festival Hall, one bong, at the end...

Sam Aaron: Was that stressful?

James Lewis: It would be great apart from the gong. The issue was the review. Who let that idiot lose on the gong?

Sam Aaron: Was that stressful though, to actually get that right?

James Lewis: It was quite stressful yeah. I mean, the reason I'm sort of bringing that up is because I don't consider myself a musician. But, you know, I did learn a lot of the formal kind of...I had sort of formal education if you like be able to read treble clef, and bass clef, and percussion and, you know, syntax and those sorts of things, right?

And when I see...I mean coming on to the tool that you've built Sonic Pi, which you use to teach a lot, and then like, DJ heads with all these really cool things.  I can sort of see when you're writing the code, when you're live coding. I can kind of translate that in a sense in my head. Do you think you would have had to have done that if you didn't know it already? Do you see what I mean?

Sam Aaron: Well, I mean, I don't think I did know it already, to be honest. I do remember, right in the early days of Sonic Pi, not knowing what chords were or scales really. I mean, I'd done enough to do to my grades, and just did the things and I was just basically listening to the teacher and printing it, but didn't really understand it. And, obviously, to code something, you have to really understand it to be able to code it properly.

So I do remember having a friend who was doing a master's in music, and bringing them over to my house and playing for beers and saying, "Tell me what a scale is?" And he would explain it to me in words I didn't understand. And I'd try and understand it and say, "No, I'm sorry, can you try again and try again."

Then when I started to feel I had some kind of grasp I would code something, and then show it to him and he was like, "Oh, it's almost." And through that process to learn what this was. So, I definitely had to do a lot of learning. I still do very much. There's still a whole ton of stuff I'm going to apply to the software I'm working on, that I don't fully understand yet.

But also, I mean, I'd like to try and possibly pick on your statement where you said that you weren't a musician because I think I hear this a lot. It's like somebody who's saying they're not a sportsperson. And if they're saying they're not a professional sportsperson, I can appreciate that as a statement if they don't make the money playing for a professional football team or whatever. But they haven't still played on a Saturday League, or they just play with the kids in the park and they just enjoy it.

So I think that there's a big stigma around people not being musicians because they don't feel that they're professional musicians. Actually, I think we've mentioned this earlier just now, music is part of humanity and we're all musicians. If you can hit a stickler table, then you're a musician. You might not be a professional musician, but I'll agree with that. But you are going from saying, you're not a musician, but then being able to read music notation that's already like...

James Lewis: Well, I suppose yeah, maybe back in the day.

Sam Aaron: But even now. So I think it was really important that people understand that everyone is a musician, and it's something that everyone can enjoy. And you don't have to be a professional in anything to enjoy that thing.

Sonic Pi- The Live Coding Music Synth for Everyone

James Lewis: And I think that brings us neatly on to the topic of...well, maybe let's talk a bit about the tool first so people who haven't come across it will sort of, I guess, understand a bit about what it is, what it does, how you use it and these kinds of things. So you started off with a library for Clojure, I believe called Overtone many years ago.

Sam Aaron: That was one of my first. I was very interested in the idea of using code for music. I was actually one of these conferences actually the GOTO conference. I was going to the party afterward and saw a band playing, I was like this is a programming language conference wouldn't be cool if people could code the music.

And, of course, then you start to do the research and realize that people have been coding music ever since computers have been around, in fact, even beforehand. And so Ada Lovelace when she was working with Charles Babbage on the difference engine, was capable of looking in this very simplistic as we see it now, but then quite sophisticated mechanical machine that was just designed to do mathematical numerical computations. And project that this is something that could potentially be used to make music from my dreams. So this is not a new idea.

And obviously, automation and music, huge history, and just like yeah, same, like, knitting or weaving. So I then found lots of other people doing it and looked at what was interesting. I was very interested at the time in the programming language Clojure, particularly for its concurrency semantics and immutable data structures seemed to be, like, very interesting to me. And was...

James Lewis: It was a huge splash when it arrived Clojure, right?

Sam Aaron: Yes.

James Lewis: I was there as well, here, actually.

Sam Aaron: It's like "Oh, this is super cool." Then I also discovered something called SuperCollider, which is an open-source synthesizer for audio synthesis, which has a network API. It's super powerful, amazing. And the programming language that the SuperCollider used for its execution model was very much related to Ruby, which I'd been playing with before and I wanted to move away from.

So I said, "Well, why would I spend time learning something that has the semantics I'm not looking for, and maybe see if I can find a way to get Clojure to talk SuperCollider because it has this network API? " And I was living in Amsterdam at the time and one of my friends said, "I met this guy called Jeff Rose, and he's working on a thing which is combining Clojure and SuperCollider." I'm like, "Okay." So we made instant friends.

We worked on this project together. And it was really fun. It was basically a really nice DSL, Lisp-based DSL, which allowed you to design synthesizers and interact with them. Essentially just programming and controlling SuperCollider. We designed it to be the most powerful programming system we could imagine, so we treated ourselves as wizards.

I have a few thoughts about this because I think that was a mistake, in essence. But what we did is we thought about it in a way that I think people traditionally talk about programming languages, which is language A is much better than language B whatever it is because the semantics are superior or you can test it better. Or you can mathematically prove that this isn't possible, its memory is safe for whatever it is that you're...or it's a more elegant solution.

We were trying to do this and that kind of guise, but for us, it was more about liveliness. We're gonna modify things in real-time. An expression in terms of it being able to not be limited to imagine new kinds of sounds and synthesis techniques, and then be able to make them to reality.

But what I didn't account for was how much complexity that included, which then became a huge barrier to entry for anybody really trying to do this because not only did you need to learn Clojure, but you also needed to learn sound synthesis.

I remember going to Clojure conferences and giving talks, and afterward, I went down the corridor, and obviously, there were other attendees from the conference there playing with Overtone making crazy funny sounds, and it's, but nothing really ever came from that. I think that was because I hadn't really understood that these social barriers are really important, but it's also the barrier of time as well as how long it takes you to learn.

James Lewis: To learn something.

Recommended talk: Software Architecture, Team Topologies & Complexity Science • James Lewis • YOW! 2022

Sam Aaron: And if you do know it, how long it takes you to build it? So if I...

James Lewis: And the affordances that you need sometimes when you're using or trying to do something that is inherently quite complicated actually, which live coding of music is clear. It can be made as simple as possible, but no simpler I think is your kind of thing, right? And you need some of these affordances to guide people along the path.

Using Sonic PI for teaching programming

James Lewis: So Overtone came first and then Sonic Pi came later. Now did Sonic Pi...so I know you do a lot of work in schools, like, helping kids learn programming, and you're deeply involved in that, deeply passionate about that. Did those things happen at the same time or did you build...what came first chicken or egg? Like Sonic Pi to help teach, or teaching and then needing a tool that you can use to teach? Do you see what I mean?

Sam Aaron: So I was working at the University of Cambridge computer labs, and my contract was about to expire. It was the same year that the Raspberry Pi Foundation spun up their labs, and they had this wildly successful, very affordable computer and a remit to teach computer science to children.

But they had a problem. There weren't really that many capable pieces of software that ran on this very low-powered computer. Most of the software that had been developed was web-based and this thing didn't really run web browsers very effectively. You tried to go to YouTube and kind of didn't do anything. Even things like Scratch at the time which was implemented in small talk didn't run effectively at all that time. I remember they hired external people to try and optimize it to actually get it to run there at all.

So this is like they got this thing selling but they didn't have any good software. There's a small budget from Broadcom Foundation that people make chips with. For a three-month project as I understood at the time, to help try and think about a piece of software that might help teach kids how to code. And I kind of waved my arm and said, "Maybe I can adapt my Overtone software and build something similar for the Raspberry Pi. So that's how that happened.

In the first meeting, I was quite shocked really because that wasn't what they wanted. What they wanted was that and a load more. So in three months, they wanted me to develop this piece of software, implement it, find a teacher to collaborate with, go into their school, teach a curriculum, a course of computer science with the software, and evaluate it in three months. So it was already very much constraints kind of project, which was like, wow, okay, so I, like, stopped everything and focused on this.

Sonic Pi was born. It took two weeks to build the first version because I knew that if I spent any longer than that, then I wouldn't have been able to do all the other stuff. I was like, what is the smallest possible thing I can build that would allow me to go and do one lesson in a classroom? And one of the things...and this is the same time that the UK had the computer science curriculum, so there was an organization called CAS, Computers at Schools, headed by Simon Peyton Jones.

James Lewis: Yes, also in Cambridge at the time I believe.

Sam Aaron: They pushed for this new curriculum. It was nice and had lots of interesting concepts like sequential programming and conditionals, variables, algorithms, and data structures. Which was the one I think probably starts with sequential programming? So that's a sequence of instructions, we start at the top and work down. So I'm gonna play a note and wait for some time and play another note and that'll be less than one. So that's what I did the first time.

So schools definitely came first. That was absolutely the focus of Sonic Pi. And I always saw it in the early days as sort of a toy system that could be interesting in a school context. But I was definitely still interested in Overtone as my primary tool for expression because at the time I was actually already starting to achieve what I wanted to do, which was performing at conferences. So I was going around the world going to conferences performing with Overtone. I even made a band with my brother-in-law, we call ourselves Meta-eX because I'm a nerd.

That was doing really well. Sonic Pi thing was kind of a way of continuing to work at university context on musical software. I implemented it so that the synthesizers the sounds in Sonic Pi, were actually this Clojure and Overtone sound. So I can actually take the work I'd be doing Sonic Pi and make use of it in my performances.

Challenges of using Sonic Pi to teach coding

James Lewis: I mean, fast forward a number of years, you've spent a lot of time developing the software. It's pretty amazing, I recommend you go and have a look. It's obviously open source. It is open source we'll come to that in a bit. It's incredible what you can do with it. Generally, it's got visualizations in it. It looks like a full-featured piece of software in order to be able to live code basically music.

I went running in circles then but then never mind. So fast forward, what were the challenges of using this software to teach children in schools, then? I mean, what are the...or rather, what constraints are placed on the design of the software then by the fact that you're using it in schools and that's the primary purpose of it if you like? Was there a set of design constraints that you had?

Sam Aaron: Absolutely and I mean, I could talk for hours about that. They really both informed the development design and philosophy of Sonic Pi but also changed fundamentally the way I approach software.

So, of course, there's obviously the main constraint that you're working with people who don't really know anything about software because you teach them how to code, that's the children. And then the teachers primarily don't necessarily have a background in computer science often. So you're dealing with beginners in many cases, and also making things as simple as possible to onboard because the number of time constraints in the classroom was one of the most surprising elements

When I went into that classroom first and saw that half an hour lesson, a 45-minute lesson, most of it is about the management of behavior, and getting everyone to listen. And then the teacher has like two minutes of their attention, What are the two-minute sentences that you're gonna say they're gonna get excited by your programming?

So already, you had to have something that gave an immediate response that was interesting. What turned out to be the most interesting focus was, it didn't turn out to be that hard to implement a language that will expose the basics of the computer science curriculum in the UK. And, in fact, you can just look at history and then just go back and look at Seymour Papert and Cindy Solomon, pick out a logo, and you can work with that.

So the hard part turned out...and the interesting part to me at least, was how do you actually engage the children? How do you maintain their engagement? How do you capture their interests? That actually was both a challenge because it's hard to do that, but also a challenge because doing that could and did...so I was really trying not to interfere with the teaching of the core computer science.

Making it something cool and fun might have been something that got in the way of teaching the variables or the data structures. If that ever happened, the teacher would never be interested in pursuing this because it was not what they were trying to do. Their learning objectives were set, we have to teach those things, and the shiny stuff sounds, it looks very nice and pretty, but the kids aren't getting the learning objectives met so you're wasting our time. So...

James Lewis: You mentioned some of that yesterday and sounded a brilliant talk yesterday where you talked a bit about concurrency and how that sort of was a bit of a barrier at the start.

Sam Aaron: No, absolutely. I mean, that was a critical one. The kids wanted to play the drums at the same time as the bass.

James Lewis: Right. Who'd have thought?

Sam Aaron: And to do that you need concurrency. But also a band itself is concurrent in nature. It's like you have multiple people playing at the same time. And that's actually much more natural and obvious than a single-person band. So concurrency is a really natural state of affairs in music, but of course, it's not on the UK computer science curriculum for schools.

When the children were asking how to play the drum at the same time as the bass, the teacher would say, "No, we can't teach that because it's not on the curriculum." So here's a really good example of where we have something that we maintain and keep where not only would it maintain the motivation, but they are already motivated enough to ask about this. Like, I want to do this interesting thing that you haven't told me is even possible.

That's amazing, like, to get the child to be in that state where they can ask that question, it's an amazing thing. And so it's a precious...it's like the seeds just, like, popped out and the flowers are just about to sprout, and then going to that child and saying, "No, you can't do it." It's like stamping on the flower and crushing it. It was really demoralizing.

I put a lot of effort into sneaking in features that maybe weren't in the remit of the project, and maybe that the funders probably weren't that interested in seeing their money being spent on, but directly helped maintain that engagement.

Also then made this concept really grow out in terms of hey, actually, if we can make it more musical, and more interesting, really, what we want to be doing is getting the artists that the kids are looking up to and appreciating to be coders. Because if that's possible...if we get Taylor Swift to make a piece of music with Sonic Pi or similar a piece of software, we'll have a million coders tomorrow.

So then I really focused on making the app not just something that was useful in the school context, but also something that was a powerful tool for expression for musical and artistic people.

And in a way that didn't sort of one-stop the other. So I was really always careful with any feature to make sure that, hey this is a really cool musical idea, I can see musicians using this in the DJ platform, but it's not getting in the way of teaching those basic computer science lessons.

That was something that I remember and still today actually a lot of people say, "Well, just that's crazy Sam. You can't build something that's both simple for kids and super sophisticated."

James Lewis: And deep enough for...

Sam Aaron: And my response to that is like "What is a guitar?" And like, okay, like, anyone could pick up a guitar and strum the strings. But then you can practice that and you can do amazing, you can be Jimi Hendrix, you can play beautiful Spanish flamenco guitar. It's those artists that people admire, are the motivation for people picking up a guitar. It's not because it's a technically sophisticated, beautiful instrument. Some people might pick it up for that, but most pick it up because they've seen their favorite artists.

James Lewis: Yes, because it's cool. They think they'll make friends.

Sam Aaron: Not because the school is telling you you must learn this in the curriculum it's because they want to make music.

James Lewis: That's why people play the trombone.

Sam Aaron: They wanna have fun. So what is the equivalent of code? What is the guitar for code? Sonic Pi is my attempt to make a guitar or a piano for code.

James Lewis: Obviously, you said we've got the GOTO Conference party tonight I believe you're playing, you're coming tonight?

Sam Aaron: Yes, the full circle. So, from saying wouldn't be cool if we could code in a party, to actually be coding in the party.

James Lewis: It's fantastic.

Sam Aaron: It's gonna be great.

Famous users of Sonic Pi

James Lewis: I'm super looking forward. I'm really looking forward to it actually. But I mean, another thing you mentioned in your talk, which I think you sort of touched on a little bit, just now is this aspirational thing. And you showed a great picture. I can't remember who the artist was, the DJ was in New York I think it was.

Sam Aaron: Oh, yeah. Sarah Davis, DJ Dave.

James Lewis: DJ Dave.

Sam Aaron: She's based in New York, she's playing huge sets now, yes. Amazing.

James Lewis: That's amazing. Being able to show children that actually this is accessible to them in a way that...actually I think is even more important from I guess, a social perspective these days than it has ever been before...has been for a long time. In the sense that many kids will have access to something that they can use Sonic Pi on, if you'd like when they might not have access to an electric guitar or a violin anymore.

Sam Aaron: Absolutely.

James Lewis: Because certainly in the UK, there's been a sort of shift in education, hasn't there? 

Sam Aaron: Across Europe in general.

James Lewis: Yes. There's not so much access to this stuff, whereas this is accessible. I mean, I don't know...I mean, that must be really pleasing for you to be able to provide that sort of...

Sam Aaron: It's depressing that schools are cutting their musical curricula. I had a teacher in the northwest of England contact me because Sonic Pi wasn't working on the machines. And we've figured out a solution and got it working. Because Windows is installed in all sorts of different ways in those schools, so trying to circumnavigate. Their problem was that they weren't allowed to run batch files. Sonic Pi uses batch files to set up the environment variables.

Their security system wasn't allowing this so I changed it so it didn't batch files. Your SCADA system doesn't seem to do much good. But anyway, I came to this teacher because I was interested in like, well, how are you using it? What are you using it for? And it turned out, she was a music teacher who was still employed by the school, but the music curriculum was being reduced, so she was taking over the craft and design technology lessons.

She was then using that as an opportunity to teach some computer science and some music at the same time to build, I think it was maybe ringtones for an object they were building. So she was sneaking it in and it's, like, it's sad because this is not a simple thing you can apply everywhere. It is frustrating to see that.

It's frustrating that typically in the music world, there are a lot of gatekeepers who decide what music is. So if I go to a computer science teacher, they're all over it. "Yes, we need anything that can get the kids excited. It sounds great."

The computer science landscape is constantly shifting. There are new things all the time and keeping up with that is a tricky job itself. Well, they used to change and churn. Whereas in the music world, they're still studying the greats, you know, and there will be in 10 years and in 20 years' time. So it's much more glacial the movement of the curriculum for music.

James Lewis: You told a funny story about jazz, I think yesterday.

Sam Aaron: I was in a lift. I was nominated for an award actually Music Teachers Award. So I went to this fancy dinner with bow ties, someone playing the piano and someone taking the Ferrero Rochers around. And I was talking to the top two people from the rule-associated board of music. So these are the people with the grades for pianos and clarinets and classical instruments in the UK, and I told them what I was doing.

They are actually really interested and it's lovely to hear their response. And they sounded really enthusiastic so I was like, "Okay, let's take a punt here." And I said, "So when might you consider introducing grades for live coding?" And they went, "Well, Sam, we've just introduced jazz." And they were really pleased about this. It's like "Jazz is just 100 years old." So yeah, we have to wait.

James Lewis: It might be a little while then.

Sam Aaron: Glacial speed.

James Lewis: As you say, it is actually aspirational when you see like a room, a club full of people jump to this stuff it's pretty amazing. Many others have taken on as well. I've been to other conferences where people are using your software to do the same thing, you know, so, super, super exciting.

Sam Aaron: It's also important to point out that mine is not the only piece of software that you can do live coding with. There are tons of really cool pieces of software out there. All have their own affordances in

the same way, a violin is slightly different from a piano, it's slightly different from a trumpet. Sonic Pi is one of many cool pieces  of software to live code with.

A simple tool designed by solving hard software problems

James Lewis: Could double down there and tell me more about the alternative software?

Sam Aaron: One of the most exciting alternatives is this piece of software called TidalCycles.

James Lewis: So one thing I've been fascinated by with Sonic Pi is that whilst it seems a simple piece of software. Generally, you know, Sonic Pi I've used it myself badly. But it is a simple piece of software to use. It's incredibly well-designed. And yet... from the surface. But then when you actually think about some of the problems you must have had to solve to build it, there must be some quite deep comp sci stuff going on under the hood. We chatted briefly yesterday, and you mentioned, you know, this is human real-time. There are difficult problems to solve. I mean can you just maybe talk about some of those things you have to solve?

Sam Aaron: I mean, time is the crucial one. If you're building a piece of software for music, it has to have good timing, otherwise, it's not a credible musical instrument. So that's a really very hard constraint. Everywhere you're making sure that the melody doesn't drift.

The very first version of Sonic Pi which I built in two weeks, didn't have good timing. So it was perfectly fine to teach that basic computer science lesson and nobody noticed. But it was actually when I introduced the concurrency that it was really obvious because you'd have two threads, making a melody, or one doing melody, one doing a drum line and there was some drift out of time. Oh, my goodness because they were doing sort of POSIX style sleeping, which says, shut the process down for some period of time.

James Lewis: For about this amount, please, but you don't have to if you don't want to, you know, you can't afford it

Sam Aaron: Absolutely.

James Lewis: ...a bit longer if you need it to be.

Sam Aaron: You say it's not about typically, it's at least plus some delta. And that's like when the schedule...when the operating system says, "Okay, are you ready to track on again?" And that delta is variable every single time, and smaller or larger depending on the operating system.

Actually, this is one of the interesting things about developing it on the original Raspberry Pi, because it's such a low-powered, low-CPU power device that all these problems really became large straightaway. So there's deltas that the scheduler took to wake up were large and variable, and they were really very noticeable very quickly. So this stems, okay, how do I fix this?

And that was interesting as well because it was about I quite like the semantics of the language. It was very simple as you describe it. The focus of it was to be able to make it so that anybody could play and work with this thing. So I was like, how do I make this language keep the same from an API perspective, the language descriptions perspective, but have better timing? And so that's sort of replacing all the underlying guts. And it took me a year actually of walking in fields to figure it out. And, of course, when the solution popped into my head, it was very simple.

James Lewis: That's true about all these people who live in Cambridge. They just walk through the fields thinking about stuff all the time.

Sam Aaron: Most of my day actually was going into the computer labs, meeting interesting people talking to them in the morning, having a coffee, and then doing my emails, then having lunch. And then from lunch till about 3 or 4, I will do a solid piece of work. And I think actually, that's all I need to do in a day is three or four hours of actual coding. It's all I can do really, I think.

And then it depended. So I would set myself a goal every day and look at the weather as well. And if the weather was good, I set myself a slightly easier goal. Then if I could meet that goal, I would allow myself to shut the laptop and go out for a walk. I walked for anywhere between half an hour to two hours, depending on what time of day it was and what the weather was like.

There were some really nice fields next to the computer labs at Cambridge that I could stroll around and that's where I did most of my thinking. Like the design process for a feature for Sonic Pi was done there where I was, like, have all these ideas I could implement.

I pull one down from the ether look at it and go "Can I complement this? No. And then maybe I can implement it. Well, does implementation tessellate well, and fit well with all the other features? No, it doesn't. And then it would be like, yes, I can implement it. Yes, it does tessellate well. Could I imagine teaching this to a 10-year-old child?"

That was every time for all the ideas I had to go through this process. And so it took a long time for any feature really to make it into the software because I had really very stringent tests based on this philosophy that things had to be simple enough for schoolchildren. Then I extrapolated to every feature in Sonic Pi whether it's the concurrency, whether it's the time state model, whether it's the event system, whether it's the IO model, it's the live coding aspects. There's a lot of cool technology.

Every aspect of that I can teach and have taught to a 10-year-old child.

James Lewis: And that's...

Sam Aaron: And that's only because I've thought about it so deeply.

James Lewis: So hard yeah. I mean, it is amazing. You were showing some of the diagrams of where like, in terms of the components of the software as is, and I think maybe for the future as well. You have to tackle pretty much all the big problems, right? I mean, you talk about things like concurrency and timing. But you also talk about IO integrating with lots of different standards so things like MIDI and all these different pieces of kits you need to talk to. I mean, this is kind of what I mean, it's remarkable that it is so simple because actually, you've solved a lot of really gnarly problems under the hood, you know?

Sam Aaron: When people talk about the idea of a swarm, right, it's looking very graceful above the water but swimming hard underneath, and Sonic Pi has been like that. And it's kind of like there are lots of different processes implemented in different languages to GUIs in C++, the Language Runtime currently is in Ruby, the iOS or Erlang both Elixir and the Erlang language itself, and the synthesizers are still designed in Clojure. Although the JVM isn't part of the runtime, it is used to compile synthesizers. And then the audio synthesis is only too early is something called SuperCollider.

So having to orchestrate those and get more runtime is kind of like having to build my own small Kubernetes to make sure the processes are running in the correct order the execution of the process is, make sure they can all talk to each other, make sure all the logging happens. So when things don't work out, people can send me the logs while I'm on that particular platform, This particular process hasn't started, like, a lot of these things are critical to implement correctly.

James Lewis: Plus you open in Kafka, apparently, as well.

Sam Aaron: So there is...turns out that actually Sonic Pi is now...the core of it, it's one of the things when you design something that is this time event state model which can be seen from multiple ways.

I actually took a bunch of different features in Sonic Pi that I needed, like, I want to be able to press a button and run a function. I wanted to be able to share messages across threads deterministically. I wanted to share time so I wanted to...I could thread barrier synchronization. So don't do this until this has happened across threads. But also past the time things happen so this can continue at the same time as this. So it's like it has a bit more constraints than thread synchronization.

I built lots of different systems for doing this. But it turned out, I can actually combine them all in this time state model as I call it, which is kind of like a tuple space where you can...just like a map. Kinda like in Python, whatever, where you say get me value foo from the map, but also you'd say at time t. Which then means that the foo is not just a single value, it's a time series. Then you can pattern match over this thing. Then you could say things like, wait for a new value in this system to come in. So events are coming into the system...

James Lewis: So it's almost like a stream processing component of a complex event processing kind of.

Recommended talk: Introduction to Stateful Stream Processing with Apache Flink • Robert Metzger • GOTO 2019

Sam Aaron: Which is the Kafka analogy, right? And Kafka does also deal with time. The time gets inserted once you get into the system and stays there for the entire time.

James Lewis: But also it doesn't coordinate across streams, right? We didn't...maybe it doesn't. That was always the thing with most of these pieces of kit because that's the hard part. But...exactly.

What’s next for Sonic Pi?

Sam Aaron: It's necessary absolutely. And so, I've implemented that and it was a terrible augmentation in Ruby and it works-ish. So that's now the next thing I'm gonna work on. I'm gonna rebuild that in Elixir.

James Lewis: Just sort of occurred to me that something else I was thinking about was testing. And this kind of how'd you know if it works-ish or not?

Sam Aaron: It sounds right. When you run the code you know what it's gonna sound like and you can hear that. And the same way the really old days, they used to listen in on the big massive mainframe computers but they were going so slowly, and they were quite mechanical. You can actually hear when they're executing correctly or not.

James Lewis: We used to be able to do that. You mentioned, but your first computer, my first computer was an Acorn Electron with an additional tape deck on the side. You used to be able to hear... You know whether your game had loaded correctly because you turned the volume up on the tape, right? So you'd be able to hear that... Yeah, exactly. You know, no Chucky egg is not loading again 

Sam Aaron: Yeah, super annoying.

James Lewis: Really, really. Obviously, it's open-source software. We've talked a little...we've mentioned that or I mentioned that. So I mean, how does this work now in terms of ongoing funding? Obviously, you're still involved with schools and teaching and things, and you're still involved in active development of [crosstalk 00:38:20.221]?

Sam Aaron: Yes, that's a hard question to ask. I mean, I was initially funded and I had a postdoc role at the university. But funding university is hard and so these things don't always work out. So that became more and more difficult to sustain and maintain. And it felt like there were more constraints on me about having to demonstrate impact and do reporting.

I'm not a person that's very good at those things. I'm a programmer and I wanted to focus my energies on that. So I had a choice either to go back to being a professional programmer, or just find a way to carry on working on this, but maintaining my independence. Since I had that independence in the early days because there was a sufficient lack of monitoring and management and they allowed me to do the walking in the hills and imagine all these features and build them and no one really sort of said no. And that allowed me to develop the software as it is and so I wanted to carry that on.

James Lewis: Sure.

Sam Aaron: Currently, the funding model is really about trying to diversify the income streams as much as possible. So I no longer want to be in a position where a single organization is controlling all the funding because that's a sort of a power position that I don't want to be into because if they wanted to pull it that puts me in a bad situation.

James Lewis: Back to building back …

Sam Aaron: Yes, and I'd rather not do that. So I'll say what I'm trying to do is diversify as much as possible. So currently, maybe people have more ideas about how I can change this. But currently, I have half my income comes from Patreon, which is a website that lets people make small donations, equivalent to a cup of coffee every month. And because that's a recurring thing that's really sustainable and it really gives me that sort of safety net, that's really critical. That was essential actually, during COVID.

James Lewis: Yes. Of course.

Sam Aaron: And allow me to survive through that terrible time. Then topping up on that so that's about half. When I say income, I'm talking about the income of an average researcher, not an average programmer or senior programmer. So I'm not attempting for that. It'd be nice, but no. And then the other half is coming from organizations, conferences, and companies hiring me to deliver party performances so I'm live coding the music, projecting it on screen, and making people dance. I can get finished programmers to dance as my claim to fame which is a cool statement to make.

And then also workshops. So I can deliver a workshop where I can go in and teach people the basics of live coding music. Because of the nature of it being for schools, that workshop works perfectly well for professional programmers who are hardened and say, "I don't need to learn anything new." And they'll still learn something new. As well as the non-programmers.

Actually, if you've got an organization, with programmers and non-programmers you want them to actually coordinate and collaborate together Sonic Pi is a great way of making that possible. And then also talk to keynotes. I can go and deliver an exciting talk about how it works and the kind of things I'm talking about now. So yeah, so those things...the money from that kind of events, they also go directly towards the continued development and maintenance. It's sort of working out, but it's very stressful.

James Lewis: Yes, I bet it is. But long, may it continue. Not being stressful, may it continue being in development as a project. I think there's one final question I had because I know we talked a lot about things like computer science and kids learning programming and how aspirational that could be if we could make it that way. So do you have any advice for the people maybe watching about how to maybe get involved or help, or what we could do actually all of us to make this more interesting for kids?

Sam Aaron: My advice generally, is if it sounds interesting to have a go, play with it, and share it, and use it to show your children or show your friends. Because it's really hard going to parties and people asking you, what you do, and you say you're a programmer, and they're not interested.

That's because we don't really have the things to talk about. You can't really talk about I'm working on this cool Java business system with distributed, whatever and Kubernetes, and whatever, serverless, blah, blah, blah. No one really knows what any of those words mean.

James Lewis: Including us, I think oftentimes.

Sam Aaron: Desktop servers where are all the servers? Whereas something like Sonic Pi actually gives you an opportunity to talk to non-programmers in a party situation about programming in an interesting way, and get them engaged. So definitely I'd give it a go, have a play.

I think actually, probably the most important thing people can do is to find their local school, and offer their help. Find the computer science teacher and say I'm willing, as a professional programmer to come into your school and run a Code club, or give a talk, or show something about the work I do trying to make it real and understandable to the children. I think schools are crying out for professional programmers to get involved with them. So that would be what I would do. And, of course, then use Sonic Pi as one of the things you're gonna share.

If you really like what I'm doing and you're supporting the approach I'm taking, then support me financially. So if you're an individual, Patreon is a great way of doing it, just $1 a month. Enough of those add up and I really don't need to stress about the money. Or if you're an organization that wants a party, or wants a workshop, then hire me and I'll be really happy to join them.

James Lewis: Perfect. Okay. Well, thank you very much, Sam. Thanks for joining us on "GOTO Unscripted." It's been an absolutely brilliant interview. Thanks, again.

Sam Aaron: Thank you for your thoughtful questions.