The Future of DevOps

The pandemic and remote work have changed the direction of the DevOps movement. Find out what lies ahead with Hanna Park, community manager at Mattermost, and PJ Hagerty, head of developer relations at Mattermost, in their discussion at GOTO Copenhagen 2021.

Hanna Park: We're live from GOTO Copenhagen. I'm Hanna Park. I work at Mattermost as the workplace community manager, and we're here interviewing PJ, our head of DevRel.

PJ Hagerty: Hey Hanna.

Hanna Park: Hey PJ. How are you?

PJ Hagerty: Good. How are you doing?

Work collaboration in COVID times

Hanna Park: I'm good. So let's just get right into it. How has work collaboration changed since COVID and we moved into after COVID?

PJ Hagerty: Well, I think that in the beginning, remote work pre-COVID was kind of seen as really nice-to-have. It was something that maybe certain people got and certain people didn't. I knew a lot of engineering teams that were definitely remote, but DevOps teams tended to have to be on-site so they could collaborate and work together closely.

As we moved into COVID and everyone had to go remote, companies like Mattermost led the way by kind of pointing out that it was possible to have a 100% remote team and still have that collaboration, still have that ability to work together securely, you know, without worrying about too many loopholes. As we move into post-COVID, it seems like there's kind of a dichotomy where on one side you have the people who said, we've really learned something here. We understand there are great ways to collaborate, communicate, keep everybody in a line across time zones around the world, and not have to have everyone come into an office every day and deal with a commute, or trains, or traffic, or anything like that.

And on the other side, there's kind of a more traditionalist point of view where people are saying, "Well, we've always worked in an office. Why wouldn't we still continue to work in an office?" And I think there are lots of things that inform that. Some of it is probably people paid rent or agreed to lease agreements that were years into the future and they're like, "Well, we don't wanna lose our investment." But I think that other places have seen the cost savings that they're getting with people working remotely, which is just kind of the first step as the tip of the iceberg when it comes to remote work. I think that with remote work, it often seems that people are given their own responsibility and they take that on. Granted to make it clear, it's not for everyone, not everyone can work remotely. I mean, beyond there being some jobs that you can't do...even in tech, there are certain people that just work better in an office. There are certain people that wanna have that daily interaction or face-to-face, time and water cooler time, and whatever.

There are startups that still want to have ping pong tournaments in their office and goofy things like that. And that's fine. But at the same time, we have to find a way to make sure that people who wanna work remotely, who benefit from working remotely, who have a good environment and can do that are able to do that continuously. And I think that that's one of the things that we're seeing develop post-COVID.

The impact of remote work on open source projects

Hanna Park: How has open source benefited from people working remotely over the last year and a half?

PJ Hagerty: Hugely. I think that a lot of people who didn't expect to get into open source have the opportunity now because I think a lot of people looked at open source as something to do in my free time, or when I have a chance, or couldn't do this in the office, so I had to wait to do it on my personal time. Well, we've upended the idea of you have to be in the office from 9 to 5 and your lunch is from 12 to 1. We have the idea that as long as you're getting the work done, that's what's important. So I think more and more people are finding opportunities to have things like Open Source Fridays or Freedom Fridays where you're able to work on open-source projects. There are opportunities for people to say, "Hey, you know, I'm home. You know, what else am I doing?"

I might as well go to work on that cool go project that I saw a part of, or I wanna get back intocoding in Ruby and help them out with some documentation. So there was so much more opportunity because people literally didn't have anything else to do. For open source developers that were already into open source, they had more time to dedicate, to open source. For people that were new to open source, all these doors were opening, all these opportunities where they could see their way into a project.

I think that even outside of that,with COVID going on, a lot of people got into Twitch streaming and YouTube videos, things that were instructional and allowed people to say,maybe I'm not the king or queen of development. Maybe I'm just getting into this, but hey, these people made a great video about how to write documentation and I know that the documentation for that project I love isn't so great.

I can get into the documentation and start contributing to that. And then they get further and deeper into it. I think there are so many more open source developers over the past year and a half than there were prior to that because those barricades are coming down, those gates are coming down. People can come in and they're finding that they have the time to do it. I think it's doing away with the idea of, "Oh, I can only do open source if I can do 40 hours of open source a week along with my 40-hour week job." That's not the case anymore. You can have very small bite-size increments where you're contributing a little bit here, contributing a little bit there, and finding a way to get that out there. So I think if there's any positive that came out of COVID, and there are very few positives that came out of COVID, but if there are any positives, it's the fact that people realize that they really can dedicate themselves to the open-source world and give back.

Hanna Park: That's cool. I think even our open source community expanded by a lot in the year and a half too, right?

PJ Hagerty: It absolutely did. We were able to see people come in from all over the world too, even just to like come in and say like, "Hey, this is a really cool project. I would love to translate it into my language." Translations are kind of tough. So it took time before, well, you have that time now. So at Mattermost, we saw all kinds of contributions. One of the biggest ones was someone who took a plugin and made it so it was possible for people who read right to left, like the Arabic languages and things like that. So they would be able to see their language in the way that they expect to see it. We were able to do that through open source, and I think that's absolutely amazing.

Is returning to the office beneficial for tech workers?

Hanna Park: Yeah. That's amazing. Do you think returning to the office is beneficial for tech workers?

PJ Hagerty: I'm gonna give a longer answer, but the short answer is no. I think that offices, especially with the way that tech offices have evolved, they're huge distraction centers. I remember going into an office at a job that I had a while ago and I was a remote worker, but they expected me to go in the office for like a couple days a month. Every time I went in, no work got done because there was always "Hey, come over here." "Hey, talk to us over here." "Oh, we're playing ping pong. These people are playing hacking seconds." Like, "Why are you doing these things here? We're supposed to be getting work done." I think that most workers, most of the average frontline, independent contributors, they realize that they can do their job anywhere. I think beyond not going back to the office, I think you're gonna see a return to the idea of, this was big like four or five years ago, the digital nomad where people realize that they can actually like work from anywhere.

The time zone's less important. People can work asynchronously. We learned that over the last year and a half. And I think that a lot of businesses like I said earlier, will be able to realize the benefit of not having an office, not paying...I mean, they don't really understand I think sometimes how much goes into having a desk at an office, like, you have to pay for furniture, there's insurance, there's rent, there's elevators, there are fees, there are all kinds of things on top. There's a power making sure everyone has a standing desk, all that jazz, all that crap that kinda gets dumped into this bucket. That's huge and huge expenses. Getting rid of that and sending people to remote work helps to kind of alleviate those expenses so you can actually spend them on things like research and development, building a better product, marketing, whatever. You can take hundreds of thousands, sometimes millions of dollars, depending on the size of the company, and reinvest that somewhere else while someone's working comfortably from their home at a place that they can really benefit from and build the things that we need to see.

Hanna Park: That's cool. I think I got into remote because of the digital nomad, work from anywhere kind of flex hours. And if I have the choice, I never wanna go back to work in an office.

PJ Hagerty: Yeah. I don't think I can.

Hanna Park: I don't think I can either.

PJ Hagerty: I mean, we both know I'm kind of social. I like to talk, I'm a bit chatty. So the idea of going into an office, I want it and then it's a distraction, but at the same time, I wanna talk to these people,what are they doing? What's their perspective? What are they up to? It's interesting to me. But a lot of offices don't want you to...and I shouldn't say don't want you to, but kind of discourage that kind of conversation.

I remember a lot of engineers who were working in offices and because of the noise around them and the people, and the sales team is doing this, the marketing team is doing this, there are meetings going on over here and you can hear everything.

I feel like the one thing that going back to offices will increase is the number of people buying noise-canceling headphones because they just don't wanna hear it. They just wanna get there and they wanna do their work and they want to come in, do it, go home. I think that we actually have seen growth in relationship building between co-workers, especially in the tech space by saying, "Hey, like maybe, I mean, you and I, we didn't meet until a year after I started with the company."

Hanna Park: Yes.

PJ Hagerty: And only because we had an offsite. A lot of the people we work with, I've never met, but I feel close to them. I feel like we're friends. I know about their kids and their significant others and what's going on in their lives because we're forced to have that communication and we can have it asynchronously. Someone can leave a message in Mattermost, and I don't have to respond to it right away. But if we need to have a conversation, we have a conversation, it just takes a little bit longer. There's nothing wrong with that. As long as the world's not on fire, there's nothing wrong with that. So I think that going back to offices just isn't gonna work.

Hanna Park: Yes. I feel I'm more productive working at home than I did at the office.

PJ Hagerty: Oh, definitely. I can get five days of work done in three days at home.

How has remote work affected DevOps teams?

Hanna Park: Have DevOps teams been affected by the move to remote work?

PJ Hagerty: I think they've been affected. I think that most people think of DevOps teams still to this day as like the emergency backup team, which is just like it's a small part of DevOps. That's your SREs, your site reliability engineers. They are the emergency fire team whenever something goes wrong. But I think that people thought, even when people started to move into the cloud, that those people had to be centrally located. They had to be together. They had to have a place to meet.

I think what we've realized as teams went remote is that that place doesn't need to be real. It could be a virtual location. So you could have somebody in a chat platform jump in and say, hey listen, AWS U.S. East has gone down because if you've worked in DevOps long enough that AWS East goes down pretty much daily. It is their oldest facility, but that's AWS's problem, not mine. You find that what they actually have is the ability to communicate and get everything done without having to like first take the time to travel into the office. So you're actually getting to the problem faster.

Working remotely has been a huge, huge benefit to DevOps teams. They realize that they don't need to be dragged out of bed, get dressed, drive down to an office, and have to deal with an emergency at 3 in the morning. They can just roll out of bed, pop open a laptop, and be like, "All right. Let's get it fixed." That's one of the key things I think is when you think about DevOps, the point of DevOps is to keep things up and running. It's a large umbrella, but under that umbrella is basically the concept of keeping things moving forward. And doing that, especially, when it comes to SREs, you really need to have the ability to communicate quickly, and having to drive and go into an office, that's not communicating quickly. That's wasting time and energy. You don't wanna waste time and energy when there's an emergency. You wanna make sure everything's laser-focused. And I think that that's the key. When you're remote, you can continue to be laser-focused, even though you're dragged out of bed.

What's the future of DevOps?

Hanna Park: That makes sense. What is the future of DevOps?

PJ Hagerty: That's a complicated question. A lot of people in DevOps will tell you that the whole point of DevOps is to automate things, to build and innovate so that things can move as smoothly as possible. That's true to a degree. I don't think that automation is quite as far progressed as most people think it is. I mean, we can use an example outside of DevOps, like automated driving. Everybody's like automated driving is the future. I'm like, "Yeah, it is the future, but it's not the now. It's not the now." All of these things that are coming out to make cars automated is wonderful and really great and it can be used for safety, but even a couple months ago, there will be a Tesla that ran through a crosswalk. It's still a problem.

Hanna Park:: Yes. My Tesla's in the shop.

PJ Hagerty: Okay. Well, that's a whole other issue. We'll talk about that off-camera. But the whole idea of automation is great, but you can't have it without the human factor. So I think DevOps is constantly evolving. There are lots of things that you can't automate, CI/CD is a, you know, great example. It used to be that, you know, in order to merge everyone's code at the end of the day, someone had that job that was literally a position where they had to look and resolve conflicts and then build, and then merge and push forward. Now with CI/CD, everything kind of comes together naturally, automatically, virtually. So you can continue to develop or deploy or whichever side of the fence you're on, you can continue to do those things without having to worry about that piece in the middle. That's key. I think the future of DevOps is also realizing that some of the direct concepts of DevOps are maybe a little bit misleading or mistaken.

And by that, I mean, the initial concept of DevOps is any one person on a product team, building a product, building the application should be able to do anything from A to Z. From writing the code to testing the code, to deploying the code, to monitoring when it's out in the world. That's great in theory and in the utopian world that would totally work, but well, we don't need the silos that we used to have where it was like, I just write code, and then I toss it over the wall to the QA team, and they toss it over the wall to the infrastructure team. I don't think you need that so much, but what you do need to do is realize that people can only train in different areas to a certain degree. The idea that was once popular of like a 10x developer, like a 10x DevOps person is preposterous. It's ridiculous. We shouldn't expect anyone to do 10 times the work at any point in time.

Focus on your research area, focus on your work area, make sure that's working well. Overlap to learn what the other teams do so that if they do need your help, you can step in. But at no point in time should we expect that someone beginning to end knows every single step of the process. Maybe they're aware of every step of the process, maybe they can help with every step of the process, but they shouldn't be in control of every step of the process. I think that's the key thing that a lot of people miss with DevOps now. Now in the future is we automate more and more things, maybe that's not necessary. Maybe we can have someone who just writes code, hits a button, it's automatically QA'ed, automatically deployed, and there's monitoring so that people can watch it. But I think that a lot of people also hear automation, they think, "Oh, we're gonna automate ourselves out of jobs." I don't think that's gonna happen any time soon.

Hanna Park: That is a concern for a lot of people.

PJ Hagerty: It's a huge concern. I know a lot of people who are in the DevOps space that say, "Oh, well, we're gonna automate ourselves out of jobs." It's like, "Well, why are you doing this?" And that seems crazy. "Why are you writing the code that's gonna make you not have a job anymore?" But at the same time, I think they realize that there has to be a human factor. We build software applications for human beings. We do not build software applications for other software yet.

I don't know if that further future will happen, but if we keep that in mind, then we need to realize there always needs to be a human factor. I think a great example of this, there was a little study done where a team had done a piece on facial recognition, and the team that did it was made up of a bunch of white guys. So when they put it out in the world, they realized that people who weren't white guys didn't get picked up by the facial recognition, they couldn't figure out what they had done wrong. They were like, what did... And that's kind of where it comes, you know, into play in DevOps is like if you don't have a human factor, you're going to miss some key piece of information that may or may not get looked at correctly.

You know, another great example is observability. A lot of people believe that observability is the way of the future in DevOps. It's a very important piece of the puzzle. You have to have proper monitoring, you have to have a proper, you know, setup.

Whether you're using as an open-source ELK Stack kind of thing, or you're using something like,Humio or something else that's observability tool, that's great. But the biggest problem with that is 9 times out of 10, you only tell it to look for what you expect to look for. You're not looking for the unexpected. So part of DevOps needs to be learning what we mean by unexpected and how to expect the unexpected, which is a cliche, but like, you have to learn that. So that's like...

Hanna Park: You're talking about AI now?

PJ Hagerty: Yes. And that's something that I think that AI and even machine learning are not ready to pick up. They're great at understanding false and simple things, but AI is definitely not to the point of making a determination like, hey, I see that on the server, 26, we've got, you know, a lot of CPU flop. That means that we're gonna have to do something because there's something going wrong there. Cool. But we're not there yet. That's something that an engineer could easily say like, "Yo, I know 26 is a problem. Twenty has always been a problem. Maybe we need to cancel that instance and bring up another instance or whatever." But it's something a human will recognize that a machine can't learn yet.

Hanna Park: I agree. I agree. Well, this has been great. It's been fun. Our positions have changed. You know, you interviewed me like back in June at MatterCon. So this has been so fun.

PJ Hagerty: So next time I'll interview you again. So we'll just keep flip-flopping back and forth.

Hanna Park: Okay. Thanks, PJ.

PJ Hagerty: Awesome. Thank you so much, Hanna.