How Team Structure Influences Code Quality

Does team structure actually have an impact on the code quality? John Le Drew and Adam Tornhill discuss the correlations between factors such as team size, structures, diversity and healthy retrospectives on both code quality and effectiveness.

Intro

John Le Drew: My name is John Le Drew. I've been working in software, in the kind of software industry in a whole bunch of different roles, for about 20 years. I spent a lot of that as a software engineer, and then in the last decade, I've sort of slowly moved from being more of a technical consultant to then being more of an agile coach, and then slowly moving far more into looking into the way we manage teams and lead teams of software engineers than necessarily the hands-on engineering. But I still do a fair bit of all of that, as well.

My talk this time was looking at collaborative product ownership. So it was looking at how we tend to have… I think it's a bit of this weird thing where, I think, in engineering, we recognize that having like a single engineer that knows all about all of the systems in every end and you have the bus problem that that's, you know, known as being a really bad idea, but apparently, it's completely okay, and it's apparently best practice even to have like a single product owner that knows all of the business knowledge for the application, but there's no challenge there. So it's kind of looking at how maybe the whole team can be involved with that product ownership and that you can share that role. So that was me.

Recommended talk: GOTO 2019 • The 5 Pillars of Collaborative Product Ownership • John Le Drew

Adam Tornhill: All right. So I'm Adam Tornhill, I'm the founder of Empear, where I develop code analysis tools. So that's one of my big interests. I'm also the author of "Software Design X-Rays" and "Your Code as a Crime Scene" on similar topics. This time, I spoke about prioritizing technical depth as if time and money mattered, which happens to be one of my big fascinations. So what I think I do differently when prioritizing technical depth is that I tend to emphasize organizational factors and social factors like teamwork over any properties of the code. So that's pretty much me

Recommend talk: GOTO 2019 • Prioritizing Technical Debt as if Time and Money Matters • Adam Tornhill

Does remote work have an impact on code quality and technical debt?

Preben Thorø: Has technical debt changed over the years? We tend to work more remotely and on a global scale now. Does that reflect technical depth in the code quality?

Adam Tornhill: Yes, I think it definitely does. So I've seen a very clear trend. So I think in general, with many software projects, they'll be getting larger and larger. We're building ever-larger software systems. 

What I tend to see is that we make many technical debt decisions, many decisions to re-design and re-factor based on properties of the code, technical properties. But I do think that when we're talking about software development at scale, factors like system mastery and team coupling, should also be drivers of re-design and refactorings to align the organization and architecture, and it's something I don't see happen that often, unfortunately.

John Le Drew: From my side, an area that I work with a lot is teams and their retrospectives. One of the interesting things you will see is the very, very common symptom of teams is that they go into their retrospective, they have their little moan about how awful everything is. They come out with some ideas for how they might change that. Then two weeks later, they go back into the room for a retrospective and they complain about how awful everything is. And they come up with the same set of things to change.

I find that the fascinating thing is, this goes back to actually the topic of your talk, is that you'll often have a, "Well, you know, we just can't get to, you know, fixing the build, or improving the speed of the build, or dealing with that area of really poor code coverage, or dealing with whatever the issue might be. We just can't get that prioritized over-delivering stuff." Then I ask, "But those things are the things stopping you from delivering stuff. They're the reason why you failed to hit that last deadline, isn't that right?" "Well, yes, but, you know, they just don't understand." "But you've told them that?" They're like, "Yes."

You'll have these conversations with product owners a lot of the time where you say, "Well, so team… you've been in the retrospective, and the team has told you, these are the things that they consider to be the most important things to them, the biggest reasons that are stopping them from delivering to your expectations, and you're not prioritizing those things." It's a very odd thing where there's a lack of trust, I think, between the engineering team, and I actually think that goes back to my talk, which is around the kind of historical links to Taylorism and Taylor and this basic idea of this division between the worker and the thinker, as Taylor would describe himself slightly arrogantly.

Adam Tornhill: Yes, I find that really interesting. So I've seen the same situation many times myself, of course. What I find fascinating is that… I have this hypothesis that the situation you described is much more likely in a larger team. Because I do think it has something to do with the team size. That’s when you run the risk of running into things like diffusion of responsibility, which social psychologists love to talk about.

John Le Drew: Yes.

The importance of retrospectives and psychological safety

Preben Thorø: One of the things I know you care very much about is psychological safety, that the team should be a safe environment for me, as a junior, to express my feelings and meanings about what we're doing. But still, I feel like doing retrospectives, it's like establishing that room once for a very short time every two weeks. If I was in a team, with all this in place, would I need these retrospectives?

John Le Drew: Linda Rising is also speaking quite a bit on the topic of continuous retrospectives. I think there's value in both things. 

GOTO 2016 • Continuous Retrospectives • Linda Rising

Something I've started doing a lot was I was getting fed up with a particular team… well, I get fed up with teams that moan all the time in the retrospective. So you go into a retro and they spend the entire two hours moaning about how awful everything is. A lot of the time, I found myself just saying, "But look, that thing that you're moaning about, like you knew about that, you know, nine days ago? Why didn't you start fixing that? Then why wait until now to deal with it?"

I  had an agreement with them that essentially when they raise an impediment if they face an impediment, we would stick it up on the board, and they would start to focus on it right then, so, whenever it's an issue with the test. So what was quite nice was after that, I don't know, somewhere between three or four weeks worth of work, we actually realized they came into a retro and they didn't have anything really negative to talk about, not kind of technical things, but some other stuff. But really, it was a really positive retro.

So we decided that the two weekly retro that we had would be an entirely positive occasion, after that point. We deal with all the negative stuff beforehand, and we use this as a two-hour session to talk about, as Woody Zuill would talk about… I think he was at GOTO Berlin this year, you know, turn up the good. That became a really nice cycle for that particular team, was that we actually began to change that. So I kind of agree and I disagree. I think that there's a real value in that cadence of a two-weekly retrospective of actually having the time to come in. And even if all you're doing is using that time to focus on celebrating your successes, that's really valuable.

Recommended talk: Let’s Make It Easy • Woody Zuill • GOTO 2021

Preben Thorø: But you can do it on a daily basis too?

John Le Drew: And you absolutely can, but there's something wonderful about making it a special occasion.

Preben Thorø: A ceremony.

John Le Drew: It's like you can tell your wife, your partner, that you love them every day of the year if you want to, but there's still something special about taking them out for their birthday or saying that on Valentine's Day. You make a ceremony out of it. I think that gives it more potency in some respects. But I definitely agree that in an ideal world, you almost make the session for the only point of fixing things, it becomes superfluous.

Dealing with difficult teams

Preben Thorø: Has any of you ever been in a team, where you said, "I've tried everything. This simply doesn't work out. I'll pack my stuff and I'll leave?"

Adam Tornhill: I've come very close a few times. Definitely. I haven't given up yet, but it definitely happens. In my experience that tended to be much more common, like 20 years ago when I started out with professional development. That was still back in the day of the traditional waterfall approach. And that could be extremely frustrating at times with the delayed feedback loops. So things are definitely… even though we might complain a lot about the state of software development, I do think we have made significant improvements in the past two decades.

John Le Drew: I completely agree. I think that even though there are some of the biggest challenges with kind of Agile adoption or moves to different mindsets, there are lots of companies, doing the ceremonies, doing the things, having the stand-ups, but not really operating in any way that could be described as operating with agility. While that's frustrating, even just the slight movement in that direction allows for a lot more breathing space than it did 20 years ago, certainly in my memory anyway. It's a lot better than it was.

Team structure and the quality of work

Preben Thorø: You often hear that we should strive to make our teams as diverse as possible in any sense. Can a team be too diverse?

John Le Drew: I don't think so. I think that diversity of thought is fairly critical to solving problems. The greater heterogeneity you have in any group, the greater your abilities to observe a problem from different perspectives you have. And as well, certainly, we're at GOTO, so these are engineering teams, you know, they're all doing creative problem-solving. So that's the thing you want to focus on. But, I would say that the skill that we seem to lack in most organizations is facilitation. You need good facilitation to support and create positive conflict because there will be positive good creative conflict in those settings. And if it's not well supported, and you don't engage people, you won't get anywhere near as effective an outcome as you might want to.

Preben Thorø: That's a brilliant way to put it, positive conflict. Adam, I see you as kind of 'Mr. Technical Dept' in a positive way.

Adam Tornhill: I created my own brand.

Preben Thorø: Have you ever done any statistics on how the composition of the team reflects in the quality of what we do?

Adam Tornhill: So, to a certain degree, yes. When I do my analysis and my research, I tend to focus mostly on the team level, where we rarely look at the individuals and how they compose into teams simply because I think the team level is most interesting. What I have seen is a very positive correlation with shorter lead times, throughput, and small teams. So that's one of the things I'm looking into more and more these days. 

I think a small team is much smaller than we would like to think. So what I see is that teams work best when they are as small as three to just four people because that helps you avoid a lot of, not only coordination overhead but also a lot of motivation losses that easily occur as soon as the team grows just a little bit bigger.

Preben Thorø: That's so small, it's almost not a team.

Adam Tornhill: It is a team. I think what tends to happen is that when you get a slightly larger team, we can speak about just eight, nine people, what you get it's no longer a team. You get a bunch of individuals with largely artificial organizational boundaries around them. It's incredibly hard to keep a team together of a larger size, in my experience.

Preben Thorø: Do you think companies should learn from that, from not just building development teams, but the way we divide companies into departments?

Adam Tornhill: Yes, I think that's actually vital. So all the stuff I've been saying with small teams, what that basically means is that the only way to scale is by having lots of really, really autonomous teams. In order to pull that off, you need to align it with, not only the software architecture but also the overall business structure of your company. That's an area where I think and hope we will see a lot of improvements over the next few years.

John Le Drew: I think that is a very interesting point around the smaller teams that you got me thinking. I've worked with a lot of companies where they have very, very large, very legacy systems. They will often have upwards of 100 people at least working on the system. Now, they're not in one big team as such. But what I've seen a lot of is, is how teams or organizations start to divide up that work and those people, you end up with epic amounts of friction between those teams, whatever that might be. That actually happens very naturally.

There's kind of an evolutionary bias for us, we like working in small groups and that's great. We have this kind of natural, very natural ability to form that. We form very cohesive groups very, very quickly. But there's, I think it's Stanley McChrystal in the book "Team of Teams," he's the guy that headed up the joint task force in Iraq. He talks about how when he started with them, essentially the squads were the edges of their equivalent of teams, outside of that, that boundary was where everyone else sucked.

So you're in a team and everyone else is terrible. I see that a lot. And something that I've seen is that the other challenge is that actually what you have is, is that, from a code quality perspective, if you're all essentially working on the same codebase, you've got 100 people kind of working on the same codebase, then you naturally have lots of integration challenges between the teams. 

Something I've experimented with, with some companies, is actually saying, "We’ll technically, rather than artificially split you all up and have lots of channels of work, we say, allow you as a team to group yourselves around the piece of work naturally."

So technically, there's one big team of 100 people. But the work comes through, there's a single backlog and they say, "Okay, we're going to form small groups of four or five people..." I'd say, often not much more than that, "... around this piece of work that might last a month or maybe even less than that." They kind of form and then they disband, and they move around that. 

I've actually found that to be quite effective. So you keep the people working close together quite small, but you lose that kind of issue. There's lots of rotation around those groups. So it means you don't have the issues of one team following a completely different coding style, standards than other teams, which sometimes happens, or other teams having very undisciplined approaches to testing before they push up to the build server and stopping the entire department.

So the team's got 100 people in it, but in reality, there's never more than four to six, I think, was probably the largest group we had working together at once for any length of time. But I've found that to be quite effective. I mean, saying that the codebase on that particular project was in quite a bad state, to begin with. So it was hard to…

Adam Tornhill: I think that's where we, as an industry, have a lot of learning to do, because we are pretty good at the technical aspects of software architecture, but we're not doing a particularly good job at organizing software architectures according to boundaries of the teams.

Preben Thorø: Absolutely.

Adam Tornhill: And when you have that misalignment, that's when things start to become really, really expensive.

John Le Drew: Was it Conway's Law, reverse Conway's Law, you know?

Adam Tornhill: Yes.

Preben Thorø: Yes, kind of.

Adam Tornhill: Yes. There's a lot of truth to it. You know, there's this really good research from, I think it's from Microsoft Research, where they point out that organizational factors like the number of developers that work on a parallel piece of code are better predictors of quality issues than any properties of the code itself.

John Le Drew: You mentioned my love of psychological safety. Obviously, Google did their Project Aristotle study and basically demonstrated that there are these five indicators around specific psychological safety being the critical one that demonstrated the kind of effectiveness of their teams. Now, this wasn't just software teams, but amazingly, when they were looking at software teams, if you have two teams, you might assume that Team A is doing really well, and Team B is not doing very well, that may be Team A had more competent engineers and more experienced engineers. But that wasn't true in any of their studies, essentially.

You could take a team of five, mid-level, or even junior engineers, and they could be more effective at delivering against expectations than in a team of five senior engineers that had low levels of psychological safety in the team and they weren't communicating effectively. 

I think that there's still a huge amount of research to do in that space because I don't think we really know. The work of Amy Edmondson and other people is looking into this.

But I think that we've kind of only scratched the surface of something that's quite important. To me, it’s incredibly profound to suggest that with research to demonstrate that you take a group of people that by all objective facts would say that these people are more effective and more experienced engineers than this group of people, that this group is more likely to be effective, even though they had less experience for a bunch of apparently unrelated traits. It seems kind of mind-blowing to me.

Adam Tornhill: Yes, it's powerful.

John Le Drew: Yes

Preben Thorø: That's pretty interesting. I think we could go on forever here. We have plenty of stuff already. Thank you so much.

John Le Drew: Thank you very much.

Preben Thorø: It was a wonderful conversation.

John Le Drew: Thanks for having us.

Adam Tornhill: Thanks.