Expert talk: What’s Next for .Net?

.NET has been undergoing a massive development since its very beginning and  Martin Thwaites, developer advocate for Honeycomb, and Hannes Lowette, head of learning and development at Axxes, enjoyed every step of the way. Join them as they reveal important milestones in .NET’s evolution as well as gain practical insights into web performance, running .NET at scale, and how to implement observability.

From .NET 1.1 to .NET 2.0: A personal perspective

Martin Thwaites: So, I'm Martin Thwaites. I'm a developer advocate for Honeycomb. I've been doing .NET for the past two decades since the start of .NET. And I'm here with  Hannes Lowette to talk about the evolution over the last two decades. Can you tell us a bit about yourself?

Hannes Lowette: Yes. I'm  Hannes Lowette. I work as the head of learning and development for a company called Axxes in Belgium. I've been doing .NET ever since 2.0. I've done 1.1 when I was in college. So, I've experienced pretty much all of the evolution of the ecosystem throughout my professional career. That brought us to a talk that we wanted to have about how we experienced that journey. Right? It's more about that than about the factual parts of everything.

Martin Thwaites: Starting back then ends-off as a college or school student doing .NET 1.1, do you see a lot of what you were doing back then in what you're doing now when we're in the sort of .NET Core and now the new .NET World?

Hannes Lowette: Well, a lot of stuff that we did back then, it still works. You can still do similar things with .NET, except you can also do a million things that you couldn't do back then. For me, the first reality check was when they broke all compatibility going from 1.1 to 2.0 because 1.1 is what we had in college. Then I arrived at my first proper job and they were going, "Oh, no. Don't do 1.1." Like, that was the broken version. They fixed a lot of it. "So, switch to 2.0. Here. Install Visual Studio 2002," or 2003 it was that introduced 2.0? Like, "Install that and work with that." Then I tried my code to compile, but, of course, all the name spaces were broken and a lot of the classes didn't exist anymore. It was like, "Okay, we're gonna start over." That was my first reality check. We had done that. It's like how I entered the work field.

Martin Thwaites: It was very similar for me. I remember the big green box of Visual Studio that I got. And I'm completely self-taught. At that point, that was my entry point.

Hannes Lowette: At that point, they were still shipping all the documentation with the box. You would get it on... Like, Stack Overflow wasn't a thing. It's like you got the MSDN documentation on DVDs when you got your Visual Studio box. And you could opt to use it from the DVD, or if you had lots of disc space and a powerful machine, you could install it to disk and get faster access to the docs. Right?

Martin Thwaites: Yes. That was a bit of a revelation when they did MSDN online, for instance, when you could actually just search it. That was that first experience of being able to install stuff off the internet.

Hannes Lowette: And you still see Microsoft's documentation efforts haven't really changed much. I mean, the documentation that they now have online, MSDN on DVD doesn't exist anymore. But they do keep their docs pretty much up to date with code samples and how to use things. You technically would not need Stack Overflow to find a certain feature of the language design and so on. You can find that in the Microsoft Docs and you'll have it in all the different versions of all the different frameworks if you want to.

Martin Thwaites: Yes. I think that's been a big push that they've made to lower that barrier to entry. Because previously, it was a very desperate bit of information that you would have to find. You'd have to go onto the MSDN Docs. I think they've made a massive push to build docs.microsoft.com as a site that is a go-to place for people onboarding. How do you feel that the onboarding experience has moved from when you were way back installing things from a CD through to sort of where we are now on that evolution?

Hannes Lowette: Oh, a lot of things happened in that period. Like, in the beginning, we were installing everything. Then what I remember is the way that you kept up to speed with things. You had a Visual Studio release which meant the .NET Framework release and all the features that were in there. You would get security fixes. But you basically had time until the next Visual Studio to learn all the new stuff right so that you were up to speed. I think they started decoupling that around 2010 with .NET 4 is when they pulled out the release cycles of a couple of .NET products, for instance, the MVC stack and the end of the framework stack. They were moved onto their own release cycles that were not paired to Visual Studio anymore. So, suddenly you were learning things all the time. And that has changed tremendously.

Also, one of the things that stood out to me is the pick-and-choose installer. In the beginning, installing Visual Studio was something that was complex and you had to know what you were going to do after you installed it, even when you were running the installer because you had to pick which stuff that you wanted to be installed with it. And now, I think it was...was it 2012? Or, no, it was 2017 when they revamped the whole installer where you could basically pick certain workflows and it would suggest like, "Okay, if you're going to do desktop development with Xamarin, you're gonna need these parts of Visual Studio." Right? All that kind of stuff became a lot easier.

Because there was this tendency of moving from smaller applications to bigger and bigger and bigger applications. And now that we're all switching back to interconnected collections of smaller applications again, you can see that the .NET ecosystem is also lowering the bar of entry to build those. Right now, with .NET 6, if you scaffold an MVC application, you're not going to get controllers anymore. That bar of entry is a lot lower than it used to be. I think that if we look at the ecosystem of a lot of programming languages, that is a really good thing because we're making it a lot easier for newcomers to get into .NET. Because we took all that stuff for granted. We've been there for the evolution. We've seen WebForms. And then, Web Forms was a wonderful attempt from Microsoft to transfer the skills that people had building desktop applications to the web. Right? It was just meant for that. It meant horrible things like ViewState and all the things that came with that that made our life hell and it made it pretty much impossible to performance tune. But it did do that for the .NET ecosystem to bring all these line of business desktop developers into the space of the internet.

Clean Architecture with ASP.NET Core 3.0 • Jason Taylor • GOTO 2019

Martin Thwaites: So, I would challenge that a little bit. ViewState was a thing with great power. It's a thing that was useful. It can be abused. And that is the big problem is that what we did at the point when we had WebForms was we made it easy to do the bad thing. We made it easy for you to take your customer object and serialize it into ViewState and have a 40-megabyte ViewState on your page. We made it easy for people to do. We didn't make it obvious.

Hannes Lowette: Exactly. That's the biggest problem. We didn't make it obvious what you were doing. And that's why a lot of people fell into that trap. It's not because they consciously abused the system, it's just that the system pushed the wrong attitude onto the developers. Right? This is why a lot of people hated WebForms. And when the MVC came along, which was a mirror of what in a lot of other web-first programming languages was happening at the time. That made a lot of sense, having this a lot cleaner server sites rendered HTML, but not necessarily thinking of, "We're going to do the same page over and over again and we have to transform the page loads and take data with us between them." You put that into the hands of the users. If you want to post something back to this controller, you're going to be making sure that it is actually in the form of data that is pushed onto that controller action. Right?

Web performance

Martin Thwaites: Yes. I think WebForms was trying to be so many things to so many people. It was trying to be that Swiss Army knife that allowed you to do absolutely everything. At the time when it came out, absolutely that was what people needed. As you started to move on, you mentioned 2017 and when we started to move towards this idea that people who developed for desktop, are not the same people who developed for web. 

That separation of that workflow within the Visual Studio stuff was a bit of a revelation really, where we started to understand that actually web development is very, very different to WinForms development, to building things for desktop, to building things for mobile. We tried to stop being one-size-fits-all for all these things. I think that was that moment where we started to think a lot about web performance and around this idea that using WebForms was great when you're doing line of business applications and you can write other applications in it. But when we start to think about pure web development where you are using a browser, we started to have the 2G, 3G, 4G, 5G sort of evolution of mobile. Where actually getting the page load times down was really, really important.

Hannes Lowette: Was important. It really was.

Lowering the bar of entry in .NET

Martin Thwaites: Thinking about controls, being purposeful about what you send back and forward over the wire, making developers really think about what's important to them. And have you seen that in the things that you've been doing recently where you've been working with .NET Core? Do you think that you think about the development a lot different now than just have everything?

Hannes Lowette: Oh, sure. Like, a lot of things have changed there as well. For instance, dependency injection was an afterthought for a long time. And in .NET Core, they made that a first-class citizen in the ecosystem, which brought a lot of the patterns and practices that people had been doing also in .NET for a long time. They brought it to the masses. Because the file Start New Project template, for a long time, if you did an MVC application, was let's just have everything in one assembly. It's really good for demo purposes, but if you want to build anything that a team is working on, you might want to maybe separate your code a little bit and make sure that you have clean dependencies so that you can get some of your logic tested. Right. That was not something they had in mind when they were doing these application templates.

I feel that they're cleaning up a lot of those things to reflect what the community is doing with .NET is being reflected back into the framework. They're also being pushed from the outside. It used to be this Windows-only. You had Mono, but it used to be this Windows-only framework built and run by Microsoft. Whereas in .NET Core, we see it runs cross-platform on everything by default. For the first time in two decades, we're seeing a third-party IDE to develop .NET. JetBrains released Rider. You can write it in Visual Studio code as well. You can basically write .NET in everything now and run the compiler tools with it. So, that is changing tremendously.

It is all coming from what are people doing with .NET out there? How can we bring that back into the whole system? Cloud was a thing. So, they needed to move cross-platform. Like, the templates that incorporated solid and dependency injection and so on, that was a thing. So, they are the default now. All those things are changing. What we're also seeing is there are so many ecosystems out there that have a very, very low bar of entry. Look at Node. Like, writing an API that responds to a request is only a couple of lines of code, wherein .NET, it used to be controllers and actions and whatever. It's like a whole...

Martin Thwaites: Bunch of files.

Hannes Lowette: It's like a whole...

Martin Thwaites: Package Jason's.

Hannes Lowette: A whole bunch of files and dependencies and so on. To do like that one thing it's like we're gonna respond to a request. And the fact that they have looked at that and reflected, how can we keep all of the power that we have in the ASP.NET Framework but lower the bar of entry so that if you want to prototype something real quick, or if you want to build something that isn't as complex as these huge enterprise services that we both probably have seen in our careers, but, like, a simple API that does a couple of things? You're gonna have something that has very little code. In my opinion, because I've been there for the journey, I've seen it all. For me, it never felt like we were doing too much work because you have these classes in your fingers. It's really quick to make it. But for a newcomer, it feels like, "Oh, I need to understand all these things" because all the code is there. Right? It's all C# file. So, if you've never seen .NET before, and you do the new project for a web project, you can see all the files and you wonder"What does this bit do? What does this bit do?" And basically, all of this is like default configuration for…

Recommended talk: Did We(b Development) Lose the Right Direction? • Stefan Judis • GOTO 2020

Martin Thwaites: Boilerplate.

Hannes Lowette: Yeah. It's like boilerplates to be able to answer an HTTP request. Now you just get a single file of code and it does just that. All the stuff is still there. It's still there behind the scenes and it's accessible if you want to do it. You just don't have to anymore.

Martin Thwaites: So, the minimal APIs is a really interesting thing because you and I both saw Twitter blow up when they started sharing the minimal APIs. The traditional 20-year .NET developers looking at this going, "I don't like it. It's wrong. This isn't .NET." And I was one of them. I looked at this code and I was like, "This isn't the .NET I know." And it took me quite a while to get around to that idea.

.NET Core, Standard and Framework

Hannes Lowette: But isn't it good now that it isn't the .NET you know?

Martin Thwaites: That was the point, that knee-jerk reaction that this isn't the .NET I know. But .NET needs to evolve. .NET needs to become more things. Now, what we didn't do is we didn't remove all of the stuff that already existed. You can still do things the way you want.

Hannes Lowette: You mean like the 2.0 to 4.8 journey, which was a decade and a half, of building stuff on top of, on top of, on top of? And the language specification basically didn't change. You had three different run times. Like, there's everything pre-4, and then there's 4, and there's everything 4.5 and up. But the language, they didn't remove anything. So, all of the stuff that was there when I did 2.0, like the first years of my career, I can still do that in a supportive version of .NET Framework today, and that code will still compile and run on the latest .NET.

Martin Thwaites: I mean, you wouldn't do that, obviously, because...

Hannes Lowette: There would be no point.

Martin Thwaites: ...nobody hates themself that much, obviously.

Hannes Lowette: No. I mean, the type of applications that we were writing was so different from what we're doing today that there would be no point to cross-compile anything from back then because those tools have long been replaced. I mean, I wasn't that good that anything survived through the ages that would still be running today. But theoretically, it could, but the fact is there is no point. And the big re-engineering came when they started thinking about, "We need this cross-platform run-time." It was like, "Okay, we have an opportunity to start over. Which bits of this old thing that we have here were good, and do we want to bring them over? Which bits could be improved, and we're gonna actually break compatibility there?" And then you get into this whole limbo of .NET Framework, .NET Standard, .NET Core. Then you now have .NET 5 and 6, which were actually .NET Core 4 and 5. but for reasons, they call them .NET 5 and 6.

Martin Thwaites: We can't call it 4 because 4 exists. So, actually 5.

Hannes Lowette: Yeah, 4 exists, but it was Framework 4 and not the Core. And they wanted to get rid of the Core thing and... See, this is the look that I get from people when they ask me, like, "What is this .NET Standard thing?" And then you have to bring them this whole explanation. It's like, okay, they had some bits that they wanted to bring over from Framework to Core. They wanted to have libraries that you could cross-compile to be used on both sides of the .NET ecosystem, which was very important for all the package makers and component vendors and so on so that they can build a single code base but run them on both sides of the...

Martin Thwaites: And you didn't have to cross-compile across multiple frameworks and have #ifdefs everywhere too, "Oh, if I'm running on the full framework, then I need this but with some other..."

Hannes Lowette: It was a good solution to the problem. But if you weren't there for that to happen, understanding how, like, Framework and Core and 5 and 6 and Standard how it all works together because now you have .NET Standard 2.1, which is not compatible with the full framework anymore.

Martin Thwaites: Yeah. But we should just be going to .NET 6 and then 7, then 8, then 9, then 10, then 11, and then Vista, and then 

Hannes Lowette: The thing is .NET Standard, which was like the way that you could write libraries that you could use in Framework and Core and Xamarin run times has now, like, ditched the whole framework compatibility in the latest version. So, even if you were used to the fact that, like, "Okay, I can make a standard assembly and have that on two sides," that's no longer true.

Martin Thwaites: I mean, let's be honest, versioning from 1.1 through to .NET 6 of where we are, and everything in between, has been a Docs defier. Now, there were multiple opportunities that we could have done it better, but we didn't. And that's the past. I think what we need to do going forward is just look at when we're talking to people about it, we talk about .NET 6 and it's just .NET 6. That's all that exists. Forget about everything that came before. Everything that came before was wrong.

Hannes Lowette: I think that's why they did the naming switch. That's why they dropped the Cores. Like, we want you to talk about this. And 5 is higher than 4.8., so you're gonna be talking about .NET 5 and .NET 6 and you're gonna forget about the old stuff. But we'll be seeing, like, Framework stuff in production for the next decade to come. I'm pretty sure.

Martin Thwaites: So, my biggest problem with the idea of moving back to .NET as the name of the thing that we're building against is hiring .NET developers. Because I've always seen the difference between a .NET Core developer, and a .NET developer. Now, we've seen the evolution and we talked about the evolution from a Web Forms-focused approach through to a .NET Core, controllers, MVC-type approach where we've got extensibility, we've got dependency injection, Standard. Everything is sort of built-in and we don't have to rebuild things.

Hannes Lowette: Well, we got extensibility quite early on. It's just not a lot of people liked it. If you think back to, like, .NET 3.0 WCF, that was like the most massively overengineered and extensible part of the framework that they ever built. And everybody hated it because it was complex and extensible.

Martin Thwaites: I loved WCF.

Hannes Lowette: I did a lot of awesome things with it. And once you understood...

Martin Thwaites: I did a lot of bad things with it.

Hannes Lowette: Yeah. Like duplex binding. So, I should never have used those. I mean, there are better solutions to that problem. But the thing is, it was a very, very, very powerful tool, and it lived on in its original state for quite long because it was so extensible.

Martin Thwaites: Which brings us nicely to backward compatibility because recently, Core WCF was released, which is a .NET Core version of WCF. And this is because we have so many people who are wedded to these technologies. They're wedded to SOP, they're wedded to this old-school idea of how these APIs work. Because like you said, we're gonna be dealing with full framework stuff for the next decade, at least.

Recommended talk: Expert Talk: gRPC, Kubernetes & .NET • Mark Rendle & Matt Turner • GOTO 2022

Hannes Lowette: Originally, they were only going to bring over the client-side of WCF. Right? We want our new services to be able to call the old stuff but we don't wanna build services this way anymore. That was the first philosophy. And then there was a whole part of the industry that revolted against that because they had built their own microservices frameworks on top of WCF. I wouldn't today. If I would have to do cross-process communication in a remote procedure call way, the way that WCF was built, there are way better options, for instance, in gRPC, which is way faster, it's a lot easier to set up. A lot fewer things that can go wrong. I dunno if you notice, you probably do because you know Mark as well, right? Mark Randall. He built, like, this whole conversion product that can take old full framework WCF codes and generate working .NET Standard assemblies that have gRPC services in them, and they don't break compatibility. So, you can basically remove your WCF services from your product, replace them with gRPC, and gain a lot of performance in the process. Because you're doing away with all the XML and all the SOP stuff.

Martin Thwaites: So, fun side quest on that one. When Mark was building that thing, he asked people to provide him with examples of WCF. I showed him the example of my very bad things in WCF and managed to crash his solution multiple times. Because it was so easy to do the bad thing in WCF. There were so many people who were, like you say, adding extensions, they were doing the things that you wouldn't imagine them doing. And then you try and then say, "Let's create a gRPC version." This is why converters don't always work is let's take that thing that was written in, you know, the "Hello, world!" example of WCF, convert that into gRPC, great, everything works. Take the 20 years developed WCF solution that somebody's got over here and try and run that through a converter and try and make that into gRPC. It's a really, really hard thing, which is why that tool exists because it's really hard to do. But it comes back to what I was saying about what we did in the past was make the bad thing really easy to do, and I think what we've done a lot of within the .NET Core world and now they've done that.

Hannes Lowette: Let's make it easier to do the right thing.

Martin Thwaites: Yes. Make the right thing the easy thing to do, and then people just go down that route. And I think we are starting to get this to this point now where we're training that out of people. We're training it out of people to say, "Oh, you want a logging framework? Well, don't write a logging framework. You've got ILogger. Just use ILogger."

Hannes Lowette: Yes. And then plug whatever logging framework into that.

Recent evolution of .NET Core

Martin Thwaites: It's one of my favorite things about .NET Core is this killing of that debate, you know. And one of the talks I give, the thing I say is when you used to start a project back in 4.8 days, the first 6 to 8 weeks of the project was deciding which logging framework you're using.

Hannes Lowette: Yeah. Like, logging framework, which DI container are we going to use? Like, all that. Which ORM?.

Martin Thwaites: And now you don't need to. Now what you start doing is saying, "I'm just gonna use ILogger. I'm just gonna use this out-of-the-box dependency injection. I'm gonna use the out-of-the-box observability solution." All of that kind of stuff is just out-of-the-box. And then later down the line...

Hannes Lowette: Running also changed tremendously. I mean, back in the days, it's like writing stuff to a file on the server that your application was running on. Right? So, you had to give your IIS service user, like, rights, access to a folder run disc so that it could dump its logs somewhere.

Martin Thwaites: But then you put it on a network share because then you had a server farm. And you have a network share between all of them.

Hannes Lowette: We've all done those things. Right. But now, I think, like, a lot of the new code that is being written is deployed in a containerized way. Whether you run it on Kubernetes or not is irrelevant. But if you're going to push it into the Cloud somewhere, chances are pretty high that you're first gonna build it into a Docker container and then run that.

Martin Thwaites: Either you run it in a Docker container, or somebody else runs it in the Docker container for you.

Hannes Lowette: Yeah. Because there are these frameworks out there and these orchestration platforms that just let you upload the zip with your code, but what they do is indeed like convert it to...

Martin Thwaites: I mean, just say app service in Azure, the Linux-based stuff is all building containers now. So, you are running it in a container. You might not be running it yourself, but you are running it in a container.

Hannes Lowette: You can just upload a container to an app service as well. So, that works perfectly. But what we do there is we have changed from this idea that our logs need to be in a file to the idea of we need to put our logs out there somewhere. And you can, either in containers, like, log to standard out and then, like, gather logs from there. The standard way of logging would be the way to do it with containers.

Martin Thwaites: Twelve-Factor App methodology. Right to out, get somebody else to do it.

Hannes Lowette: Right to out. And then like, ParaStat. Right? But what we've also been seeing, and this was something that was pretty big outside the .NET ecosystem before it came into it, you had these vendors like New Relic, for instance, that were looking at like, okay, we have these logs in these distributed applications and there are so many things that we want to know. We don't only wanna know what the developer thought that he had to write to logs, we also wanna look at, like, how long do these things take? Which database queries are being executed in the back end and take a long time? What is the load on the machine that this is running on? And, like, correlate all that data into, like, something that you can actually use to troubleshoot because reading logs is hard. We've all done it. If they give you that plain text file, like, finding out what's going on is a lot harder if you don't have context. If you add context to things, it becomes a lot easier to process, and you can actually figure out what's going on with your system.

Martin Thwaites: So, we're about to open the door into a ramp that I will probably spend the next four hours on.

Hannes Lowette: Okay. I don't think we have four hours, though.

Running .NET at scale

Martin Thwaites: So, you know, what you're talking about there, the idea that we are starting to now, in the .NET world, think more about our production systems. We're really starting to think now about it might work locally. It might work on our Visual Studio instance when we hit debug. But we now need to start thinking because we're running it in containers, we're running it in the Cloud, we're running it in server farms. We're running it, not on our machine, we're running it at load, at scale. Those are things that we really now need to start thinking about the operability of our platform. Do we need to start thinking about how do we get that information out of that platform where logging doesn't work on a file system anymore?

Hannes Lowette: It doesn't get us where we need to be. We know why we want to logging. We want to figure out what's wrong. But logging is maybe not the tool that we needed to solve that problem. And I think credit where credit is due, I think a lot of the awareness in the .NET ecosystem came from Microsoft including Application Insights into the default application template so that developers were pushed to go and take a look. They were at least triggers to go and take a look, like what is this all about? Why is this here? What can a platform like this do for us? And in the early days, Application Insights into a lot of things. But it offered you some out-of-the-box visibility on what your code was doing in production even without properly writing any... Because that was the thing with logging. If you wanna write something to the file, it's a conscious decision. Whereas, like, metrics and performance numbers and all that sort of stuff, you're not writing that out by default. But it's very interesting for you to know. So, you want to have more information than that, but you don't wanna spend all the extra work of instrumenting all your code with all those log statements, right?

Martin Thwaites:  I think that's where devs' tools come in.

Hannes Lowette: And that's where these tools come in.

Martin Thwaites: This is driven by the DevOps movement. I think .NET was behind in the DevOps movement. Where it was very much, I've done my code, I've pushed it up to SourceSafe because, you know, that's what we used to use.

Hannes Lowette: Yeah. I remember. Have you ever used ClearCase?

Martin Thwaites: I have not. I don't believe I have.

Hannes Lowette: Okay. It was like a...Oh, Keflin has, he's sitting over there. It came on a DVD because it was too big to fit on a CD. So, it was like a 900-megabytes source control system that was built for the Enterprise.

Martin Thwaites: Ah, for the Enterprise.

Hannes Lowette: Yes. It was glorious. Because it also came with ClearCase administrators. So, if you wanted to make a branch, you would call somebody up and they would make a branch for you, that sort of stuff. Lovely.

Martin Thwaites: But, yeah, you know, this idea, the .NET world was very much in this idea of, "Well, I build my application, I send it off to somebody else. Somebody else builds it, and there's a package, and then somebody else goes and deploys it." I think what .NET Core has done is brought those developers into this world where you need to actually care about your production environment. You need to care what it runs on. You need to care how it's gonna log its stuff out. You need to care about the outputs of this to the APM concern that you were talking about, where that idea of somebody caring about the request times, caring about the counters, and stuff like that. They really didn't, and now it is for them to care about.

Recommended talk: Nullable Reference Types in C# 8 • Jon Skeet • GOTO 2019

Hannes Lowette: For me, one of the big eye-openers was, at some point we were struggling with production performance problems, right? They came back with every release. We didn't have good visibility of what our system was doing. It was built as a distributed system. It had a bunch of like synchronism APIs, and had a bunch of queuing in there. We basically didn't know which code was causing it to be slow, which was a bad thing. And I started doing research, and it was in a time when Application Insights was still crap. But we convinced management to get us a New Relic license. And at that time, they were ahead of the game in a lot of the programming languages that they supported.

Martin Thwaites: They were the first to market with a lot of APM. Yeah.

Hannes Lowette: And we just dropped that. We included the packages and installed them on our servers. And that was basically all the effort that we did in configuring that product. We learned so much about the system that we had been building for a long time in those first two weeks that we managed to half the load on the database server by just performance tuning two stored procedures. Because that's what the system told us, like, "Hey, you're calling this a lot and it might not be as fast as you need it to be." Whereas, we always thought that it was not a stored procedure that was actually really slow. That was the one that we believed to be slow in our mind, whereas when we first were presented with the evidence, we learned that, okay, it's this one that we thought was really fast that's actually slow, and we can easily performance tune that. So, all this limbo of sticking a finger in the air and figuring out where your problems are coming from, for me, this was a turning point in my career because now, like, visibility is the first thing that I do when I set up a new project just to be able to know, like, when I drop this in production, I wanna know what it's doing. I wanna know where my problems are coming from.

Observability in .NET

Martin Thwaites: So, in Honeycomb, we refer to this as the MTT WTF.

Hannes Lowette: MTT...Okay.

Martin Thwaites: So, it's the meantime to go, and what the...is happening on this particular system? And that's what observability gives you. That's what some of the APM tools used to do. That was the table stakes now. That's your auto stuff. And I think what we're getting into now, especially in the .NET world, is people understanding that actually, now it's up to me to actually say what's interesting about my code and now do some manual instrumentation. And I think that's where we're heading now with all of those tools that were the auto things like let's just install the New Relic agent. Let's just install the app.

Hannes Lowette: Well, you can install instrument stuff manually because we were using on service. So, I instrumented all of the handlers, because that's like a relevant scope of processing something. But even that didn't take too much effort to do. Instrumenting codes is a lot easier than logging everything that you think might be interesting in the future to a logging platform. Whereas, you take the opposite approach, like, let's instrument, like, a lot of things, and then distill later what we need from this data. And that's stuff that we're really going to need if we move forward with these distributed systems. Because back when we started writing software, you would have a desktop application that would often contact the database directly. So, you had like a UI in the database. That was the complexity of the system. So, debugging and performance tuning, that was easy.

Now you have an authentication service and you have all these services that are calling each other, and they're synchronous calls going on and there are queues in between. and so on. And often, it's very nice to build on top of these Cloud platforms that provide you all of this plumbing code and this stuff that we had to configure manually as a service so that you can use it out-of-the-box. But it makes the complexity of the entire system a lot larger and it makes getting the observability right a lot harder.

Martin Thwaites: So, all of that stuff that you're talking about sounds incredibly complex. Do you think we're getting to a point now where everybody's overengineering everything? Because everybody believes that I need a token server, I need five APIs.

Hannes Lowette: The whole identity server discussion comes to mind.

Martin Thwaites: But do you think that's where we're heading?

Hannes Lowette: I think we are as an industry prone to over-engineering. My credo is still like as long as you can get away with it, just run a monolith. I know that's a very unpopular opinion in the industry, but I think it...

Martin Thwaites: Popular opinion in the room, but maybe unpopular outside.

Hannes Lowette: I've always been a firm believer in solving problems when they present themselves because you can pre-optimize for a certain problem that you think you're going to have in the future, but you're almost certainly going to have a different one. Right. So, I think, like, keep it as simple as possible. And that's why I really like that we're bringing back simpler services in .NET as well. But we cannot hide the fact that building on top of Cloud where you get a lot of your components as a service and the fact that the potential for the number of users of the average application has grown tremendously. When we were first building software, you were writing a line of business stuff inside an enterprise. These companies don't grow that fast. And when they grow...

Martin Thwaites: You know your users. You know where they sit, you know how many there are because it's gonna take six months to get through procurement for 100 more staff. So, you knew.

Recommended talk: :Observability Engineering • Charity Majors, Liz Fong-Jones & George Miranda • GOTO 2022

Hannes Lowette: And Moore's Law plays to your advantage because the company rarely grows as quickly as CPU power increases. So, what you could do is like, okay, we have this thing that we build and it's struggling, but we have, like, two years of server evolution, let's just buy a faster server. That's the thing we could do. 

With the web, that is no longer possible. You need to have scale-out scenarios. You're running on top of other people's infrastructure. Things are going to be more complex by default. Even if you're shipping a monolith in a container to Azure, there are, like, way more moving parts than you previously had. And it's not necessarily a bad thing because it has commoditized a lot of things that we used to spend tons of time on. Like, setting up a new project, as you earlier described, like bringing all the packages. I think setting up something new, including deployment pipelines and monitoring and everything, that's like less than a day now.

Martin Thwaites: Absolutely. Absolutely. That is…

Hannes Lowette: You can go from nothing to having source control, deployment pipelines, automated tests, everything. That is because of the efforts in the industry to commoditize all these things. And I think that's good. And the same thing is happening with the way we look at how code runs in production. That is also becoming easier. I'm excited to see what is going to happen in that space. I really like the fact that OpenTelemetry is catching traction, not just in the .NET ecosystem, but that we're seeing something that is going to standardize what we...the same thing that we talked about with ILogger, right? We have a standardized way of putting all the data into a system, and then we have a way of building all the monitoring dashboards that we want to have on top of that. And those two things can be decoupled from each other. That's something we've been lacking and that is now coming, and it's going to revolutionize how we look at our code in production, I think.

Martin Thwaites: So, let's round it out with, are you excited about what .NET is gonna be in the next decade once we have got rid of those decades worth of legacy applications we've been talking about?

Hannes Lowette: Oh, absolutely. It's running faster than ever. It's moving faster than ever. It's hard for old guys like us to keep up with all the new stuff happening.

Martin Thwaites: I'm not old. I'm not old, just to be clear.

Hannes Lowette: Well, old-ish. There's some grey in that beard. Yeah. No, it's very exciting times to be in .NET. I think they're moving in the right direction for a world that is also moving really quickly. Good stuff.

Martin Thwaites: Well, thank you for your time. And I hope somebody got something out of this. I certainly did.

Hannes Lowette: Happy to be here. And hope to see you at the next GOTO.

Martin Thwaites: Indeed. Well, follow us on Twitter.