Adriana geeks out with Doug Ramirez of Uplight. Doug shares stories of his experiences being on the national ski patrol and how they can be applied to the software industry. Key takeaways include not falling prey to overly-complicated solutions, how standardization helps decrease cognitive load, and staying sharp by refreshing and retraining.
About our guest:
Doug Ramirez is a Principal Architect at Uplight, where he aligns his passion for software and the planet.
Find our guest on:
Find us on:
Show Links:
Transcript:
ADRIANA: Hey, y'all, welcome to Geeking Out, the podcast about all geeky aspects of software delivery, DevOps, Observability, reliability, and everything in between. I'm your host, Adriana Villela. Coming to you from Toronto, Canada. And geeking out with me today is Doug Ramirez. Is welcome, Doug.
DOUG: Hi. It's nice to see you again.
ADRIANA: Nice to see you, too. Where are you calling in from today, Doug?
DOUG: I'm in central Virginia. I live in a small town called Crozet. It's a French word. It was named after a guy who was an engineer and built or architected a tunnel through the Blue Ridge Mountains, which is right in my backyard here.
ADRIANA: Oh, cool.
DOUG: Yeah, small town, pretty cool. Come visit.
ADRIANA: Awesome. And I'm excited because we're in the same time zone. A lot of folks that I interact with, even for this podcast, are like west coast. I've had a few in Europe, but, yeah.
DOUG: So it's like, yeah, I'm constantly doing more and more these days just doing time zone math because there's...
ADRIANA: Oh, yeah, right?
DOUG: ...the proliferation of telecommunications.
ADRIANA: Very true. Yeah. I even know now what my time zone is in GMT.
DOUG: Wow, that's impressive.
ADRIANA: Yes, I finally remember it now. Okay, well, let us get started with the lightning round questions. Are you ready?
DOUG: I'm ready.
ADRIANA: Okay, first question. Are you a lefty or a righty?
DOUG: I'm a righty.
ADRIANA: All right. Do you prefer iPhone or Android?
DOUG: iPhone. Yeah. We're definitely an Apple family here, so right, wrong, or indifferent? We're in the iCloud.
ADRIANA: I feel you. Yeah. I think my daughter has never used a Windows machine.
DOUG: Oh, interesting. Yeah, my kids have. So they have Apple machines, laptops, but they also get issued machines from school that are Windows. And they're always asking me for help. I'm like, where's the start button?
ADRIANA: I know, right? Yeah. I haven't touched a Windows machine in so long. At my daughter's school, they give Chromebooks for all of high school.
DOUG: Oh, interesting. Yeah.
ADRIANA: But my daughter refuses. She's like, no, I'll stick with my MacBook Air.
DOUG: Yeah. Honestly, I don't think my kids even think about it in terms of a preference. I think that they're accustomed to phones and tablets and laptops, of being different varieties. So I think it's normalized for them.
ADRIANA: Yeah. It's funny, when I was growing up, it was Windows or DOS. Linux was not a thing. You wanted Unix, go to a server room somewhere. Like, desktop Unix? Ha ha! Or like one of those Sun SPARCstations workstations that we used to have in my school, my university.
DOUG: Wow.
ADRIANA: Not high school, but university. We had Sun SPARCstations.
DOUG: I've worked on those before. I've actually written code on those.
ADRIANA: My first foray into the world of Unix was like my first year university computers class and I didn't even know there was other operating systems besides Windows. And I go into the computer lab, I'm like, what is this? Staring blankly at the screen and it was like, I think the C-shell prompt. It was like a percent sign. I'm like, what do I do with this? And they're like, "Oh, just type pine. You can see your email." I'm like, "Oh cool."
DOUG: Oh my God, I forgot about pine. Wow.
DOUG: My first professional experience with Unix was actually writing some code on a PDP-11 at General Electric. PDP-11 is like one of the old machines that was very popular at that time, but it was old when I was at GE and I just felt like I had reached this level of accomplishment in my computer career when I got given an account on the PDP-11 and I could log in and throw a little C at the compiler and do something fun.
ADRIANA: That's awesome. And now look where we are.
DOUG: I know. It's so amazing.
ADRIANA: Cloud. Provisioning infrastructure through code. Like, what?
DOUG: Yeah, I still actually do kind of have these moments of awe and giddiness, I guess to a certain degree when I think about how cool it is now. Like the things that we can do is just phenomenal compared to what it used to be.
ADRIANA: It's mind blowing. My dad, he turned 70 this year and he learned how to program on punch cards. He's evolved along with the technology, like for funsies. Last year he decided to learn Rust and back in the day he learned SmallTalk and he was like, object-oriented programming is the bees knees. And then a few years later, after all the hype, after Java, he's like, Java and object-oriented programming is an anti-pattern. I think it's kind of interesting because I think people who are more established in their careers might not remain so technical later in their careers and he's remained, I would say, ahead of me in terms of learning programming languages in that sense. And it's kind of mind blowing to see. I kind of feel like I see the evolution of computers through him as I grew up.
DOUG: You know, it's interesting to think, know, like you mentioned, OOP. And I was, I was sharing these stories with some folks that I work with recently and this idea of Gen AI and what's going to happen and I tend to lean back into my experiences which have always been pretty positive when it's been as a professional developer, through Y2K. Like, I remember how scary that was. They're different things, right? Gen AI and that. But just this idea of object-oriented programming was a thing that came about and it really did kind of catapult people's ability to deliver software solutions faster. And I remember when component based, VisualBasic and Delphi, some of these other tools that came out, right. It was this idea of like, well, developers are not going to have a job anymore because you just pull these things off of a palette and drop them onto a canvas and the software builds itself and even things like rational rows, you just draw circles and lines and this thing will generate all this code for you, you don't need. I think Gen AI is different, right? I think it is a different thing.
But I do think that at the end of the day, what I think is going to happen, and what I hope is going to happen is that it just is going to give us the ability to invite more people into this craft and develop solutions faster. I don't see it replacing developers. I think it might create the opportunity for more people to learn how to express sets of instructions through a language that a computer understands, right?
ADRIANA: Yeah, definitely. I see it very much as an aid, right? Let it take care of the stuff that isn't as interesting and let's focus on the things that are more interesting. It kind of relieves a little bit of that mental burden. But I think it also makes me think back to a conversation that I had with somebody earlier in the podcast where it was almost like a caution because I think they were saying that it's very important to have that base knowledge as well, right? So that you don't take it for granted, right? Rather than don't rely on AI as a crutch. Still have those first principles, fundamentals, and then it can really give you those powers because I think that's the risk that we can potentially run, especially like kids who are growing up in an AI-powered world where they might not have that incentive to learn the fundamentals as a result.
DOUG: Yeah, I think that is one of the main challenges that we face, right? Just being responsible with it. Yeah, we'll see.
ADRIANA: Yeah, indeed. Okay, next question on our list. What is your favorite programming language?
DOUG: I would say my favorite programming language right now is definitely Python. I've been doing this for a long time. I started writing software when I was a little kid, as a matter of fact, in the late '70s, and I was just a very unique opportunity or place to be where I had two parents who were educators and we had access to computers before most people didn't. Although the computers were large, like mainframes that were on the grounds of the University of Virginia where my dad worked. And so I really have written in so many different languages. I think it's like 30 different languages I've written code in. And I've gone through these periods of my career when I've thought about software in the language that I was writing in. Like, I can think back to when I was at General Electric.
I used to think about software in C, and also I would dream, and sometimes I still do dream about software and trying to express something in a language. Anyway, so it was C for a while, it was C++ for a while, Java, C#, some smattering of languages in between. But I definitely now am at the point where Python is my go-to. So if I ever want to do anything outside of work, let's say if I just want to write a script or, I don't know, just have, like, I'm usually pulling repos from GitHub that are written in Python, and that's kind of the language that I prefer. I'm still able to read code in a lot of other languages, but I think that I'm at the point now where the half life on, let's say, my C++ experience is such that I would probably have to RTFM and take my time and ask copilot for some help.
ADRIANA: I feel you. I did Java programming for a really long time, and I found that if I was ever away from it for a stretch, even if it was like a couple of months, it was always one of those cases of like, oh my God, I don't remember how to set up my JVM or whatever. How do I do this again? How do I find the length of a string or whatever the hell? Like how do FOR loops work again?
DOUG: Yeah, I know, that's funny.
ADRIANA: I was going to ask, you said you started working with computers at a really young age. What was your first programming language?
DOUG: I would say BASIC. BASIC for sure. Yes, I taught myself basic when I was a kid. My dad had some books around, and back then, I guess the body of knowledge was just so small. The books were very prescriptive. It showed you everything you needed to do. And they were dry. They weren't really about programming. It's just like, let's build.
ADRIANA: This is how you do this.
DOUG: Ten pages of all the things you have to do.
ADRIANA: Yeah.
DOUG: And I learned a little bit of assembler. I taught myself a little bit of assembler when I was in middle school just because, I don't know, I think my dad told me that assembler was, like, the language you really need to use if you want to do something with the hardware of the computer.
ADRIANA: Right.
DOUG: And so I just remember getting to the point where I could have a small bit of assembly code that would turn the screen blue or red. And I just thought that that was the coolest thing ever. And I was a little kid when I was doing that.
ADRIANA: That is so cool. I think everyone who's learning how to code should learn assembler because it gives you an appreciation for memory allocation.
DOUG: Yeah. And I think for me, it gave me an appreciation for garbage collectors when they first came out.
ADRIANA: Yes, that's the other thing.
DOUG: I was definitely one of those people where it's like, there's no way I'm letting something else decide when memories being reclaimed. Like, I'm not letting it do that. And then after a while, I remember thinking that this is actually kind of cool. I don't have to worry about allocating and deallocating memory anymore. Again, it goes back to this idea that, like, you know, I'm. I'm lucky that I have been able to be...been around and seen...
ADRIANA: Yeah, yeah.
DOUG: ...these...these incremental steps and, you know, the evolution of software and, like, garbage collection was one of them. Like, man, now I can spend. You know, I can. I can get that 33% of my time back. For some people, it might have been 50%, some people might have been 20%, but still, it's like, I don't have to do that thing anymore, and I can still focus on my craft and I can still solve the problems. This is a win for everybody.
ADRIANA: Yeah. In some ways, it kind of even ties to the paradigm shift of, say, the DevOps movement or automation in general, right? We're talking about people are so afraid, oh, this is going to take my job. And it's like, no, you get to focus on the cool shit now. This is where it's at. And it's the same sort of idea, but you still understand fundamentally how it works, which is really important. I remember I learned C in university, and then it wasn't until I learned assembler for a course in my fourth year where I'm like, oh, I get it. I wish I learned assembler before I learned C, because I think I would have had a much better appreciation for all the things. And I think that's the type of thing that I feel like that's the kind of knowledge that we still need to impart on folks in school.
DOUG: Yeah, you know, it reminds and on and off for the last few years I've been trying to finish this book called Code by Charles Petzold, I want to say. It explains the history of...it basically says, okay, today we have computers that do this, right? Yeah, let's go back into time and he walks you through the evolution of languages and numbers and basic math, and eventually you get to the point where you start to understand how computers really work.
ADRIANA: Right.
DOUG: Like this idea of gates and relays and switches. Computers are physical, but the idea that you can build like a very rudimentary thing that would show you how to make a decision and you could walk up to it and touch it. I don't know if everybody needs to know that level of detail, but I do appreciate it when I'm talking to other engineers and they can have that mental model. In a cosmic way it lets you kind of bond.
ADRIANA: Totally. I love it. Okay, next question in our series is do you prefer dev or ops?
DOUG: Do I prefer dev or ops? I would say dev for sure, although I have lots of opinions about ops.
ADRIANA: Feel free to share.
DOUG: It's interesting. I don't know what everybody's experience is like, but I feel like when the ops movement came around DevOps, I think some organizations got it right and the ones that did really achieved massive benefit from.
ADRIANA: Yeah, yeah, totally.
DOUG: But in my experience, I feel like leaning back again in my history, like the idea of a DBA is there. The DBA is ostensibly there to help you go faster, right? It's kind of like, what is it in team topologies when you have the group of people that are kind of the experts and, man, I can see the book on my shelf. Anyway, they're like the consultants, they're the experts there. They're just going to make you go faster. But what inevitably happened is the DBA just became a bottleneck, right?
And I'm kind of aging myself out here because I think the idea of DBA is going away. That's probably for another conversation. But in any event, I feel like that's kind of happened with Ops and DevOps, where this idea of like, we're here to provide platforms and tooling and we're going to abstract all this stuff away from you so that you, a developer, you can go flat out. You're in your flow state and you're just shipping code. But I think more often than not what happens is that the DevOps and Ops organizations often become a bottleneck.
ADRIANA: Yeah. Which is ironic because they were meant to take away that bottleneck.
DOUG: Exactly. Yeah. I feel for the folks that work in these arenas because as I see more and more companies moving to the cloud and more and more companies being in multiple clouds, their jobs have gotten really complicated. And the cloud providers do provide a lot of stuff out of the box. But if you look at most DevOps people and the skills that they have to use on any given day, particularly for multi cloud organizations, it's a long list of things that they have to know and the context switching between this tool at this cloud provider, this tool at this cloud provider. I can appreciate how that can. It's a lot of toil.
ADRIANA: Yeah. It's like a bigger operational burden on the operators, basically, whereas the devs are still sitting pretty doing their thing. I was actually having a discussion earlier with someone where I was saying how for me, I find it funny that I started in a development background and then for me, I made it my business to learn how to package containerized applications and deploy them and know how to deploy cloud infrastructure, IAC, all that stuff. But for me, it was a bit of mind boggling to realize that that is not a thing that most developers concern themselves with, kind of by design. And I thought that was kind of funny because I'm like, I don't know, I like this stuff. I want to know how everything works end to end. Yeah.
DOUG: You know, it's, you know, for both the, for both the Ops, DevOps orgs and the developer orgs, like the one thing that I am becoming more and more convinced about is just this idea of standards. You and I were talking earlier and I mentioned the fact that I'm on the national ski patrol. I think there's some really amazing things that the ski patrol has kind of figured out about how they train and trained patrollers and how patrollers operate and deal with emergencies. And I think about this a lot when I think about DevOps and developers and kind of the struggle that often exists. The thing that the ski patrol does really well is it has some very highly prescribed tools and processes, like the types of splints that we use and how you apply. The sprint is very prescribed to the point that, and I've actually done this before, I was not patrolling, but I came upon a car accident and somebody else who was an EMT was there and we were able to essentially stabilize this person and get them to be handed off to an ambulance.
ADRIANA: Yes.
DOUG: And what was interesting is that at the end of dealing with the acute emergency. When the ambulance left this guy, we kind of shook each other's hands like, man, I'm so glad we're both here.
ADRIANA: Yeah.
DOUG: And we were kind of laughing because as we were moving the patient around and putting them on the backboard and applying a C collar for neck mobilization, we never said a word to each other. We had been trained on the tool.
ADRIANA: Right.
DOUG: Like the backboard, the spider straps, and the C collar. And we knew the process like we knew which straps to apply first.
ADRIANA: Right, right. And you didn't know each other.
DOUG: Never knew each other. Never. Nothing. And so what I've been preaching a lot at where I am right now is I've been thinking about this stuff with the national ski patrol, and to me, that's a beautiful example of why standards and conventions really do help everybody, right? If we were to say, for any low latency OLTP concerns, we are going to use Postgres. If you think about the ripple effects that that has on reduction of cognitive load, it becomes really quite interesting. Think about disaster recovery, right? Your database choice informs your ability to recover quickly in the event of a disaster in a significant way. Right?
ADRIANA: Yeah.
DOUG: The reason why I came upon that accident and jumped out of my car and was able to help that person, not to say that we saved their lives, but we probably made them more comfortable and got them to the next level of care. We were able to do that so efficiently. There was no cognitive load like this person and I were focusing on solving the problem of this person who had been in a bad accident.
ADRIANA: Right.
DOUG: And when I think about, to me, it's like, okay, that's interesting. It's a really interesting organizational problem. Why was that beneficial? Oh, the standard tool and the standard process removed all cognitive load. What if we did that at work? What if we said, hey, you know what? Let's just...I mean, we can argue about SQL Server, Oracle, we can argue about it, but if we pick a tool, then the unintended but kind of intended consequences become material. Like, all of a sudden we have one backup tooling. We have one recovery tooling. If I get paged in the middle of the night and it's because my coworker's out and I have to jump into a sev one, I know how to backup restore postgres. Right. Like, if it were Cassandra or something else, it'd be like, I have no idea. Anyway, I'm obviously pretty passionate about this.
ADRIANA: I love this so much. I have so many follow ups to this. Yeah. Because it reminds me because at the previous... and we know each other from a previous organization where we worked together, and I do recall that at this previous organization, it was more of a, like, you work with the technologies that you're most comfortable with, sort of approach to things, which I think can be very interesting and effective until it stops being effective. I think especially if you're starting to grow the organization to a significant size. I definitely learned both there and at a previous organization that being able to standardize and to standardize by codifying things really went a long way, because to a certain extent, we got to save people from themselves.
DOUG: The codifying things...here's another great example. In the ski patrol, it is interesting, this idea, just to kind of zoom out a little bit, this idea of the developer experience and developer creativity. We want small autonomous teams because our hypothesis is that collectively they will deliver more value. So we talk about things like full service ownership. You build it, you run it. These are wonderful ideas. But at the end of the day, the software organization isn't there to satisfy all their curiosity.
Most places they're there because they have a mission, a purpose, a goal, and oftentimes it is to make money. So one of the things that, another thing we do on the ski patrol, which I love this idea, and this kind of goes back to this idea of like, standards creating an environment where you still have autonomy and creativity to solve problems. We all know that there's a lot of creativity in software engineering, even though at the end of the day, it's just a set of instructions. I think there's a way to allow for both. But I think that it's important for software developers and DevOps people, all of us, right, to understand that the things that we might be locally optimizing for are not part of the global optimization, right? Like my desire to introduce Rust into a service because I think it is better, and it could possibly be better for that service. It's not in service of the larger goals. So on the ski patrol, if I'm on scene with somebody who's been badly injured and they have to be taken down the mountain, they're not able to ski or snowboard anymore. We put them in a sled, a toboggan.
Whenever a toboggan arrives with another patroller, there's something called a sled pack. And the sled pack has the tools that you need to probably address the person that's injured who has to be put into a toboggan and sent away. So we have some blankets, some splints, some devices to move people around, things like that. Every time the toboggan arrives, when I open up the sled pack, everything is there and it's always in the right order. And, you know, it's to me it's like, it's another fabulous...I'm sure that if I went to any other mountain in the US, and if I was a bystander and somebody said, hey, can you help? I'd be like, yep, I'm a member of the SB, blah, blah, blah. The toboggan arrives and I go to open up the sled pack, I guarantee you it'll be all those things in order.
ADRIANA: Yeah.
DOUG: And when I was getting trained...sorry, that was my cat.
ADRIANA: That's all right!
DOUG: When I was getting trained on the ski patrol, it never thought to offer a different solution or to suggest something different or to sneak in a different thing. Because I think part of the training is like, the whole idea here is to remove the cognitive load so that when you're in a crisis, you can focus on solving the problem, which is making sure the person can breathe, they're not bleeding to death, and that anything is not straight. Gets straightened. For me, when I think about this at work, it's things like, all right, if I'm going to go look at a microservice, and we've all agreed that we're going to use Python and FastAPI or Goengen, right? Every repo I go to, I want it to be the sled pack experience. I want to clone the repo, and I want to open up the top level folder, and I want to see all the elements that are there, right. And they should be immediately recognizable. And it's interesting, if you talk about these ideas with some developers, they're kind of like, well, that seems overly prescriptive. Why would you want to do that? That might not always work. Valid arguments, but I'm always like, well, what's the benefit of having it different every time? And I go back to this idea also of that code is read, I think, like 100 times more than it's written. Like, the time human beings spend with code, it's mostly reading it.
ADRIANA: Yeah.
DOUG: So why not make it easier for the next person? Like, agree on a standard. We'll use Postgres, agree on a convention. These are folder structures for our microservices. To me, that creates that sled pack experience. Like, I'm on scene, this person's really hurt. I've got to get them to a next level of care immediately. I don't want to think about opening up the sled pack and being like, where's the litter? I don't see...why isn't the splint here? Because, you know, Doug decided that there was a better way.
ADRIANA: Yeah.
DOUG: And he liked that idea. Having said all this, though, the one thing I will say is that going back to some of my other stories is I'm old school. My first job at a school, there was never a conversation about languages or naming conventions. It was all written down. It's General Electric. They're very thorough company. Anyway. So I tend to find some comfort in thinking about things like, we have a language, we have a storage technology. We've got this mechanism for asynchronous communication. Just use it.
ADRIANA: Yeah. It's interesting because this is basically like a platform engineering problem when you look at it, right? Because you are looking to standardize across the board, not just the development experience, but the overall standardization of your infrastructure and all that. It's very cool. I think it's so great to have to be able to standardize on that, but also give some wiggle room for creativity. But not so much that you end up sort of ruining the flow, because I think that's really the thing. But I think then what helps make it effective, and I'm sure, especially when you're doing ski patrol, you have a guide, right? You have this documented somewhere how these things work. And I think that's where a lot of companies kind of miss the mark, is because these procedures, these standards, aren't necessarily documented or well documented.
DOUG: Yes. So here's the other thing that, another thing that I really appreciate about the ski patrol, national ski patrol, and something I really do want to bring into where I work. So getting certified to be a ski patroller involves typically two things. One is about six or nine months of classroom training where you effectively become an EMT. But it's called an OEC. It's outdoor emergency care. So you're trained on outdoor emergencies. So it's kind of the basics, right? Like, I think of it like computer science, computer information systems, whatever.
ADRIANA: Yeah.
DOUG: So you finish that up, you're like, okay, I've got a whole bunch of academic knowledge. Then each mountain will do what they call on the hill. So there's another couple of months of training where you are now taking kind of your academic experience and actually applying it. For us, it would be a cold weather environment, so we have classrooms on the slopes, and we practice things out in the cold, in the snow. And so the on the hill training is really important. And once you finish the on the hill training, you are very proficient. It's not like you're a noob. As a matter of fact, most people who come off of their, when they just get their certification of patrol, you almost want to go to those people because they remember things that you don't. Or I did.
ADRIANA: Yeah.
DOUG: Anyway, so to me, this idea of on the hill seems really important. And I almost envision, I don't know, every six months or maybe every quarter, we take our new developers, our new engineers across all the departments, and we basically run them through on the hill. So, like, this is our software development lifecycle, right? This is our documentation. We're going to give you some sample code, like some scenarios to run. Join us for a sev 1. So that basically after, I don't know, some number of weeks, let's say those people are very familiar with how that organization works. How does it ship code?
ADRIANA: Yeah.
DOUG: Right? So a very focused, very well thought out, thorough training program. It's on the hill training. Like you've taken your academic experiences or whatever you had before, and we're going to show you how we do them here at our mountain with our specific protocols. And then when you're done with this, the entire patrol knows that you're ready and you can go and you can work accidents and you will be trusted and you will be successful.
ADRIANA: Yeah.
DOUG: So the other thing that's related to this, too, is that another thing that I really appreciate about the NSP is that we have refreshers. So every year. So every year we get refreshed and we get retrained and retested and recertified on essentially a third of all the things you have to know as a ski patroller. So we go through cycles a, b, and c every three years. But the great thing about it is that, and this is something else I would want to do. So I would run a run cohorts of engineers through a refresher just to say to people like, okay, we know you know it, but let's to be like, let's go back through and talk about our SDLC. Let's have some people join some, like, let's understand what it's really like to work here and just recertify everybody on the process, to give people an opportunity to know that they can participate in the process so that we know that we're doing a good job and training people and making them be successful. So I really like this idea of this refresher that happens every year in the ski patrol, where you go and you spend some time rtfming.
Like, we all big medical manual that we read. We go and we meet people. It's usually in the fall, but we'll go to the resort and we'll do some dry runs of a lift evacuation. So we'll load people up on the lift and we'll evacuate them. So it's almost like, hey, let's pretend to have a sev 1 or a tabletop sev 1. That would be part of the refresher, like, so and so database clusters down. What are you going to do, right. And give people some time to kind of step back and lean back into whatever the company is doing and how it works so that people aren't under duress, if that makes sense.
ADRIANA: Yeah, and it makes a lot of sense because in many organizations, especially, like the really large ones, you already do things like DR testing, which in my experience, it's a very important thing to do. But I will say that we did it twice a year, I think, at one of the organizations where I was at, and everyone's sort of, like, rolling their eyes, like, oh, why do I have to do this thing? Why am I the one who's pulled into helping out with the DR test? So I think finding a way to it almost like, in some ways felt like we were going through the motions, which kind of defeats the whole purpose of the whole thing. But then there's the other side of it, too. Oftentimes when new engineers are onboarded onto a company or even onto a different team, unfortunately, they're kind of just thrust in, right? It's like, oh, here, read some stuff, and then we expect you to be productive in a week.
Good luck to you. And that's the type of culture that we need to aim to change, because it can be very frustrating if you're new to an organization and especially, oh, my God, imagine you're fresh out of school. If you've done this enough times in your career, like, whatever. I guess that's just the way to the poor kid fresh out of school, they're like, what's going on?
DOUG: Yeah. It's curious to me because you'd think that the company, the business, the organization would intuit the value of knowing how to recover during a disaster. In my experience, I want to say there's only a handful of places that I think really understood that and made the investment and the training and the technology to be ready.
ADRIANA: Yeah. More than just going through the motions. More than just like, this was a compliance thing where we need to tick off this item, right? Yeah. I think the companies that care about doing chaos engineering tests and game days and that sort of thing. I think those are the organizations where you're probably going to see success in terms of low panic factor, perhaps during incident responses, right?
DOUG: Well, kind of going back to one of our threads from earlier, this idea of ski patrol, we use one splint to deal with femur fractures, a sega splint, there's other splints, but that's what we use. Going back to this idea of choose a database for all of your low latency OLTP concerns, maybe even HTAP concerns, hybrid transactional analytical processing concerns. But in any event, pick a database. Not only does it have this interesting effect on reducing cognitive load, but when you start to think about the idea of chaos engineering, if you're in an environment where you're storing data in, I don't know, Kafka, Cassandra, mongo, some relational database, another relational database. Right. Like chaos engineering starts to become incredibly daunting, almost impractical or impossible. Not impossible, but like, impractical. Again, when I think about the ski patrol, it's like there's lots of different splints. Everyone knows this one, and we are all trained on it.
ADRIANA: Yeah.
DOUG: And you can put it on and off blindfolded because it's in service of a larger goal, which is to help people and hopefully save their lives.
ADRIANA: Yeah.
DOUG: If you think about this idea of really trying to be keeping your technology stack simple and having standards and really adhere to them, the unintended consequences of that start to bleed into all these other concerns. Retention, recruiting, hiring.
ADRIANA: So true.
DOUG: I love this idea of where I am at uplight right now. We have a lot of python experience. We have a lot of fast API experience. It would be easy for me to turn to another team and say, can you help for like two weeks, one sprint? And most of the people around me would be like, yes, I can do it. They know the language, they know the process, they know the framework. And I wish more organizations saw that benefit, and I wish more businesses would lean into and kind of lean into engineering to say, we're running a business, we have a mission. We understand that you have this creative thing that you do. We don't know what it is, but you build stuff for us, keep it simple.
ADRIANA: Yeah.
DOUG: Because we have these larger goals that we need to meet. And part of it is hiring people, recruiting people, having agility within our software teams. We want to have small autonomous teams that come together, build a solution, and then they dissolve, like, over and over and over again. And the way to do that is by having those standards, picking good tools, using them until they can't serve you anymore, which for the most part is never the case. To me that is a real interesting recipe, like taking those concepts from the ski patrol. I just want to inject them into software organizations.
ADRIANA: Yeah, it makes so much sense to me because then when you strip everything down, you're left with, I get to solve cool problem. And the other thing too is you can apply that to another aspect that kind of is like the bane of every developer's existence, which is like compliance and security, right? Having that standardization, working together with our friends in security, which doesn't always happen, but if you have the standardization in place, codifying the standardization, then again you're taking away the mystery and make it easy to implement the security policies is the other thing.
DOUG: Yeah, again, being the older programmer in the room, and admittedly a bit of a curmudgeon to me, just the simple technology stacks always win. Yeah, and they win because they help the people in process, people process and technology, the simple technology stacks make those other things come into focus and be successful. I see it over and over and over in my career. The simple solutions, the simple technology solutions always win.
ADRIANA: And it's funny too, because as developers we are drawn to shiny new objects and sometimes shiny things can be overly complicated. It even reminds me one time at a previous job, I was on a release engineering team and we were using Azure for our cloud and we were looking at possibly moving to Kubernetes, but we were using before Azure container services to manage all our microservices. And quite frankly it was doing a very good job. And yet they were looking at moving to Kubernetes because, well, Kubernetes come on, of course. But it was funny because in the end they made the decision of like, yeah, this thing's too complicated. Azure container services is doing the job. Shit's not breaking. We are happy. So why are we overly complicating the situation, right? Which I think was a very, for me in Kubernetes I was sad, but when you look at it very objectively, it made a lot of sense.
DOUG: Yeah, it's funny when I hear these stories about people using some very sophisticated, complicated things like Kubernetes and struggle with it. And I'm like, I have to remind myself, like, okay, we've shipped code before, right? Without all this stuff. I'm pretty sure I remember shipping code to a computer. So there's a way. But I feel like we've strayed from the path. Like how do we get back onto the path of, like, let's ship hello, world?
ADRIANA: Yeah.
DOUG: And keep it simple.
ADRIANA: Yeah. And I think that is the key, right? Keeping it simple because we like to overcomplicate for the sake of, because we want to solve a cool problem. I think at the end of the day, we're engineers. We want to solve really cool problems. So let's find a complex way to do it. And then you see the simple solution. You're like, oh, yeah, I guess.
DOUG: I think that's an interesting way that you've described that we have to go against our own nature. Most of us got into this because we're geeks. We're nerding out, and we're learning new stuff, right? All the time.
ADRIANA: Yeah.
DOUG: But it's almost like now you have to step into this work environment and say, I'm going to have to deny my true nature.
ADRIANA: Yeah.
DOUG: And I'm just going to, to a certain degree, not assume that everything's a nail and you have a hammer, but you almost want to say, like, okay, I've got four tools in this little toolbox. I've got to keep it simple. I need to focus on the larger goal here. What is our goal? We need software to solve this problem. Great. I've got these tools. I'll keep it simple. But for so many of us, it's like, but that's not why I'm here.
DOUG: I got here because of my ability to pick up new things and be curious and dig and learn.
ADRIANA: Yeah. And so you're like creating a new challenge for yourself so that you can kind of keep the brain active.
DOUG: Yeah.
ADRIANA: It actually reminds me.
DOUG: Sorry, have you heard that before, developer catnip?
ADRIANA: No, I have not.
DOUG: I heard that from somebody I work with about a year ago. I hadn't heard it before, but I thought it was just a great expression, this idea of developer catnip. Like we see something and we smell something and then we're all into it. It's like, no, go back over here.
ADRIANA: So true. It actually reminds me of something I experienced fairly early in my career where we were writing code to convert some stuff and pop it into an oracle database. And the vendor we were using, they had a Java API for it. And I was super excited. Like, Java was still new. This is like early 2000s. I'm like, oh, this is awesome. Great problem to solve. And then we're running some performance tests and the Java code was slow as fuck. It was ridiculous. But they also had like a PL/SQL API. And so we started doing some tests running the same kind of code in PL/SQL and it runs in the database. Of course it's going to be faster, like Java, extra overhead bloat. And I remember being so sad. I was like, I have to learn this crappy language. I'm going to be like, I'm not going to have. But objectively thinking back, it had to be that solution. Otherwise the PL/SQL code was so much more performant than the Java code. And I had to put my pride aside and suck it up because that was the better solution. It was the more logical solution.
DOUG: Wow. Yeah, that's awesome. I'm sitting here thinking like there's definitely cohorts of developers who have had a schema handed to them in an ORM and might not have ever really written SQL.
ADRIANA: Yeah, I feel like SQL was like my life for the first chunk of my career.
DOUG: Yeah, definitely.
ADRIANA: I don't know if I can write the complex SQL that I used to back in the day, but it was fun while it lasted. Yeah, cool. Well, we are coming up on time, but before we go, I was wondering if you have any final parting words that you would like to share with our audience.
DOUG: I would be curious to hear from other people if they have experience where they've either as a hobby or a volunteer or a second job. I would love to hear other people how they could take something that they're doing. The national ski patrol and software engineering. What in the world do they have in common? And I think the answer is quite a bit. And I think that the software engineering world could learn a lot from the ski patrol. So I guess my parting thought would be very curious to hear from other people. Like, do you have a former job or some experience where it just seems orthogonal to what we do, but actually could help.
ADRIANA: Yeah, so true. And I've talked to a bunch of people about that. Like even someone, they were in incident response. They used to be in the service industry, like work at restaurants. And she was telling me how she had to learn to manage customers and stuff, deal with them and have unpleasant conversations and how so many parallels between that and what she did in incident response. And I'm like, after we finished recording the podcast, I'm like, you have to write a blog post about this. Which she did. But yeah, it's great that let's draw more on our real life experiences and find parallels in the software world because it's so relevant.
DOUG: Yeah, for sure. I think if we can be humble and not just assume that because we're the nerds that we know the answer.
ADRIANA: Yes, absolutely. Yeah. This was really great. I'm so glad that you brought this topic up because I think it's so relevant. It's very eye opening and kind of sobering in a way. Let's take a step back and evaluate. What are we doing here?
DOUG: Yeah, I think that's the beginning of the adventure.
ADRIANA: That's awesome. Well, thank you so much, Doug, for geeking out with me today. Y'all don't forget to subscribe and be sure to check out the show notes for additional resources and to connect with us and our guests on social media. Until next time...
DOUG: Peace out and geek out.
ADRIANA: Geeking Out is hosted and produced by me, Adriana Villela. I also compose and perform the theme music on my trusty clarinet. Geeking Out is also produced by my daughter, Hannah Maxwell, who incidentally, designed all of the cool graphics. Be sure to follow us on all the socials by going to bento.me/geekingout.
ADRIANA: Hey, y'all, welcome to Geeking Out, the podcast about all geeky aspects of software delivery, DevOps, Observability, reliability, and everything in between. I'm your host, Adriana Villela. Coming to you from Toronto, Canada. And geeking out with me today is Doug Ramirez. Is welcome, Doug.
DOUG: Hi. It's nice to see you again.
ADRIANA: Nice to see you, too. Where are you calling in from today, Doug?
DOUG: I'm in central Virginia. I live in a small town called Crozet. It's a French word. It was named after a guy who was an engineer and built or architected a tunnel through the Blue Ridge Mountains, which is right in my backyard here.
ADRIANA: Oh, cool.
DOUG: Yeah, small town, pretty cool. Come visit.
ADRIANA: Awesome. And I'm excited because we're in the same time zone. A lot of folks that I interact with, even for this podcast, are like west coast. I've had a few in Europe, but, yeah.
DOUG: So it's like, yeah, I'm constantly doing more and more these days just doing time zone math because there's...
ADRIANA: Oh, yeah, right?
DOUG: ...the proliferation of telecommunications.
ADRIANA: Very true. Yeah. I even know now what my time zone is in GMT.
DOUG: Wow, that's impressive.
ADRIANA: Yes, I finally remember it now. Okay, well, let us get started with the lightning round questions. Are you ready?
DOUG: I'm ready.
ADRIANA: Okay, first question. Are you a lefty or a righty?
DOUG: I'm a righty.
ADRIANA: All right. Do you prefer iPhone or Android?
DOUG: iPhone. Yeah. We're definitely an Apple family here, so right, wrong, or indifferent? We're in the iCloud.
ADRIANA: I feel you. Yeah. I think my daughter has never used a Windows machine.
DOUG: Oh, interesting. Yeah, my kids have. So they have Apple machines, laptops, but they also get issued machines from school that are Windows. And they're always asking me for help. I'm like, where's the start button?
ADRIANA: I know, right? Yeah. I haven't touched a Windows machine in so long. At my daughter's school, they give Chromebooks for all of high school.
DOUG: Oh, interesting. Yeah.
ADRIANA: But my daughter refuses. She's like, no, I'll stick with my MacBook Air.
DOUG: Yeah. Honestly, I don't think my kids even think about it in terms of a preference. I think that they're accustomed to phones and tablets and laptops, of being different varieties. So I think it's normalized for them.
ADRIANA: Yeah. It's funny, when I was growing up, it was Windows or DOS. Linux was not a thing. You wanted Unix, go to a server room somewhere. Like, desktop Unix? Ha ha! Or like one of those Sun SPARCstations workstations that we used to have in my school, my university.
DOUG: Wow.
ADRIANA: Not high school, but university. We had Sun SPARCstations.
DOUG: I've worked on those before. I've actually written code on those.
ADRIANA: My first foray into the world of Unix was like my first year university computers class and I didn't even know there was other operating systems besides Windows. And I go into the computer lab, I'm like, what is this? Staring blankly at the screen and it was like, I think the C-shell prompt. It was like a percent sign. I'm like, what do I do with this? And they're like, "Oh, just type pine. You can see your email." I'm like, "Oh cool."
DOUG: Oh my God, I forgot about pine. Wow.
DOUG: My first professional experience with Unix was actually writing some code on a PDP-11 at General Electric. PDP-11 is like one of the old machines that was very popular at that time, but it was old when I was at GE and I just felt like I had reached this level of accomplishment in my computer career when I got given an account on the PDP-11 and I could log in and throw a little C at the compiler and do something fun.
ADRIANA: That's awesome. And now look where we are.
DOUG: I know. It's so amazing.
ADRIANA: Cloud. Provisioning infrastructure through code. Like, what?
DOUG: Yeah, I still actually do kind of have these moments of awe and giddiness, I guess to a certain degree when I think about how cool it is now. Like the things that we can do is just phenomenal compared to what it used to be.
ADRIANA: It's mind blowing. My dad, he turned 70 this year and he learned how to program on punch cards. He's evolved along with the technology, like for funsies. Last year he decided to learn Rust and back in the day he learned SmallTalk and he was like, object-oriented programming is the bees knees. And then a few years later, after all the hype, after Java, he's like, Java and object-oriented programming is an anti-pattern. I think it's kind of interesting because I think people who are more established in their careers might not remain so technical later in their careers and he's remained, I would say, ahead of me in terms of learning programming languages in that sense. And it's kind of mind blowing to see. I kind of feel like I see the evolution of computers through him as I grew up.
DOUG: You know, it's interesting to think, know, like you mentioned, OOP. And I was, I was sharing these stories with some folks that I work with recently and this idea of Gen AI and what's going to happen and I tend to lean back into my experiences which have always been pretty positive when it's been as a professional developer, through Y2K. Like, I remember how scary that was. They're different things, right? Gen AI and that. But just this idea of object-oriented programming was a thing that came about and it really did kind of catapult people's ability to deliver software solutions faster. And I remember when component based, VisualBasic and Delphi, some of these other tools that came out, right. It was this idea of like, well, developers are not going to have a job anymore because you just pull these things off of a palette and drop them onto a canvas and the software builds itself and even things like rational rows, you just draw circles and lines and this thing will generate all this code for you, you don't need. I think Gen AI is different, right? I think it is a different thing.
But I do think that at the end of the day, what I think is going to happen, and what I hope is going to happen is that it just is going to give us the ability to invite more people into this craft and develop solutions faster. I don't see it replacing developers. I think it might create the opportunity for more people to learn how to express sets of instructions through a language that a computer understands, right?
ADRIANA: Yeah, definitely. I see it very much as an aid, right? Let it take care of the stuff that isn't as interesting and let's focus on the things that are more interesting. It kind of relieves a little bit of that mental burden. But I think it also makes me think back to a conversation that I had with somebody earlier in the podcast where it was almost like a caution because I think they were saying that it's very important to have that base knowledge as well, right? So that you don't take it for granted, right? Rather than don't rely on AI as a crutch. Still have those first principles, fundamentals, and then it can really give you those powers because I think that's the risk that we can potentially run, especially like kids who are growing up in an AI-powered world where they might not have that incentive to learn the fundamentals as a result.
DOUG: Yeah, I think that is one of the main challenges that we face, right? Just being responsible with it. Yeah, we'll see.
ADRIANA: Yeah, indeed. Okay, next question on our list. What is your favorite programming language?
DOUG: I would say my favorite programming language right now is definitely Python. I've been doing this for a long time. I started writing software when I was a little kid, as a matter of fact, in the late '70s, and I was just a very unique opportunity or place to be where I had two parents who were educators and we had access to computers before most people didn't. Although the computers were large, like mainframes that were on the grounds of the University of Virginia where my dad worked. And so I really have written in so many different languages. I think it's like 30 different languages I've written code in. And I've gone through these periods of my career when I've thought about software in the language that I was writing in. Like, I can think back to when I was at General Electric.
I used to think about software in C, and also I would dream, and sometimes I still do dream about software and trying to express something in a language. Anyway, so it was C for a while, it was C++ for a while, Java, C#, some smattering of languages in between. But I definitely now am at the point where Python is my go-to. So if I ever want to do anything outside of work, let's say if I just want to write a script or, I don't know, just have, like, I'm usually pulling repos from GitHub that are written in Python, and that's kind of the language that I prefer. I'm still able to read code in a lot of other languages, but I think that I'm at the point now where the half life on, let's say, my C++ experience is such that I would probably have to RTFM and take my time and ask copilot for some help.
ADRIANA: I feel you. I did Java programming for a really long time, and I found that if I was ever away from it for a stretch, even if it was like a couple of months, it was always one of those cases of like, oh my God, I don't remember how to set up my JVM or whatever. How do I do this again? How do I find the length of a string or whatever the hell? Like how do FOR loops work again?
DOUG: Yeah, I know, that's funny.
ADRIANA: I was going to ask, you said you started working with computers at a really young age. What was your first programming language?
DOUG: I would say BASIC. BASIC for sure. Yes, I taught myself basic when I was a kid. My dad had some books around, and back then, I guess the body of knowledge was just so small. The books were very prescriptive. It showed you everything you needed to do. And they were dry. They weren't really about programming. It's just like, let's build.
ADRIANA: This is how you do this.
DOUG: Ten pages of all the things you have to do.
ADRIANA: Yeah.
DOUG: And I learned a little bit of assembler. I taught myself a little bit of assembler when I was in middle school just because, I don't know, I think my dad told me that assembler was, like, the language you really need to use if you want to do something with the hardware of the computer.
ADRIANA: Right.
DOUG: And so I just remember getting to the point where I could have a small bit of assembly code that would turn the screen blue or red. And I just thought that that was the coolest thing ever. And I was a little kid when I was doing that.
ADRIANA: That is so cool. I think everyone who's learning how to code should learn assembler because it gives you an appreciation for memory allocation.
DOUG: Yeah. And I think for me, it gave me an appreciation for garbage collectors when they first came out.
ADRIANA: Yes, that's the other thing.
DOUG: I was definitely one of those people where it's like, there's no way I'm letting something else decide when memories being reclaimed. Like, I'm not letting it do that. And then after a while, I remember thinking that this is actually kind of cool. I don't have to worry about allocating and deallocating memory anymore. Again, it goes back to this idea that, like, you know, I'm. I'm lucky that I have been able to be...been around and seen...
ADRIANA: Yeah, yeah.
DOUG: ...these...these incremental steps and, you know, the evolution of software and, like, garbage collection was one of them. Like, man, now I can spend. You know, I can. I can get that 33% of my time back. For some people, it might have been 50%, some people might have been 20%, but still, it's like, I don't have to do that thing anymore, and I can still focus on my craft and I can still solve the problems. This is a win for everybody.
ADRIANA: Yeah. In some ways, it kind of even ties to the paradigm shift of, say, the DevOps movement or automation in general, right? We're talking about people are so afraid, oh, this is going to take my job. And it's like, no, you get to focus on the cool shit now. This is where it's at. And it's the same sort of idea, but you still understand fundamentally how it works, which is really important. I remember I learned C in university, and then it wasn't until I learned assembler for a course in my fourth year where I'm like, oh, I get it. I wish I learned assembler before I learned C, because I think I would have had a much better appreciation for all the things. And I think that's the type of thing that I feel like that's the kind of knowledge that we still need to impart on folks in school.
DOUG: Yeah, you know, it reminds and on and off for the last few years I've been trying to finish this book called Code by Charles Petzold, I want to say. It explains the history of...it basically says, okay, today we have computers that do this, right? Yeah, let's go back into time and he walks you through the evolution of languages and numbers and basic math, and eventually you get to the point where you start to understand how computers really work.
ADRIANA: Right.
DOUG: Like this idea of gates and relays and switches. Computers are physical, but the idea that you can build like a very rudimentary thing that would show you how to make a decision and you could walk up to it and touch it. I don't know if everybody needs to know that level of detail, but I do appreciate it when I'm talking to other engineers and they can have that mental model. In a cosmic way it lets you kind of bond.
ADRIANA: Totally. I love it. Okay, next question in our series is do you prefer dev or ops?
DOUG: Do I prefer dev or ops? I would say dev for sure, although I have lots of opinions about ops.
ADRIANA: Feel free to share.
DOUG: It's interesting. I don't know what everybody's experience is like, but I feel like when the ops movement came around DevOps, I think some organizations got it right and the ones that did really achieved massive benefit from.
ADRIANA: Yeah, yeah, totally.
DOUG: But in my experience, I feel like leaning back again in my history, like the idea of a DBA is there. The DBA is ostensibly there to help you go faster, right? It's kind of like, what is it in team topologies when you have the group of people that are kind of the experts and, man, I can see the book on my shelf. Anyway, they're like the consultants, they're the experts there. They're just going to make you go faster. But what inevitably happened is the DBA just became a bottleneck, right?
And I'm kind of aging myself out here because I think the idea of DBA is going away. That's probably for another conversation. But in any event, I feel like that's kind of happened with Ops and DevOps, where this idea of like, we're here to provide platforms and tooling and we're going to abstract all this stuff away from you so that you, a developer, you can go flat out. You're in your flow state and you're just shipping code. But I think more often than not what happens is that the DevOps and Ops organizations often become a bottleneck.
ADRIANA: Yeah. Which is ironic because they were meant to take away that bottleneck.
DOUG: Exactly. Yeah. I feel for the folks that work in these arenas because as I see more and more companies moving to the cloud and more and more companies being in multiple clouds, their jobs have gotten really complicated. And the cloud providers do provide a lot of stuff out of the box. But if you look at most DevOps people and the skills that they have to use on any given day, particularly for multi cloud organizations, it's a long list of things that they have to know and the context switching between this tool at this cloud provider, this tool at this cloud provider. I can appreciate how that can. It's a lot of toil.
ADRIANA: Yeah. It's like a bigger operational burden on the operators, basically, whereas the devs are still sitting pretty doing their thing. I was actually having a discussion earlier with someone where I was saying how for me, I find it funny that I started in a development background and then for me, I made it my business to learn how to package containerized applications and deploy them and know how to deploy cloud infrastructure, IAC, all that stuff. But for me, it was a bit of mind boggling to realize that that is not a thing that most developers concern themselves with, kind of by design. And I thought that was kind of funny because I'm like, I don't know, I like this stuff. I want to know how everything works end to end. Yeah.
DOUG: You know, it's, you know, for both the, for both the Ops, DevOps orgs and the developer orgs, like the one thing that I am becoming more and more convinced about is just this idea of standards. You and I were talking earlier and I mentioned the fact that I'm on the national ski patrol. I think there's some really amazing things that the ski patrol has kind of figured out about how they train and trained patrollers and how patrollers operate and deal with emergencies. And I think about this a lot when I think about DevOps and developers and kind of the struggle that often exists. The thing that the ski patrol does really well is it has some very highly prescribed tools and processes, like the types of splints that we use and how you apply. The sprint is very prescribed to the point that, and I've actually done this before, I was not patrolling, but I came upon a car accident and somebody else who was an EMT was there and we were able to essentially stabilize this person and get them to be handed off to an ambulance.
ADRIANA: Yes.
DOUG: And what was interesting is that at the end of dealing with the acute emergency. When the ambulance left this guy, we kind of shook each other's hands like, man, I'm so glad we're both here.
ADRIANA: Yeah.
DOUG: And we were kind of laughing because as we were moving the patient around and putting them on the backboard and applying a C collar for neck mobilization, we never said a word to each other. We had been trained on the tool.
ADRIANA: Right.
DOUG: Like the backboard, the spider straps, and the C collar. And we knew the process like we knew which straps to apply first.
ADRIANA: Right, right. And you didn't know each other.
DOUG: Never knew each other. Never. Nothing. And so what I've been preaching a lot at where I am right now is I've been thinking about this stuff with the national ski patrol, and to me, that's a beautiful example of why standards and conventions really do help everybody, right? If we were to say, for any low latency OLTP concerns, we are going to use Postgres. If you think about the ripple effects that that has on reduction of cognitive load, it becomes really quite interesting. Think about disaster recovery, right? Your database choice informs your ability to recover quickly in the event of a disaster in a significant way. Right?
ADRIANA: Yeah.
DOUG: The reason why I came upon that accident and jumped out of my car and was able to help that person, not to say that we saved their lives, but we probably made them more comfortable and got them to the next level of care. We were able to do that so efficiently. There was no cognitive load like this person and I were focusing on solving the problem of this person who had been in a bad accident.
ADRIANA: Right.
DOUG: And when I think about, to me, it's like, okay, that's interesting. It's a really interesting organizational problem. Why was that beneficial? Oh, the standard tool and the standard process removed all cognitive load. What if we did that at work? What if we said, hey, you know what? Let's just...I mean, we can argue about SQL Server, Oracle, we can argue about it, but if we pick a tool, then the unintended but kind of intended consequences become material. Like, all of a sudden we have one backup tooling. We have one recovery tooling. If I get paged in the middle of the night and it's because my coworker's out and I have to jump into a sev one, I know how to backup restore postgres. Right. Like, if it were Cassandra or something else, it'd be like, I have no idea. Anyway, I'm obviously pretty passionate about this.
ADRIANA: I love this so much. I have so many follow ups to this. Yeah. Because it reminds me because at the previous... and we know each other from a previous organization where we worked together, and I do recall that at this previous organization, it was more of a, like, you work with the technologies that you're most comfortable with, sort of approach to things, which I think can be very interesting and effective until it stops being effective. I think especially if you're starting to grow the organization to a significant size. I definitely learned both there and at a previous organization that being able to standardize and to standardize by codifying things really went a long way, because to a certain extent, we got to save people from themselves.
DOUG: The codifying things...here's another great example. In the ski patrol, it is interesting, this idea, just to kind of zoom out a little bit, this idea of the developer experience and developer creativity. We want small autonomous teams because our hypothesis is that collectively they will deliver more value. So we talk about things like full service ownership. You build it, you run it. These are wonderful ideas. But at the end of the day, the software organization isn't there to satisfy all their curiosity.
Most places they're there because they have a mission, a purpose, a goal, and oftentimes it is to make money. So one of the things that, another thing we do on the ski patrol, which I love this idea, and this kind of goes back to this idea of like, standards creating an environment where you still have autonomy and creativity to solve problems. We all know that there's a lot of creativity in software engineering, even though at the end of the day, it's just a set of instructions. I think there's a way to allow for both. But I think that it's important for software developers and DevOps people, all of us, right, to understand that the things that we might be locally optimizing for are not part of the global optimization, right? Like my desire to introduce Rust into a service because I think it is better, and it could possibly be better for that service. It's not in service of the larger goals. So on the ski patrol, if I'm on scene with somebody who's been badly injured and they have to be taken down the mountain, they're not able to ski or snowboard anymore. We put them in a sled, a toboggan.
Whenever a toboggan arrives with another patroller, there's something called a sled pack. And the sled pack has the tools that you need to probably address the person that's injured who has to be put into a toboggan and sent away. So we have some blankets, some splints, some devices to move people around, things like that. Every time the toboggan arrives, when I open up the sled pack, everything is there and it's always in the right order. And, you know, it's to me it's like, it's another fabulous...I'm sure that if I went to any other mountain in the US, and if I was a bystander and somebody said, hey, can you help? I'd be like, yep, I'm a member of the SB, blah, blah, blah. The toboggan arrives and I go to open up the sled pack, I guarantee you it'll be all those things in order.
ADRIANA: Yeah.
DOUG: And when I was getting trained...sorry, that was my cat.
ADRIANA: That's all right!
DOUG: When I was getting trained on the ski patrol, it never thought to offer a different solution or to suggest something different or to sneak in a different thing. Because I think part of the training is like, the whole idea here is to remove the cognitive load so that when you're in a crisis, you can focus on solving the problem, which is making sure the person can breathe, they're not bleeding to death, and that anything is not straight. Gets straightened. For me, when I think about this at work, it's things like, all right, if I'm going to go look at a microservice, and we've all agreed that we're going to use Python and FastAPI or Goengen, right? Every repo I go to, I want it to be the sled pack experience. I want to clone the repo, and I want to open up the top level folder, and I want to see all the elements that are there, right. And they should be immediately recognizable. And it's interesting, if you talk about these ideas with some developers, they're kind of like, well, that seems overly prescriptive. Why would you want to do that? That might not always work. Valid arguments, but I'm always like, well, what's the benefit of having it different every time? And I go back to this idea also of that code is read, I think, like 100 times more than it's written. Like, the time human beings spend with code, it's mostly reading it.
ADRIANA: Yeah.
DOUG: So why not make it easier for the next person? Like, agree on a standard. We'll use Postgres, agree on a convention. These are folder structures for our microservices. To me, that creates that sled pack experience. Like, I'm on scene, this person's really hurt. I've got to get them to a next level of care immediately. I don't want to think about opening up the sled pack and being like, where's the litter? I don't see...why isn't the splint here? Because, you know, Doug decided that there was a better way.
ADRIANA: Yeah.
DOUG: And he liked that idea. Having said all this, though, the one thing I will say is that going back to some of my other stories is I'm old school. My first job at a school, there was never a conversation about languages or naming conventions. It was all written down. It's General Electric. They're very thorough company. Anyway. So I tend to find some comfort in thinking about things like, we have a language, we have a storage technology. We've got this mechanism for asynchronous communication. Just use it.
ADRIANA: Yeah. It's interesting because this is basically like a platform engineering problem when you look at it, right? Because you are looking to standardize across the board, not just the development experience, but the overall standardization of your infrastructure and all that. It's very cool. I think it's so great to have to be able to standardize on that, but also give some wiggle room for creativity. But not so much that you end up sort of ruining the flow, because I think that's really the thing. But I think then what helps make it effective, and I'm sure, especially when you're doing ski patrol, you have a guide, right? You have this documented somewhere how these things work. And I think that's where a lot of companies kind of miss the mark, is because these procedures, these standards, aren't necessarily documented or well documented.
DOUG: Yes. So here's the other thing that, another thing that I really appreciate about the ski patrol, national ski patrol, and something I really do want to bring into where I work. So getting certified to be a ski patroller involves typically two things. One is about six or nine months of classroom training where you effectively become an EMT. But it's called an OEC. It's outdoor emergency care. So you're trained on outdoor emergencies. So it's kind of the basics, right? Like, I think of it like computer science, computer information systems, whatever.
ADRIANA: Yeah.
DOUG: So you finish that up, you're like, okay, I've got a whole bunch of academic knowledge. Then each mountain will do what they call on the hill. So there's another couple of months of training where you are now taking kind of your academic experience and actually applying it. For us, it would be a cold weather environment, so we have classrooms on the slopes, and we practice things out in the cold, in the snow. And so the on the hill training is really important. And once you finish the on the hill training, you are very proficient. It's not like you're a noob. As a matter of fact, most people who come off of their, when they just get their certification of patrol, you almost want to go to those people because they remember things that you don't. Or I did.
ADRIANA: Yeah.
DOUG: Anyway, so to me, this idea of on the hill seems really important. And I almost envision, I don't know, every six months or maybe every quarter, we take our new developers, our new engineers across all the departments, and we basically run them through on the hill. So, like, this is our software development lifecycle, right? This is our documentation. We're going to give you some sample code, like some scenarios to run. Join us for a sev 1. So that basically after, I don't know, some number of weeks, let's say those people are very familiar with how that organization works. How does it ship code?
ADRIANA: Yeah.
DOUG: Right? So a very focused, very well thought out, thorough training program. It's on the hill training. Like you've taken your academic experiences or whatever you had before, and we're going to show you how we do them here at our mountain with our specific protocols. And then when you're done with this, the entire patrol knows that you're ready and you can go and you can work accidents and you will be trusted and you will be successful.
ADRIANA: Yeah.
DOUG: So the other thing that's related to this, too, is that another thing that I really appreciate about the NSP is that we have refreshers. So every year. So every year we get refreshed and we get retrained and retested and recertified on essentially a third of all the things you have to know as a ski patroller. So we go through cycles a, b, and c every three years. But the great thing about it is that, and this is something else I would want to do. So I would run a run cohorts of engineers through a refresher just to say to people like, okay, we know you know it, but let's to be like, let's go back through and talk about our SDLC. Let's have some people join some, like, let's understand what it's really like to work here and just recertify everybody on the process, to give people an opportunity to know that they can participate in the process so that we know that we're doing a good job and training people and making them be successful. So I really like this idea of this refresher that happens every year in the ski patrol, where you go and you spend some time rtfming.
Like, we all big medical manual that we read. We go and we meet people. It's usually in the fall, but we'll go to the resort and we'll do some dry runs of a lift evacuation. So we'll load people up on the lift and we'll evacuate them. So it's almost like, hey, let's pretend to have a sev 1 or a tabletop sev 1. That would be part of the refresher, like, so and so database clusters down. What are you going to do, right. And give people some time to kind of step back and lean back into whatever the company is doing and how it works so that people aren't under duress, if that makes sense.
ADRIANA: Yeah, and it makes a lot of sense because in many organizations, especially, like the really large ones, you already do things like DR testing, which in my experience, it's a very important thing to do. But I will say that we did it twice a year, I think, at one of the organizations where I was at, and everyone's sort of, like, rolling their eyes, like, oh, why do I have to do this thing? Why am I the one who's pulled into helping out with the DR test? So I think finding a way to it almost like, in some ways felt like we were going through the motions, which kind of defeats the whole purpose of the whole thing. But then there's the other side of it, too. Oftentimes when new engineers are onboarded onto a company or even onto a different team, unfortunately, they're kind of just thrust in, right? It's like, oh, here, read some stuff, and then we expect you to be productive in a week.
Good luck to you. And that's the type of culture that we need to aim to change, because it can be very frustrating if you're new to an organization and especially, oh, my God, imagine you're fresh out of school. If you've done this enough times in your career, like, whatever. I guess that's just the way to the poor kid fresh out of school, they're like, what's going on?
DOUG: Yeah. It's curious to me because you'd think that the company, the business, the organization would intuit the value of knowing how to recover during a disaster. In my experience, I want to say there's only a handful of places that I think really understood that and made the investment and the training and the technology to be ready.
ADRIANA: Yeah. More than just going through the motions. More than just like, this was a compliance thing where we need to tick off this item, right? Yeah. I think the companies that care about doing chaos engineering tests and game days and that sort of thing. I think those are the organizations where you're probably going to see success in terms of low panic factor, perhaps during incident responses, right?
DOUG: Well, kind of going back to one of our threads from earlier, this idea of ski patrol, we use one splint to deal with femur fractures, a sega splint, there's other splints, but that's what we use. Going back to this idea of choose a database for all of your low latency OLTP concerns, maybe even HTAP concerns, hybrid transactional analytical processing concerns. But in any event, pick a database. Not only does it have this interesting effect on reducing cognitive load, but when you start to think about the idea of chaos engineering, if you're in an environment where you're storing data in, I don't know, Kafka, Cassandra, mongo, some relational database, another relational database. Right. Like chaos engineering starts to become incredibly daunting, almost impractical or impossible. Not impossible, but like, impractical. Again, when I think about the ski patrol, it's like there's lots of different splints. Everyone knows this one, and we are all trained on it.
ADRIANA: Yeah.
DOUG: And you can put it on and off blindfolded because it's in service of a larger goal, which is to help people and hopefully save their lives.
ADRIANA: Yeah.
DOUG: If you think about this idea of really trying to be keeping your technology stack simple and having standards and really adhere to them, the unintended consequences of that start to bleed into all these other concerns. Retention, recruiting, hiring.
ADRIANA: So true.
DOUG: I love this idea of where I am at uplight right now. We have a lot of python experience. We have a lot of fast API experience. It would be easy for me to turn to another team and say, can you help for like two weeks, one sprint? And most of the people around me would be like, yes, I can do it. They know the language, they know the process, they know the framework. And I wish more organizations saw that benefit, and I wish more businesses would lean into and kind of lean into engineering to say, we're running a business, we have a mission. We understand that you have this creative thing that you do. We don't know what it is, but you build stuff for us, keep it simple.
ADRIANA: Yeah.
DOUG: Because we have these larger goals that we need to meet. And part of it is hiring people, recruiting people, having agility within our software teams. We want to have small autonomous teams that come together, build a solution, and then they dissolve, like, over and over and over again. And the way to do that is by having those standards, picking good tools, using them until they can't serve you anymore, which for the most part is never the case. To me that is a real interesting recipe, like taking those concepts from the ski patrol. I just want to inject them into software organizations.
ADRIANA: Yeah, it makes so much sense to me because then when you strip everything down, you're left with, I get to solve cool problem. And the other thing too is you can apply that to another aspect that kind of is like the bane of every developer's existence, which is like compliance and security, right? Having that standardization, working together with our friends in security, which doesn't always happen, but if you have the standardization in place, codifying the standardization, then again you're taking away the mystery and make it easy to implement the security policies is the other thing.
DOUG: Yeah, again, being the older programmer in the room, and admittedly a bit of a curmudgeon to me, just the simple technology stacks always win. Yeah, and they win because they help the people in process, people process and technology, the simple technology stacks make those other things come into focus and be successful. I see it over and over and over in my career. The simple solutions, the simple technology solutions always win.
ADRIANA: And it's funny too, because as developers we are drawn to shiny new objects and sometimes shiny things can be overly complicated. It even reminds me one time at a previous job, I was on a release engineering team and we were using Azure for our cloud and we were looking at possibly moving to Kubernetes, but we were using before Azure container services to manage all our microservices. And quite frankly it was doing a very good job. And yet they were looking at moving to Kubernetes because, well, Kubernetes come on, of course. But it was funny because in the end they made the decision of like, yeah, this thing's too complicated. Azure container services is doing the job. Shit's not breaking. We are happy. So why are we overly complicating the situation, right? Which I think was a very, for me in Kubernetes I was sad, but when you look at it very objectively, it made a lot of sense.
DOUG: Yeah, it's funny when I hear these stories about people using some very sophisticated, complicated things like Kubernetes and struggle with it. And I'm like, I have to remind myself, like, okay, we've shipped code before, right? Without all this stuff. I'm pretty sure I remember shipping code to a computer. So there's a way. But I feel like we've strayed from the path. Like how do we get back onto the path of, like, let's ship hello, world?
ADRIANA: Yeah.
DOUG: And keep it simple.
ADRIANA: Yeah. And I think that is the key, right? Keeping it simple because we like to overcomplicate for the sake of, because we want to solve a cool problem. I think at the end of the day, we're engineers. We want to solve really cool problems. So let's find a complex way to do it. And then you see the simple solution. You're like, oh, yeah, I guess.
DOUG: I think that's an interesting way that you've described that we have to go against our own nature. Most of us got into this because we're geeks. We're nerding out, and we're learning new stuff, right? All the time.
ADRIANA: Yeah.
DOUG: But it's almost like now you have to step into this work environment and say, I'm going to have to deny my true nature.
ADRIANA: Yeah.
DOUG: And I'm just going to, to a certain degree, not assume that everything's a nail and you have a hammer, but you almost want to say, like, okay, I've got four tools in this little toolbox. I've got to keep it simple. I need to focus on the larger goal here. What is our goal? We need software to solve this problem. Great. I've got these tools. I'll keep it simple. But for so many of us, it's like, but that's not why I'm here.
DOUG: I got here because of my ability to pick up new things and be curious and dig and learn.
ADRIANA: Yeah. And so you're like creating a new challenge for yourself so that you can kind of keep the brain active.
DOUG: Yeah.
ADRIANA: It actually reminds me.
DOUG: Sorry, have you heard that before, developer catnip?
ADRIANA: No, I have not.
DOUG: I heard that from somebody I work with about a year ago. I hadn't heard it before, but I thought it was just a great expression, this idea of developer catnip. Like we see something and we smell something and then we're all into it. It's like, no, go back over here.
ADRIANA: So true. It actually reminds me of something I experienced fairly early in my career where we were writing code to convert some stuff and pop it into an oracle database. And the vendor we were using, they had a Java API for it. And I was super excited. Like, Java was still new. This is like early 2000s. I'm like, oh, this is awesome. Great problem to solve. And then we're running some performance tests and the Java code was slow as fuck. It was ridiculous. But they also had like a PL/SQL API. And so we started doing some tests running the same kind of code in PL/SQL and it runs in the database. Of course it's going to be faster, like Java, extra overhead bloat. And I remember being so sad. I was like, I have to learn this crappy language. I'm going to be like, I'm not going to have. But objectively thinking back, it had to be that solution. Otherwise the PL/SQL code was so much more performant than the Java code. And I had to put my pride aside and suck it up because that was the better solution. It was the more logical solution.
DOUG: Wow. Yeah, that's awesome. I'm sitting here thinking like there's definitely cohorts of developers who have had a schema handed to them in an ORM and might not have ever really written SQL.
ADRIANA: Yeah, I feel like SQL was like my life for the first chunk of my career.
DOUG: Yeah, definitely.
ADRIANA: I don't know if I can write the complex SQL that I used to back in the day, but it was fun while it lasted. Yeah, cool. Well, we are coming up on time, but before we go, I was wondering if you have any final parting words that you would like to share with our audience.
DOUG: I would be curious to hear from other people if they have experience where they've either as a hobby or a volunteer or a second job. I would love to hear other people how they could take something that they're doing. The national ski patrol and software engineering. What in the world do they have in common? And I think the answer is quite a bit. And I think that the software engineering world could learn a lot from the ski patrol. So I guess my parting thought would be very curious to hear from other people. Like, do you have a former job or some experience where it just seems orthogonal to what we do, but actually could help.
ADRIANA: Yeah, so true. And I've talked to a bunch of people about that. Like even someone, they were in incident response. They used to be in the service industry, like work at restaurants. And she was telling me how she had to learn to manage customers and stuff, deal with them and have unpleasant conversations and how so many parallels between that and what she did in incident response. And I'm like, after we finished recording the podcast, I'm like, you have to write a blog post about this. Which she did. But yeah, it's great that let's draw more on our real life experiences and find parallels in the software world because it's so relevant.
DOUG: Yeah, for sure. I think if we can be humble and not just assume that because we're the nerds that we know the answer.
ADRIANA: Yes, absolutely. Yeah. This was really great. I'm so glad that you brought this topic up because I think it's so relevant. It's very eye opening and kind of sobering in a way. Let's take a step back and evaluate. What are we doing here?
DOUG: Yeah, I think that's the beginning of the adventure.
ADRIANA: That's awesome. Well, thank you so much, Doug, for geeking out with me today. Y'all don't forget to subscribe and be sure to check out the show notes for additional resources and to connect with us and our guests on social media. Until next time...
DOUG: Peace out and geek out.
ADRIANA: Geeking Out is hosted and produced by me, Adriana Villela. I also compose and perform the theme music on my trusty clarinet. Geeking Out is also produced by my daughter, Hannah Maxwell, who incidentally, designed all of the cool graphics. Be sure to follow us on all the socials by going to bento.me/geekingout.