Adriana geeks out with longtime friend and fellow Cloud Native fan, Robert Golabek, CEO of Translucent Computing. They talk about Rob's Cloud Native journey, and the challenges of being an early Docker and Kubernetes adopter. Rob also touches on the exciting possibilities of Conversational AI and its integration with Kubernetes. Finally, Rob turns the tables on Adriana, as they chat about managing infrastructure on-premise vs public cloud, and running on-premise Observability tooling vs software-as-a-service (SaaS) tooling.
About our guest:
Rob Golabek is Chief Architect & CEO at Translucent Computing. A thought leader and insightful tech visionary with over 20 years of experience, Rob is a Cloud Native expert specializing in App Modernization. Leveraging data, AI & cloud for digital transformation, he provides expert guidance to clients navigating the complex, ever-changing cloud-native landscape.
Rob has shaped the technology landscape through his work at Translucent, progressing from software development to architecture and leadership roles. His expertise in cloud-native technologies, DevOps practices, infrastructure tooling, and tailored consulting approach helps clients drive toward cloud-native success, including observability and robust cloud foundation building.
Rob also leads the ExecutiveEspresso Series, where he contributes to fueling business growth and inspiring the next generation of innovation.
Find our guest on:
Find us on:
Show Links:
Transcript:
ADRIANA: Hey, y'all. Welcome to Geeking Out, the podcast about all geeky aspects of software delivery, DevOps, Observability, Reliability, and everything in between. I'm your host, Adriana Villela, coming to you from Toronto, Canada. And geeking out with me today is my good friend Robert Golabek. Welcome, Rob!
ROB: Hey, nice to be here.
ADRIANA: Super nice to have you on. And full disclosure, Rob and I have known each other for a really long time, like since, what...2000...I want to say 2005? It's been a while. We've known each other for a really long time in a past life, in our past lives as Java developers, which is really awesome. So Rob, for starters, where are you calling from?
ROB: I am from the deep west Toronto Etobicoke.
ADRIANA: Yay. Fellow Canadians.
ROB: Yeah, people don't know Etobicoke is a borough of Toronto, so some people call it Toronto, some people don't. For some people, I heard it's really far. For me, it's actually the perfect balance. Twenty minutes from Toronto. But yeah, get some kind of space. So yeah, that's kind of where I'm from.
ADRIANA: Cool. Awesome. Awesome. All right, so we're going to start with some rapid fire questions. Are you ready? I promise it won't hurt. All right, number one, are you a lefty or a righty?
ROB: Righty. And happy Lefty Day. I saw that post, so yes, your superpower...I was going to respond post of my right-handed rights.
ADRIANA: I always forget to acknowledge Left-Handed Day. And then this year I'm like, "I am going to schedule this post so I don't forget." And then when it popped up the next day, like on Monday when I was back at the office, I'm like, "Oh, Lefty Day passed. Oh, I remembered post on that."
ROB: It so the reason it's close to me is my dad is left-handed, right?
ADRIANA: Awesome.
ROB: And for some weird reason, it was weird when he was growing up to be left-handed. So they tried to even make him write it with the right hand, and it was kind of, you know, so yeah, it's dear to me.
ADRIANA: Yeah, totally. Yeah, my mom too, she was left-handed and she was subjected to people trying to make her write with her right hand. And she was one of those non-functioning-with-her-right-hand lefties...everything with the left hand. So she's like, "No." I can manage with some right handed stuff, but lefty and proud. All right, next question. iPhone or Android?
ROB: Android
ADRIANA: All right. Mac, Linux, or Windows for development?
ROB: Windows.
ADRIANA: Awesome. Favorite programming language?
ROB: The one I know, I got to say Java
ADRIANA: All right. Dev or Ops?
ROB: DevOps
ADRIANA: Yeah, I've gotten a few of those answers before. It's very PC. DevOps. All right. JSON or YAML?
ROB: Depends on the situation.
ADRIANA: All right, fair enough.
ROB: All right.
ADRIANA: Fair enough. All right. And then final question: do you prefer to consume content through video or text?
ROB: Text.
ADRIANA: All right. Yeah, the text people are winning so far. Most people are like, "Text." I'm right there with you.
ROB: And if there was a video, I watch it on mute. I like the writing.
ADRIANA: Do you read the subtitles?
ROB: Yeah...I don't know, I'm not a video person.
ADRIANA: Yeah, I know, right?
ROB: So kind of my age, I guess.
ADRIANA: My daughter Hannah, she's like video, no question about it. I'm like, really?
ROB: Yeah, I'm the same way.
ADRIANA: But yeah, maybe it is an age thing. I don't mind...I'll watch video with subtitles or I will just put on the audio and walk around the house and have it on YouTube...video on my phone, walk around the house with just the audio, and that I can consume...but I can't just sit there and watch a video. Especially for tech stuff.
ROB: Yeah, my my attention span is like, really, like, short. I want to kind of go to the end ofthe video, and I just want to read it very quickly because I usually skim through it and then I read the most interesting part in video. It's like, okay, where's the climax? You can't really find it.
ADRIANA: Yeah, I'm exactly the same, so I totally feel you. All right, so now that we're warmed up, let's geek out on some stuff. So I guess first things first. So why don't you share with everyone what you do? Because you've come, I guess, a long way from our early days in our earlier careers of being the lowly Java devs.
ROB: So yeah, so maybe start from the beginning, you know? '96, '97, '95. I don't know, just kind of coding. And at that point was involved with wires and illegal streaming. Got me interested. Kind of made some money from there. Very quickly. Went to Sheridan in 2000. Within a year, kind of graduated and then got my first job at SickKids Hospital as a reports developer that turned into Java developer, that kind of turned into architecture. It was pretty cool. And after that, that's when I went to Metavante, or is that the right name? I don't know. I think...
ADRIANA: I don't know what they're called anymore because it was like when I joined, it was called GHR, and then Metavante ate them up and then I don't know what happened after that.
ROB: Yeah. So I don't know. In between that, I was kind of in the Canadian Army Reserves too. So kind of got some discipline there. So yeah, it's kind of put me straight as an arrow. Got me kind of healthy and got some responsibility skills. That was from like '99 to 2004 while I was at SickKids. And then between '99 and 2010 while I was still working, I had a side business. So funny story, is in my first resume that I submitted to, I had like, a quote where I want to have a worldwide business where I kind of want to dominate and kind of provide value to people. And when I gave the resume to the SickKids people, they laughed. Right. And I'm like, was that naïve or was that aspiration to kind of something greater. So the entrepreneurship was always there, looking backwards. Maybe a little naïve but kind of inspiring to something greater was kind of my goal. That was kind of my beginnings of trying to take over the world. Pinky and the Brain 2006 that's kind of where I met you in Metavante. Worked there for three years. Went to Point Click Care. I don't know, I think everybody kind of knows here in Canada. Point Click Care one of the kind of unicorns in healthcare.
So I was there for six months. Sad story is I joined, somebody got the bonus for referring me - it was my brother - who's kind of with the company as me and then six months later I left
ADRIANA: Right when he got his bonus I'll bet.
ROB: And they changed the rule after me. I think they even call it the Rob Rule that referral...you got to work there a little bit longer. I didn't do it on purpose. That's kind of when I started my business after Point Click Care. Got my first contract kind of working and actually was with SickKids too, developing their platform and that's kind of where my journey started. And we're here today. And what we do is right now we matured and kind of through the innovation that we do and putting engineering before sales, which I don't always advise because if you have passion for engineering and you want to do everything right, it might hurt sales. But we're proud of that. We run the business our way. So because of that we always kind of innovate, not always to the benefit of kind of sales, but it got us to the journey of early adaptation of Docker, Kubernetes, Cloud Native always early adapters and now we're Cloud Native experts specializing in app modernization, trying to kind of build for the Cloud and the beauty of Cloud Native and optimization, which I love, is it's ever changing, right? So before was moving to the Cloud was legacy software. Now it's kind of the hot take is how do you add AI to software that already kind of are out there, right? In a few years it's going to be something else. So really love what I do, kind of giving the Cloud Native expertise and kind of sharing my wisdom with people.
And through that, sorry, I started Executive Espresso series where I started kind of like, you know what I love kind of talking to people. So started posting information, just kind of sharing on Cloud Native expertise and kind of the different aspects...Kubernetes, Observability...one thing that's challenging, which I tell you and it hurts me, is to be Cloud Native expert. I keep reminding myself how big the space is and like DevOps, Observability, Platform Engineering and cloud foundations, it takes a lot of learning and knowing and talking to people like you and different spaces. So I find that really challenging. But I enjoy that because in my DNA it's kind of learning. So combining all those things is pretty cool.
ADRIANA: And you touched on something really important, which is like the Cloud Native space is ginormous and technology is ginormous and there's a new thing out all the time, so then you can't stay on top of everything. So how do you pick what you focus on as a result of that?
ROB: So we can bring up maybe when you're doing the edits, you can bring up the landscape of the Cloud Native landscape. And I don't know how many tools they have now. Maybe 200, 300, a lot. So what we focus on is opinated experience technologies that we use. So we call it our Tek Stack, kind of powered by open source software. And we chose some tools right as the starting point. Now when we go to clients and kind of try to kind of give our opinions, it's based on that. Now it's also being open to other tools. But when you choose a tool, let it be mature, let it be kind of used by people, let it be a supporting community.
We did a mistake before in the past, where we were too early of an adapter and you pay the price. I think we did it with Angular 2. We did it way too fast. When Angular kind of went through versions of one to two was Angular 1, then it was 2, then it jumped all the way to 5, was too early. I wish we waited a little bit and kind of used it maybe a little bit later. And same with these tools.
So we broke it down into different tools for security. It's Falco, Consul, Vault, KeyCloak, kind of maybe HashiCorp kind of world. And then for kind of cluster resources, Postgres, Redis, OpenSearch, Kafka.
So you can see it's like main kind of tools that we kind of use.
And Observability: Prometheus stack...Kubernetes Prometheus stack, Sentry, Jaeger, Loki...Kind of making sure that we center on those tools and then making sure that adding principal infrastructure as code kind of on top of that and on top of Google, that's kind of how we chose the tools.
And that's like the starting point, right. You can see for Observability, I think it's a very similar stack as Logz.io uses or anybody kind of those seem to be the main kind of open source tools that are out there, and there's a lot of support for them. So that's kind of the biggest kind of aspect of selecting them. And they're really good, right?
So, yeah, that's kind of how we use them. But the biggest thing is through clients and through conversations, you always learn about the new tools. So best way is to throw your tools out there and then tell you some.
The conference you went to, I think, was from you. You threw one tool, I forget the name of it, that I never knew about. And I was like, okay, it was for platform engineering, I think, or I can't remember which tool it was. But you were using your presentation. It'll come to me.
ADRIANA: Oh, yeah. Kratix.
ROB: Kratix, right, right. I never knew that tool before because I never came across it. Right. But then you use that and then it kind of opens up, and then I can query you and be like, hey, how do you use it? Where's the support? So learning from the community and kind of expanding it and then making a selection, hey, is the tool that we want to use or not? Do we want to add it to our stack? Right. So that's pretty cool.
And I'll finish with this. That's kind of where the new thing of platform engineering is, I think. How can we best, to your question, select the best tools that maybe if I was going to propose to you, but also switch the game, what are you comfortable with and building around that most important part?
ADRIANA: Yeah, that's so true. It's so important because there's nothing worse...and I've been in the consultancy space before...and there's nothing worse than coming in and saying your stuff sucks and then you're just going to hurt their feelings. It's like basically saying you have an ugly baby and no one wants to hear that they have an ugly baby. You got to be gentle and understand. What are you comfortable with, what are you using? Hey, would you be open to switching over to this? If you're familiar with this, maybe this might be the thing for you. And I think that's very important, especially in consultancy, because you're essentially trying to help companies do things better. But there can be a lot of resistance to change, so you have to be very gentle with them.
ROB: It yeah, I don't know if I'm an engineer anymore. I'm ex engineer. I love engineering, but I spend more time doing non-engineering stuff. But there's one thing, right, that I always notice with engineers kind of myself, too, not excluding myself. There's that ego, right? I selected, I know the tools better. Prove me wrong. Why are you using this tool? And I don't like taking that conversation there. I'd rather being like, hey, if it's tools great, let's use it, let's improve it. Let's build what you guys need.
Right? But engineers are smart people. I'm going to say that they're usually intellectually smart, so they know what they're talking about. And you got to come with a game, too, to say that you know what you're talking about. So that's kind of where the conversation goes.
ADRIANA: Yes, absolutely. Is definitely a fine line. And I think one of the things in engineering, that engineering is an art form, really. And I think that goes to say for any type of art form is that sometimes we tend to fall in love with our code, with our technology, with the things that we create. But the best thing that we can do for our art is to give it some sort of a seed so that it can grow, whether it's like, hey, that sparks another idea where someone's like, hey, you know what? You could do this a little bit better. I like where you started, but I think this is how it can be improved. And being able to let go of your initial notions and be open-minded to other ideas, other ways of improving it, honestly, I think that's what open source is all about and I think that's what makes also for very successful organizations and very successful teams that you have to check your ego at the door.
It's hard though, because sometimes you're working on a thing and it's like it's your baby. You've put a lot of TLC into it only to have someone say, well, I found a better way of doing it. And it pretty much scraps all the stuff that you did that can hurt. But also recognizing that maybe your initial work, even though it's being discarded, inspired somebody to come up with a better way of doing things.
ROB: Yeah. I was going to ask you, what do you think is the best way of judging that? Right. How do you best put it out there? You kind of answered, I guess, open source. Right. Kind of let the community play with it. Any other kind of ways you would kind of try it out. Kind of let kind of people give you opinions in a non hateful control fashion.
ADRIANA: Yeah, generally just having conversations. I think it all comes back to community, whether it's putting it out there through open source or writing about it in a blog post or having a conversation with somebody. Finding ways to make those connections, I think is probably the best way but you can't do that without it being out there in some form or another, I think.
ROB: Yeah, I really like kind of in my recent time writing, right. So got me thinking and expressing and talking to people. Right. And then the biggest thing is taking that feedback in a positive way.
ADRIANA: Yeah.
ROB: First reactions like, oh, man, why did he say it that way? But then it's like, why did he say it that way? Maybe explore that a little bit more. Right. And then you meet the person, and then you have a different kind of perspective, and then you can change or you don't have to change. The biggest thing is to agree with them
ADRIANA: Yeah, absolutely. But I think the most important thing is that someone offering an opinion forces you to take a step back and rethink it. And it's like what you said, I'll either agree or, hey, there's something to that statement. Maybe I'll tweak it or take some of that into consideration, or like, no, I actually think my way is the better way. I've given it some thought and that's perfectly all right.
ROB: Yeah. And you might have different motives too, right. It could be business case, could be technology, could be different case. Just recently, we had somebody come in and they had an objective of looking from this kind of zoom...was monetary zoom. And it's like that's one way of looking at it. Right. And because it's a business client, they're going to push it in that way. Now, as an engineer, the most frustrating part is let go of your best practices. And then because most of the times client is right. Quote. You try to kind of make them happy but you got to really put your ego away and also put away I told you so because I believe even in that scenario business person could be right because now they're coming from their perspective with and they might have limits.
So you got to look at from that angle ego from technology, from business and kind of move the conversation forward.
ADRIANA: That makes a lot of sense. I think at the end of the day, you just have to be open-minded. So of all the technologies that you've been working with, what's the one that's really exciting you right now?
ROB: I love Conversational AI
ADRIANA: Oh, yeah. Cool.
ROB: So...and applying it to any domain. We're just working actually with Pat, working on Conversational Kube Bot, where you can talk to it in human language and get a response.
ADRIANA: Oh, nice. Is that something you guys are developing?
ROB: Yeah, we're developing it. I want to release it. It's kind of started as a kind of small project because we're in a grander schemes working on Enterprise search, and we call it Conversational Enterprise Search, and we call it, like, Next Knowledge Base Economy, where knowledge is king. And how can you take that knowledge and how can you converse with it right. At a basic level. Right. And applying it in Kubot, hey, get all the resources, all the material from kind of Kube Bot and then suck it in. Use kind of NLU, NLP, NLG, kind of all the kind of natural processing, human and language kind of processing. So you're able to create something where it's your human assistant. Right. So my goal is, like, I never want to remember a Kubernetes command. And with this, we already have a prototype where it's like, hey, tell me the status of the system and let's see all the pods or something, right?
ADRIANA: Oh, my God, that's so cool. I cannot tell you how many times, if I'm away from Kubernetes for a while, I have to Google this stuff. Or now I have a GitHub repo where I just have a README with all of my go to Kubernetes commands because I forget that stuff, especially the gnarly ones. Like, how do you freaking go through your logs in Kubernetes? Or how do you log into your pod? Into your container in your pod? Or be like, yeah...
ROB: I'm with you, man. I have a folder with documents, and it's for, like, Kubernetes, Docker this, and I'm like, get lost in those. And then it's like searching through those commands. So we're applying this Enterprise search and conversational search to Kubernetes and Observability. And it ties into AI Ops. So I'm in awe in how powerful large language models are and applied in the right case, I'm going to write a blog. It's on my to-do. I kind of have a draft format where take it from a different angle, right.
ADRIANA: Yeah, yeah.
ROB: How these AI tools can help the world, right. I see too much boom and doom, kind of, hey, they're going to break this, break that. It's going to take cover, control. Yes, everything, right? We have the biggest case of nuclear power. It's for good, it's for bad. It's our human choice to use it for the good. Right. And I'm always optimist.
So I love it because it can apply to so many...We just combined the few elements, AI search and Observability and Kubernetes and boom. That's something we're working on.
So that goes back to engineering and working cool stuff. So that's kind of what I really enjoy.
ADRIANA: That is super cool. Yeah, it's funny because I think AI has definitely become a hot topic because it's come up more than once in this podcast. I think my first dabbling into AI was, like, using DALL-E for generating images for my presentations. That was kind of my first one where I'm like, oh, my God, this is the coolest thing. I can tell it to generate pictures of llamas doing funny things. What? Or my favorite, I have this love for capybaras now because Instagram one day decided to serve me pictures and videos of capybaras.
And I'm like, oh, my God, this is such a glorious animal, you know, DALL-E has generated me a bunch of images of these things for my presentations as well. So I'm like, "Shit, that is some really rad stuff."
And then further leveraging ChatGPT for even certain things, where you find yourself in a position where I need to reword this thing. My brain is fried. "ChatGPT, just take the sentence that I wrote and make it a little bit shorter," because I don't have the brain power to try to think of five different ways of saying this word and conveying this thing, right?
ROB: So you're touching on something pretty cool, right? So it takes you to the next level. And some people say it actually does it for you. It doesn't yeah, it's going to be mind-blowing. It doesn't, because I can smell, like when a marketing person talks about technology thing, and it kind of doesn't make sense. And then when a techie will use the same kind of and they will just rephrase it. There's a difference.
So it's such a helpful...I love it. It's been changing. So we applied it kind of all over the place. Again, combining AI, Observability, DevOps like...crazy.
ADRIANA: It's going to be mind-blowing. And I think people forget that it's not like AI, as you said, AI is not going to do all the work for you. You still need the human touch to guide it in the direction, and then you still have to vet it because sometimes AI spits out some dumb-ass shit and you're like, "No, I do not want this." And then you just rephrase the question.
At first when I heard the term "prompt engineer", I'm like, "Ha ha. That's so hokey."
But we've been prompt engineers for a while now, if you think about it, in software, because that is essentially what we do when we do a Google search, especially when we're trying to solve a gnarly-ass problem and you enter a particular search term, and then you're like refining, refining, refining, until you're like, oh, you know what? That's not even the right question that I have to ask, but now I've got enough information that I know the right question to ask, and that's essentially what a prompt engineer does. It's just now the floodgates have opened in terms of what it provides you right. It's more than just those Google search results. It's more contextual information.
ROB: So, you know, I agree with you. 100%. So I didn't know what prompt engineering was. That I was doing prompt engineering, right? Before it was...because I was doing what you said. It was kind of like the engineering brains, like, okay, I'm going to do it this way. I want to ask it that way. Oh, it's pretty cool. And then you start learning from it and then yeah, you were engineering a prompt, right? As a CEO, write me an email on this promotion.
ADRIANA: Make it sound more beautiful. Another thing that I want to ask is, we both came about in technology before there was such a thing as Cloud, Kubernetes...We are children of the monolithic era of Java enterprise servers, which are no longer I don't know if I miss it or if I'm glad that that stuff's gone. What was your foray into Kubernetes? What led you in that path?
ROB: So I was doing consulting in Montreal, this is...whenever Docker 1.1 came out and was lucky enough that the company was kind of looking and really trying to find solutions around Docker. And we use Docker Compose and Docker Compose is kind of limiting solution.
And from there, just bringing the Docker world, we kind of started working with it. We had a few implementation of Docker Compose for clients, and then Kubernetes came, right? Early adapters...and kind of jumped on that because there was a limitation of controlling and deploying Dockers without a kind of orchestration platform. So we kind of started building for Google and Kubernetes and creating our own kind of platform with the CLI on how to deploy, ordering...
So we were kind of early kind of working on it and the tools that we have right now weren't there. We kind of build them ourselves. So that's how we jumped on on it. So it was through a client and then just thinking that it's really cool how you can kind of abstract to OS level virtualize, a little kind of component.
It was just kind of groundbreaking even though Linux had it before putting it in kind of element where, hey, we're using Eclipse. And then instead of deploying MySQL on my Windows box at the time we deployed it in Kubernetes...sorry...in Docker, which you can kind of start and turn on and off.
It wasn't some kind of heavy windows or Mac installation.
That kind of it was just bring it up, the Docker is there, and connect to it. And I was like, man, that's pretty cool, right? And I'm like, man, I don't know. As an engineer, it was kind of groundbreaking tech porn.
ADRIANA: Yeah, I totally agree.
ROB: I'm like, oh, my God, what can you do? And then not a lot of people were working on it, but we had some solutions, and then Kubernetes was the next kind of level.
ADRIANA: Yeah.
ROB: Kubernetes is complex, right. It's not easy. I wouldn't say for everybody to use it. There's a good case for it, but those benefits that it brought were pretty cool in terms of kind of working with containers and providing the networking and deploying. So kind of building around that. That's kind of our first foray to it. And it just continues until now.
ADRIANA: I think that's such a really good point on the containerization is the gateway drug, right to Kubernetes. I mean, it really is. Docker in itself was awesome. And then you're like, oh, shit. Now I've got to manage these Docker containers in tandem and figure out all this stuff, the networking and stuff between them. And then Docker Compose kind of helps you with that, and you're like, okay, that's better. And then you realize I need a little more, umph.
Oh, Kubernetes is like, the next natural evolution of it, where you're like, oh, my God, this makes things so much easier. But then at the same time, it's like, my life is hell. It's like, you can't win, right? It solves a problem, but then it brings on additional complexity because it is such a complex tool. But so cool.
ROB: Yeah, it I keep following kind of...some questions. Once use Kubernetes, and people are against it and big projects, small projects, I have a simple answer. The community of tools is so big right now, you got to use it because everybody's kind of working towards one goal, and that's the beauty of it, right? Yes. It's complex. Yes. It's hard. Yes. You got to have that's what we try to make it easier. Yes. You got to remember that managed Kubernetes is a little bit easier, but dealing with it overall, it brings complexities. But having every single tool, like Cloud Native tool, you go into a landscape, every single tool is deployable on Kubernetes, right?
So having that power and building from infrastructure-as-code and kind of Helm Chart and combining it all together, the power is there. That's kind of what I think is the biggest benefit. So, yeah, use it and then use it smartly. If somebody asks you when to use it or if it's good or bad, man, that's the wrong question. You find a problem, and there's solutions for it.
And if you want to build a WordPress site, build it on Wordpress.org or something, right? Or if you want to deploy WordPress and Kubernetes, deploy in Kubernetes. What is your need? What is your problem?
ADRIANA: I totally agree. And it's funny because I was having a similar discussion with folks today where I was chatting about Kubernetes and Nomad and how a lot of people talk about it in terms of a versus thing. But it's like, what is your use, case? When I worked at Tucows, it was a Nomad shop.
And it made sense because they had their own data centers, which meant that when they tried to start up their own Kubernetes clusters in their own data center, that's like you are creating your clusters from scratch, which is a horrible, horrible experience. Versus if they were using Public Cloud and have access to managed Kubernetes, maybe that would have changed the conversation.
But at the time, using data centers well, between running Nomad in a data center versus running Kubernetes in a data center, it's a lot easier to manage a Nomad cluster compared to a Kubernetes cluster. But then also, I guess some organizations might not need the additional complexity that you get with Kubernetes, and so they might choose Nomad or whatever other product because there's like, for example, VMware Tanzu, right?
They're a competitor as well in the space. I've not played with it, I've just heard of them, and that is the extent of my knowledge. But it's interesting to know that there are other competitors in the space that solve the problem, but in a different manner. And maybe that suits your use case better.
ROB: Yeah. So, when we were working, like, a few years ago, I felt it was nightmare to have Kubernetes on premises or data center, to your point, right?
No matter something, the tools were not baked in. Now it's easier. But that one leads me to a question for you. What do you think of...I read some articles that were kind of, I guess, headline grabbers. Cloud is dead. People are going back on premises or data centers.How do you feel where the world is going to go? Like, having a crystal ball...Cloud versus maybe people going back to their own data centers or hybrid. Any ideas there?
ADRIANA: Yeah, I think it's going to be a hybrid thing because here's my take on Cloud. I think Cloud abstracts a lot of the complexity that you would have for managing your own data center. And I think to a certain extent you can even manage the complexity of running your own data center through tools like OpenStack. And I think Azure has a thing called Azure Stack, and I'm sure the other Cloud providers have their own thing as well.
So you're basically having the same nice little infrastructure-as-code convenience in your data center rather than hosting in Public Cloud. Now. I think a lot of people treated like there was this mega rush to public cloud, I think because A, it was easy, and B, there was a lot of hype.
And then people forgot to look at the cost, where they're like, oh, this stuff is limitless. No, until you get your first cloud bill and you're like, "Shit, that was a massive cloud build."
Did I actually need all that stuff? But in terms of leaving it to somebody else to manage your infrastructure, awesome. But you have to be super mindful of your costs. Whereas when you're running your own data center, you are so mindful of your costs because you are keeping an eye on that budget like a hawk. Right? It's like, no, I do not have extra rack. Like, I ran into an issue when I worked at Bank of Montreal where we were setting up...we had to buy new physical server.
There was no rack space. They had to buy a rack. And because there was no rack space, they had to lay in the electrical work to be able to rack up that server. There was all this stuff that you take for granted when you're, when you're running in Public Cloud.
ROB: Yeah, I think it's going to be both. I'm just a proponent that it's really hard for data centers, specifically the smaller ones replicate the security, right? How do you do that, right? So you have these billion dollar clouds and their day in, day out as they go to an office and they think how to make it better, how to make it better. And over there where yes, now there's great tools from a lot of clouds to have infrastructure kind of as code infrastructures, cloud foundations for your data center. But that investment, continuous investment into securing it, that's what worries me, right?
Maybe like to your point, I'm hearing horror stories with managed services and cloud and cloud bills that it might be more kind of cost efficient to have that data center. Right. Because the cloud costs are so huge. Right. I think we're kind of still in the early stages, but I think it's going to be hybrid. I just don't know how cool will solve that issue of having a secure data center. So is it going to be your own data center stack?
So maybe know Bank of Montreal or the big banks or Canadian Tire might have their own because they have money for it. Right. And then you might have data centers that are kind of from the old age where they host stuff for you and you just have your rack in there. We might solve the cost savings issue, but if we don't, we're going to see some bigger blowback. But I just don't see yet how other companies can replicate that heavy investment those big three are doing into that security or whatever security or the future tools or that's kind of where the word is going to be. So I'm going to see where it's gonna go.
ADRIANA: Yeah, it'll be interesting and in the same way way that you're kind of keeping an eye on the whole data center situation. On prem or Cloud. I think we'll see a similar movement with the monolith versus the microservice, because, again, a lot of organizations rushed into the microservices model thinking, this is going to solve my problems. And then now they're rethinking it, which is rather interesting, which I'm not surprised by, because, again, it's like a lot of hype and a lot of people just did very knee-jerk reactions, rather than, is this actually going to do the thing that it's supposed to do for me?
ROB: I think it is going to go somewhere in between. As long as you work towards the scalability and elastic nature of the cloud, build it for that, right? So microservices are good for that if built well because you can isolate the problem, right?
If you can have a monolith, make sure you can do the same thing. Make sure you can scale the biggest thing of monolith. Once it couldn't scale and you had your 30 different features in one set, and then what, right. So there's room for both. And it's an architectural pattern they want to use.
I agree. But it's the same answer. The Kubernetes answers. It's the same kind of answers. When I see kind of, hey, I go to Reddit and I'm Kubernetes and I'm here and I'm there, it's the same thing. Oh, I would never use monolith. I'm like, man, that's not the right answer. Be more a little bit critical of what you're trying to say, why you fail with your problem. It's not like brass stroke for everything is the same. So room for both.
Got a question for you. Can I ask? Okay. Observability...what do you think of same thing with the cost because you're in the space and I think we had a conversation on it before, manage versus kind of in your stock because just an example, DataDogs and all that stuff, the same thing. You start slowly and then there's a boom, a bill, right? Is that bill justifiable millions of dollars? Where do you stand in that Observability world and what do you think about open source or in your kind of open source entire Kubernetes or kind of powered by open source versus kind of the fully managed solutions and the benefits kind of of that. Where do you stand with that?
ADRIANA: Yeah. So I'm going to put on my not a "I work for an Observability company hat," but my "I was in the position of managing an Observability team hat," and from that perspective...so when I worked at Tucows, I came in to manage two teams, a platform team and an Observability team. And the Observability team at the time, their function was basically managing tools and not focusing on practices. But we were also using a SaaS vendor. So internally managed tools plus SaaS vendor. I'm like, you know what, you've already got the contract with the SaaS vendor. Let's use that as the standard. Let's ditch the internal tools so then we can focus on practices and focus then on making sure that people are doing Observability properly and making sure that we standardize on the OpenTelemetry.
Because this was like the early days of OpenTelemetry, so traces weren't even general availability. Now we're at the point where traces are general availability, metrics are general availability and I think logs are stable, but depending on the language, it's like the specification is stable, but it's on a per language basis, like where things are. But long story short, OpenTelemetry has evolved a lot and for me it was more important coming into that team making sure that the organization was doing Observability properly rather than focusing on maintaining tools. Because if you're so focused on maintaining tools, then what's to say that you're actually doing Observability properly? So we wanted to set out a set of best practices across the org.
Now, we did run into cost overruns with the vendor that we were using, but the nice thing about using OpenTelemetry is it gave us this opportunity to...because my focus was, let's make sure that the organization instruments everything in OpenTelemetry. And they were not. They were using vendor SDKs at the time. But my goal was let's inform people on making sure that they adopt OpenTelemetry so that if you're stuck with a vendor that way you're not stuck with a vendor that's going to cost you a gajillion dollars.
Right now you have that flexibility of going to another SaaS vendor or...you know what, now you have the flexibility too. If you want to go the self-hosted model you have that kind of flexibility. But yeah, I feel like when you're evaluating vendor, you have to know what you're getting in bed with. Because as soon as with that particular vendor, we started moving away from their SDKs and started using OpenTelemetry, the cost shot up because they supported OpenTelemetry, but they treated the OpenTelemetry stuff as like extra I don't know, extra nodes or whatever, extra containers or some I forget what it was, but our costs shot up. It was shockingly horrifyingly expensive as a result.
So I think you need to understand the cost model up front. Unfortunately a lot of vendors have very complex costing models which then that makes it a little bit tricky. Yeah.
ROB: So when you said that if you design it properly, do you think you can very easily exchange the tools because your best practices are kind of build on OpenTelemetry and then you can kind of go from tool to tool? Is that kind of what you mean by best practices?
ADRIANA: Yeah, so best practices means...because the idea of Observability is your system is emitting enough information so that even without knowing the inner workings of the system, you have enough information so you can tell what's happening, right? So yeah, you can use OpenTelemetry but if your system is not emitting the right stuff then so what, right?
And it's a combination of emitting the right stuff and also making sure that the vendor is representing the information. So then when you instrument using OpenTelemetry the thing that differentiates the vendors is how they render that information. Is this going to be useful to you? So it's a combination of making sure that the code is instrumented properly and also is this thing showing up in a way that's useful to you so that you can troubleshoot. Right, so that I think becomes the trick.
ROB: Yeah, that's good, right. It's kind of like what we're concentrating with kind of our stack. But the journey is not understood, right? And I feel some vendors are overselling the promise because the tool will not solve everything and you can just get into a really bad practice of paying a lot because you're going to be searching for what to collect and just scraping everything possible. So that best practice we're talking about and then emitting the data, collecting data. That's a very important piece.
So back to the other question. So we have the practices and kind of OpenTelemetry and kind of instrumenting the code. Where do you find then after that's done, the SaaS model vendors and I don't want to pick on DataDog, there's a few others of them that are there. Where do you feel they fit into that once you have that set up the internal platform versus external SaaS model.
ADRIANA: In terms of what specifically?
ROB: For Observability. So comparing having Prometheus stacking your Kubernetes versus maybe connecting to again Logs.io, just say, right? Because they're kind of API based and kind of instrumenting kind of thing. Where do you think do you have an approach or preference towards one or you think it depends on the situational and company?
ADRIANA: Yeah, I think at the end of the day it just depends on your situation. When I started my Observability journey, my dream was to have a tool that took care of all the things. So in my ideal world you could do away with Prometheus because you can emit those Prometheus style metrics and then just ingest them into whatever system and you'll have a place that displays your metrics, your logs and your traces and they're all correlated nicely.
ROB: Right.
ADRIANA: I don't think that any one vendor does that well right now it's interesting too, like for example in OpenTelemetry there's a way right now to correlate your traces and your logs which is currently being implemented. There's a way to correlate your traces to your metrics. It's called a trace exemplar. But when you look under the covers...so a lot of people talk about trace exemplars. You look under the covers. It's not been implemented for a lot of languages. I think the only one that's actually been implemented for is Java. So then you'll see actually a lot of vendors that will do that correlation in the tool itself and not use OpenTelemetry for it, which is quite interesting. So there's still some work to be done. It'll be interesting to see where things go.
ROB: That's an interesting problem that I feel we always face because they're so wide to kind of adapt to so many different languages and tools and stuff and open it up and making sure can one company be doing everything well? It goes back to kind of can Apple do everything well? Can Microsoft do everything well? At what point can you invest in everything, right?
So that's going to be interesting to see when I was talking to somebody at a conference, what's going to happen eventually is people are going to be really buying out each other, right? We're going to reach that level where they're going to be eating up and then, hey, these guys are doing good. This level Observability, combine it together and then see if that works.
I spoke to you about it as well, kind of because where you are so that's going to happen. That's good and bad. Because that will kind of go to your point where maybe somebody's going to be able to create that kind of one tool by waiting to see if there's going to be enough appetite and investment to make those different parts of the tool well structured. So it's pretty cool. Pretty cool.
This whole Observability is just so crazy, so vast. You can spend just like and you can spend a world and all your time reading about it and you still can kind of tackle the fraction of it, right?
ADRIANA: Oh, yeah, absolutely. I know. I do this for a living, and I'm like, I've barely scratched the surface. Well, cool. We are just coming up on time. So, for parting words, do you have any awesome advice that you want to share with our lovely audience?
ROB: Go slow, talk to experts. If you do things, try to do them right the first time, but don't be afraid to fail. And iterate, right? So it's kind of challenging aspect there, but yeah, maybe for people that are starting out, touch technology, it's here with us. For AI, embrace it, don't hate it. It's here with us. There's ways of things, figuring it out. As long as we have a positive outlook for what we want to do, we're humans are very smart, we're going to solve it. So that's kind of the approach I take too. All these different things that are coming out, maybe because we're techies, we enjoy it more because we see the potential of it and I see huge potential and just where the world's going in a very good way, very positive way.
ADRIANA: Totally. That's awesome. Those are great words of wisdom. Well, thanks so much, Rob, for geeking out with me today, y'all. Don't forget to subscribe. And be sure to check the show notes for additional resources and to connect with us and our guests on social media. Until next time.
ROB: Peace out, and geek out.
ADRIANA: Geeking Out is hosted and produced by me, Adriana Villela. I also compose and perform the theme music on my trusty clarinet. Geeking Out is also produced by my daughter, Hannah Maxwell, who, incidentally, designed all of the cool graphics. Be sure to follow us on all the socials by going to bento.me/geekingout.
ADRIANA: Hey, y'all. Welcome to Geeking Out, the podcast about all geeky aspects of software delivery, DevOps, Observability, Reliability, and everything in between. I'm your host, Adriana Villela, coming to you from Toronto, Canada. And geeking out with me today is my good friend Robert Golabek. Welcome, Rob!
ROB: Hey, nice to be here.
ADRIANA: Super nice to have you on. And full disclosure, Rob and I have known each other for a really long time, like since, what...2000...I want to say 2005? It's been a while. We've known each other for a really long time in a past life, in our past lives as Java developers, which is really awesome. So Rob, for starters, where are you calling from?
ROB: I am from the deep west Toronto Etobicoke.
ADRIANA: Yay. Fellow Canadians.
ROB: Yeah, people don't know Etobicoke is a borough of Toronto, so some people call it Toronto, some people don't. For some people, I heard it's really far. For me, it's actually the perfect balance. Twenty minutes from Toronto. But yeah, get some kind of space. So yeah, that's kind of where I'm from.
ADRIANA: Cool. Awesome. Awesome. All right, so we're going to start with some rapid fire questions. Are you ready? I promise it won't hurt. All right, number one, are you a lefty or a righty?
ROB: Righty. And happy Lefty Day. I saw that post, so yes, your superpower...I was going to respond post of my right-handed rights.
ADRIANA: I always forget to acknowledge Left-Handed Day. And then this year I'm like, "I am going to schedule this post so I don't forget." And then when it popped up the next day, like on Monday when I was back at the office, I'm like, "Oh, Lefty Day passed. Oh, I remembered post on that."
ROB: It so the reason it's close to me is my dad is left-handed, right?
ADRIANA: Awesome.
ROB: And for some weird reason, it was weird when he was growing up to be left-handed. So they tried to even make him write it with the right hand, and it was kind of, you know, so yeah, it's dear to me.
ADRIANA: Yeah, totally. Yeah, my mom too, she was left-handed and she was subjected to people trying to make her write with her right hand. And she was one of those non-functioning-with-her-right-hand lefties...everything with the left hand. So she's like, "No." I can manage with some right handed stuff, but lefty and proud. All right, next question. iPhone or Android?
ROB: Android
ADRIANA: All right. Mac, Linux, or Windows for development?
ROB: Windows.
ADRIANA: Awesome. Favorite programming language?
ROB: The one I know, I got to say Java
ADRIANA: All right. Dev or Ops?
ROB: DevOps
ADRIANA: Yeah, I've gotten a few of those answers before. It's very PC. DevOps. All right. JSON or YAML?
ROB: Depends on the situation.
ADRIANA: All right, fair enough.
ROB: All right.
ADRIANA: Fair enough. All right. And then final question: do you prefer to consume content through video or text?
ROB: Text.
ADRIANA: All right. Yeah, the text people are winning so far. Most people are like, "Text." I'm right there with you.
ROB: And if there was a video, I watch it on mute. I like the writing.
ADRIANA: Do you read the subtitles?
ROB: Yeah...I don't know, I'm not a video person.
ADRIANA: Yeah, I know, right?
ROB: So kind of my age, I guess.
ADRIANA: My daughter Hannah, she's like video, no question about it. I'm like, really?
ROB: Yeah, I'm the same way.
ADRIANA: But yeah, maybe it is an age thing. I don't mind...I'll watch video with subtitles or I will just put on the audio and walk around the house and have it on YouTube...video on my phone, walk around the house with just the audio, and that I can consume...but I can't just sit there and watch a video. Especially for tech stuff.
ROB: Yeah, my my attention span is like, really, like, short. I want to kind of go to the end ofthe video, and I just want to read it very quickly because I usually skim through it and then I read the most interesting part in video. It's like, okay, where's the climax? You can't really find it.
ADRIANA: Yeah, I'm exactly the same, so I totally feel you. All right, so now that we're warmed up, let's geek out on some stuff. So I guess first things first. So why don't you share with everyone what you do? Because you've come, I guess, a long way from our early days in our earlier careers of being the lowly Java devs.
ROB: So yeah, so maybe start from the beginning, you know? '96, '97, '95. I don't know, just kind of coding. And at that point was involved with wires and illegal streaming. Got me interested. Kind of made some money from there. Very quickly. Went to Sheridan in 2000. Within a year, kind of graduated and then got my first job at SickKids Hospital as a reports developer that turned into Java developer, that kind of turned into architecture. It was pretty cool. And after that, that's when I went to Metavante, or is that the right name? I don't know. I think...
ADRIANA: I don't know what they're called anymore because it was like when I joined, it was called GHR, and then Metavante ate them up and then I don't know what happened after that.
ROB: Yeah. So I don't know. In between that, I was kind of in the Canadian Army Reserves too. So kind of got some discipline there. So yeah, it's kind of put me straight as an arrow. Got me kind of healthy and got some responsibility skills. That was from like '99 to 2004 while I was at SickKids. And then between '99 and 2010 while I was still working, I had a side business. So funny story, is in my first resume that I submitted to, I had like, a quote where I want to have a worldwide business where I kind of want to dominate and kind of provide value to people. And when I gave the resume to the SickKids people, they laughed. Right. And I'm like, was that naïve or was that aspiration to kind of something greater. So the entrepreneurship was always there, looking backwards. Maybe a little naïve but kind of inspiring to something greater was kind of my goal. That was kind of my beginnings of trying to take over the world. Pinky and the Brain 2006 that's kind of where I met you in Metavante. Worked there for three years. Went to Point Click Care. I don't know, I think everybody kind of knows here in Canada. Point Click Care one of the kind of unicorns in healthcare.
So I was there for six months. Sad story is I joined, somebody got the bonus for referring me - it was my brother - who's kind of with the company as me and then six months later I left
ADRIANA: Right when he got his bonus I'll bet.
ROB: And they changed the rule after me. I think they even call it the Rob Rule that referral...you got to work there a little bit longer. I didn't do it on purpose. That's kind of when I started my business after Point Click Care. Got my first contract kind of working and actually was with SickKids too, developing their platform and that's kind of where my journey started. And we're here today. And what we do is right now we matured and kind of through the innovation that we do and putting engineering before sales, which I don't always advise because if you have passion for engineering and you want to do everything right, it might hurt sales. But we're proud of that. We run the business our way. So because of that we always kind of innovate, not always to the benefit of kind of sales, but it got us to the journey of early adaptation of Docker, Kubernetes, Cloud Native always early adapters and now we're Cloud Native experts specializing in app modernization, trying to kind of build for the Cloud and the beauty of Cloud Native and optimization, which I love, is it's ever changing, right? So before was moving to the Cloud was legacy software. Now it's kind of the hot take is how do you add AI to software that already kind of are out there, right? In a few years it's going to be something else. So really love what I do, kind of giving the Cloud Native expertise and kind of sharing my wisdom with people.
And through that, sorry, I started Executive Espresso series where I started kind of like, you know what I love kind of talking to people. So started posting information, just kind of sharing on Cloud Native expertise and kind of the different aspects...Kubernetes, Observability...one thing that's challenging, which I tell you and it hurts me, is to be Cloud Native expert. I keep reminding myself how big the space is and like DevOps, Observability, Platform Engineering and cloud foundations, it takes a lot of learning and knowing and talking to people like you and different spaces. So I find that really challenging. But I enjoy that because in my DNA it's kind of learning. So combining all those things is pretty cool.
ADRIANA: And you touched on something really important, which is like the Cloud Native space is ginormous and technology is ginormous and there's a new thing out all the time, so then you can't stay on top of everything. So how do you pick what you focus on as a result of that?
ROB: So we can bring up maybe when you're doing the edits, you can bring up the landscape of the Cloud Native landscape. And I don't know how many tools they have now. Maybe 200, 300, a lot. So what we focus on is opinated experience technologies that we use. So we call it our Tek Stack, kind of powered by open source software. And we chose some tools right as the starting point. Now when we go to clients and kind of try to kind of give our opinions, it's based on that. Now it's also being open to other tools. But when you choose a tool, let it be mature, let it be kind of used by people, let it be a supporting community.
We did a mistake before in the past, where we were too early of an adapter and you pay the price. I think we did it with Angular 2. We did it way too fast. When Angular kind of went through versions of one to two was Angular 1, then it was 2, then it jumped all the way to 5, was too early. I wish we waited a little bit and kind of used it maybe a little bit later. And same with these tools.
So we broke it down into different tools for security. It's Falco, Consul, Vault, KeyCloak, kind of maybe HashiCorp kind of world. And then for kind of cluster resources, Postgres, Redis, OpenSearch, Kafka.
So you can see it's like main kind of tools that we kind of use.
And Observability: Prometheus stack...Kubernetes Prometheus stack, Sentry, Jaeger, Loki...Kind of making sure that we center on those tools and then making sure that adding principal infrastructure as code kind of on top of that and on top of Google, that's kind of how we chose the tools.
And that's like the starting point, right. You can see for Observability, I think it's a very similar stack as Logz.io uses or anybody kind of those seem to be the main kind of open source tools that are out there, and there's a lot of support for them. So that's kind of the biggest kind of aspect of selecting them. And they're really good, right?
So, yeah, that's kind of how we use them. But the biggest thing is through clients and through conversations, you always learn about the new tools. So best way is to throw your tools out there and then tell you some.
The conference you went to, I think, was from you. You threw one tool, I forget the name of it, that I never knew about. And I was like, okay, it was for platform engineering, I think, or I can't remember which tool it was. But you were using your presentation. It'll come to me.
ADRIANA: Oh, yeah. Kratix.
ROB: Kratix, right, right. I never knew that tool before because I never came across it. Right. But then you use that and then it kind of opens up, and then I can query you and be like, hey, how do you use it? Where's the support? So learning from the community and kind of expanding it and then making a selection, hey, is the tool that we want to use or not? Do we want to add it to our stack? Right. So that's pretty cool.
And I'll finish with this. That's kind of where the new thing of platform engineering is, I think. How can we best, to your question, select the best tools that maybe if I was going to propose to you, but also switch the game, what are you comfortable with and building around that most important part?
ADRIANA: Yeah, that's so true. It's so important because there's nothing worse...and I've been in the consultancy space before...and there's nothing worse than coming in and saying your stuff sucks and then you're just going to hurt their feelings. It's like basically saying you have an ugly baby and no one wants to hear that they have an ugly baby. You got to be gentle and understand. What are you comfortable with, what are you using? Hey, would you be open to switching over to this? If you're familiar with this, maybe this might be the thing for you. And I think that's very important, especially in consultancy, because you're essentially trying to help companies do things better. But there can be a lot of resistance to change, so you have to be very gentle with them.
ROB: It yeah, I don't know if I'm an engineer anymore. I'm ex engineer. I love engineering, but I spend more time doing non-engineering stuff. But there's one thing, right, that I always notice with engineers kind of myself, too, not excluding myself. There's that ego, right? I selected, I know the tools better. Prove me wrong. Why are you using this tool? And I don't like taking that conversation there. I'd rather being like, hey, if it's tools great, let's use it, let's improve it. Let's build what you guys need.
Right? But engineers are smart people. I'm going to say that they're usually intellectually smart, so they know what they're talking about. And you got to come with a game, too, to say that you know what you're talking about. So that's kind of where the conversation goes.
ADRIANA: Yes, absolutely. Is definitely a fine line. And I think one of the things in engineering, that engineering is an art form, really. And I think that goes to say for any type of art form is that sometimes we tend to fall in love with our code, with our technology, with the things that we create. But the best thing that we can do for our art is to give it some sort of a seed so that it can grow, whether it's like, hey, that sparks another idea where someone's like, hey, you know what? You could do this a little bit better. I like where you started, but I think this is how it can be improved. And being able to let go of your initial notions and be open-minded to other ideas, other ways of improving it, honestly, I think that's what open source is all about and I think that's what makes also for very successful organizations and very successful teams that you have to check your ego at the door.
It's hard though, because sometimes you're working on a thing and it's like it's your baby. You've put a lot of TLC into it only to have someone say, well, I found a better way of doing it. And it pretty much scraps all the stuff that you did that can hurt. But also recognizing that maybe your initial work, even though it's being discarded, inspired somebody to come up with a better way of doing things.
ROB: Yeah. I was going to ask you, what do you think is the best way of judging that? Right. How do you best put it out there? You kind of answered, I guess, open source. Right. Kind of let the community play with it. Any other kind of ways you would kind of try it out. Kind of let kind of people give you opinions in a non hateful control fashion.
ADRIANA: Yeah, generally just having conversations. I think it all comes back to community, whether it's putting it out there through open source or writing about it in a blog post or having a conversation with somebody. Finding ways to make those connections, I think is probably the best way but you can't do that without it being out there in some form or another, I think.
ROB: Yeah, I really like kind of in my recent time writing, right. So got me thinking and expressing and talking to people. Right. And then the biggest thing is taking that feedback in a positive way.
ADRIANA: Yeah.
ROB: First reactions like, oh, man, why did he say it that way? But then it's like, why did he say it that way? Maybe explore that a little bit more. Right. And then you meet the person, and then you have a different kind of perspective, and then you can change or you don't have to change. The biggest thing is to agree with them
ADRIANA: Yeah, absolutely. But I think the most important thing is that someone offering an opinion forces you to take a step back and rethink it. And it's like what you said, I'll either agree or, hey, there's something to that statement. Maybe I'll tweak it or take some of that into consideration, or like, no, I actually think my way is the better way. I've given it some thought and that's perfectly all right.
ROB: Yeah. And you might have different motives too, right. It could be business case, could be technology, could be different case. Just recently, we had somebody come in and they had an objective of looking from this kind of zoom...was monetary zoom. And it's like that's one way of looking at it. Right. And because it's a business client, they're going to push it in that way. Now, as an engineer, the most frustrating part is let go of your best practices. And then because most of the times client is right. Quote. You try to kind of make them happy but you got to really put your ego away and also put away I told you so because I believe even in that scenario business person could be right because now they're coming from their perspective with and they might have limits.
So you got to look at from that angle ego from technology, from business and kind of move the conversation forward.
ADRIANA: That makes a lot of sense. I think at the end of the day, you just have to be open-minded. So of all the technologies that you've been working with, what's the one that's really exciting you right now?
ROB: I love Conversational AI
ADRIANA: Oh, yeah. Cool.
ROB: So...and applying it to any domain. We're just working actually with Pat, working on Conversational Kube Bot, where you can talk to it in human language and get a response.
ADRIANA: Oh, nice. Is that something you guys are developing?
ROB: Yeah, we're developing it. I want to release it. It's kind of started as a kind of small project because we're in a grander schemes working on Enterprise search, and we call it Conversational Enterprise Search, and we call it, like, Next Knowledge Base Economy, where knowledge is king. And how can you take that knowledge and how can you converse with it right. At a basic level. Right. And applying it in Kubot, hey, get all the resources, all the material from kind of Kube Bot and then suck it in. Use kind of NLU, NLP, NLG, kind of all the kind of natural processing, human and language kind of processing. So you're able to create something where it's your human assistant. Right. So my goal is, like, I never want to remember a Kubernetes command. And with this, we already have a prototype where it's like, hey, tell me the status of the system and let's see all the pods or something, right?
ADRIANA: Oh, my God, that's so cool. I cannot tell you how many times, if I'm away from Kubernetes for a while, I have to Google this stuff. Or now I have a GitHub repo where I just have a README with all of my go to Kubernetes commands because I forget that stuff, especially the gnarly ones. Like, how do you freaking go through your logs in Kubernetes? Or how do you log into your pod? Into your container in your pod? Or be like, yeah...
ROB: I'm with you, man. I have a folder with documents, and it's for, like, Kubernetes, Docker this, and I'm like, get lost in those. And then it's like searching through those commands. So we're applying this Enterprise search and conversational search to Kubernetes and Observability. And it ties into AI Ops. So I'm in awe in how powerful large language models are and applied in the right case, I'm going to write a blog. It's on my to-do. I kind of have a draft format where take it from a different angle, right.
ADRIANA: Yeah, yeah.
ROB: How these AI tools can help the world, right. I see too much boom and doom, kind of, hey, they're going to break this, break that. It's going to take cover, control. Yes, everything, right? We have the biggest case of nuclear power. It's for good, it's for bad. It's our human choice to use it for the good. Right. And I'm always optimist.
So I love it because it can apply to so many...We just combined the few elements, AI search and Observability and Kubernetes and boom. That's something we're working on.
So that goes back to engineering and working cool stuff. So that's kind of what I really enjoy.
ADRIANA: That is super cool. Yeah, it's funny because I think AI has definitely become a hot topic because it's come up more than once in this podcast. I think my first dabbling into AI was, like, using DALL-E for generating images for my presentations. That was kind of my first one where I'm like, oh, my God, this is the coolest thing. I can tell it to generate pictures of llamas doing funny things. What? Or my favorite, I have this love for capybaras now because Instagram one day decided to serve me pictures and videos of capybaras.
And I'm like, oh, my God, this is such a glorious animal, you know, DALL-E has generated me a bunch of images of these things for my presentations as well. So I'm like, "Shit, that is some really rad stuff."
And then further leveraging ChatGPT for even certain things, where you find yourself in a position where I need to reword this thing. My brain is fried. "ChatGPT, just take the sentence that I wrote and make it a little bit shorter," because I don't have the brain power to try to think of five different ways of saying this word and conveying this thing, right?
ROB: So you're touching on something pretty cool, right? So it takes you to the next level. And some people say it actually does it for you. It doesn't yeah, it's going to be mind-blowing. It doesn't, because I can smell, like when a marketing person talks about technology thing, and it kind of doesn't make sense. And then when a techie will use the same kind of and they will just rephrase it. There's a difference.
So it's such a helpful...I love it. It's been changing. So we applied it kind of all over the place. Again, combining AI, Observability, DevOps like...crazy.
ADRIANA: It's going to be mind-blowing. And I think people forget that it's not like AI, as you said, AI is not going to do all the work for you. You still need the human touch to guide it in the direction, and then you still have to vet it because sometimes AI spits out some dumb-ass shit and you're like, "No, I do not want this." And then you just rephrase the question.
At first when I heard the term "prompt engineer", I'm like, "Ha ha. That's so hokey."
But we've been prompt engineers for a while now, if you think about it, in software, because that is essentially what we do when we do a Google search, especially when we're trying to solve a gnarly-ass problem and you enter a particular search term, and then you're like refining, refining, refining, until you're like, oh, you know what? That's not even the right question that I have to ask, but now I've got enough information that I know the right question to ask, and that's essentially what a prompt engineer does. It's just now the floodgates have opened in terms of what it provides you right. It's more than just those Google search results. It's more contextual information.
ROB: So, you know, I agree with you. 100%. So I didn't know what prompt engineering was. That I was doing prompt engineering, right? Before it was...because I was doing what you said. It was kind of like the engineering brains, like, okay, I'm going to do it this way. I want to ask it that way. Oh, it's pretty cool. And then you start learning from it and then yeah, you were engineering a prompt, right? As a CEO, write me an email on this promotion.
ADRIANA: Make it sound more beautiful. Another thing that I want to ask is, we both came about in technology before there was such a thing as Cloud, Kubernetes...We are children of the monolithic era of Java enterprise servers, which are no longer I don't know if I miss it or if I'm glad that that stuff's gone. What was your foray into Kubernetes? What led you in that path?
ROB: So I was doing consulting in Montreal, this is...whenever Docker 1.1 came out and was lucky enough that the company was kind of looking and really trying to find solutions around Docker. And we use Docker Compose and Docker Compose is kind of limiting solution.
And from there, just bringing the Docker world, we kind of started working with it. We had a few implementation of Docker Compose for clients, and then Kubernetes came, right? Early adapters...and kind of jumped on that because there was a limitation of controlling and deploying Dockers without a kind of orchestration platform. So we kind of started building for Google and Kubernetes and creating our own kind of platform with the CLI on how to deploy, ordering...
So we were kind of early kind of working on it and the tools that we have right now weren't there. We kind of build them ourselves. So that's how we jumped on on it. So it was through a client and then just thinking that it's really cool how you can kind of abstract to OS level virtualize, a little kind of component.
It was just kind of groundbreaking even though Linux had it before putting it in kind of element where, hey, we're using Eclipse. And then instead of deploying MySQL on my Windows box at the time we deployed it in Kubernetes...sorry...in Docker, which you can kind of start and turn on and off.
It wasn't some kind of heavy windows or Mac installation.
That kind of it was just bring it up, the Docker is there, and connect to it. And I was like, man, that's pretty cool, right? And I'm like, man, I don't know. As an engineer, it was kind of groundbreaking tech porn.
ADRIANA: Yeah, I totally agree.
ROB: I'm like, oh, my God, what can you do? And then not a lot of people were working on it, but we had some solutions, and then Kubernetes was the next kind of level.
ADRIANA: Yeah.
ROB: Kubernetes is complex, right. It's not easy. I wouldn't say for everybody to use it. There's a good case for it, but those benefits that it brought were pretty cool in terms of kind of working with containers and providing the networking and deploying. So kind of building around that. That's kind of our first foray to it. And it just continues until now.
ADRIANA: I think that's such a really good point on the containerization is the gateway drug, right to Kubernetes. I mean, it really is. Docker in itself was awesome. And then you're like, oh, shit. Now I've got to manage these Docker containers in tandem and figure out all this stuff, the networking and stuff between them. And then Docker Compose kind of helps you with that, and you're like, okay, that's better. And then you realize I need a little more, umph.
Oh, Kubernetes is like, the next natural evolution of it, where you're like, oh, my God, this makes things so much easier. But then at the same time, it's like, my life is hell. It's like, you can't win, right? It solves a problem, but then it brings on additional complexity because it is such a complex tool. But so cool.
ROB: Yeah, it I keep following kind of...some questions. Once use Kubernetes, and people are against it and big projects, small projects, I have a simple answer. The community of tools is so big right now, you got to use it because everybody's kind of working towards one goal, and that's the beauty of it, right? Yes. It's complex. Yes. It's hard. Yes. You got to have that's what we try to make it easier. Yes. You got to remember that managed Kubernetes is a little bit easier, but dealing with it overall, it brings complexities. But having every single tool, like Cloud Native tool, you go into a landscape, every single tool is deployable on Kubernetes, right?
So having that power and building from infrastructure-as-code and kind of Helm Chart and combining it all together, the power is there. That's kind of what I think is the biggest benefit. So, yeah, use it and then use it smartly. If somebody asks you when to use it or if it's good or bad, man, that's the wrong question. You find a problem, and there's solutions for it.
And if you want to build a WordPress site, build it on Wordpress.org or something, right? Or if you want to deploy WordPress and Kubernetes, deploy in Kubernetes. What is your need? What is your problem?
ADRIANA: I totally agree. And it's funny because I was having a similar discussion with folks today where I was chatting about Kubernetes and Nomad and how a lot of people talk about it in terms of a versus thing. But it's like, what is your use, case? When I worked at Tucows, it was a Nomad shop.
And it made sense because they had their own data centers, which meant that when they tried to start up their own Kubernetes clusters in their own data center, that's like you are creating your clusters from scratch, which is a horrible, horrible experience. Versus if they were using Public Cloud and have access to managed Kubernetes, maybe that would have changed the conversation.
But at the time, using data centers well, between running Nomad in a data center versus running Kubernetes in a data center, it's a lot easier to manage a Nomad cluster compared to a Kubernetes cluster. But then also, I guess some organizations might not need the additional complexity that you get with Kubernetes, and so they might choose Nomad or whatever other product because there's like, for example, VMware Tanzu, right?
They're a competitor as well in the space. I've not played with it, I've just heard of them, and that is the extent of my knowledge. But it's interesting to know that there are other competitors in the space that solve the problem, but in a different manner. And maybe that suits your use case better.
ROB: Yeah. So, when we were working, like, a few years ago, I felt it was nightmare to have Kubernetes on premises or data center, to your point, right?
No matter something, the tools were not baked in. Now it's easier. But that one leads me to a question for you. What do you think of...I read some articles that were kind of, I guess, headline grabbers. Cloud is dead. People are going back on premises or data centers.How do you feel where the world is going to go? Like, having a crystal ball...Cloud versus maybe people going back to their own data centers or hybrid. Any ideas there?
ADRIANA: Yeah, I think it's going to be a hybrid thing because here's my take on Cloud. I think Cloud abstracts a lot of the complexity that you would have for managing your own data center. And I think to a certain extent you can even manage the complexity of running your own data center through tools like OpenStack. And I think Azure has a thing called Azure Stack, and I'm sure the other Cloud providers have their own thing as well.
So you're basically having the same nice little infrastructure-as-code convenience in your data center rather than hosting in Public Cloud. Now. I think a lot of people treated like there was this mega rush to public cloud, I think because A, it was easy, and B, there was a lot of hype.
And then people forgot to look at the cost, where they're like, oh, this stuff is limitless. No, until you get your first cloud bill and you're like, "Shit, that was a massive cloud build."
Did I actually need all that stuff? But in terms of leaving it to somebody else to manage your infrastructure, awesome. But you have to be super mindful of your costs. Whereas when you're running your own data center, you are so mindful of your costs because you are keeping an eye on that budget like a hawk. Right? It's like, no, I do not have extra rack. Like, I ran into an issue when I worked at Bank of Montreal where we were setting up...we had to buy new physical server.
There was no rack space. They had to buy a rack. And because there was no rack space, they had to lay in the electrical work to be able to rack up that server. There was all this stuff that you take for granted when you're, when you're running in Public Cloud.
ROB: Yeah, I think it's going to be both. I'm just a proponent that it's really hard for data centers, specifically the smaller ones replicate the security, right? How do you do that, right? So you have these billion dollar clouds and their day in, day out as they go to an office and they think how to make it better, how to make it better. And over there where yes, now there's great tools from a lot of clouds to have infrastructure kind of as code infrastructures, cloud foundations for your data center. But that investment, continuous investment into securing it, that's what worries me, right?
Maybe like to your point, I'm hearing horror stories with managed services and cloud and cloud bills that it might be more kind of cost efficient to have that data center. Right. Because the cloud costs are so huge. Right. I think we're kind of still in the early stages, but I think it's going to be hybrid. I just don't know how cool will solve that issue of having a secure data center. So is it going to be your own data center stack?
So maybe know Bank of Montreal or the big banks or Canadian Tire might have their own because they have money for it. Right. And then you might have data centers that are kind of from the old age where they host stuff for you and you just have your rack in there. We might solve the cost savings issue, but if we don't, we're going to see some bigger blowback. But I just don't see yet how other companies can replicate that heavy investment those big three are doing into that security or whatever security or the future tools or that's kind of where the word is going to be. So I'm going to see where it's gonna go.
ADRIANA: Yeah, it'll be interesting and in the same way way that you're kind of keeping an eye on the whole data center situation. On prem or Cloud. I think we'll see a similar movement with the monolith versus the microservice, because, again, a lot of organizations rushed into the microservices model thinking, this is going to solve my problems. And then now they're rethinking it, which is rather interesting, which I'm not surprised by, because, again, it's like a lot of hype and a lot of people just did very knee-jerk reactions, rather than, is this actually going to do the thing that it's supposed to do for me?
ROB: I think it is going to go somewhere in between. As long as you work towards the scalability and elastic nature of the cloud, build it for that, right? So microservices are good for that if built well because you can isolate the problem, right?
If you can have a monolith, make sure you can do the same thing. Make sure you can scale the biggest thing of monolith. Once it couldn't scale and you had your 30 different features in one set, and then what, right. So there's room for both. And it's an architectural pattern they want to use.
I agree. But it's the same answer. The Kubernetes answers. It's the same kind of answers. When I see kind of, hey, I go to Reddit and I'm Kubernetes and I'm here and I'm there, it's the same thing. Oh, I would never use monolith. I'm like, man, that's not the right answer. Be more a little bit critical of what you're trying to say, why you fail with your problem. It's not like brass stroke for everything is the same. So room for both.
Got a question for you. Can I ask? Okay. Observability...what do you think of same thing with the cost because you're in the space and I think we had a conversation on it before, manage versus kind of in your stock because just an example, DataDogs and all that stuff, the same thing. You start slowly and then there's a boom, a bill, right? Is that bill justifiable millions of dollars? Where do you stand in that Observability world and what do you think about open source or in your kind of open source entire Kubernetes or kind of powered by open source versus kind of the fully managed solutions and the benefits kind of of that. Where do you stand with that?
ADRIANA: Yeah. So I'm going to put on my not a "I work for an Observability company hat," but my "I was in the position of managing an Observability team hat," and from that perspective...so when I worked at Tucows, I came in to manage two teams, a platform team and an Observability team. And the Observability team at the time, their function was basically managing tools and not focusing on practices. But we were also using a SaaS vendor. So internally managed tools plus SaaS vendor. I'm like, you know what, you've already got the contract with the SaaS vendor. Let's use that as the standard. Let's ditch the internal tools so then we can focus on practices and focus then on making sure that people are doing Observability properly and making sure that we standardize on the OpenTelemetry.
Because this was like the early days of OpenTelemetry, so traces weren't even general availability. Now we're at the point where traces are general availability, metrics are general availability and I think logs are stable, but depending on the language, it's like the specification is stable, but it's on a per language basis, like where things are. But long story short, OpenTelemetry has evolved a lot and for me it was more important coming into that team making sure that the organization was doing Observability properly rather than focusing on maintaining tools. Because if you're so focused on maintaining tools, then what's to say that you're actually doing Observability properly? So we wanted to set out a set of best practices across the org.
Now, we did run into cost overruns with the vendor that we were using, but the nice thing about using OpenTelemetry is it gave us this opportunity to...because my focus was, let's make sure that the organization instruments everything in OpenTelemetry. And they were not. They were using vendor SDKs at the time. But my goal was let's inform people on making sure that they adopt OpenTelemetry so that if you're stuck with a vendor that way you're not stuck with a vendor that's going to cost you a gajillion dollars.
Right now you have that flexibility of going to another SaaS vendor or...you know what, now you have the flexibility too. If you want to go the self-hosted model you have that kind of flexibility. But yeah, I feel like when you're evaluating vendor, you have to know what you're getting in bed with. Because as soon as with that particular vendor, we started moving away from their SDKs and started using OpenTelemetry, the cost shot up because they supported OpenTelemetry, but they treated the OpenTelemetry stuff as like extra I don't know, extra nodes or whatever, extra containers or some I forget what it was, but our costs shot up. It was shockingly horrifyingly expensive as a result.
So I think you need to understand the cost model up front. Unfortunately a lot of vendors have very complex costing models which then that makes it a little bit tricky. Yeah.
ROB: So when you said that if you design it properly, do you think you can very easily exchange the tools because your best practices are kind of build on OpenTelemetry and then you can kind of go from tool to tool? Is that kind of what you mean by best practices?
ADRIANA: Yeah, so best practices means...because the idea of Observability is your system is emitting enough information so that even without knowing the inner workings of the system, you have enough information so you can tell what's happening, right? So yeah, you can use OpenTelemetry but if your system is not emitting the right stuff then so what, right?
And it's a combination of emitting the right stuff and also making sure that the vendor is representing the information. So then when you instrument using OpenTelemetry the thing that differentiates the vendors is how they render that information. Is this going to be useful to you? So it's a combination of making sure that the code is instrumented properly and also is this thing showing up in a way that's useful to you so that you can troubleshoot. Right, so that I think becomes the trick.
ROB: Yeah, that's good, right. It's kind of like what we're concentrating with kind of our stack. But the journey is not understood, right? And I feel some vendors are overselling the promise because the tool will not solve everything and you can just get into a really bad practice of paying a lot because you're going to be searching for what to collect and just scraping everything possible. So that best practice we're talking about and then emitting the data, collecting data. That's a very important piece.
So back to the other question. So we have the practices and kind of OpenTelemetry and kind of instrumenting the code. Where do you find then after that's done, the SaaS model vendors and I don't want to pick on DataDog, there's a few others of them that are there. Where do you feel they fit into that once you have that set up the internal platform versus external SaaS model.
ADRIANA: In terms of what specifically?
ROB: For Observability. So comparing having Prometheus stacking your Kubernetes versus maybe connecting to again Logs.io, just say, right? Because they're kind of API based and kind of instrumenting kind of thing. Where do you think do you have an approach or preference towards one or you think it depends on the situational and company?
ADRIANA: Yeah, I think at the end of the day it just depends on your situation. When I started my Observability journey, my dream was to have a tool that took care of all the things. So in my ideal world you could do away with Prometheus because you can emit those Prometheus style metrics and then just ingest them into whatever system and you'll have a place that displays your metrics, your logs and your traces and they're all correlated nicely.
ROB: Right.
ADRIANA: I don't think that any one vendor does that well right now it's interesting too, like for example in OpenTelemetry there's a way right now to correlate your traces and your logs which is currently being implemented. There's a way to correlate your traces to your metrics. It's called a trace exemplar. But when you look under the covers...so a lot of people talk about trace exemplars. You look under the covers. It's not been implemented for a lot of languages. I think the only one that's actually been implemented for is Java. So then you'll see actually a lot of vendors that will do that correlation in the tool itself and not use OpenTelemetry for it, which is quite interesting. So there's still some work to be done. It'll be interesting to see where things go.
ROB: That's an interesting problem that I feel we always face because they're so wide to kind of adapt to so many different languages and tools and stuff and open it up and making sure can one company be doing everything well? It goes back to kind of can Apple do everything well? Can Microsoft do everything well? At what point can you invest in everything, right?
So that's going to be interesting to see when I was talking to somebody at a conference, what's going to happen eventually is people are going to be really buying out each other, right? We're going to reach that level where they're going to be eating up and then, hey, these guys are doing good. This level Observability, combine it together and then see if that works.
I spoke to you about it as well, kind of because where you are so that's going to happen. That's good and bad. Because that will kind of go to your point where maybe somebody's going to be able to create that kind of one tool by waiting to see if there's going to be enough appetite and investment to make those different parts of the tool well structured. So it's pretty cool. Pretty cool.
This whole Observability is just so crazy, so vast. You can spend just like and you can spend a world and all your time reading about it and you still can kind of tackle the fraction of it, right?
ADRIANA: Oh, yeah, absolutely. I know. I do this for a living, and I'm like, I've barely scratched the surface. Well, cool. We are just coming up on time. So, for parting words, do you have any awesome advice that you want to share with our lovely audience?
ROB: Go slow, talk to experts. If you do things, try to do them right the first time, but don't be afraid to fail. And iterate, right? So it's kind of challenging aspect there, but yeah, maybe for people that are starting out, touch technology, it's here with us. For AI, embrace it, don't hate it. It's here with us. There's ways of things, figuring it out. As long as we have a positive outlook for what we want to do, we're humans are very smart, we're going to solve it. So that's kind of the approach I take too. All these different things that are coming out, maybe because we're techies, we enjoy it more because we see the potential of it and I see huge potential and just where the world's going in a very good way, very positive way.
ADRIANA: Totally. That's awesome. Those are great words of wisdom. Well, thanks so much, Rob, for geeking out with me today, y'all. Don't forget to subscribe. And be sure to check the show notes for additional resources and to connect with us and our guests on social media. Until next time.
ROB: Peace out, and geek out.
ADRIANA: Geeking Out is hosted and produced by me, Adriana Villela. I also compose and perform the theme music on my trusty clarinet. Geeking Out is also produced by my daughter, Hannah Maxwell, who, incidentally, designed all of the cool graphics. Be sure to follow us on all the socials by going to bento.me/geekingout.