We're back after our summer break, and we have a great guest to kick things off for us! Liz Fong-Jones, Field CTO of Honeycomb, geeks out with Adriana on what it's like to be a Field CTO. Liz talks about the transition from DevRel to Field CTO, and the differences between the two roles. She also shares some 🌶️🌶️🌶️ takes on the Gartner Magic Quadrant, and provides insights on the evaluation process.
About our guest:
Liz is a developer advocate, labor and ethics organizer, and Site Reliability Engineer (SRE) with nearly two decades of experience. She is currently the Field CTO at Honeycomb, and previously was an SRE working on products ranging from the Google Cloud Load Balancer to Google Flights.
Find our guest on:
Find us on:
Show notes:
Additional notes:
Transcript:
ADRIANA: Hey, fellow geeks, welcome to Geeking Out, the podcast about all geeky aspects of software delivery DevOps, Observability, reliability, and everything in between. I'm your host, Adriana Villela, coming to you from Toronto, Canada, and geeking out with me today, I have Liz Fong Jones of Honeycomb. Welcome, Liz.
LIZ: G'day, Adri, from Sydney, Australia.
ADRIANA: Thank you for waking up early to record.
LIZ: That's kind of my life these days, given that I work with people from the US and Canada, start early, take a break midday, and then work late to catch the UK.
ADRIANA: Oh, damn. Wow, that is a lot.
LIZ: You know, that's a voluntary choice that I made to move to Australia, so I fully accept that.
ADRIANA: Are you permanently moved to Australia? Because before, I remember, you were splitting your time between.
LIZ: I'm still splitting my time, but, you know, I have a house here, I have clients here, so I'm spending several months a year here.
ADRIANA: Oh, nice, nice. And hopefully...how's the weather down under right now?
LIZ: A little bit chilly and rainy, but, you know, not by Canadian standards, right?
ADRIANA: True, true.
LIZ: People are complaining. Oh, like, you know, it's like, you know, 10 degrees or 15 degrees, and I'm just like, yeah, whatever, it's fine.
ADRIANA: I know, right?
LIZ: I have a jacket.
ADRIANA: There you go. Yeah, we've had, um, kind of, we've had a hot summer in Toronto, actually. Like, like Brazil hot, which is where I'm from originally. And, yeah, I've...I have a pretty good heat tolerance, but I have been melting, so...
LIZ: Yeah, yeah, it's fun to, you know, it's like the sauna to the, to the ice. To the ice bath, right? Like going back and forth. You get used to rapid climate changes in addition to time zones. That's something that no one tells you about is climate change.
ADRIANA: Yeah, yeah, exactly, exactly. Well, I'm super excited to have you on...on the show and, and for folks who have been listening to Geeking Out, Liz was On-Call Me Maybe way back when, when I used to host that with Ana. And so I'm very excited that you've agreed to come on. Now, before we start off, I always like to start my guests off with some icebreaker questions. So are you ready?
LIZ: As ready as I'm going to be.
ADRIANA: All right, first question. Are you a lefty or a righty?
LIZ: I am a righty, like a majority of the population.
ADRIANA: All right, do you prefer iPhone or Android?
LIZ: I am an Android user because Google gave them to me for free for about a decade. I was one of the early, one of the Android beta testers. So that meant that I got free phones that might break. And that habit has carried on since after I left Google.
ADRIANA: Oh, that's so cool. How was it like having used like the early Android phones? What kind of experience was that?
LIZ: Yeah, you get them early in the technical validation process and you help carry them through all the way to production. Because of NDA, I can't talk exactly about what the experience was like. But yeah, no, it's cool having access to, to the latest hardware. It's actually though, it makes you weirdly paranoid because you have to hide your phone from being photographed by others. You have to slip it in your pocket at all times. You can't just leave it out on the table. So it does introduce some interesting complications to your life. But it was worth it at the time to get access to the latest and greatest hardware and to give the team feedback on it.
ADRIANA: Wow, that's so cool. Okay, next question. Do you prefer using Mac, Linux, or Windows?
LIZ: I am a hardcore Linux user with one exception. Well, it's technically still Linux on the desktop. I am a ChromeOS user for my laptop, again like habit from my Google days. But yes, I do my development in a VM on that ChromeOS machine. So it's Linux. I'm talking to you from a Linux machine. I have a habit of building mini ITX PCs that are all Linux based. I think I've got four little computers running around, each of which is its own independent Linux system.
ADRIANA: So are you then a lifelong Linux user? Did you ever dabble in Windows?
LIZ: I've been using Linux since I was 16, since 2003, 2004, cool. Yeah, yeah, yeah. But it's been my primary environment since, since college. So since 2005.
ADRIANA: I was, I guess forced...DOS was forced upon me because I mean like when Windows...
LIZ: Oh yes, of course, right, like MS DOS. Yeah, no, no, no, as a kid. Yes. Yeah, QBasic. Yeah, yeah.
ADRIANA: QBasic, oh my God, yes! Exactly!
LIZ: I knew I was destined to become a programmer when, in I think fourth grade, I wrote a program that would take three sets of coordinates and solve the quadratic equations.
ADRIANA: That's so cool. My dad got me into BASIC. He pulled me aside when I was ten and he's like, how'd you like to see something cool? I'm like, all right.
LIZ: Yeah. Yep. It runs in my biological family. I've got uncles and aunts who work in IT. So yeah.
ADRIANA: That's awesome. Okay, next question. What's your favorite programming language?
LIZ: Ooh, favorite. That's an interesting question. So I'm somewhat known for solving Advent of Code every year. I've been doing Advent of Code every year since it started in goodness, I don't even remember when it started, but like 2017 or something like that. So I've been doing Advent of Code in Go publicly on stream when I can for the better part of a decade now. So definitely Go is a programming language...I even use in recreation, the language that I use professionally. But it's really hard to pick favorites because, you know, I work with clients, clients use all kinds of languages.
I have to be a little bit of a polyglot as a result. So like, I do, I have a project that's written in typescript, for instance. I think it's really important to not, you know, just settle into rut and be like, I am a Java programmer, right? I think you kind of have to see and experience kind of what's going on. So at some point I will pick up Rust, I am sure, and become the prototypical stereotype of a trans cat girl who programs in Rust and has stripey socks, but that's not going to be today.
ADRIANA: Fair enough, fair enough. I do really like what you said about just broadening your horizons and learning other languages because I was actually like a Java developer for like 15,16 years and that was like my whole life. And then a friend introduced me to Python and, you know, I was like in my 30s at the time and I'm like, oh, so cool. Like, you know, it was for the first time, like since, you know, my BASIC days I did QBasic, Visual Basic, that I was like actually picking up another language and I'm like freaking cool. I got to do more of that. And yeah, and that was like my first sort of like, oh my God, you can learn another language at the same time, which is ridiculous when you think about it. Of course you can. Next question. Do you prefer dev or ops?
LIZ: I prefer ops. I still, despite my railing on about how you shouldn't try to be a hero, I personally enjoy that feeling of solving the problem. I'm not going to say that I enjoy necessarily being the hero, but I definitely enjoy being, being the person who has the insight that solves a problem, right? Like, you know, it's weird, right? Like, you know, when you, when you're doing dev stuff, like, you know, there, there is some degree of, you know, I'm, I have a start of the problem. I have the end of the problem, I fill in the stuff in the middle, right? Like you have some idea as to how it's going to go because you've decomposed the pro- the problem enough, right? Yeah, I think with ops it's a little bit more unpredictable. There's a little bit more novelty. Right? Because you don't know what's going to happen when you open up the box, right? And I, and I think that's, that's, that's the fun thing.
ADRIANA: Yeah. Yeah.
LIZ: So I'm not going to say it's necessarily about, you know, the esteem of having people be like, Liz, you solved it. Like...But it's much more about the, you know, I find that it's really interesting to do the ops and, and to, and to find new things.
ADRIANA: Yeah, absolutely. Totally agree with you. All right, next question. Do you prefer JSON or YAML?
LIZ: I think I prefer JSON because it is not whitespace sensitive and, you know, you can pretty print JSON if you need to. That being said, my pet peeve about JSON is the fact that they do not support the trailing comma in lists and that peeves me off to...like nothing else. But no, I have to interact with YAML because of Kubernetes manifests and CircleCI configs and I have broken enough YAML configs. Oh, and the hand handling of floats and the handling of like, variables. Can we not just quote all of the keys and call it a day? Right? Like, it's stuff like that that just drives me up the wall.
ADRIANA: Yeah, I feel, ya. I've heard a couple of horror stories with YAML. Like someone was telling me the other day, like, "ON", which is the abbreviation for Ontario, is interpreted by YAML as "on", "true". So it's like...I know, right? So it's like all these little nuances in YAML where you have to be like extra careful. Plus the white space. I still like YAML myself because I find it a little bit more readable than JSON because of like my Java days. All the curly braces in JSON just kind...
LIZ: Just feed it to JQ and you'll be good, right? That's literally what I do anytime I encounter anything that's in JSON is I immediately pretty print it. You're right.
ADRIANA: Oh, yeah. I have to pretty print it because I just cannot function otherwise. I feel you. Okay, next question. Do you prefer spaces or tabs?
LIZ: I am a Go programmer, so I am obliged to tell you tabs. But let's be real. I think that spaces work a little bit better in text editors because they actually run consistently. Like, I have to manually configure, like my tab with in Nano, my favorite text editor, in order to, you know, whenever I set up a new machine, because it defaults to eight spaces for a tab and that just eats your screen, right? Two, two, right? Like, so, yeah. I personally would prefer spaces, except for Go makes me use tabs.
ADRIANA: Ah, fair enough, fair enough. Okay, two more questions to go. Do you prefer to learn through video or text?
LIZ: I am a text person. I use captions whenever I can because I just read so much faster than I audio process. Or rather, if I'm going to be listening to video, I have to listen to it at 1.75 x or 2x. I just have to. It's not that my audio processing is slow. It's actually the opposite of that. It's that interacting with video at 1.0x is painful, and I will often multitask something else with it. If you make me watch a video at 1.0x. Cue the obligatory HR videos where you have to sit...sit and watch them, and like, click the little spinner at the end of 1 minute precisely. And it's just like, yeah, so, yeah, text. Because I can read it however fast I want without having to wait for the speaker to deliver the words.
ADRIANA: Yeah, I agree with you. I actually watch TV with captions on, and it's just so...
LIZ: I watch TV with captions on and I do something else that if I, rightly, because I process not quite at 2.0x, I process at 1.75x. So as a result, like, if I do two things at the same time, I'm going to miss a little bit of each one. But the captions help me stay like...
ADRIANA: Yeah, yeah, yeah, absolutely, absolutely, yeah.
LIZ: ADHD high 5.
ADRIANA: One thing that I've never, like, been able to do is actually listen to stuff at like, high speed. Like faster than 1x. Like, when I'm listening to a podcast, if I accidentally hit a button that makes it 1.25x, it actually drives me crazy. Like, they're talking too fast. Like, it breaks my brain. I don't know what it is, but so many people I talk to, they're like, I can only listen to it at, like, listen to stuff at super high speed. I'm like, more power to you all. Does not work for me at all.
Okay, final question. What is your superpower?
LIZ: What is my superpower? My superpower is I am...are you familiar with Dungeons and Dragons?
ADRIANA: Yes. High-level familiar.
LIZ: Okay...there is a class that was added on afterwards that's called the Factotum, and the Factotum is able to emulate the ability of any other class by using a certain number of knowledge points. I am a Factotum, right? Like, I am specialized in nothing in particular, but I am really good at being able to do things that specialists in other fields would do, but only one or two of them, and otherwise being able to communicate with specialists in other fields, right?
ADRIANA: Yeah.
LIZ: So I can get people to talk to each other who don't necessarily kind of share common ground or share kind of skills. I can pick up those people's skills and use them, at least for a time. So that's my superpower.
ADRIANA: That is such a great superpower. And this leads nicely into our main, or one of our main topics of conversation, which is, I think when you and I met, you were at Honeycomb working as a DevRel. But in the last couple of years, you've transitioned over to being a Field CTO. And for folks in our audience, it would be great if you could explain what that entails and also what prompted you to make that change.
LIZ: Yeah. So Field CTO is a role that is relatively new in the industry, and it really depends from Field CTO to Field CTO, company to company. Um, so actually, the one of the field CTOs at Confluent, um, whose name is Kai, wrote a piece about it. But in essence, you know, regardless of what someone's background is and where they come, come to being a Field CTO from being a Field CTO is about interacting with customers who are making very sophisticated use of your technology or otherwise have really interesting and gnarly technical problems or social problems. Honestly, the social problems are the more interesting ones that they need the help of someone who is an expert and an expert at the executive level to solve, right? So unlike a DevRel, right? Like, you know, I don't necessarily...
I don't necessarily write "how to" blogs anymore. I don't necessarily, you know, yes, I do speak at the occasional conference, but more and more of my time is spent on site with customers. And I think that is, you know, interacting one on one with customers is something that I really, really treasure because it means that I get to see all the cool things they're doing with Honeycomb. And the other piece of my job that I really enjoy is going back to the product development team. And actually, I try to, as best as I can, carry water and chop wood for them and also help solve technical problems that our customers are having at scale in our tool. So, for instance, I am working this week on something where a customer was like, we want more than 100 group by fields in a Honeycomb query. And I'm like, okay, I'll see what I can do. Let's talk to the team to see if it's possible. Let's try it out. I've been Field CTO for about two years now, coming up on two years in October, and it's been super, super rewarding to now be at the executive level in Honeycomb, to have the opportunity to interact with the executives at other companies. It's weird. There's not like, you know, a sudden transition in job responsibilities. I think it was more like, there's this funny thing that happened. In July of 2022, I was invited by Amazon to give the keynote at, or one of the customer keynotes at AWS Summit, New York. And they gave me, the AWS PR team, gave me a lot of side eye about, oh, you're, you know, not an executive.
You're not a CTO or VP. Like, you know, what are you doing up on stage as a principal engineer, right? And that was kind of a catalyst of, okay, fine. Like, you know, titles do matter at some point, right? So I didn't change what I was doing overnight, but instead, I kind of gradually fell into the role, and then the job titles changed afterwards.
ADRIANA: Oh, that's so cool.
LIZ: Yeah.
ADRIANA: So did you find then, because it was more of a gradual change. Like, there was, like, less. Like, was. Was there anything still, like, super jarring that stood out for you when you. When you made that change? Or. Or.
LIZ: I think the main thing that has really been kind of challenging is, you know, when I was DevRel, I was part of the Devrel team, right? Like, you know. You know Jessitron, you know Martin Thwaites, right? Like, you know Martin Dot Net, right? Like, so, you know, they're an amazing team, but I'm not part of that team anymore, right? Like, Charity and I are off on our own as the office, the CTO. And I think that that is a little bit of a change in that. I'm not part of marketing anymore.
I kind of don't have a department. So I work across all the departments, but there's not necessarily anyone I can lean on who's like, you're working with me. Let's do this. So I kind of have to beg and borrow to work with people. And, of course, people are happy to have the opportunity to work with me, but I'm nothing part of their planning processes, right? If I show up and, you know, as happened the past six months, right? Like, you know, if...
If I show up and say, you know, hey, by the way, we're going to be trying out Graviton 4,right? Like, you know, that's...that's something where I either need to drive myself or, you know, I need to just find someone who wants to geek out about it with me.
ADRIANA: Yeah. Yeah. So does part of your job then entail, like, you coming up with some interesting use cases to try out? Or are you driven more by customer asks, or is it a bit of both?
LIZ: Customer and partner asks? But I do think that they're, for instance, the thing with Graviton 4, that's Amazon, who's a partner of ours. They asked us to try it and I said yes. You don't in general say no when Amazon asks you, do you want to try this shiny cool thing? But, yeah, I think that majority of what I do is driven by what large customers are experiencing or what I can see they will be experiencing, right? Like, I think there is some room for thought leadership, right? There is some room for, like, looking ahead of where things are. But historically, as a company, Honeycomb has trended always, you know, too far ahead of where the ball is today.
Right? Where people can't necessarily see. This is, you know, how it aligns with what they're, what they're doing today. And we're trying to course correct that now and meet people, people where they're at now. So that's where I find myself spending a majority of time now is pragmatically connecting where people are with the challenges that Honeycomb helps solve for them. And also seeing these are the integration points that we're going to need, right? So one of the projects I'm working on is relating to better log support in Honeycomb.
Because it turns out that despite Charity and me saying, you know, throw your logs in the bin. Actually, no, you can't. Kubernetes emits logs. You're not going to throw Kubernetes in the bin. So what do we do about that? What do we do about your legacy applications? Looking at that is something I'm contributing to, and that's really driven by what customers ask me about every day.
ADRIANA: Right. That's so cool.
LIZ: Yeah. In terms of superpowers, another superpower...we were just...also, how I consume...I read text incredibly fast. As a result, I'm in several hundred different Slack channels and I read them all and I can just do that. It's great.
ADRIANA: Oh, damn. That is a superpower.
LIZ: Yeah. I don't listen to all of this calls that our sales team have with clients. But, boy, do I ever read the gong summaries of all of the call recordings.
ADRIANA: So it's funny because when you were describing the nature of the job, initially, it almost felt like. Almost like a consultant role, but non technical. But it is totally not that, because there's definitely. It sounds like there's some very, very technical aspects. So you're...you're kind of like a...a super tactical consultant who is working with, like, very high level, like, executives kind of thing.
LIZ: Executives and principal engineers. Yeah, right. Like, you know, that's that first point of call of, there's this really interesting or weird customer who's asking this question they've never seen before, right? Like, hit me with it. Like, I've been around the system long enough, and also, I'm aware of what the best practices are around observability.
ADRIANA: So does this. I wonder, like, does part of your job entail also, like, working with some of the solutions engineers? Like, pairing with them on that?
LIZ: Yeah. So I work with our solutions engineers. I work with our customer support and our customer success team. I work with customer architects. I work with software developers. That's why I say I'm a Factotum. I have to be able to speak sales. I have to be able to speak engineering.
I have to be able to speak marketing. I have to be able to communicate with all of these people and collaborate with them daily.
ADRIANA: What's been your favorite part of being Field CTO so far?
LIZ: I think my favorite part is the variety of it. No customer is alike. I think that's a lot of fun. I think the gratification and the payoff of this is what we're building. This is how people are really, actually leveraging it. I think that's also really, really satisfying.
ADRIANA: Is there any sort of thing that you've been working on that you're allowed to talk about where you're like, oh, my God, this has been, like, the coolest thing I've gotten to do.
LIZ: Yeah. I think one of my on and off fascinations is continuous profiling, and it is very, very weird in that, you know, it interacts with the very, very guts of the kernel, of the runtimes. So getting to interact with one of the Go subsystem maintainers, cherrymui, and sending her crash reports when the profiling doesn't work completely according to plan, getting to work on our integration with profiling that we developed a couple years back and that we continue to use ourselves, I think that's a lot of fun because it shows how much depth there is if you really, really, really want to get into understanding the performance of your system. I do not necessarily recommend that our clients do this. There's so much low hanging fruit to find just via tracing, but we aim to be cost effective. We aim to be fast. And part of how you get there is by looking at continuous profiling and looking at the data down to the kind of nearest, nearest line of code. And I think that's a lot of fun.
It's just maybe not quite at the level of application where everyone should be doing it, but that's kind of a thing that I've worked on, on and off for the past two years that I found to be just, it's so much fun. And that engineer geek brain of, I want to optimize the heck out of this. That's the thing that it really satisfies for me.
ADRIANA: And speaking of profiling, now, profiling is actually one of the newest OTel signals, which is extremely exciting.
LIZ: Yes. I was one of the people who nudged the Pyroscope team to start to form the SIG, and then people from all the profiling vendors joined, and it was wonderful. Yes. So I am really thrilled by it. I wanted to congratulate the people who work on it. And, yeah, having kind of that singular profiling agent contribute by Elastic, like, that's. That's going to be...that's going to be so amazing in terms of just, you know, standardizing the format, standardizing how we can produce the data and then leaving it to vendors and open source solutions for people to look at it.
Right? Like that kind of really, really opens up that opportunity for people to start using it in anger a little bit more, I think.
ADRIANA: Yeah, absolutely. And just getting that extra little bit of insight now that, you know, it's been standardized, which is super cool.
LIZ: Yeah.
ADRIANA: Now, I wanted to switch gears a little bit because before we started recording, I asked if you had any interesting hot takes to share. So I will let you share your...one o...one of your hot takes.
LIZ: Yeah. So as of when we're recording this, the Gartner Magic Q uadrant just came out. And I was actually just on Reddit, like, you know, talking to a bunch of SREs who...and I think that it's interesting in that, you know, the SREs are saying Gartner got it wrong. And, you know, I may or may not have some spicy opinions about, about the way that the Gartner Magic Quadrant shook out, but I think it's really interesting to see. It's almost like a Rorschach test, right? Like, you look at it and you see what you want to see, right? So, yeah, my spicy take is that because I interact with enterprise buyers, SREs are not the enterprise buyer, right? So I saw SREs just slagging Gartner, right? And it's like, no, the Gartner analysts that I speak to are very smart. They know what they're doing.
And their audience is executives. Their audience is executives at fortune hundred companies, right? Like, so, you know, you an individual contributor SRE at some cool startup. The Gartner magic quadrant is not for you. So if you're complaining, you know, oh, like, you know, why didn't. Isn't Grafana ahead of Datadog and, and Dynatrace? The answer is that Grafana is maybe not quite as batteries included as you know, that large enterprise really wants it to be, right? Like, you know, that's, you know, sure, you can set up Grafana. That's great for you, but that doesn't mean that it's going to be the best choice for a big enterprise. So, yeah, people, you know, were like, oh, my God, like, you know, Gartner's so pay to play, and it's like, no, like, you know, Gartner does a fair job. Like, you know, sure, you can buy their attention to listen to you, but that doesn't necessarily guarantee they're going to, you know, say good things about you.
So, you know, you can get Gartner to, you know, even mention your name in the quadrant, but that doesn't guarantee that you're going to score well according to their evaluation criteria. That being said, you can game their evaluation criteria. So I think that's spicy take number two is I was actually looking at LinkedIn and I saw, you know, Rob Skillington, one of the co founders of Chronosphere, you know, bragging about, you know, how well they placed and also saying, like, you know, they spent, you know, hundreds of hours, you know, a thousand, a thousand hours working on, you know, on making sure that they had every single, like, you know, qualifying attribute of the Gartner magic quadrant precisely shown in a, in a, in a, in a demo video snippet, right? If you try super, super hard and, you know, you curate your example to, you know, demonstrate narrowly what Gartner's asking for, sure, you can do really well, but I think competitively in the field, my own experience is that we do not tend to encounter chronosphere in terms of it being a competitor we've run into in APM competitive situations. They're primarily a metrics vendor and newly logging vendor with their calyptia acquisition. And it seems very weird to me until I saw Rob Skillington's post.
You know, it seemed very weird to me that a competitor that was so weak in the APM and tracing space that very publicly trashed tracing and trashed OpenTelemetry that they could score so well in the Gartner Magic Quadrant. And then the pieces clicked together when I saw that they basically curated the view that they wanted Gartner to see. Whereas I can say my team and the extended team in Honeycomb, we put in a good effort and we showed the product as it is. We didn't invest a bunch of effort in polishing it, and I think that reflects it. Gartner is tough. Gartner is fair. I don't dispute where we placed in the Gartner Magic Quadrant. I think their criticisms of us were spot on, and those are things that we actually happen to be working on.
You know, I think the Gartner Magic Quadrant is a useful tool. You know, I think that it should be taken with skepticism and a grain of salt, but it is not pay to play. It is. If you make one criticism, it's that, you know, you can put in a lot of effort to, like, look super sparkly, but that it is a fair perspective as to how the enterprise market perceives, perceives companies, whether it be observability or a different magic quadrant. So sorry, SREs, you're wrong. Gartner is not being unfair to Grafana. Gartner is not being "pay to play". But you are not the audience for the Gartner Magic Quadrant. Right?
ADRIANA: That's super fair. And I have a follow up question on that, which is, you know, how...what's the process of being like, one of the vendors that Gartner evaluates is that do they look at all the vendors in the space, or do you come to them?
LIZ: They look at all the vendors in the space. Although obviously some of the additions to that are a little bit weird. Like, in past years, they've had Alibaba Cloud on there, and it's like, who? Right? And that might be an example of, okay, this is a really niche thing that they were forced to add for one reason or another. But no, every major player in the space gets given an invitation to participate. But as a criteria for inclusion, you are obliged to...you are obliged to submit proof that you have a certain minimum number of customers. You are required to submit confidential proof of your top line revenue and the growth year on year. And if you do not meet those criteria, you are not included.
They actually added a note in their report saying Observe Inc. was not included in the report because they. Not because they failed to meet the functional criteria, but because they failed to meet the non functional revenue, revenue and customer criteria, which was super spicy, but, right. Like, so, yeah, it is a well rounded set of the industry. Obviously a vendor can choose not to participate. I don't know why they would do that, but, yeah. So your employer, ServiceNow, Lightstep, is on the Magic Quadrant. I truthfully think you should have placed higher, but, you know, I wasn't privy to what you submitted to them.
So, yeah, that's kind of how it goes. Lots and lots and lots and lots of spreadsheets, lots and lots of recording demo videos. And, you know, it's up to you how much time you want to invest in it. We are a 200 person startup. We decided to do a good enough job and not necessarily. Not necessarily clip all the rough edges off.
ADRIANA: Right, right. It's interesting because it almost sounds like, you know, the type of process that you, you go through for an audit. Obviously not, not quite as, as much scrutiny, I would imagine, as doing an audit, but you have to put in the work.
LIZ: Yep. Yep.
ADRIANA: There is another question that I want to ask. You know, having, now that you're, you're interacting a fair bit with, with enterprise customers, what's, how has it been in terms of like, differences that you've noticed between interacting with enterprise versus non enterprise customers?
LIZ: People are a little bit scared by the deploy on Fridays thing. It still is a little spooky to people. Right. Like, and it's understandable that if your deploys break regularly and break after a time, time delay of 24 to 48 hours, that you would be spooked about deploying on Fridays. Right. So I kind of have to dial back the, you know, Charity and Liz, like, you know, break all the things rabble rousing. And, you know, I focus on stability, I focus on speed. And then I'm like, okay, now that you have stability and speed, like, you know, let's, let's talk, let's talk about revisiting Friday deploys.
Right. Similarly, like, I've had to caveat the, you know, test in production to like, you know, you test and you, you test in production whether you admit it or not. Right? Like, we're not saying don't test in staging. We're saying, you know. Right. Yeah. So I think that's definitely changed. I think, you know, the enterprises are not necessarily quite as willing to make large bets with the exception of kind of innovation units in startups or, sorry, innovation units and enterprises.
Right. Like, so they spin up an internal team, they give them resources to work with public cloud, to work with the latest technology. Right. Like those teams are the teams that are more willing to be game to try, to try and experiment.
ADRIANA: Yeah, absolutely. And let's not forget also the inordinately long process of getting approvals for anything enterprise related.
LIZ: Firewall holes. Firewalls are my new enemy.
ADRIANA: Oh my God. I, when I worked at Bank of Montreal for eleven-ish years, and I think one of the most annoying things that I had to do in my time there was making firewall rules, request changes. It was such a process. Such a process. And I swear it, like changed every time I did it. I just wanted to like pull my hair out. It was. Yeah...
LIZ: I know. And we live in the world of public cloud, right? Like, I use ALBs, the IP addresses of my ALBs, I cannot guarantee. Right? Like, you know, we have private link. That's how we solve that problem for a majority of cases. Right. Like, because people don't understandably don't want to open a firewall hole to all of the us east. One EC2 public IPs.
ADRIANA: Yeah, I feel ya. I feel ya. One thing that I also wanted to ask you spent many years at Google as an SRE. Do you miss it? Do you miss the SRE work?
LIZ: I get to work with the SRE team at Honeycomb and they are so incredibly talented and sharp and I love working with them. No, I don't necessarily get to do that much SRE work myself anymore, but I get to help and work with SREs across many different companies. Right. So I'm kind of a meta SRE now. I've come to terms with that. In terms of Google. Yes, I miss my Google colleagues, but increasingly, whether due to layoffs or voluntary turnover, there's been this diaspora and it's really nice to get to interact with them and potentially even work with them. At Honeycomb, we just had a former Google SRE who became a platform engineering manager at Honeycomb.
Right. Like, so. Yeah. So, you know, I do miss some. I do miss the people, but many of them have followed over into startup world, which is exciting.
ADRIANA: Yeah, that's awesome. And one other question that I wanted to ask with regards to your role as Field CTO, do you find coming in to an organization, you know, when you're proposing certain changes, how open are folks to making those changes?
LIZ: It's a self selecting bias in that the people that I speak to are the people who have already chosen to engage Honeycomb or to do a trial with Honeycomb or are otherwise investigating us. That means that a leader has a mandate for some kind of change. It may or may not be the change that we're proposing, but they do have a mandate for change. So that means that there is some appetite, at least by leadership. Yes. The people they are leading may or may not want to go along with that change, but that's kind of their job as a leader, is to have the trust of their organization and drive the change through, through the organization. So, yes, I think one of the best times to approach someone on behalf of our sales team is when someone's just made a job change, right. When they've just come in as a director or VP or CTO somewhere.
Right. Like, that means that they have a mandate to bring in new practices. And Honeycomb, OpenTelemetry can be some of those new practices.
ADRIANA: Yeah. So, so true. And speaking of OpenTelemetry, what's. What's your involvement with OpenTelemetry these days?
LIZ: I'm an emeritus governance committee member. So, right now, you know Austin Parker very well. So they're serving as, as a OpenTelemetry governance committee member, and they're very easily accessible to me. As you know, we're both Honeycomb employees. The governance committee state belongs to the individual, not to the company. But I don't see a reason for duplication, though, of having multiple people who work at the same company being on the GC. So I haven't felt the need to stand for the GC.
I recently submitted some pull requests to the OpenTelemetry Go project. So, you know, I'm still...I explicitly said I do not want my approver status back. Thank you very much. I don't have enough time to contribute. But, hey, by the way, here's a drive by performance fix.
ADRIANA: Yeah.
LIZ: Whenever I see a problem that I can help a customer address with my familiar with OpenTelemetry, I'll do it. But we have an entire team, engineering team, that's dedicated to working on OpenTelemetry. Now, I don't have to do that change unless it's something that's super quick and easy for me.
ADRIANA: Yeah, yeah. That's great. And I think it's so great that so many of the observability vendors out there have dedicated teams to work on OpenTelemetry, which I think really speaks to the staying power of OTel, and that they collectively, everybody wants OTel to succeed. And I absolutely love that.
LIZ: Yeah. Right. Like, it is our SDK. Right. Like, you know, it is our SDK that we collectively have to maintain in order to make sure all of our customers have a good experience. You know, it's a little bit decentralized, but it means that we're working on the same project despite having our paychecks paid by different people. And that's okay.
ADRIANA: Yeah, totally. And one thing that, you know, I always say over and over is I really appreciate the vendor neutrality aspect of OTel because, you know, I interact with folks in OTel who are from different companies, and I don't look at them as competitors. They're just, like, friends, people I work with. Like, we're all working towards the same goal and. And that it's so deliberate that, you know, anytime there's, like, a hint of, like, this might not be vendor neutral, people are like, you might want to reconsider, like, rewording it or, I'm sorry, we can't accept this because it violates our vendor neutrality policy. Super fair. Super fair. And I love that.
LIZ: Yeah. The only bug there has been when someone's marketing department releases something without the. Without checking it first with the OTel team at that vendor. Right. When there's no. Yeah, right. People are pretty good about self policing. Unless, you know, unless there's just a lack of communication. Right. And you could say that about engineering, too, right? Like, you know, lack of communication. That's what causes, like, things to go awry more often than not.
ADRIANA: Yeah, absolutely. Totally agree with you. Well, we are coming up on time, so before we wrap up, I was wondering if you had any parting words of wisdom that you wanted to share with folks.
LIZ: I think my parting word of wisdom is always be trying new things. And if that new thing is OpenTelemetry, great. The starting experience is super easy. But no, but, yeah, just keep on learning. Never just be like, I'm in my abroad, and this is what I do.
ADRIANA: Yeah, I love that. And so important for tech as well, right? I mean, you either learn new stuff or you wither away from the industry. Well, thank you so much, Liz, for Geeking Out with me today. Y'all don't forget to subscribe and be sure to check the show notes for additional resources and to connect with us and our guests on social media. Until next time...
LIZ: Peace out and geek out.
ADRIANA: Geeking Out is hosted and produced by me, Adriana Villela. I also compose and perform the theme music on my trusty clarinet. Geeking Out is also produced by my daughter, Hannah Maxwell, who incidentally, designed all of the cool graphics. Be sure to follow us on all the socials by going to bento.me/geekingout.
ADRIANA: Hey, fellow geeks, welcome to Geeking Out, the podcast about all geeky aspects of software delivery DevOps, Observability, reliability, and everything in between. I'm your host, Adriana Villela, coming to you from Toronto, Canada, and geeking out with me today, I have Liz Fong Jones of Honeycomb. Welcome, Liz.
LIZ: G'day, Adri, from Sydney, Australia.
ADRIANA: Thank you for waking up early to record.
LIZ: That's kind of my life these days, given that I work with people from the US and Canada, start early, take a break midday, and then work late to catch the UK.
ADRIANA: Oh, damn. Wow, that is a lot.
LIZ: You know, that's a voluntary choice that I made to move to Australia, so I fully accept that.
ADRIANA: Are you permanently moved to Australia? Because before, I remember, you were splitting your time between.
LIZ: I'm still splitting my time, but, you know, I have a house here, I have clients here, so I'm spending several months a year here.
ADRIANA: Oh, nice, nice. And hopefully...how's the weather down under right now?
LIZ: A little bit chilly and rainy, but, you know, not by Canadian standards, right?
ADRIANA: True, true.
LIZ: People are complaining. Oh, like, you know, it's like, you know, 10 degrees or 15 degrees, and I'm just like, yeah, whatever, it's fine.
ADRIANA: I know, right?
LIZ: I have a jacket.
ADRIANA: There you go. Yeah, we've had, um, kind of, we've had a hot summer in Toronto, actually. Like, like Brazil hot, which is where I'm from originally. And, yeah, I've...I have a pretty good heat tolerance, but I have been melting, so...
LIZ: Yeah, yeah, it's fun to, you know, it's like the sauna to the, to the ice. To the ice bath, right? Like going back and forth. You get used to rapid climate changes in addition to time zones. That's something that no one tells you about is climate change.
ADRIANA: Yeah, yeah, exactly, exactly. Well, I'm super excited to have you on...on the show and, and for folks who have been listening to Geeking Out, Liz was On-Call Me Maybe way back when, when I used to host that with Ana. And so I'm very excited that you've agreed to come on. Now, before we start off, I always like to start my guests off with some icebreaker questions. So are you ready?
LIZ: As ready as I'm going to be.
ADRIANA: All right, first question. Are you a lefty or a righty?
LIZ: I am a righty, like a majority of the population.
ADRIANA: All right, do you prefer iPhone or Android?
LIZ: I am an Android user because Google gave them to me for free for about a decade. I was one of the early, one of the Android beta testers. So that meant that I got free phones that might break. And that habit has carried on since after I left Google.
ADRIANA: Oh, that's so cool. How was it like having used like the early Android phones? What kind of experience was that?
LIZ: Yeah, you get them early in the technical validation process and you help carry them through all the way to production. Because of NDA, I can't talk exactly about what the experience was like. But yeah, no, it's cool having access to, to the latest hardware. It's actually though, it makes you weirdly paranoid because you have to hide your phone from being photographed by others. You have to slip it in your pocket at all times. You can't just leave it out on the table. So it does introduce some interesting complications to your life. But it was worth it at the time to get access to the latest and greatest hardware and to give the team feedback on it.
ADRIANA: Wow, that's so cool. Okay, next question. Do you prefer using Mac, Linux, or Windows?
LIZ: I am a hardcore Linux user with one exception. Well, it's technically still Linux on the desktop. I am a ChromeOS user for my laptop, again like habit from my Google days. But yes, I do my development in a VM on that ChromeOS machine. So it's Linux. I'm talking to you from a Linux machine. I have a habit of building mini ITX PCs that are all Linux based. I think I've got four little computers running around, each of which is its own independent Linux system.
ADRIANA: So are you then a lifelong Linux user? Did you ever dabble in Windows?
LIZ: I've been using Linux since I was 16, since 2003, 2004, cool. Yeah, yeah, yeah. But it's been my primary environment since, since college. So since 2005.
ADRIANA: I was, I guess forced...DOS was forced upon me because I mean like when Windows...
LIZ: Oh yes, of course, right, like MS DOS. Yeah, no, no, no, as a kid. Yes. Yeah, QBasic. Yeah, yeah.
ADRIANA: QBasic, oh my God, yes! Exactly!
LIZ: I knew I was destined to become a programmer when, in I think fourth grade, I wrote a program that would take three sets of coordinates and solve the quadratic equations.
ADRIANA: That's so cool. My dad got me into BASIC. He pulled me aside when I was ten and he's like, how'd you like to see something cool? I'm like, all right.
LIZ: Yeah. Yep. It runs in my biological family. I've got uncles and aunts who work in IT. So yeah.
ADRIANA: That's awesome. Okay, next question. What's your favorite programming language?
LIZ: Ooh, favorite. That's an interesting question. So I'm somewhat known for solving Advent of Code every year. I've been doing Advent of Code every year since it started in goodness, I don't even remember when it started, but like 2017 or something like that. So I've been doing Advent of Code in Go publicly on stream when I can for the better part of a decade now. So definitely Go is a programming language...I even use in recreation, the language that I use professionally. But it's really hard to pick favorites because, you know, I work with clients, clients use all kinds of languages.
I have to be a little bit of a polyglot as a result. So like, I do, I have a project that's written in typescript, for instance. I think it's really important to not, you know, just settle into rut and be like, I am a Java programmer, right? I think you kind of have to see and experience kind of what's going on. So at some point I will pick up Rust, I am sure, and become the prototypical stereotype of a trans cat girl who programs in Rust and has stripey socks, but that's not going to be today.
ADRIANA: Fair enough, fair enough. I do really like what you said about just broadening your horizons and learning other languages because I was actually like a Java developer for like 15,16 years and that was like my whole life. And then a friend introduced me to Python and, you know, I was like in my 30s at the time and I'm like, oh, so cool. Like, you know, it was for the first time, like since, you know, my BASIC days I did QBasic, Visual Basic, that I was like actually picking up another language and I'm like freaking cool. I got to do more of that. And yeah, and that was like my first sort of like, oh my God, you can learn another language at the same time, which is ridiculous when you think about it. Of course you can. Next question. Do you prefer dev or ops?
LIZ: I prefer ops. I still, despite my railing on about how you shouldn't try to be a hero, I personally enjoy that feeling of solving the problem. I'm not going to say that I enjoy necessarily being the hero, but I definitely enjoy being, being the person who has the insight that solves a problem, right? Like, you know, it's weird, right? Like, you know, when you, when you're doing dev stuff, like, you know, there, there is some degree of, you know, I'm, I have a start of the problem. I have the end of the problem, I fill in the stuff in the middle, right? Like you have some idea as to how it's going to go because you've decomposed the pro- the problem enough, right? Yeah, I think with ops it's a little bit more unpredictable. There's a little bit more novelty. Right? Because you don't know what's going to happen when you open up the box, right? And I, and I think that's, that's, that's the fun thing.
ADRIANA: Yeah. Yeah.
LIZ: So I'm not going to say it's necessarily about, you know, the esteem of having people be like, Liz, you solved it. Like...But it's much more about the, you know, I find that it's really interesting to do the ops and, and to, and to find new things.
ADRIANA: Yeah, absolutely. Totally agree with you. All right, next question. Do you prefer JSON or YAML?
LIZ: I think I prefer JSON because it is not whitespace sensitive and, you know, you can pretty print JSON if you need to. That being said, my pet peeve about JSON is the fact that they do not support the trailing comma in lists and that peeves me off to...like nothing else. But no, I have to interact with YAML because of Kubernetes manifests and CircleCI configs and I have broken enough YAML configs. Oh, and the hand handling of floats and the handling of like, variables. Can we not just quote all of the keys and call it a day? Right? Like, it's stuff like that that just drives me up the wall.
ADRIANA: Yeah, I feel, ya. I've heard a couple of horror stories with YAML. Like someone was telling me the other day, like, "ON", which is the abbreviation for Ontario, is interpreted by YAML as "on", "true". So it's like...I know, right? So it's like all these little nuances in YAML where you have to be like extra careful. Plus the white space. I still like YAML myself because I find it a little bit more readable than JSON because of like my Java days. All the curly braces in JSON just kind...
LIZ: Just feed it to JQ and you'll be good, right? That's literally what I do anytime I encounter anything that's in JSON is I immediately pretty print it. You're right.
ADRIANA: Oh, yeah. I have to pretty print it because I just cannot function otherwise. I feel you. Okay, next question. Do you prefer spaces or tabs?
LIZ: I am a Go programmer, so I am obliged to tell you tabs. But let's be real. I think that spaces work a little bit better in text editors because they actually run consistently. Like, I have to manually configure, like my tab with in Nano, my favorite text editor, in order to, you know, whenever I set up a new machine, because it defaults to eight spaces for a tab and that just eats your screen, right? Two, two, right? Like, so, yeah. I personally would prefer spaces, except for Go makes me use tabs.
ADRIANA: Ah, fair enough, fair enough. Okay, two more questions to go. Do you prefer to learn through video or text?
LIZ: I am a text person. I use captions whenever I can because I just read so much faster than I audio process. Or rather, if I'm going to be listening to video, I have to listen to it at 1.75 x or 2x. I just have to. It's not that my audio processing is slow. It's actually the opposite of that. It's that interacting with video at 1.0x is painful, and I will often multitask something else with it. If you make me watch a video at 1.0x. Cue the obligatory HR videos where you have to sit...sit and watch them, and like, click the little spinner at the end of 1 minute precisely. And it's just like, yeah, so, yeah, text. Because I can read it however fast I want without having to wait for the speaker to deliver the words.
ADRIANA: Yeah, I agree with you. I actually watch TV with captions on, and it's just so...
LIZ: I watch TV with captions on and I do something else that if I, rightly, because I process not quite at 2.0x, I process at 1.75x. So as a result, like, if I do two things at the same time, I'm going to miss a little bit of each one. But the captions help me stay like...
ADRIANA: Yeah, yeah, yeah, absolutely, absolutely, yeah.
LIZ: ADHD high 5.
ADRIANA: One thing that I've never, like, been able to do is actually listen to stuff at like, high speed. Like faster than 1x. Like, when I'm listening to a podcast, if I accidentally hit a button that makes it 1.25x, it actually drives me crazy. Like, they're talking too fast. Like, it breaks my brain. I don't know what it is, but so many people I talk to, they're like, I can only listen to it at, like, listen to stuff at super high speed. I'm like, more power to you all. Does not work for me at all.
Okay, final question. What is your superpower?
LIZ: What is my superpower? My superpower is I am...are you familiar with Dungeons and Dragons?
ADRIANA: Yes. High-level familiar.
LIZ: Okay...there is a class that was added on afterwards that's called the Factotum, and the Factotum is able to emulate the ability of any other class by using a certain number of knowledge points. I am a Factotum, right? Like, I am specialized in nothing in particular, but I am really good at being able to do things that specialists in other fields would do, but only one or two of them, and otherwise being able to communicate with specialists in other fields, right?
ADRIANA: Yeah.
LIZ: So I can get people to talk to each other who don't necessarily kind of share common ground or share kind of skills. I can pick up those people's skills and use them, at least for a time. So that's my superpower.
ADRIANA: That is such a great superpower. And this leads nicely into our main, or one of our main topics of conversation, which is, I think when you and I met, you were at Honeycomb working as a DevRel. But in the last couple of years, you've transitioned over to being a Field CTO. And for folks in our audience, it would be great if you could explain what that entails and also what prompted you to make that change.
LIZ: Yeah. So Field CTO is a role that is relatively new in the industry, and it really depends from Field CTO to Field CTO, company to company. Um, so actually, the one of the field CTOs at Confluent, um, whose name is Kai, wrote a piece about it. But in essence, you know, regardless of what someone's background is and where they come, come to being a Field CTO from being a Field CTO is about interacting with customers who are making very sophisticated use of your technology or otherwise have really interesting and gnarly technical problems or social problems. Honestly, the social problems are the more interesting ones that they need the help of someone who is an expert and an expert at the executive level to solve, right? So unlike a DevRel, right? Like, you know, I don't necessarily...
I don't necessarily write "how to" blogs anymore. I don't necessarily, you know, yes, I do speak at the occasional conference, but more and more of my time is spent on site with customers. And I think that is, you know, interacting one on one with customers is something that I really, really treasure because it means that I get to see all the cool things they're doing with Honeycomb. And the other piece of my job that I really enjoy is going back to the product development team. And actually, I try to, as best as I can, carry water and chop wood for them and also help solve technical problems that our customers are having at scale in our tool. So, for instance, I am working this week on something where a customer was like, we want more than 100 group by fields in a Honeycomb query. And I'm like, okay, I'll see what I can do. Let's talk to the team to see if it's possible. Let's try it out. I've been Field CTO for about two years now, coming up on two years in October, and it's been super, super rewarding to now be at the executive level in Honeycomb, to have the opportunity to interact with the executives at other companies. It's weird. There's not like, you know, a sudden transition in job responsibilities. I think it was more like, there's this funny thing that happened. In July of 2022, I was invited by Amazon to give the keynote at, or one of the customer keynotes at AWS Summit, New York. And they gave me, the AWS PR team, gave me a lot of side eye about, oh, you're, you know, not an executive.
You're not a CTO or VP. Like, you know, what are you doing up on stage as a principal engineer, right? And that was kind of a catalyst of, okay, fine. Like, you know, titles do matter at some point, right? So I didn't change what I was doing overnight, but instead, I kind of gradually fell into the role, and then the job titles changed afterwards.
ADRIANA: Oh, that's so cool.
LIZ: Yeah.
ADRIANA: So did you find then, because it was more of a gradual change. Like, there was, like, less. Like, was. Was there anything still, like, super jarring that stood out for you when you. When you made that change? Or. Or.
LIZ: I think the main thing that has really been kind of challenging is, you know, when I was DevRel, I was part of the Devrel team, right? Like, you know. You know Jessitron, you know Martin Thwaites, right? Like, you know Martin Dot Net, right? Like, so, you know, they're an amazing team, but I'm not part of that team anymore, right? Like, Charity and I are off on our own as the office, the CTO. And I think that that is a little bit of a change in that. I'm not part of marketing anymore.
I kind of don't have a department. So I work across all the departments, but there's not necessarily anyone I can lean on who's like, you're working with me. Let's do this. So I kind of have to beg and borrow to work with people. And, of course, people are happy to have the opportunity to work with me, but I'm nothing part of their planning processes, right? If I show up and, you know, as happened the past six months, right? Like, you know, if...
If I show up and say, you know, hey, by the way, we're going to be trying out Graviton 4,right? Like, you know, that's...that's something where I either need to drive myself or, you know, I need to just find someone who wants to geek out about it with me.
ADRIANA: Yeah. Yeah. So does part of your job then entail, like, you coming up with some interesting use cases to try out? Or are you driven more by customer asks, or is it a bit of both?
LIZ: Customer and partner asks? But I do think that they're, for instance, the thing with Graviton 4, that's Amazon, who's a partner of ours. They asked us to try it and I said yes. You don't in general say no when Amazon asks you, do you want to try this shiny cool thing? But, yeah, I think that majority of what I do is driven by what large customers are experiencing or what I can see they will be experiencing, right? Like, I think there is some room for thought leadership, right? There is some room for, like, looking ahead of where things are. But historically, as a company, Honeycomb has trended always, you know, too far ahead of where the ball is today.
Right? Where people can't necessarily see. This is, you know, how it aligns with what they're, what they're doing today. And we're trying to course correct that now and meet people, people where they're at now. So that's where I find myself spending a majority of time now is pragmatically connecting where people are with the challenges that Honeycomb helps solve for them. And also seeing these are the integration points that we're going to need, right? So one of the projects I'm working on is relating to better log support in Honeycomb.
Because it turns out that despite Charity and me saying, you know, throw your logs in the bin. Actually, no, you can't. Kubernetes emits logs. You're not going to throw Kubernetes in the bin. So what do we do about that? What do we do about your legacy applications? Looking at that is something I'm contributing to, and that's really driven by what customers ask me about every day.
ADRIANA: Right. That's so cool.
LIZ: Yeah. In terms of superpowers, another superpower...we were just...also, how I consume...I read text incredibly fast. As a result, I'm in several hundred different Slack channels and I read them all and I can just do that. It's great.
ADRIANA: Oh, damn. That is a superpower.
LIZ: Yeah. I don't listen to all of this calls that our sales team have with clients. But, boy, do I ever read the gong summaries of all of the call recordings.
ADRIANA: So it's funny because when you were describing the nature of the job, initially, it almost felt like. Almost like a consultant role, but non technical. But it is totally not that, because there's definitely. It sounds like there's some very, very technical aspects. So you're...you're kind of like a...a super tactical consultant who is working with, like, very high level, like, executives kind of thing.
LIZ: Executives and principal engineers. Yeah, right. Like, you know, that's that first point of call of, there's this really interesting or weird customer who's asking this question they've never seen before, right? Like, hit me with it. Like, I've been around the system long enough, and also, I'm aware of what the best practices are around observability.
ADRIANA: So does this. I wonder, like, does part of your job entail also, like, working with some of the solutions engineers? Like, pairing with them on that?
LIZ: Yeah. So I work with our solutions engineers. I work with our customer support and our customer success team. I work with customer architects. I work with software developers. That's why I say I'm a Factotum. I have to be able to speak sales. I have to be able to speak engineering.
I have to be able to speak marketing. I have to be able to communicate with all of these people and collaborate with them daily.
ADRIANA: What's been your favorite part of being Field CTO so far?
LIZ: I think my favorite part is the variety of it. No customer is alike. I think that's a lot of fun. I think the gratification and the payoff of this is what we're building. This is how people are really, actually leveraging it. I think that's also really, really satisfying.
ADRIANA: Is there any sort of thing that you've been working on that you're allowed to talk about where you're like, oh, my God, this has been, like, the coolest thing I've gotten to do.
LIZ: Yeah. I think one of my on and off fascinations is continuous profiling, and it is very, very weird in that, you know, it interacts with the very, very guts of the kernel, of the runtimes. So getting to interact with one of the Go subsystem maintainers, cherrymui, and sending her crash reports when the profiling doesn't work completely according to plan, getting to work on our integration with profiling that we developed a couple years back and that we continue to use ourselves, I think that's a lot of fun because it shows how much depth there is if you really, really, really want to get into understanding the performance of your system. I do not necessarily recommend that our clients do this. There's so much low hanging fruit to find just via tracing, but we aim to be cost effective. We aim to be fast. And part of how you get there is by looking at continuous profiling and looking at the data down to the kind of nearest, nearest line of code. And I think that's a lot of fun.
It's just maybe not quite at the level of application where everyone should be doing it, but that's kind of a thing that I've worked on, on and off for the past two years that I found to be just, it's so much fun. And that engineer geek brain of, I want to optimize the heck out of this. That's the thing that it really satisfies for me.
ADRIANA: And speaking of profiling, now, profiling is actually one of the newest OTel signals, which is extremely exciting.
LIZ: Yes. I was one of the people who nudged the Pyroscope team to start to form the SIG, and then people from all the profiling vendors joined, and it was wonderful. Yes. So I am really thrilled by it. I wanted to congratulate the people who work on it. And, yeah, having kind of that singular profiling agent contribute by Elastic, like, that's. That's going to be...that's going to be so amazing in terms of just, you know, standardizing the format, standardizing how we can produce the data and then leaving it to vendors and open source solutions for people to look at it.
Right? Like that kind of really, really opens up that opportunity for people to start using it in anger a little bit more, I think.
ADRIANA: Yeah, absolutely. And just getting that extra little bit of insight now that, you know, it's been standardized, which is super cool.
LIZ: Yeah.
ADRIANA: Now, I wanted to switch gears a little bit because before we started recording, I asked if you had any interesting hot takes to share. So I will let you share your...one o...one of your hot takes.
LIZ: Yeah. So as of when we're recording this, the Gartner Magic Q uadrant just came out. And I was actually just on Reddit, like, you know, talking to a bunch of SREs who...and I think that it's interesting in that, you know, the SREs are saying Gartner got it wrong. And, you know, I may or may not have some spicy opinions about, about the way that the Gartner Magic Quadrant shook out, but I think it's really interesting to see. It's almost like a Rorschach test, right? Like, you look at it and you see what you want to see, right? So, yeah, my spicy take is that because I interact with enterprise buyers, SREs are not the enterprise buyer, right? So I saw SREs just slagging Gartner, right? And it's like, no, the Gartner analysts that I speak to are very smart. They know what they're doing.
And their audience is executives. Their audience is executives at fortune hundred companies, right? Like, so, you know, you an individual contributor SRE at some cool startup. The Gartner magic quadrant is not for you. So if you're complaining, you know, oh, like, you know, why didn't. Isn't Grafana ahead of Datadog and, and Dynatrace? The answer is that Grafana is maybe not quite as batteries included as you know, that large enterprise really wants it to be, right? Like, you know, that's, you know, sure, you can set up Grafana. That's great for you, but that doesn't mean that it's going to be the best choice for a big enterprise. So, yeah, people, you know, were like, oh, my God, like, you know, Gartner's so pay to play, and it's like, no, like, you know, Gartner does a fair job. Like, you know, sure, you can buy their attention to listen to you, but that doesn't necessarily guarantee they're going to, you know, say good things about you.
So, you know, you can get Gartner to, you know, even mention your name in the quadrant, but that doesn't guarantee that you're going to score well according to their evaluation criteria. That being said, you can game their evaluation criteria. So I think that's spicy take number two is I was actually looking at LinkedIn and I saw, you know, Rob Skillington, one of the co founders of Chronosphere, you know, bragging about, you know, how well they placed and also saying, like, you know, they spent, you know, hundreds of hours, you know, a thousand, a thousand hours working on, you know, on making sure that they had every single, like, you know, qualifying attribute of the Gartner magic quadrant precisely shown in a, in a, in a, in a demo video snippet, right? If you try super, super hard and, you know, you curate your example to, you know, demonstrate narrowly what Gartner's asking for, sure, you can do really well, but I think competitively in the field, my own experience is that we do not tend to encounter chronosphere in terms of it being a competitor we've run into in APM competitive situations. They're primarily a metrics vendor and newly logging vendor with their calyptia acquisition. And it seems very weird to me until I saw Rob Skillington's post.
You know, it seemed very weird to me that a competitor that was so weak in the APM and tracing space that very publicly trashed tracing and trashed OpenTelemetry that they could score so well in the Gartner Magic Quadrant. And then the pieces clicked together when I saw that they basically curated the view that they wanted Gartner to see. Whereas I can say my team and the extended team in Honeycomb, we put in a good effort and we showed the product as it is. We didn't invest a bunch of effort in polishing it, and I think that reflects it. Gartner is tough. Gartner is fair. I don't dispute where we placed in the Gartner Magic Quadrant. I think their criticisms of us were spot on, and those are things that we actually happen to be working on.
You know, I think the Gartner Magic Quadrant is a useful tool. You know, I think that it should be taken with skepticism and a grain of salt, but it is not pay to play. It is. If you make one criticism, it's that, you know, you can put in a lot of effort to, like, look super sparkly, but that it is a fair perspective as to how the enterprise market perceives, perceives companies, whether it be observability or a different magic quadrant. So sorry, SREs, you're wrong. Gartner is not being unfair to Grafana. Gartner is not being "pay to play". But you are not the audience for the Gartner Magic Quadrant. Right?
ADRIANA: That's super fair. And I have a follow up question on that, which is, you know, how...what's the process of being like, one of the vendors that Gartner evaluates is that do they look at all the vendors in the space, or do you come to them?
LIZ: They look at all the vendors in the space. Although obviously some of the additions to that are a little bit weird. Like, in past years, they've had Alibaba Cloud on there, and it's like, who? Right? And that might be an example of, okay, this is a really niche thing that they were forced to add for one reason or another. But no, every major player in the space gets given an invitation to participate. But as a criteria for inclusion, you are obliged to...you are obliged to submit proof that you have a certain minimum number of customers. You are required to submit confidential proof of your top line revenue and the growth year on year. And if you do not meet those criteria, you are not included.
They actually added a note in their report saying Observe Inc. was not included in the report because they. Not because they failed to meet the functional criteria, but because they failed to meet the non functional revenue, revenue and customer criteria, which was super spicy, but, right. Like, so, yeah, it is a well rounded set of the industry. Obviously a vendor can choose not to participate. I don't know why they would do that, but, yeah. So your employer, ServiceNow, Lightstep, is on the Magic Quadrant. I truthfully think you should have placed higher, but, you know, I wasn't privy to what you submitted to them.
So, yeah, that's kind of how it goes. Lots and lots and lots and lots of spreadsheets, lots and lots of recording demo videos. And, you know, it's up to you how much time you want to invest in it. We are a 200 person startup. We decided to do a good enough job and not necessarily. Not necessarily clip all the rough edges off.
ADRIANA: Right, right. It's interesting because it almost sounds like, you know, the type of process that you, you go through for an audit. Obviously not, not quite as, as much scrutiny, I would imagine, as doing an audit, but you have to put in the work.
LIZ: Yep. Yep.
ADRIANA: There is another question that I want to ask. You know, having, now that you're, you're interacting a fair bit with, with enterprise customers, what's, how has it been in terms of like, differences that you've noticed between interacting with enterprise versus non enterprise customers?
LIZ: People are a little bit scared by the deploy on Fridays thing. It still is a little spooky to people. Right. Like, and it's understandable that if your deploys break regularly and break after a time, time delay of 24 to 48 hours, that you would be spooked about deploying on Fridays. Right. So I kind of have to dial back the, you know, Charity and Liz, like, you know, break all the things rabble rousing. And, you know, I focus on stability, I focus on speed. And then I'm like, okay, now that you have stability and speed, like, you know, let's, let's talk, let's talk about revisiting Friday deploys.
Right. Similarly, like, I've had to caveat the, you know, test in production to like, you know, you test and you, you test in production whether you admit it or not. Right? Like, we're not saying don't test in staging. We're saying, you know. Right. Yeah. So I think that's definitely changed. I think, you know, the enterprises are not necessarily quite as willing to make large bets with the exception of kind of innovation units in startups or, sorry, innovation units and enterprises.
Right. Like, so they spin up an internal team, they give them resources to work with public cloud, to work with the latest technology. Right. Like those teams are the teams that are more willing to be game to try, to try and experiment.
ADRIANA: Yeah, absolutely. And let's not forget also the inordinately long process of getting approvals for anything enterprise related.
LIZ: Firewall holes. Firewalls are my new enemy.
ADRIANA: Oh my God. I, when I worked at Bank of Montreal for eleven-ish years, and I think one of the most annoying things that I had to do in my time there was making firewall rules, request changes. It was such a process. Such a process. And I swear it, like changed every time I did it. I just wanted to like pull my hair out. It was. Yeah...
LIZ: I know. And we live in the world of public cloud, right? Like, I use ALBs, the IP addresses of my ALBs, I cannot guarantee. Right? Like, you know, we have private link. That's how we solve that problem for a majority of cases. Right. Like, because people don't understandably don't want to open a firewall hole to all of the us east. One EC2 public IPs.
ADRIANA: Yeah, I feel ya. I feel ya. One thing that I also wanted to ask you spent many years at Google as an SRE. Do you miss it? Do you miss the SRE work?
LIZ: I get to work with the SRE team at Honeycomb and they are so incredibly talented and sharp and I love working with them. No, I don't necessarily get to do that much SRE work myself anymore, but I get to help and work with SREs across many different companies. Right. So I'm kind of a meta SRE now. I've come to terms with that. In terms of Google. Yes, I miss my Google colleagues, but increasingly, whether due to layoffs or voluntary turnover, there's been this diaspora and it's really nice to get to interact with them and potentially even work with them. At Honeycomb, we just had a former Google SRE who became a platform engineering manager at Honeycomb.
Right. Like, so. Yeah. So, you know, I do miss some. I do miss the people, but many of them have followed over into startup world, which is exciting.
ADRIANA: Yeah, that's awesome. And one other question that I wanted to ask with regards to your role as Field CTO, do you find coming in to an organization, you know, when you're proposing certain changes, how open are folks to making those changes?
LIZ: It's a self selecting bias in that the people that I speak to are the people who have already chosen to engage Honeycomb or to do a trial with Honeycomb or are otherwise investigating us. That means that a leader has a mandate for some kind of change. It may or may not be the change that we're proposing, but they do have a mandate for change. So that means that there is some appetite, at least by leadership. Yes. The people they are leading may or may not want to go along with that change, but that's kind of their job as a leader, is to have the trust of their organization and drive the change through, through the organization. So, yes, I think one of the best times to approach someone on behalf of our sales team is when someone's just made a job change, right. When they've just come in as a director or VP or CTO somewhere.
Right. Like, that means that they have a mandate to bring in new practices. And Honeycomb, OpenTelemetry can be some of those new practices.
ADRIANA: Yeah. So, so true. And speaking of OpenTelemetry, what's. What's your involvement with OpenTelemetry these days?
LIZ: I'm an emeritus governance committee member. So, right now, you know Austin Parker very well. So they're serving as, as a OpenTelemetry governance committee member, and they're very easily accessible to me. As you know, we're both Honeycomb employees. The governance committee state belongs to the individual, not to the company. But I don't see a reason for duplication, though, of having multiple people who work at the same company being on the GC. So I haven't felt the need to stand for the GC.
I recently submitted some pull requests to the OpenTelemetry Go project. So, you know, I'm still...I explicitly said I do not want my approver status back. Thank you very much. I don't have enough time to contribute. But, hey, by the way, here's a drive by performance fix.
ADRIANA: Yeah.
LIZ: Whenever I see a problem that I can help a customer address with my familiar with OpenTelemetry, I'll do it. But we have an entire team, engineering team, that's dedicated to working on OpenTelemetry. Now, I don't have to do that change unless it's something that's super quick and easy for me.
ADRIANA: Yeah, yeah. That's great. And I think it's so great that so many of the observability vendors out there have dedicated teams to work on OpenTelemetry, which I think really speaks to the staying power of OTel, and that they collectively, everybody wants OTel to succeed. And I absolutely love that.
LIZ: Yeah. Right. Like, it is our SDK. Right. Like, you know, it is our SDK that we collectively have to maintain in order to make sure all of our customers have a good experience. You know, it's a little bit decentralized, but it means that we're working on the same project despite having our paychecks paid by different people. And that's okay.
ADRIANA: Yeah, totally. And one thing that, you know, I always say over and over is I really appreciate the vendor neutrality aspect of OTel because, you know, I interact with folks in OTel who are from different companies, and I don't look at them as competitors. They're just, like, friends, people I work with. Like, we're all working towards the same goal and. And that it's so deliberate that, you know, anytime there's, like, a hint of, like, this might not be vendor neutral, people are like, you might want to reconsider, like, rewording it or, I'm sorry, we can't accept this because it violates our vendor neutrality policy. Super fair. Super fair. And I love that.
LIZ: Yeah. The only bug there has been when someone's marketing department releases something without the. Without checking it first with the OTel team at that vendor. Right. When there's no. Yeah, right. People are pretty good about self policing. Unless, you know, unless there's just a lack of communication. Right. And you could say that about engineering, too, right? Like, you know, lack of communication. That's what causes, like, things to go awry more often than not.
ADRIANA: Yeah, absolutely. Totally agree with you. Well, we are coming up on time, so before we wrap up, I was wondering if you had any parting words of wisdom that you wanted to share with folks.
LIZ: I think my parting word of wisdom is always be trying new things. And if that new thing is OpenTelemetry, great. The starting experience is super easy. But no, but, yeah, just keep on learning. Never just be like, I'm in my abroad, and this is what I do.
ADRIANA: Yeah, I love that. And so important for tech as well, right? I mean, you either learn new stuff or you wither away from the industry. Well, thank you so much, Liz, for Geeking Out with me today. Y'all don't forget to subscribe and be sure to check the show notes for additional resources and to connect with us and our guests on social media. Until next time...
LIZ: Peace out and geek out.
ADRIANA: Geeking Out is hosted and produced by me, Adriana Villela. I also compose and perform the theme music on my trusty clarinet. Geeking Out is also produced by my daughter, Hannah Maxwell, who incidentally, designed all of the cool graphics. Be sure to follow us on all the socials by going to bento.me/geekingout.