In this episode I talk with Roy Tomeij, co-founder of AppSignal, about how AppSignal started in 2012 as a Rails-only APM tool when everyone used New Relic, and how it grew to support Elixir and other languages. Roy explains how its first customers came from co-founder Thijs biking around Amsterdam, why usability was the big selling point, and why aiming for "80% of what competitors have" works. We also talk about brand marketing for small companies: sponsoring conferences like Rails World, handing out stroopwafels, and investing in community without measurable ROI. Links: - AppSignal - Roy Tomeij's Website - Roy Tomeij on LinkedIn - Nonsense Monthly
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
SPEAKER_01Hey, it's Jason, host of the Code with Jason podcast. You're a developer. You like to listen to podcasts. You're listening to one right now. Maybe you like to read blogs and subscribe to email newsletters and stuff like that. Keep in touch. Email newsletters are a really nice way to keep on top of what's going on in the programming world. Except they're actually not. I don't know about you, but the last thing that I want to do after a long day of staring at the screen is sit there and stare at the screen some more.
SPEAKER_01That's why I started a different kind of newsletter. It's a snail mail programming newsletter. That's right. I send an actual envelope in the mail containing a paper newsletter that you can hold in your hands. You can read it on your living room couch, at your kitchen table, in your bed, or in someone else's bed. And when they say, What are you doing in my bed? You can say, I'm reading Jason's newsletter. What does it look like? You might wonder what you might find in this snail mail programming newsletter.
SPEAKER_01You can read about all kinds of programming topics like object-oriented programming, testing, DevOps, AI. Most of it's pretty technology agnostic. You can also read about other non-programming topics like philosophy, evolutionary theory, business, marketing, economics, psychology, music, cooking, history, geology, language, culture, robotics, and farming. The name of the newsletter is Nonsense Monthly.
SPEAKER_01Here's what some of my readers are saying about it. Helmut Kobler from Los Angeles says, thanks much for sending the newsletter. I got it about a week ago and read it on my sofa. It was a totally different experience than reading it on my computer or iPad. It felt more relaxed, more meaningful, something special and out of the ordinary. I'm sure that's what you were going for, so just wanted to let you know that you succeeded.
SPEAKER_01Looking forward to more.
SPEAKER_01Thank you for this. Can't wait for the next one. Dear listener, if you would like to get letters in the mail from yours truly every month, you can go sign up at nonsensemonthly dot com. That's nonsensemonthly dot com. I'll say it one more time. NonsenseMonthly dot com. And now without further ado, here is today's episode.
SPEAKER_02Hey Jason, thanks for having me.
SPEAKER_01Thanks for being here. Uh last time you and I met in person, which was also the first time you and I met in person, we shared a snack. I gave you some beef jerky, which I had illegally smuggled across the Canadian border. It was a nice uh it was it was a nice thing to be able to enjoy together.
SPEAKER_02Uh absolutely, and I appreciate you um being willy to uh to face jail time just to uh to smuggle me some snacks. Oh yeah. Those were great and I'm the sucker for smoked meats and and beef jerky, so uh double appreciated.
SPEAKER_01Um does the Netherlands have any kind of like uh renowned domestic uh smoked meats?
SPEAKER_02Smoked fish, I would say. That's a thing. Smoked eel, things like that. Uh not a fan. Uh I'm a Dutch guy, but I always say like I'm more like a Texan at heart. Um appreciate my hot smoked meats. Um the Netherlands is mostly cold smoked. Uh so yeah, uh not that much here.
SPEAKER_01Have you been to Texas?
SPEAKER_02I have. Uh AppSignals dual headquartered both in Amsterdam and in Austin, so I do go there semi-regularly. Oh wow, okay.
SPEAKER_01I used to live in Austin. Oh, cool. So yeah, it's not out of the question that you and I have eaten at some of the same barbecue restaurants.
SPEAKER_02Could be, could be. Uh I've been to Franklin's, uh obviously Terry Black's and things like that. Uh La Barbecue long ago, long ago. Um so yeah, I've been around, but definitely, definitely lots, lots more spots to uh to cover.
SPEAKER_01I first went to Terry Black's just a few weeks ago. I went back to Austin. I live in Michigan, as you may know, um, but I visited Austin, took my family to Terry Black's, which I had never been to before, but it was like nearby us. And of course, I've been to Rudy's many times. Perhaps you have too.
SPEAKER_02I have not yet. Oh, okay. But definitely on the list. I'm going to be there in September for Reals World, so um lots of opportunities to find some Ruby friends to hang out with at some of the uh the great barbecue joints.
SPEAKER_01Well, Roy, let's make this plan right now here on the spot. Um let's, you and me and some other people, let's let's go get some barbecue when we're Austin. I'll be there too.
SPEAKER_02Yeah, well, that sounds great. Let's have a group. Um yeah, make it official. It's gonna be the official, the official, unofficial uh Reals World barbecue hangout. Let's do it.
SPEAKER_01Absolutely. Um Okay. I I feel I feel great about these plans that that we're forming. Um before we go much further, uh, do you want to tell us a little bit about yourself and your background and that stuff?
SPEAKER_02Sure. Uh well, Roy Temei, as you said, I'm one of the founders of AppSignal, uh, that some of your listeners will probably know as an ABM solution, application performance monitoring, error tracking, uptimes, data spaces, things like that. Uh co-founded that back in 2012, uh, and currently I'm serving there still as a brand ambassador. So no longer involved heavily operationally, but I just get to do the fun stuff, which is going to conferences, meeting people, um making plans for grabbing barbecue together, things like that.
SPEAKER_01Are you still stand are you still sending Stroopwaffles?
SPEAKER_02Uh still sending Stroopwaffles. Um not always arriving where they should arrive, uh depending on customs. We have a bunch stuck in the UK currently uh that were meant for last week's Brighton Ruby, dating this episode now. Um those will be returned to sender, so um they'll eventually end up somewhere uh where they're loved.
SPEAKER_01Yeah, you sent some to SinCity Ruby, which were much much appreciated. But uh I love those things. And it's just a nice gesture.
SPEAKER_02It's a small thing, right? But it's what we're known for, and we shipped over half a metric ton uh last year to customers and mostly conferences globally. Um yeah, it's it's a fun thing. Not the core business, obviously, but something for people to uh to remember as by.
SPEAKER_01I've I've wanted to copy you for a long time and send some kind of edible treat to conferences, but I couldn't I haven't been able to figure anything out yet. Because like Stroopwaffles, I understand, are a Dutch thing, and so it's kind of like that makes sense to send those from the Netherlands. Um, but I don't know if I have anything that is like emblematically uh regional or something like that that I can send.
SPEAKER_02Yeah, I can imagine. Like Schuwaffels are well, they're dry goods individually packed, they're flat, so easy to ship. Um so we got lucky that we just happen to have a local delicacy. That's just uh also logistically easy to uh uh to deal with. I I reckon that beef jerky or anything similar uh will run you into issues with basically any customs enforcement agency, I guess.
SPEAKER_01Yeah, that's probably true. And the the cost might be prohibitive. Um Stroopwaffles are a probably a more affordable investment. Are are you able to, you know, probably very indirectly, but are are you able to measure any kind of ROI from your Stroopwaffle activities? Can you trace that back to any business activity?
SPEAKER_02No, not at all. Uh but it's there's never been a reason to stop sending them, luckily. Um I didn't know. Like we just believe in general that it's like a karma thing. Uh what goes around comes around. We just try to be nice by just having those true puffers out there, and also sponsoring all the events we sponsor and things like that. Just lots of community focused initiatives that's really important to us, and ROI is hard, if not impossible, to measure.
SPEAKER_02But we believe that like good things come to good people, and then I just strive to be good people, basically.
SPEAKER_01Yeah, I agree. Um, you know, the the the idea of brand advertising doesn't apply to small businesses in most ways. Um, like a mom and pop restaurant probably shouldn't put up a billboard on the freeway or something like that. But I do think that brand advertising can apply to small businesses in the way that you've done it. Um, and I I've looked to you and a couple other people and organizations as kind of a model.
SPEAKER_01Um, you along with Adrian Marin from Avo and Arena and the Evil Martians people, they really do a great job of uh of making themselves known and familiar in the Ruby community and and radiating goodwill and you know becoming a a an entity that people know and like.
SPEAKER_02Absolutely. Uh and that those are two great examples you just mentioned. Uh like Adrian with like the Ruby passports for people who don't know. At Ruby conferences globally, you can get a Ruby passport, and then various events have their own stamp. So you can get your passport stamped. Um, obviously uh Ethel Martians, lots of great content to put out there, etc. etc. It's all very direct. It's not the billboard on the highway.
SPEAKER_02It's very much here is something valuable for you. Give me your Ruby passport. Here is a through buffal for you, here's some excellent content for you uh and aimed at you instead of just putting up a sign buy stuff.
SPEAKER_01Yeah, exactly.
SPEAKER_02Yeah, I would say like I I don't sell stuff. I'm just connecting with people or hope to connect with people, and then the stuff will sell itself eventually.
SPEAKER_01I read this book some years ago called Scientific Advertising, which is not very scientific, but it has some good advice in it, um, which is like when you're advertising, don't be selfish and just talk about yourself and say, hey, hey, buy my product instead of the other guy's product. Um, offer service, which I think is great advice. And so the advertising that I do for Saturn CI and the other stuff that I've done, I try to package it in something valuable, like almost like a uh benevolent Trojan horse.
SPEAKER_01Like, here is a genuinely useful blog post on such and such topic. If you happen to look at the URL bar, it says SaturnCI.com, and so now you know about it. And the Evil Martians does this, you do this, I believe. Um you put out blog post after blog post, yeah, and eventually, you know, maybe the first time somebody sees one of your blog posts, they don't pay much attention to the origin of it. But if they see blog posts month after month, year after year, you become a known entity.
SPEAKER_02Absolutely. And what if you what we've what we have learned with our business is that when it comes to error tracking tooling specifically, people don't usually change that mid-project. They may do so if it's uh like the project has been in production for years and at some point they're open to switching. Uh-huh. But especially during an active development phase, people are less likely to switch. So for us, it's uh nigh impossible to predict like when to talk to you to say buy our product because you're most oftentimes you're not ready to buy.
SPEAKER_02So we try to do is just being in the consideration set as people start a new project, start working on a new code base, uh, ask an LLM what are my options. Um, and then we hope that people just can remember, oh, there was this bold Dutch guy with his troopwaffles. Um what was the thing called again? And I think if you actually just Google APM troopwaffles, I'm sure you'll find AppSignal.
SPEAKER_01Right. Um, okay, so I'm gonna I'm gonna indulge my own personal curiosity. Uh listener be damned. Um uh because because I have I I I have uh all sorts of curiosities about the business side of AppSignal. Um, and I know that some listeners are interested in this topic too. Um so first of all, there's often advice to like go after a narrow niche. And APM is not exactly a narrow niche.
SPEAKER_01That I I think that's pretty broad. Um and then for my product, CI, not a narrow niche. That's pretty broad too. And so in that sense, you have gone kind of against the grain, and I'm going kind of against the grain. Um, and I'm curious in your case, like what gave you what gave you the nerve to go after a broad category like that instead of something like safer seeming and more narrow.
SPEAKER_02Right. So I do think we're targeting a niche in a way, but I'll get to that in a bit. So when we started doing this, there you basically had neuralic. Like nowadays, there's Datadoc, Sentry, and all the other ones. Back then, like I said, this is like 2012-ish, uh, everyone used New Relic. And with everyone, I mean people I knew which were people using Ruby on Rails. Um we had a Ruby on Rails agency before this. I think we were one of the first in Europe to start using Ruby on Rails in production.
SPEAKER_02Uh Rails version 0.12. Uh in hindsight, not a version to run in production. Uh apologies to all our former customers. Um and at some point we were like, agency work is great, but not that great because you need to keep feeding the beast. You need more projects as you take on more projects, you need a larger team. When the project ends, you quickly need a larger project because you have to still pay your team, etc. etc. So you want to get into SaaS, right?
SPEAKER_02Build it once and then sit on a beach and drink cocktails. Um turns out build it once is not a thing, as you know. It's continuous building for 60 hours a week, uh, and I never saw any cocktails on the beach. But that's fine. That's all good. But then we said we want to do APM, but specifically for Ruby on Rails, and Ruby on Rails only. Back then. Right now it's it's uh uh there's many more frameworks and languages we support. Back then it was just Ruby on Rails.
SPEAKER_02And then uh Elixir came along, and we saw a lot of people moving from or at least the people who started using Elixir were often former Ruby developers. And we were Ruby developers, so for us it was uh easy-ish to target them because it's your peers, you know what they like, what they dislike in general, what the market likes. Um and then we said, well, we can apply the same to Elixir, because it's generally the same people. So then we added Elixir, and then obviously if you want to be competitive over time, you need to add support for more languages as well, which we did.
SPEAKER_02But it's still not for everything. There is we're working with um open telemetry implementations to more quickly support more languages currently. Uh huh. But also like you can't use AppSignal for everything you can use Sentry for, for instance. And that's fine. We've learned there are some markets that are really hard for us to penetrate. Like looking at the Node.js market, there is very little like community-driven events. Like you have tons of conferences globally that are run by volunteers in the Ruby space.
SPEAKER_02For Node.js, that's a lot of commercial ones. But companies who organize like 5,000 people Node.js conferences that they need to make money off of, and then sponsorships start at amounts that are way over budget for us. Um so some markets uh such as Ruby, but also Python to an extent and Elixir are very approachable. And we're very like the people in those markets, because we're developers ourselves. Um so still a niche in the way that we're not targeting Java, and we're not going like we want to support Java, want to support PHP, but currently that's not where all our marketing efforts lie.
SPEAKER_02That's still within that core of Ruby, Elixir, and like neighboring languages, really.
SPEAKER_01And I'm curious how you got your early customers. It's a tough thing. Um, you know, one of the advantages of doing something like what you did and what I'm doing is it's not a novel new product type that you have to explain to people. That part of it I found to be great. It's like, what is it? It's a CI platform. Like, you know what CI is, you know, GitHub Actions, Circle CI, etc. If you know those, then you know my product category at least.
SPEAKER_00Yeah.
SPEAKER_01Um, but you know, kind of the flip side of that is that everybody who needs it is probably already using something else. And in the early days, your offering is surely not as feature-rich as the competitors who have been around for a long time. And so it's like, hey, switch from your mature product that you've been using to my relatively untested, very new, unproven, blah, blah, blah thing.
SPEAKER_01Um, that that can be a little bit of a tough sell, but obviously not impossible. You made it work and and I'm making it work. Um and and one more part of that question is like there's the concept of like a buying window, where, like you mentioned, at the time you meet somebody, it's very improbable that they will happen to be in the buying window at the exact time you meet. And I'm curious what that buying window looks like. So a lot of questions in there, but take that whatever direction you like.
SPEAKER_01Sure.
SPEAKER_02So, like you said, I I think it's advantageous being in a space that people know. So instead of saying it does this and this and this and this, people just ask you, how does this how is this different than insert whatever tool they're using at the time. Um and for us, and I think the same goes uh for your product, but you get to be opinionated. You say, I know what all those other tools do, and some of what I do is great, but some of this we really dislike for whatever reason, and that's not going to be in our product.
SPEAKER_02And we always say, or I always say, AppSignal has like 80% of what a lot of our competitors have, and that's great for 80% of the market. And 20% will have will say, Well, I want this other tool because there's this one thing that I've been using for over a decade and it's super specific, and I can't live without it. Well, that's fair. Uh Then we are not for you. Like not everyone is our customer. Um I do think that we tend to have customers who appreciate that opinion we have and that we feel like we're not doing this.
SPEAKER_02Like on purpose, we're we're not building this feature. Even if five people ask for it or ten people ask for it. Like at some point obviously you have to ask yourself if dozens of people ask for it, should we add it? Uh but for a long time what you say, well, no, we we're not adding it because uh we have a reason for this not being part of the product, because you want to steer people in a different direction, etc. etc. So you need to have a certain vision for your product. Um but like I said, just uh it's good to be to have competitors because people know the space, and you can say, well, we're different because of this product pricing, uh whatever.
SPEAKER_02So I would totally prefer this over anything like totally new and trying to explain to people why you need this. Because lots of our customers know why they need a solution like CI or like APM. They just need they don't they're not sure about what's which specific solution, but they know they need something. Um I think it's very useful.
SPEAKER_01And what do you what do you say when people ask, like, okay, what specifically is different? Like what would make me want to switch from what I'm using now to your product.
SPEAKER_02I'll try not to make this uh a sales spiel, but it's zero effort setup. So we guarantee that you can get it app set up and running within five minutes. And that's like without using LLM, like just manually uh including if we're if there's about Ruby on Rails, just including the gem, um setting up your your API key and that's it. And then if you're using Kubernetes, we'll detect you're using Kubernetes and sp and come up with a dashboard for that. If you're using Sidekick, we'll detect you're using Sidekick and add a dashboard for that.
SPEAKER_02Obviously, you can also just do all those manual dashboards and things like that. But we just hope or we aim for it to work out of be valuable out of the box. So also when a more junior developer comes in, they can enable AppSignal and get valuable insights within five minutes and say, Oh, I need to fix this. That's slow, there's an error there. Uh, why isn't my site working when or slow when viewed from Southeast Asia? Whatever.
SPEAKER_02And then with your power user, you can add lots of customization and custom dashboards and custom metrics and have alerting on that. Uh but for us, the most important is turn it on and instant gratification basically.
SPEAKER_01Interesting. Okay, that does seem compelling. Um I'm I'm also curious. Okay, I I'll I'll I'll share what I'm going through, what what I've gone through so far in my sales and marketing journey. Um, early on, I asked myself, like, okay, at this point in time I have zero customers, who are my first few customers going to be? Um, and my guess was that it was gonna be mostly just people I know personally, because again, it it's a leap of faith to use somebody's unproven product, especially if you're the first customer, nobody wants to be first.
SPEAKER_01Yeah, um, and is this thing gonna be around in six months still? All these risks, and so it really helps if if there's someone who you have an existing trusting relationship with uh who can be that early customer, and that did well, it kind of turned out to be the case. Um, my first paying customer was a friend of mine who I've known for years, um, but he signed up for the product and started paying it, but then paying for it, but then never used it.
SPEAKER_01And then he sold his business, and then you know, I'm like, hey, do you want me to like stop the stripe payments for this because you've never used it? Uh anyway, that was kind of a false start. And then my first real customer who both paid for it and used it, he was somebody I wasn't personally acquainted with before, but he was aware of me. He had bought my Rails testing book some years ago, and I searched for him in my email newsletter list to see if he was there, and sure enough, he was there, he's been subscribed for some number of years.
SPEAKER_01So even though I didn't know him before, there was a pre-existing relationship there, and then I have a handful of people coming on board now, and I have had a handful of people try the product in the past, and most of them have just been people I know almost trying it out just to be nice, like not really. There there was a need there. I think there was only one guy who did it just because, like, he was like, Hey, I don't really need this, but I'll do it just to support you or whatever.
SPEAKER_02Take my money.
SPEAKER_01Yeah, so that's that's helpful. But I'm I'm curious what was the case for you and how you found your very first few customers.
SPEAKER_02So, for the very first few, uh one of my co-founders, Thijs, just hopped on his bike and went to other businesses in Amsterdam uh to talk about Ruby businesses. Ask them, like, hey, what are you using? Um, what do you dislike? What if we built this thing? And we had a demo back then, um, not even MVP maybe, but at least this is how we feel what it should be instead of of what New Relic is in that case.
SPEAKER_02And we told them if you pay us a thousand euros up front, you can use it for a year when we're done. Uh and then like a handful just immediately said yes.
SPEAKER_01What?
SPEAKER_02So apparently we were solving like because we felt that's that pain as well, and they were very similar to us. Um we were like, we have something.
SPEAKER_01And then for absolute I'm sorry, I I didn't quite catch like what the like what was so compelling that they were willing to pay a thousand bucks up front for this thing that it at this point did it not even exist yet, or did it exist? It was a demo-ish kind of situation.
SPEAKER_02So we would go to them and then ask them basically, so what do you like about New Relic? What what specific thing do you like about New Relic? And we knew New Relic was notoriously bad for surfacing those features and making them accessible and findable. People were like, yeah, there's this great chart. I'd love that it should be in every APM product, let me show it. And then they'd be lost for five minutes trying to search for it, but they couldn't find it.
SPEAKER_02So I would say usability alone was a huge, huge selling factor back then. Obviously, we knew those people from Ruby meetups and things like that, so he's also friendly with them, which made it easier to say, hey, well, sure, we'll give it a try. And 90%, and that's still the case, 98% of F-Signal customers are B2B SaaS, or SaaS in general, B2C and B2B. Meaning that if uh 98% of our customers, if they have a problem, that will cost them money or headaches because of support, etc.
SPEAKER_02etc. So for us it's also important to stress the uh return on using a very uh usable and friendly tool that will proactively alert you about things with like actionable insights instead of there's an issue, make it as specific as possible to people that just dive in and fix it. We have a ton of customers who rarely visit the app but just rely on email notifications because we just provide most of the contacts, if not all the contacts needed to fix an error, for instance.
SPEAKER_02Um so we had that and uh more broadly speaking, so outside of that first handful of customers, back then I did a lot of conference speaking. Uh I done about 50 uh uh conference talks and workshops over the years, so it had a huge network, especially in the in the Ruby community. And I'm not saying we sold App Signal because of me, but it definitely made it easier to make people see that App Signal existed, right?
SPEAKER_02Make them aware of what we're building. Um, and in hindsight, it's just hard to tell. We messed up lots of things. Um at some points we could have could have gone ten times faster. Uh, other times we did a great job. Um, there's some survivorship bias here, seeing where we are currently. Um but I think just being part of the community worked for us really well back then, and I think that's also one of the reasons why we still want to be part of that community.
SPEAKER_02Because if it worked back then for 10 people, then why won't it work right now for 100,000 people?
SPEAKER_01Yeah. And it's crazy how you can keep broadcasting yourself as aggressively as you can for years and years, and still there's way more people who haven't heard of you than who have heard of you. Um, like Yeah, that's painful.
SPEAKER_02I find it painful. Like this year is gonna be the fourth year where the Palatinum sponsor for RailsWorld, and there are still people walking up to us uh like at the booth asking, so AppSignal, so what do you do? And I it's easy to think, well, what rock did you crawl out under uh out of? But it's that's not that's not true. That's on us, like somehow we just keep hitting the same group of people or whatever. Um, but it's kind of painful if you've been doing this for so long and and maybe you recognize it as well.
SPEAKER_02At some point you think, well, everyone has heard of AppSignal. Um, but that's just that's not how it works.
SPEAKER_01Yeah, there are people I have known for a long time, and know people I know pretty well, and then at some point I'll say something about my podcast, and I'll be like, Oh, you have a podcast? What's it about? And I'm just like, what the fuck? Yeah, like how do you not know about this? And there there were a couple times I went on a round of job interviews, and like nobody knew who I was, and I'm like, this is bullshit. Um, because it's it's really interesting, you you have your natural social circles that you move around in, and it gives you a certain sense of reality or whatever, but it's not always very accurate.
SPEAKER_01Um, people often accuse me of being very famous, but I I don't necessarily feel that way all the time. Like when I go to a job interview and the the people interviewing me have not heard of me. Um and it's really funny, like I've shared those stories of how I've had a really hard time uh uh getting a job, and I've had a hard time interviewing and stuff like that, and people are like, wow, like with all your advantages and your name recognition and stuff like that, uh I thought it would be super easy for you to get a job, and I'm just like, well, do you have a job for me?
SPEAKER_01Um it's it's like a relatively small number of people who who are aware, I think.
SPEAKER_02Absolutely, yeah. We're both world famous uh within a very small subset of the world's population, right? Umbling. Um but it's also funny that if people don't know you, you do get like better feedback. I remember sitting at uh at lunch uh a few months ago at um uh Rubicon in Vienna. And so where do you work? Well, work at AppSignal.
SPEAKER_02Like, oh okay, cool, and then people start talking about stuff, and then at some point someone mentions, so you're one of the founders, right? And there's eight people at the table, and then the other six are like okay, so they're a well we're saying we're I think we're a successful company. Um and then the tone kind of shifts, right? People, I'm not saying they're star tr star struck because I'm not that important, uh, nor do others think that's the case. But it's going to be then more about them asking me questions, which is totally fine, but less room for me to just ask others questions and have others speak what you want to speak about.
SPEAKER_02So that's uh sometimes it's okay. Nowadays I don't say like I'm a founder of AppSignal. People who know know, and otherwise I'll just say I work at AppSignal.
SPEAKER_01Oh, interesting. Okay. Yeah. Yeah, I can see how that would be the case. Um it's hard to get, in fact, I might even say it's impossible to get truly unfiltered feedback. The best way, I'm sure you've discovered this too, the best way to get feedback on your product is to actually just observe somebody using it in person because people can tell white lies to be nice and say, like, oh yeah, this is really easy to understand.
SPEAKER_01I love it. And maybe they even really believe that, but then you actually observe them, and they well, I don't know if you've ever heard the expression uh reaching over your shoulder to scratch your ass. They'll like do this thing and do like seven clicks to get there, and it's like, what? Like, you can just go right here, you know, and do this. They'll abuse your product in ways you never could have anticipated, and you'll never know it until you sit literally in person with them and watch them use it.
SPEAKER_02Yeah, I I think if you have a product that has been around long enough, at some point you'll fix a bug or at least a UX issue, and people will complain because now the thing doesn't work anymore. And we're like, no, this is the first time it actually works well. And then it just turns out that the bug enabled them to take some shortcut that we never meant to be there. Um it has become part of their daily routine and how they use the product, so then the bug becomes a feature, and that's uh interesting.
SPEAKER_02Yeah. If you're sitting looking at a recording or something, like click the thing. No, you need to click the thing. Why is he not clicking the thing? Like it to just it's so obvious, like we build it for you to click the thing, uh, but like you said, it will take them seven clicks, and then that's that's still on us for not properly educating people or nudging them to use it in a way that's we feel it it works better.
SPEAKER_02But maybe we're wrong, maybe they have a way that works better for them.
SPEAKER_01Right. Yeah, when I was new to usability testing, I would show people my designs and give them a list of tasks tasks to carry out using the design, and the first person would use it wrong. I'd be like, Well, this person is dumb, and then I would show it to a second person, and they would use it wrong in the same exact way, and then show it to a few more people, they would use it wrong in the same exact way, and it's like, oh, wait a second.
SPEAKER_01It's not that all these people are dumb, it's that my design is no good, and it needs to be fixed to match the way people are actually going to use it.
SPEAKER_02Absolutely. And in my experience, that means making serious changes, like just adding a label to a button for an onboarding tour that says click here, that doesn't work. Because if it if it's not intuitive, it's not intuitive, right? You can slap on as many red pulsating dots as you as you want. Um, in the end, you want something that without those nudges, people are just able to figure it out. Um, and that that's a constant challenge.
SPEAKER_01Yeah, I remember about 10, no, about 13 years ago, um, I had this job and it and we were we were building this product and we had a designer, and I remember being so depressed because the designer came up with all these designs where the form fields or buttons would just have like a small paragraph of instructions next to it, and I'm just like, dude, if your design requires a paragraph of instructions or like really any text beyond like one word, like that's no good.
SPEAKER_01That's that's that's a sign that it's not a good design.
SPEAKER_02Yeah, absolutely agree. And we do spend a lot of time and effort on design, just UI, UX, what it looks like and how it works. That's what sets us. That's also one of the things that sets us apart from others. Um and just that's just part of our DNA as well. We also just want to like want it to look good, basically. Because that's a start for also having it be usable, right?
SPEAKER_01Yeah, that is great to hear, and that's so refreshing to hear. I think a lot of companies will say that they care about usability, because who's gonna go and say that they don't? But there's a difference between saying it and actually like um you know, there's a there's a saying, don't ask a man what he cares about, watch how he spends his time and money. So, like with your limited resources, what do you choose to spend your resources on?
SPEAKER_01Do you choose to spend some of that on um usability? Most places don't. And if you don't spend some of your resources on it, then you don't really actually care about it. And most places don't, which is really frustrating, and it's very evident in the low usability of so many products that we use every day. Um, but it's to me, it's just so satisfying to create something that is highly usable and and not just that functions, like it technically allows you to somehow achieve the thing you need to do, but it's actually like a pleasure to use.
SPEAKER_02Absolutely. I think it's super important, but it's also hard to not just add stuff, like you can't keep adding stuff without ever taking a step back and you saying, okay, we've added all these features. Now how to ensure, like, is this still cohesive? If not, how we how do we tie this together? Like you could just add like more tabs, but it doesn't make it better. It makes it like more complete in the sense that it has more features, but it doesn't make it better or more pleasurable to use, right?
SPEAKER_02So sometimes you take a step back and like it now first, uh like up front it's do you really want to add this? Will this add enough value? If you like balance adding this feature that provides value, but also cluttering the interface? Um like how does the how do those two like balance, right? Uh should we add this? Or are we just making it too complicated? Who are we adding this for? Like, is this for like for two power users?
SPEAKER_02Do you really want to confuse the thousands of others? Um, so that that's a constant struggle. And it's a hit and miss as well, right? You can do your research and have great ideas and then build something, and people are like, yeah, liked it better a month ago. Um, well, yeah, back to the drawing board.
SPEAKER_01And it takes a lot of discipline to spend a lot of time and effort and care on something, put it out there, and then admit that it's not justifying its own existence and it needs to be ripped out as extreme.
SPEAKER_02And then when you do, there's like three people who relied on it daily. Oh no. And those will be super vocal, right? I remember like back in the day we added app Dex, like a health score for apps. People were like, well, we need that, because SLA reporting, things like that. But then as we tracked usage over time, there was like two people using it. And we're like talking tens of thousands of developers being active on the platform, and then two you like navigating to that app decks uh page.
SPEAKER_02So we yanked it out. Um yeah, and then those two are gonna be vocal. Um but that's also too bad. Right. And they explain it to them, and they either they understand or they don't, and both is okay, right? You you take something away from someone. So even if that's like point one percent of customers, like it impacts them. So I totally get where they're coming from if they're unhappy with that. So best we can do is explain, well yeah, for others which is confusing.
SPEAKER_02People didn't know what it meant. Um no one was using it. We do need to maintain it and ensure that it's accurate and works. So instead of keep doing that, we decided to decide to remove it.
SPEAKER_01Um yeah, um I I recently read about this concept. It was a concept I was familiar with, but it it put a name to it. Um there are some things in the world which we're just stuck with because they were created at one point in time and now they become like load-bearing, and we can never really go away from it again. Like the US using Imperial measurements instead of metric.
SPEAKER_01Like metric is objectively better in many ways, but we're never gonna switch. Just too much stuff depends on Imperial now. Um, the author uh whose book I read on this calls it a QWERTY problem, you know, because the QRI keyboard layout was intentionally designed so that the the keys that you use frequently in quick succession are far apart from each other instead of right next to each other.
SPEAKER_01So it was like intentionally designed to be inefficient. Um, but at this point, we're not gonna switch to a different keyboard layout because the cost of that would just be so enormous relative to the payoff, and there's never gonna be like a trigger that forces the whole world to do that somehow. Like it's just not gonna happen. Um and products have the same things happen. I was talking with a founder friend of mine last night, and he said how like there are fundamental things that he would change about his product that he could have changed early on, but now it's really hard to do that because it's gonna affect so many customers if he did it, and the risk would just be enormous.
SPEAKER_01There are changes that he could make that could inadvertently wipe out some of his customers' whole business if he did it wrong. Um, so he can't just go and like make fundamental changes to the product now. So that is a lesson that I've been very conscious of. Because there's my friend and I talked about this too. There's common advice in the startup industry to just create a mess and then clean it up later with regard to the code and how the product works and stuff like that.
SPEAKER_01But like not everything can be cleaned up later. A lot of stuff is expensive to clean up later and often prohibitively expensive. So it really does pay off to if if if you know there's a decision of a design aspect of the product that's that's wrong, it pays off to change that early.
SPEAKER_02I totally agree. Especially when you have like a 14-year-old code base like we have. Um if you look at the like the user, the customer-facing app, that's a Rails app, and that start it has like the commits. The first commit was, I believe, August 2012, but it still has the code from that commit. And obviously we've upgraded Rails versions, Ruby versions, etc. etc. So it's all up to date. But if you start off messy and you don't clean it up, four years in, it's gonna be impossible to clean up.
SPEAKER_02And then at some point it's gonna be the mythical like big rewrite, which is gonna be a throwaway and rebuild, which is in the end much more expensive than just about to say getting it right the first time. That's also a myth. But at least attempting to get it right the first time. And then if you figure out later you got it wrong, clean it up. Um, like Boy Scout rule, leave it better than you found it.
SPEAKER_01Yeah. I I have a slightly different rule that is much less well known. I I I haven't gotten an uptake on it yet, but I I keep talking about it in hopes that maybe I will. Um so the the the gripe that I have with the Boy Scout rule is that sometimes the change that's needed is like very broad and fundamental and requires like the input of other people and stuff like that.
SPEAKER_01And so you can't just like unilaterally go and make that change. Like maybe uh the you you can clean up the campsite, but then there's also like these invasive beetles in the forest that are uh getting in your food, and and so you can't just like clean that up. You need to go and like do this whole program throughout the whole forest, and that's not something you can just do the morning that you pack up and go away and stuff like that. Um so my rule for myself is that I'm allowed to commit as many sins as I feel like, um, as long as I do an equal amount of atonement.
SPEAKER_01And the atonement doesn't have to be related, it just has to be an equal amount.
SPEAKER_02Interesting. Well, you're absolutely right in in in saying that you just can't fix everything all at once. There's always issues to fix, and you could have like a whole development team just fixing all the things that are okay, tidy enough, uh, but could be better. Um that's absolutely true. And you also don't want to have like, oh, uh there's this one line actual code change for this feature, but then this commit also has a thousand other uh uh lines with edits because I replaced steps with two spaces or uh or whatever.
SPEAKER_02Or we're now using this approach, I rewrote all these methods to do this other thing we discussed, right? Um so yeah, it there is a there's gray area. Uh let's then narrow down a bit. If you see a super messy campusite, at least uh clean it up enough so you s see what's underneath the mess or the initial mess. And if that's if that's invasive beatles, uh write an issue about it, and then if you go through the backlog once in a while and it's uh you can st is then still consider it an important problem, then just plan for fixing it.
SPEAKER_02But on the other hand, most of those issues go to silently die on some GitHub board, and then two years later I delete it.
SPEAKER_01That's true. My I I I tweeted about this the other day. My Saturn CI backlog is a written list on a sheet of paper, and as a matter of principle, I regularly just throw the sheet away and forget everything that was on it. Because if something's really important, it'll come up again. And I've worked so many places where they will like they'll they'll come across a problem and they'll choose not to address it, they'll choose instead to add it to the backlog, but we add things to the backlog faster than we cross things off the backlog.
SPEAKER_01And so it's like, hang on, if you're adding three things a day to the backlog, but only accomplishing one thing a day off of the backlog, what is gonna happen to that backlog? Like it's gonna grow and you're never gonna you're never gonna finish it. And as time passes, like your um the proportion of things you're getting done on your backlog compared to how many things that are on the backlog is gonna get tinier and tinier because your your bandwidth is probably gonna stay about the same.
SPEAKER_01You know, we'd we'd knock off one ticket a day, but then for every ticket we complete, we're adding two, and so in a hundred days, we will have completed a hundred things, but then there will be two hundred things that that haven't been done. So the day after that, we'll still just get one thing done, but there will be a hundred things done that we haven't uh haven't gotten to. And so now instead of like it's one thing to get one thing out of three done, but then a hundred days from now we're gonna get one thing out of a hundred done.
SPEAKER_01Yeah, and it's just not a good feeling, like it feels so bad to have this huge backlog and be like, oh, we have a hundred things and we only got one thing done. So I say just delete the backlog. I know 37 Signals is a fan of this policy too, just delete the backlog, but it's controversial. And whenever I suggest it, people are like, that's insane and stupid. Let's let's move on. Ignore JSON.
SPEAKER_02Uh yeah, yeah. So what we tend to do is if something if you create an issue for something, unless it's like an immediate issue, it needs to be addressed now because customers are having issues because of it, uh, let it sit for for a bit, right? Something that seems important today. In the grand scheme of things, if you look at it in 14 days, you may just delete it because it seems uh unnecessary. And then on the other hand, we use this concept of uh maturity.
SPEAKER_02We may have uh something, like an idea, but we can't do that right now because uh we don't really know enough about like who needs it, why, how it should work. And it's not important for uh enough for us to start investigating that. Uh-huh. But if you then circle back to that like once every uh six months or so, uh you'll just see things that have matured uh like organically. Like instead you haven't really done research into them, but just because of other things you've been doing over the past six months.
SPEAKER_02You just kind of see, huh? This is interesting. We could actually do this now because of uh X, Y, and Z that we did that was unrelated. Um or not at all, and we still throw it away, obviously. Um but just having something sit there and quietly mature, that has worked well for us as well.
SPEAKER_01Yeah, that's really smart. Um, you know, the it's this is this is analogous to like sometimes they're like, Man, I really want to buy a new guitar. But it's like, hang on, let's let's not buy it today. Let's wait a month. And if I still want a new guitar in a month, then I'll buy it. And it's often the case that you wait a month and it's like, eh, I actually don't want it that much. There are other things I'd rather have instead. And it's easy to succumb to the uh excitement of adding a new feature or making some change when the idea is new and it seems very worthwhile, but then you have a cool down period, and a few days or weeks later you're like, eh, okay, that like in in the scheme of things, like again, we have a finite budget, like, do we really want to spend it on that?
SPEAKER_01I think we can live without that. Let's do this other thing instead. I think that's a really good idea.
SPEAKER_02And there's never enough people, right? You can have we're a small company, uh, whether you have one developer, five developers, ten developers, twenty developers, there's ultimately comp companies with thousands of developers, and they're all always busy building something. Uh and sometimes for some products you wonder like how can so many developers work on this project without any visible, meaningful uh progression. Um but then I've also learned that there's lots of stuff going on um under the hood, and sometimes like I've really spent like a s like a six-week cycle with five people working on this thing that's uh invisible.
SPEAKER_02So we try to be really uh uh cognizant of that as well, right? So we should never uh focus a cycle uh only around things that are important to us but invisible to a customer. Because people just expect from SaaS they're paying an amount of money a month or a year, they just expect new features, improvements, etc. etc. So we try to always mix that in with yes, we have kind of like we have to do some maintenance and things like that, but we also want to have stuff that we can have like meaningful things to add to the change log.
SPEAKER_01Yeah, it is crazy how profligately so many teams spend their limited budget. It's like I I I shared the other day that scene from uh Dumb and Dumber, if you've seen that movie, where they're going on a road trip, these two guys, and they have very little money, they stop at a convenience store, and then one guy says, Okay, remember, only the essentials. We we don't have much money. And the guy goes in and he comes out with like a cowboy hat and one of those paddle games with a string and ball and some magazines and just all this stuff.
SPEAKER_01And so many teams behave like that, where they just like they they they get big eyes for all these new features and they make all these changes, and they're not even like bad ideas, maybe they're good ideas, um, but they they don't step back and say, okay, we have these things that are good ideas and worth doing, but we have a thousand things that are good ideas worth doing. This week we can only do like one, and so which things are we gonna do and which things are we gonna not do?
SPEAKER_01That level of like thoughtfulness and discipline is unfortunately pretty rare.
SPEAKER_02Uh yeah, I think so. And then you and then you have to depict not the things you want to do, but the things that bring value to a customer, which uh I just mean I don't mean to say like you have to build everything that customers want built, but assuming you're you've already groomed your backlog and it's only things that customers need, and you also feel like this would this would add value to the product, um then pick the ones that bring the most value to the largest amount of people, right?
SPEAKER_02Um there's lots of cool stuff you can build because and we're developers, right? We like to build new things and there's new technology, and we can oh we can swap out this and then use this uh swap out this queuing or this ingestion system and introduce Kafka, etc. etc. etc. And that uh and then just nerd out for a few weeks. And then in the end, there's this proof of concept that can replace the current system for about 80%. Uh so you can't replace it, and if you do, there's not really an immediate upside, and a customer is wondering what those people are doing.
SPEAKER_02Um so yeah, there's always fun stuff to do. Uh yeah, it's very strict with yourself to say, yeah, I would love to spend time on this, but instead I'm going to fix this, download your invoice button.
SPEAKER_01Right. Yeah, how many projects? Because people need those invoices. Right. How many projects have gotten 80% of the way done and then canned, and you get zero value out of all that all that effort that was spent?
SPEAKER_02Um most five-coded projects currently, probably.
SPEAKER_01Right, exactly. Um, well, this hour has really flown by. Um it's it's almost time for us to wrap up, but I want to touch a little bit on events and stuff like that that are that are coming up before we go. So you're going to Rails World, um, and as I mentioned, I am too. Are you going to any other Ruby events that are coming up this year?
SPEAKER_02So I've been to Rubicon Italy, um, Rubicon in uh Vienna, and Brighton Ruby in the UK, and then together with uh Rails World, that's gonna be my four uh for this year. Yeah, how those three were great.
SPEAKER_01Yeah, how was Rubicon Italy? I desperately wanted to go, but it just was not in the cards.
SPEAKER_02Yeah, it was the first time they were doing it in Rimini, um, a coastal town, very touristy. They were very early in the season, so we're just kind of like getting everything set up there in terms of uh well beach umbrellas, things like that. Um first time organizing, and it's always hit and miss, right, for first time. Like, do people show up? Is it any good? Uh let's just say it's the I've been to a lot of conferences, and this was the first time where for lunch we were ushered across the street to a different hotel where we had a three-course sit-down meal for lunch.
SPEAKER_02Uh that pretty much is set like that and that was like if that's keep that in mind is a benchmark for everything at the event. So uh really happy for us to be a sponsor for that as well. Uh just taking a leap. Uh that was a really, really good event. Good talks as well, obviously. Not too many people. I always like love Rails World, but also love an event with just 150 attendees where you can actually like shake hands with everyone, basically, and really connect.
SPEAKER_02Um they had a toga party on the beach and things like that. It was it was amazing.
SPEAKER_01I I did see some pictures, and at least one person was wearing a toga, and I was like, hmm, what is what is that about? So apparently toga party on the beach.
SPEAKER_02Yeah, lots of hotels with missing bed sheets, probably.
SPEAKER_01Ha ha, yeah. Um okay. Well, this has been an absolute pleasure. Uh before we go, any links that you would like to share for the show notes and such.
SPEAKER_02There's appsignal.com. If you like reading, there's our blog which has lots of not stuff about appsignal, but like real in-depth articles on Ruby and and other uh languages. Um for me personally, uh my website, probably the easiest, shortest to remember is Roy.io, which will take you to uh roythome.com with more information about me and my LinkedIn probably.
SPEAKER_01Okay, and dear listener, if you want to enjoy some tasty meet treats uh down in Austin with Roy and I, just come find us at the conference. We'll be easy to find because you just find the two handsomest guys around, and and that's me and Roy. Um we are near. Yeah, and and then we uh we'll get together and and we'll do it. And Roy, uh again, it's it's been a delight to have you on the show, and thanks so much for coming on the show.
SPEAKER_02Yeah, thanks for having me. Uh it was two years in the making, but uh we finally made it and um really happy, happy with that.
In this episode I talk with Joel Hawksley, Staff Software Engineer at GitHub and creator of ViewComponent, about whether AI is making us dumber, why blocking code reviews struggle under a flood of AI-generated code, and...
In this episode I talk with John Athayde, designer-developer and co-author of The Rails View, about what architecture school taught him about software design. John tells the story of a user test at LivingSocial that...
In this episode I talk with Riccardo Carlesso, Google developer advocate and co-founder of Rubycon, a Ruby conference held in Rimini, Italy in May 2026. Riccardo explains how a joke about PyCon became the conference's...
In this episode I talk with Travis Dockter, organizer of Blastoff Rails in Albuquerque. When Sin City Ruby and RailsConf both ended, Travis decided to start his own conference. We discuss how cheaply you can run a...
In this episode I talk with Julien Danjou, CEO of Mergify, about merge queues: why PRs that pass CI separately can still break main together, and how testing changes in a queue keeps main green. We discuss how Mergify...