283 - Tom Akehurst, CTO and Co-Founder at WireMock
1 hr 1 minAudio
In this episode I talk with Tom Akehurst, CTO and Co-founder at WireMock, about API mocking, testing philosophy (verification vs specification, contracts, the testing pyramid), inner vs outer loop development, and MCP (Model Context Protocol) for integrating AI coding tools with external services. Links: WireMock WireMock on YouTube Tom Akehurst on LinkedIn Nonsense Monthly
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
SPEAKER_00Hey, it's Jason, host of the Code with Jason podcast. You're a developer. You like to listen to podcasts. You're listening to one right now. Maybe you like to read blogs and subscribe to email newsletters and stuff like that. Keep in touch. Email newsletters are a really nice way to keep on top of what's going on in the programming world. Except they're actually not. I don't know about you, but the last thing that I want to do after a long day of staring at the screen is sit there and stare at the screen some more.
SPEAKER_00That's why I started a different kind of newsletter. It's a snail mail programming newsletter. That's right, I send an actual envelope in the mail containing a paper newsletter that you can hold in your hands. You can read it on your living room couch, at your kitchen table, in your bed, or in someone else's bed. And when they say, What are you doing in my bed? You can say, I'm reading Jason's newsletter. What does it look like? You might wonder what you might find in this snail mail programming newsletter.
SPEAKER_00You can read about all kinds of programming topics like object-oriented programming, testing, DevOps, AI. Most of it's pretty technology agnostic. You can also read about other non-programming topics like philosophy, evolutionary theory, business, marketing, economics, psychology, music, cooking, history, geology, language, culture, robotics, and farming. The name of the newsletter is Nonsense Monthly.
SPEAKER_00Here's what some of my readers are saying about it. Helmut Kobler from Los Angeles says thanks much for sending the newsletter. I got it about a week ago and read it on my sofa. It was a totally different experience than reading it on my computer or iPad. It felt more relaxed, more meaningful, something special and out of the ordinary. I'm sure that's what you were going for, so just wanted to let you know that you succeeded.
SPEAKER_00Looking forward to more. Drew Bragg from Philadelphia says Nonsense Monthly is the only newsletter I deliberately set aside time to read. I read a lot of great newsletters, but there's just something about receiving a piece of mail, physically opening it, and sitting down to read it on paper that is just so awesome. Feels like a lost luxury. Chris Sonnier from Dickinson, Texas says just finished reading my first nonsense monthly snail mail newsletter and truly enjoyed it.
SPEAKER_00Something about holding a physical piece of paper that just feels good. Thank you for this. Can't wait for the next one. Dear listener, if you would like to get letters in the mail from yours truly every month, you can go sign up at nonsense monthly dot com. That's nonsensemonthly dot com. I'll say it one more time nonsense monthly. And now without further ado, here is today's episode.
SPEAKER_00Hey, today I'm here with Tom Akehurst. Tom, welcome. Hey, thanks for having me on the show. So you're the CTO of WireMak, is that right? That's right, yeah. Um tell me a little bit about yourself and WireMak and all that.
SPEAKER_01So I am uh a career software engineer. Um I have been at it for well over 20 years now. Um I have worked in a lot of kind of uh sort of enterprise integration heavy kind of contexts, I guess. Um and about 14 years ago, I started an open source project called WanMock, which um started out as a uh kind of a side project while I was consulting. Um I like a lot of developers kind of scratched my own itch.
SPEAKER_01I kept finding myself in this situation where uh I was building systems that needed to talk to lots of different APIs that were often being built by other people, sometimes they they weren't complete or they weren't working properly, or uh the sandboxes being provided were weren't very good. Um and it was this you know this persistent problem that that used to happen on every project where you'd be kind of slowed down and made unproductive by these things.
SPEAKER_00That's weird. Every single time I've worked with a third-party API, it's been great. I I'm kidding, it's been awful every single time. It's just terrible.
SPEAKER_01Yeah, it's um it's it's a mixed experience. I mean, there are companies that um that work very hard on their developer experience, you know, that um are uh you know are very intentional about making it easy to adopt their API to integrate with it and someone like that. But um I think that's uh that's often the exception, and the vast majority, yeah, the majority of APIs that that are built are are done um as an afterthought, or maybe they they're very old and and you know sort of predate uh you know the idea of good developer experience and all that sort of thing.
SPEAKER_01So there's there's sort of numerous ways that uh it's possible to have bad experiences, I think, of um uh you know, uh working with other people's APIs. And obviously, you know, a lot of modern systems are um uh kind of microservice-based, you know, actually the majority of APIs are kind of internal rather than the you know the the external kind that we um you know we often associate with with sort of public APIs, which tend to be better because they're often part of a company's product offering. You know, you work with internal APIs and and there just isn't the money and an engineering effort to invest in making them a great developer experience, and and often uh working with them is very difficult.
SPEAKER_01And this was very much the experience I had. So this was this was also quite a long time ago, you know, when um the sort of microservices is a term I think hadn't even been coined, you know, when um when I initially got started on this stuff. But there was starting to be these sort of microservices like these sort of large-scale kind of SOA with rest type projects that sort of morphed into to microservices eventually. And and I was involved in work working on one of those sort of quite um yeah, quite an early one. And it was just a it was a really acute problem, you know, it was a a real productivity sink, kind of not being able to really make any progress testing the you know testing the code that we were building and um having to sort of you know resort to sort of hacks and shortcuts and um things that didn't work particularly well in order to uh to to try and move things forward in the in the project that we were in.
SPEAKER_01And um Yeah.
SPEAKER_00Yeah, I want to talk about this some more because that's a common challenge. Um, you know, I I teach testing a lot. Um historically, I've taught testing a lot over the last five, ten years, something like that. Um I have a book on Ruby on Rails testing. Um and and the sad truth about testing is it's not very widely practiced. And when it is practiced, it's often practiced kind of perfunctorily. Um people write tests just to check the box and say, I wrote a test, but the tests aren't particularly meaningful.
SPEAKER_00Uh the tests just verify that the code you wrote is the code you wrote, and it doesn't give you the um the protection against regressions that it ought to. Tests are often very tightly coupled with the implementation details, meaning that you can't really refactor your code because refactoring your code would mean basically making all the same corresponding changes to your tests, and that kind of defeats the purpose because ideally your tests are sufficiently loosely coupled from your implementation details that you can freely change your design without touching your tests at all.
SPEAKER_00Um, so that's the unfortunate state of affairs with testing. Um, and the reason I say that is because testing where third-party APIs are involved is a particularly tricky area of testing. And most people have troubles with just the basics of testing, and so this kind of advanced topic of testing around APIs, that's something that is very seldom practiced uh with any proficiency at all.
SPEAKER_00Um so I feel like it's in order to start right from the beginning with this, not that we have to get into all of it right now, um, but talking about first of all, when you're when you're writing tests for behavior that touches APIs, what should that look like? What kinds of things are we writing tests for?
SPEAKER_01So yeah, so that's a great question. I think there's there's a uh there are several different classes of tests that that'll fit under that kind of that that umbrella. Um I mean I guess the the the way I look the the way I like to the way I advise people to use API mocks is um to uh you you what you're kind of saying is that that you you have an idea of a kind of an you know you have an assumption about a contract with this API that you're integrating with. And so assuming that you know that your understanding of the contract is okay, then you can focus on testing the you know the the functionality, you know, where the the risk that you're you're testing for is the kind the the the uh is around the behavioral correctness of the thing that you're testing.
SPEAKER_01Um how how it in it yeah, how it integrates with the with with that contract, you know, where where those things are the risk, you know, that's that's where um I I think you know the the I I guess the majority of the useful testing happens in you know in in this kind of way.
SPEAKER_00Exactly. Um and that's a super, super important point, and frankly one that is is often missed. And so I think it's worth actually going over that one more time. Um when you talk about contracts, can can you explain what you mean? And and just go over this distinction again of okay, something I've seen a lot, which which is kind of funny and kind of sad, is like uh somebody writes a test where they call the API and they mock the API response, and then they assert that the API response is the same thing that they mocked.
SPEAKER_00It's like, well, you're not accomplishing anything by that. Um, and that obviously belies a misunderstanding of what we're going for with these sorts of tests. Um so in case anybody's not familiar, can you clarify what what you mean by this contract idea?
SPEAKER_01So um so what I think of as a contract is the the um uh the a description of the of a set of constraints around how you know how you can operate across an interface. So it's um uh you know usually it's it's a definition of a set of operations and the um the structure of the data or constraints on the structure of the data um that that you know can be exchanged during those operations. It's an interesting topic in itself. I mean, I I guess one example of what I would think of a contract is an open API definition.
SPEAKER_01Um so uh you know, we work with this quite a lot. It's the yeah, by far the the the sort of biggest um API definition standard out there. So um so you know we've built functionality into our product, for instance, that that allows you to um use that as a contract and kind of verify your your traffic against it. Um it's an interesting topic because I I think there are things that are um that can be defined within within sort of contract expression languages, and then there are things that sort of in practice can't be.
SPEAKER_01Um just to kind of I suppose elaborate on the earlier point about sort of you know types of testing that that um uh sort of can be achieved using using mocked APIs and and types that can't. Yeah, one of the pushbacks you get when you're you're sort of suggesting to people that they use mocks for their testing is um well, you know, there are if I do that, then then I get false positives. You know, things work fine when I'm testing against mocks, and then they they break when I go into my staging environment or my integration environment. So uh so that kind of invalidates the whole exercise.
SPEAKER_01And the the response I usually give to that is if if your approach to testing is just to kind of blast the system with a load of you know sort of simulated user behavior um sort of blindly and and then say, well, I'll just do this and then I'll I'll go and look for things going wrong. Um and where when you're designing a test, you're you're you're essentially you're not really explicitly thinking about what risk you're managing with this test. It's just like I'm simulating a bit of user behavior and I'm gonna see if anything goes wrong, um, then you're gonna have cases where the the risk is around the integration between two components in a way that can't be expressed by a contract.
SPEAKER_01And and those are the kind of things where I'd say, you know, it integration testing is is absolutely the right thing to do. You know, you do need to do real integration testing between components. But you should if you do everything that way, then you'll have very big, very slow, very you know, unwieldy um uh test suites that that you know become a huge amount of baggage. Whereas if you can separate those two things and say, I'm gonna isolate the risks that that are primarily about sort of um functional correctness of the component I'm working on versus the risks which are genuinely about the uh aspects of integration that can't be expressed completely in a contract, yeah, what you find is you have this small suite of tests that are managing that integration risk, and then you have a very large suite of tests that are are managing the functional risk.
SPEAKER_01Yeah, there's there's so many important points in here.
SPEAKER_00Um I I I want to pick apart a few of these things. Um, first, going back to the contract idea, I think of a contract in the very most like abstract sense of it, it it's like here's what I expect of you and you can expect of me. Okay, now what can I expect of you and what do you expect of me? And of course, two people can make like a legal or business contract or whatever, or two systems can have a contract saying, here's what I expect, here's what you can expect from me, and vice versa.
SPEAKER_00Um, just in case anybody's confused about like, okay, why what contract? Why that word? Um and I want to mention about this idea of the real behavior of the API in production versus the behavior under test. Um there's a a super important distinction in testing about verification versus specification.
SPEAKER_00And probably the number one mistake that I see in testing is is having a verification mindset rather than a specification mindset. Um the the verification mindset is like you're you're checking to make sure that it worked. Um what you have in mind is this particular run or this particular instance. Um whereas the the specification mindset, you're not you're not asking, did this does this particular one work?
SPEAKER_00It's not a QA check. Um you're you're constraining the behavior to make it adhere to the specifications that you want. Um I use an analogy of like a factory that's making products. You can do QA checks on the products and like have a pile of the defective ones, and whenever you're checking a particular product, you're checking it for defects, and if it's defective, you you toss it aside because it can't be sold.
SPEAKER_00But that's that's verification, that's a QA check. Um specification is more like you're inspecting you you're inspecting the same artifact, but with a different target in mind. You're saying, okay, does do these products in general, not individual products, but do all these products, are they meeting their design specifications? In other words, is the machinery that's producing these products um is it working such that the end result meets our specifications?
SPEAKER_00And that's what we're doing with automated tests. Um application code is factory machinery and a page load, an individual page load is a product. And so you're not doing QA checks to see like, is this particular page load working? You're you're asserting the specifications uh on the machinery, which is a very different thing. And so that's what leads people to be like, okay, did this API call work?
SPEAKER_00But that's not the question. The question is not, did this API call work? The question is, is my program interacting with this third-party API in the way that I specify? So you're not testing the round trip, you're you're making assertions like when the third A third-party API gives a 200 response and a valid payload, here's what I expect. And for example, when I get a 500 response from the third-party service, um, here's the behavior that I expect.
SPEAKER_00I expect this different behavior, and so on. And that, of course, is a much different thing. Okay, so this idea about like, well, the test environment is different than the behavior in production, it's like, okay, but a couple things. One is that that's a different thing. Um, we're talking about, again, the difference between verification, QHX, and behavior specification. And the other thing is if you experience different behavior in production than you do in the test environment, well, that's probably a sign that there are more uh scenarios that you don't have in your test suite yet.
SPEAKER_01Yeah, I mean, this is something I've observed a lot, is that I I think there is a um, you know, put per your analogy, you know, that there's a there's a mindset of saying I'm going to make explicit and I'm going to control all of my all the preconditions for my tech for the check that I write. You know, I'm going to specify these are the conditions in which this thing that I'm going to check should be true. And I'm going to enumerate those things and I'm going to describe them. And that includes the um uh you know things that happen via interactions with external APIs.
SPEAKER_01Uh, and the the the the thing you see sort of um, I guess, you know, maybe teams practicing sort of older older practices, I suppose I'll say, will will sometimes sort of take the view that the system is a black box. There are essentially pre there are kind of test preconditions lurking in there that aren't really repeatable or being controlled for, and they and they're not understood by the um you know by the person doing the work. They they're just kind of implicitly floating around in there, and then a test is performed. And and like you say, then then things go into production and those sort of invisible preconditions change in some way, and and people are surprised by it.
SPEAKER_01Um and I I think there is a there's definitely this um and it probably go, I mean, it goes hand in hand to some extent with the sort of TDD versus testing after the fact mindset. I mean, I I'm personally a big advocate of of TDD and um you know using it precisely to drive that mindset. Because if you're if you're writing a test first, then uh amongst all the other benefits of it, it kind of forces you to think about the what are all of the preconditions that have to be met in order for this thing to be to be right.
SPEAKER_01Um and um it's uh you know, I part I suppose part of what's driven uh WireMox evolution, you know, sort of particularly um uh you know, particularly kind of in the earlier days, uh, was that um there were there were cases where you'd sort of think about, well, I you know, but some particular error could occur, you know, an exception could be thrown somewhere in the code. And I wanted to think about, well, I I don't want to just kind of do you know mock something in memory and sort of make a make an exception be thrown because because the code says it can.
SPEAKER_01I actually want to create the um some something as realistically as possible, the you know, with the the um that will will create the condition for real that would would make that exception be thrown. And um and it it was it you know you ended up producing much more robust software that way because it you know if you were thinking in terms of well what what real-world condition could trigger this thing to happen, and I you know I need to sort of capture this and and and bottle it and and describe it explicitly in the test that I write.
SPEAKER_00Yeah, interesting. Um there was another thing. Um well what what you just said I I think is uh a deep topic because there's this question of how do I write tests the right way? Um like if I'm testing an API, how do I do it the right way? Um people always know always want to know about the right way, but what I always teach people is it's not exactly about the right way, it's it's about um it's it's about doing it in a way that's advantageous.
SPEAKER_00Um and and and it's often not about correct or incorrect, it's about uh costs and benefits and trade-offs. So like for example, um if you if you have a very low-level test um that exercises something, uh it's a deep part of your code, um, and and you're not testing an integration, you're just testing something in total isolation, there's costs and benefits to that.
SPEAKER_00A benefit is that there's a lot of stuff that you don't have to bring into the picture and you don't have to worry about, and so maybe your test can be simpler, and also because you're not bringing other stuff into the picture, your test can execute a lot faster, and for the same reason your test can be cheaper to write in terms of developer time, um, and and those are very real advantages. Um and so I personally do write a lot of tests like that because there's great benefits to those, but it's a trade-off.
SPEAKER_00Um if you're not testing a lot of things together, then there's a lot of stuff that's not being tested, and so you can go to the other end of the spectrum and write an integration test, um, and that has the advantage of demonstrating that everything works together, and that's a great benefit. But again, those tests can be quite expensive to write because there's more stuff involved, and so it's gonna be more expensive in terms of developer time, and also those tests tend to be more expensive to run, they're slower.
SPEAKER_00Um, and what's the right way? That's like the wrong question. Um the the question is like, what are the relative costs and benefits of these various things, and what mix do I want to have in my test suite? I think that's the question.
SPEAKER_01Yeah, yeah, I would agree with that. Um there was a thing I wrote a few months ago about the testing pyramid, actually, that sort of echoes that sentiment, which is that the you know, for a long time people were walking around with this idea of um that you know the testing pyramid being this heuristic they needed to follow, that you know, they needed to build lots and lots and lots of of those vast, very narrow um, you know, sort of truly unit tests, you know, you might might call them. Um bit you know, bit because of the you know, essentially be uh, in my view anyway, because of uh kind of the economics of writing tests you know 20 years ago when that that um uh metaphor was first kind of uh developed.
SPEAKER_01And I I think the um over time those economics have changed, and also I think practices have evolved, and it's possible to take a much more nuanced view of um when you're choosing to test something in a particular way. Yeah, do you know what I mean? You've got this kind of spectrum of possibilities, and exactly as you say, there's there there are trade-offs between um sort of speed and and specificity versus um the um you know a number of other favorable characteristics, like the you know, the likelihood of serendipicly discovering other problems that you weren't directly checking for, you know, the um confidence you get in the integration between your components, you know, the um you know, whether you're your you know the sort of infrastructural part of your your your code base, you know, so if you're on top of a web framework, you're you know, if you're on on Rails, you know, you've got the Rails stuff kind of configured correctly as well as your code being correct, all of that kind of thing.
SPEAKER_01And uh yeah, there's no there's no one right answer for any given test case. And you know, in different domains, I think that distribution probably ought to vary quite a lot. I think if you're if you're working in a um a really sort of high uh logic density kind of you know, complex like financial trading system or something like that, where you know the inputs and outputs of the system are relatively simple, but the the internal logic is incredibly complicated, then I would expect it to look test pyramid like. There'd be lot lots and lots of very fast, um, very isolated test cases to to exercise that.
SPEAKER_01But likewise, you know, the other end of the scale, if you've got a microservice which is doing um uh acting mostly as kind of an orchestrator, you know, which is very heavily integrated with other services and relatively light on logic, then I would almost expect the you know to you to turn that the other way up because the the integration logic is where all the risk is. And so you you you'd want the the bulk of the testing to occur there.
SPEAKER_00Yeah, that's an interesting thing you say. The integration logic is where all the risk is. What do you mean by that?
SPEAKER_01Well, I I think I mean, like I say, there are some systems that have um comparatively simple, you know. So, like I say, that you you you I I mean, I haven't personally worked on financial trading systems like this, but being kind of London-based, you know, end up around a lot of people who have. And um, yeah, a lot of the time you you have sort of you know messages in and messages out on a single queue, and that's pretty much um the sort of sum total of the the integration issue. But then you have you know this very very complex trading logic kind of running in the in the um uh at the core of it, and that's where all the risk is.
SPEAKER_01You know, that's where you really don't want to be making errors and and you want to ensure that you're um you've very thoroughly specified and tested what you're doing. Um but you know, convert conversely, as I say, great where you have very distributed systems um that are heavily interconnected, you know, the the amount of logic, the amount of code, you know, the serialization logic, the um you know, the the you know, the um the way um integration points are kind of exercised by the code, you know, that that you know that can be the kind of the locus of risk, I guess, in um in a lot of modern systems.
SPEAKER_01And that's where if you you know, so so if you you know if you go for a sort of hexagonal architecture in that in in uh in that kind of context, so that you can isolate and test the logic in the middle, then you know, the the useful bit of testing you do in the middle is tiny, and then meanwhile you've kind of under tested this very thick outer layer of uh of sort of ports and adapters that are actually really doing you know 80% of the work in your system. Um so yeah, I would kind of say that the the you know um the strategy you choose, or or rather where you know where the the sort of weight of testing happens uh is is very um you know, it just depends on the shape of your application in that regard.
SPEAKER_01And you know, hopefully if you're experienced doing these things, this what it should happen naturally. You know, you should find yourself think you know naturally thinking, well, this is the type of risk that is present in this this particular thing I'm doing, so I'll write this type of test. And over time you kind of accumulate a kind of test type distribution that you know that sort of reflects the risk distribution within the system you're building.
SPEAKER_00Um I want to ask you about WireMark specifically. Um, and I know we already touched on it a little bit, but if you can go over it again, what exactly is wire mark and what are the use cases, how do you use it, that kind of stuff?
SPEAKER_01Sure. So it's uh at its core, it's uh it's an open source um uh open source library uh written in Java. So with apologies to your your Ruby audience.
SPEAKER_00Um by the way, uh I'm very careful to say this is not a Ruby podcast. It's it's not it used to be called Rails with Jason, now it's just called Code with Jason. So please do not apologize that it's written in Java. That is great.
SPEAKER_01Yeah, I like to think that sort of um yeah there was a certain rivalry between the two languages for a while I think but I I feel like it's kind of it's uh it's died down and uh and people have moved on from that now. But um I mean certainly I yeah I mean it's a bit of an aside but there's I think there's been a lot of good cross pollination between the the uh the the two stacks sort of over the over the years. But yeah anyway so it's it's it's a Java library it can be it can be run as a kind of um standalone process. It has its own API for everything so you can drive it from a a file base or a sort of a you know kind of its own REST API.
SPEAKER_01There's a bunch of SDKs in other languages including Ruby um for for driving that API. Then you know we have a a set of commercial products around that so there's a a um SaaS hosted version that kind of gives you yes a sort of enterprise version of it. So you you get a a um uh sort of collaborative you know editing UI that you can go and work in and um uh you know a bunch of other stuff like the ability to work with with sort of open API based prototyping workflows um there's some sort of advanced uh stateful features that aren't present in the open source and something like that.
SPEAKER_01So um so there's a sort of full spectrum of uh like I say the the open source part which is really focused on you know it's the kind of the distinction I like to make now is the the open source is really focused on the sort of developers in a loop. So um if you're you're test driving something where if you're you're yeah you're you're building your application and writing um using Wire mock to support your um uh you know the the the the uh building of your microservice or your app or whatever and writing tests around it then that's kind of its um its sweet spot but it's it's very flexible I mean you do see the inner loop versus outer loop in case anybody's not familiar sure yeah so yeah definitely a term worth uh worth defining I mean the the the inner loop is really the I I guess you define it as all all the stuff that happens prior to you sort of going commit and push in git.
SPEAKER_01So um when you're um you're working in that tight loop of writing you're doing TDD at least you know you you write a test you write some code to make it pass um you um you maybe commit to that point and you know and refactor and um you know and even if you're if you're not doing that but maybe you're kind of you're building a web application and you're kind of hitting refresh in the browser and and tweaking a few things hitting refresh again yeah whatever whatever your kind of feedback loop is um it's it's that thing you do as a developer kind of in in this very tight loop before you you push anything to a uh you know out to your your git repository um whereas the outer loop is kind of the the stuff that happens after that so things that happen in your CI C D pipelines you know the um maybe the sort of tech the integration test suites that you you don't run as a developer before you push the um code scanning static analysis the um uh linting all of that kind of thing that that that kind of that it is are all necessary for the path to production for that that piece of code um and I suppose in our case we kind of think about that as the the activities that happen sort of um between engineers and between teams as well so obviously APIs are inherently this kind of um you know they're an integration mechanism they're about um connecting different parts of systems and and um indirectly you know sort of the means by which kind of human beings you know engineering those systems kind of kind of interact with each other as well so part of the outer loop is kind of the um you know coming to consensus and figuring out you know figuring out the design of those things that cut across you know more than one individual or individual's work or individual engineering teams work.
SPEAKER_00And okay so you say that wire mark fits into the developer's inner loop um what does that look like I'd say the open source tool fits in is is with the the inner loop and then the commercial tools are focused on the the the outer loop.
SPEAKER_01And then the out one other point is worth worth making as well is that the outer loop tends to be um the the the size and complexity of sort of outer loop activities tend it varies a lot depending on the type of organization you're in. You know if you're in a a small startup with you know with sort of a four-person engineering team then nearly everything you do will be kind of in a loop development. But if you're in a in a large enterprise where you've got you know you know 20, 30, 50 teams all doing um you know only a number of microservices each, then you're uh and you know you have security and compliance issues you have to deal with you have to make sure that you're you know you're um very you know you have very robust um quality control and and um you know sort of uh processes in general for for for shipping good quality stuff to production if you're in that kind of environment then your outer loop becomes very big you know you tend to be you know a lot a lot of activity and a lot of the sort of lead time for getting code into production is um is what I would I would class as as out of loop stuff.
SPEAKER_01So so this is this is why we kind of why we make this distinction that the we we sell our commercial product to large organizations who recognize the need for uh mocking and simulating APIs kind of at the at their scale and and and to support those kind of outer loop activities. Whereas you know the open source is about the it's focused on the individual developer and that can be um you know the the the whole range you know it supports developers in in enterprises doing their doing their inner loop work but then also you know the hobbyist developer who's just mocking up a a side project can use it just as happily as well.
SPEAKER_00Yeah um so what does that look like like when I'm writing tests for something that involves API uh interaction or any kind of third party service um first of all I have a tool that will block all external network requests because I don't want that to happen anywhere in my test suite. And then at the point where my code makes an external network request I will stub that response um so so that we say when we try to hit this specific URL instead of actually making a network request just just uh intercept that and return this precanned value um and and then I I can do what I need with my test because then I don't have to worry about that part of the um that part of the behavior making a network request and potentially introducing um indeterminacy to my test suite.
SPEAKER_00And then also you know I'm probably gonna want to say okay stub this one to be a 200 stub this one to be a 500 so that I can exercise the various behaviors that I need um often there's not a lot to that part. It's like okay stub a a a 200 with this like fairly empty body because that's not the part that I'm concerned with. So I'm curious where where does the open source tool come into the picture there?
SPEAKER_00What problem does it help with specifically?
SPEAKER_01So so it supports what you've just described. I mean what what you've just described there is the the essence of how um how WireMock works um you know how how mocking tools in general work. But there's within that there's you know that there's a lot of um there's you know nuance and richness even when you're you're just focused on the the kind of unit like testing scenarios. So WireMock has quite a rich request matching um system. So the that that part when you say when you're stubbing um when you're you know if you're serious about testing and you you know you're testing a uh you know often a whole bunch of different um variations of a particular piece of behavior say um even just within the space of returning a an okay response you could have a resource that goes through a number of different states you know you could have outputs that vary a lot based on the inputs you've given it all of that kind of thing.
SPEAKER_01So so WireMock has a matching system that allows you to be very precise about if I get um you know this this uh input or set of inputs then I then uh then you should match to this output. I'll give an example so it's a dates are always a uh you know I mean for programmers generally right dates are always a pain. They're a pain in this context as well because you if you have a um a stub that you've set up that says um yeah particularly if you use the record feature and there was a a date being sent on the input it might be that the time you recorded it that date was in the future but then um you know some time will pass and that date will be in the past and in the context of your the system you're testing that could mean entirely different things you know you could be triggering entirely different behavior.
SPEAKER_01So the one of the examples I always give is um the um sorry it's not quite the same thing but but similar enough I suppose is that this this train travel API um there's a really good open API spec that that's been been published just as kind of an exemplar and I I use this a lot for sort of um showing mocking concepts but there's you can search for for train trips by by date and um yeah there's a fairly obvious business rule in there which is that you only want to see ones in the future and to try and search for ones in the past is an error because you can't book train trips in the past.
SPEAKER_01So you get very very different API behavior based on the the date but it's obviously the date relative to right now. And if you if you make everything static like you would in a recording then that uh you have a very limited window in which that stub is valid and then it will cease to be valid uh you know as time passes. Whereas if you can work with dates dynamically and offsets and that kind of thing then then you it will sort of continue to work going forward. So um so that's there's a there's a load of stuff in there like that that sort of supports those um you know those kind of more you know those non-trivial cases where just matching like the URL and the method is not enough.
SPEAKER_01You know you really need to get into the guts of the um the request content to match it accurately. So that's one thing. There's the ability to kind of dynamically produce responses using templating. Sometimes this is necessary because you've got things like correlation IDs, you know, sort of headers that have to you know bits of information that need to be passed around between request and response and so it has to be dynamic. If it's static then it you know if the response doesn't respond with whatever was in the request then it just kind of breaks the whole flow. There's things like that there's wire mock will do it will simulate statefulness in a number of ways.
SPEAKER_01So there's kind of a finite state machine model. So if you want to simulate uh you know I take an action and then you know so I read something and it says one thing I take an action that mutates that state and then I read it again and I get something different back. So simple mocking tools won't do that. They're completely stateless and the the response you get back is sort of entirely a function of the the request. And most of the time that's fine but there are there are certain cases where um where you do need that additional ability to kind of simulate statefulness. So there's there's that aspect of it as well.
SPEAKER_01There's a few other things like being able to simulate low-level faults so you can WireMark can do all sorts of really nasty things like sending a reset packet instead of responding when you you give it a particular request or you know dropping the connection sending kind of garbage HTTP which is not valid. So you can you can as well as kind of returning non-2XX HTTP responses you can actually really break things down at the the network level or you can add latency sort of can you know conditionally in some cases. So it supports essentially a whole load of like resilience testing sort of non-functional testing cases in a in a very you know you can move the testing of those things to you can shift them left you know I guess is the way you put it you can do them in a sort of unit like context rather than waiting until you've kind of got everything deployed into a production like environment maybe with a sort of chaos testing system in there is the the um the way people will commonly test that kind of thing.
SPEAKER_00So I want to use our last few minutes to to ask you a fun question, at least I think it's fun, um and we can take this in any direction you want. So at this moment, you know we can call it the mid-2020s um we're experien we're experiencing this speciation event um where ai has gotten good enough to to be useful it's it's good enough to be um disruptive or whatever it's it's not just a novelty anymore it's engaging with our work in a meaningful way.
SPEAKER_00And and I call it a speciation event because you know it's it's gonna drive a whole bunch of things extinct it's it's gonna give birth to new kinds of things. And it it's my experience is a lot of the people I talk with um all these things that are happening are giving them tons of ideas. And well for example one one thought that I had was like when I first saw ChatGPT and Claude and stuff like that, I'm like this is amazing I love this but it feels like really uh clunky to have to go back and forth between the editor and the browser I bet this is gonna go away.
SPEAKER_00I I bet eventually we won't have to do this anymore and we'll be like man remember when we used to have to go back and forth between the editor and the browser? And sure enough now we have tools that work right in the editor like a Supermaven and uh GitHub Copilot and stuff like that. And I also saw like there's a lot more like like this mode of interaction where we just like copy paste stuff or it's kind of a fancy autocomplete like that is certainly not the ultimate utility of AI.
SPEAKER_00I even wrote this little tool where I could say okay write a test for me in in Vim and then in my right hand Tmux pane run the test then take a look at that test output look at the failure message then adjust my test to make uh or adjust my code to make that test pass and then write the next test case and stuff like that. And so it was kind of doing its own TDD and I was just doing a very high level orchestration.
SPEAKER_00My tool that I wrote was super clunky and not actually useful. It was just kind of a proof of concept type thing. But the concept I think was sound and sure enough now we have clawed code and you can do just that you can say like hey write me a certain test then run the test and write the code to make it pass and it's way better than the crappy little plug-in I wrote it's like actually useful um and then I have a friend who started this business she she shut it down now um she was on a previous episode Colleen Schnettler um she she wrote this tool where you could basically ask questions of your database and it would translate your natural language question to a database query and then give you the answer and I'm working on a speaking of CI CD all this stuff I'm working on a building a CI tool myself it's called Saturn CI and I'm I'm imagining all these possibilities with AI um like if you have a database of all your past test runs then you can ask all kinds of questions of it like hey show me all the failed tests from the last 24 hours or something like that.
SPEAKER_00I don't have to build in these features individually you can just point AI at it and and interact with it using natural language.
SPEAKER_01Yeah so I I think CloudBeas are started so CloudBeas bought an AI company called launchable recently and I and I think um that's that's very much the kind of thing that you know they but yeah using AI for kind of insights into CI and and to to automate um all kinds of optimizations in the you know in the CI process based on sort of AI analyzing your you know your test histories your failures all of that kind of thing.
SPEAKER_00Yeah yeah and what I was going to ask you is like just very broadly um I I'm sure AI has been on your mind like it's been on mine and I'm willing to bet that you see AI and you see wire mock and you're like wow like I see all these possibilities of of things we can do. Has has that happened in your mind or or maybe you've even started to work on things?
SPEAKER_01Yeah so we we started building an MCP server very quickly after the MCP protocol was was released by by Anthropoc. You know that that was a a moment where we kind of thought okay this is this is the thing that we needed um because we we recognize that you know our audience is developers they're working in AI developer tools you know they don't want to be working in in you know AI like text boxes kind of out floating around in some product somewhere they they want the work to happen kind of in their you know their their core AI tools so mcp was a real um was it was a gift to to sort of um be able to to make ourselves available in that workflow.
SPEAKER_01So yeah I mean we we're still very much at the stage of experimenting with it but there's uh yeah WiMock cloud has got a bunch of MCP as a as an MCP server and a bunch of mcp functions today and um you know there's some very interesting stuff you can do with it. You can um you can support the kind of workflow you're talking about. So you can kind of say I I want to build this new feature and it's going to need this new API endpoint and this doesn't exist yet but I want to to uh to you to simulate it and then build the feature that integrates with it so it kind of validates the design um of both the feature and the API correct and they work together.
SPEAKER_00Sorry I want to interrupt you a second because not everybody might be familiar with MCP. So this is model context protocol um this this protocol that's been put out by Anthropic um and and I went to the documentation and and started studying it and quickly I realized like whoa there's actually a lot to this and unfortunately there wasn't like a one paragraph explanation that like gave a a high level overview of of what it is and so I can't really understand so I'm in no position to like explain what it is myself but maybe you can help us understand that.
SPEAKER_01Yeah it's funny if you if you're if you're building developer tools you end up so steeped in it you forget that other people you know haven't haven't been exposed to it quite this much. And yeah it is it is a bit confusing.
SPEAKER_00I mean I'd say in in essence it's a a um it's another type of API um standard which is is designed to allow AI tools to connect to you know a yeah AI coding assistants AI agents to connect to external tools in a standardized way so the the the main use case for you know for us is to expose a load of wire mock functions so the ability to create stubs the ability to pull the request journal to you know validate uh interactions all of that kind of thing to make all of those directly available to your AI coding tool so it it's it works with clawed code it works with um GitHub Copilot Cursor Windsurf a few others um and the data's a significant part of it right like if we break down model context protocol it's like okay this thing is a protocol it's it's like a standard for how to write these systems um context that I think means like the the data just like the knowledge that the uh that the model is going to need you know the model being like the AI agent or whatever um this is a way to give context data or whatever to the model so that you can do not just the like pre-canned generic stuff that any AI agent can do but it can be aware of the stuff that you want it to be aware of so it can have capabilities that are tailored to your needs.
SPEAKER_00Is that kind of the right understanding?
SPEAKER_01Yeah exactly it can it can operate things in sort of you know outside of its own little bubble of yeah because it you you know your coding agent normally it operates on on the stuff within your editor and your project it can edit it can work on your files but that's kind of about it. If you want to you know for instance there's a there's a GitHub MCP server. So if you want if you want to say to it I want you to go and read this it this um this issue ticket um generate me a you know if it's a bug ticket go and go and uh generate me a fix for it and then when you've done it I want you to write some commentary back onto the ticket saying what you did and what the problem was and all that kind of thing.
SPEAKER_00Wait what does that mean that there's a a GitHub MCP server is that like a service that GitHub offers that you can like plug into like what form does that take?
SPEAKER_01Yeah so so MCP servers I mean essentially they're just another API that that could you know so so whereas you know GitHub's already got a REST API and in fact they've also got a GraphQL API essentially now it means they've also got an MCP API. And it's it's fundamentally the same the same idea you know it exposes a bunch of it they're essentially like RPC calls but but it's it's just tailored for AI um tools to be able to use so you um you install the slightly quirky thing about it you can run mcp servers locally so you just run them as a you know your your editor will sort of spawn it as a process and then attach to it via a stream um and then use it that way and then and then it will probably just be calling the REST API as usual.
SPEAKER_01That's how ours works as well. You can have remote ones as well but the the um you know essentially the idea is your yeah so if you type something into GitHub copilot or clawed code and you say go and fetch me this GitHub issue fix the problem with it update the issue saying you fixed it. And your agent will say okay well I I can because you've read you so you've registered the MCP server with your agent so it knows what tools are available. It knows what calls it can make and each of those this is the the key part each of those tools has got a a detailed sort of AI focused description on it that says this is what the tool is this is when you should use it.
SPEAKER_01And the all of that stuff is kind of in the in the context in the background somewhere so the the when you when you give your your coding tool an instruction a prompt uh it will say oh okay I've got a tool for that I I've you know there's the the um the you know get GitHub issue tool is available though you know it got registered when I when I started up so I'm gonna go and call that tool with the issue ID that you gave me or with the search text that you gave me or whatever and go and find it. And then I'm gonna you know I'm gonna read it back and then likewise there'll be another like update issue tool that I can use later to to write to it.
SPEAKER_01And um so it's just allowing it to directly integrate with something outside of your environment you know be it another data source, another SAS product uh you know in our case uh you know an ex a an externally hosted developer tool. So in our case you can say things like I want you to create me a new mock API, I want it to be for reading and writing invoices you know here are some fields that I want to have in it. Go make me a mock and publish it and and put it on an endpoint and then integrate my test code with it. And this is the great thing about it all happening in your coding tool is you can issue these instructions that that are that cut across multiple places potentially you know they they they might affect your production code and your test code and your your mocking setup and you know maybe a database migration and you can kind of um you can create a prompt which is sort of a composite of all of these things and you're kind of saying go and sort all this stuff out for me rather than you're hopping around between different tools you know maybe even different AI enabled tools but but typing prompts and saying I just want you to do this bit here and then this bit here and then this bit here by sort of centralizing it all and using MCP to integrate it all into your coding assistant you can you can give this kind of overall high level instruction to to the tool and it will go off and and do things.
SPEAKER_00So like I say one I was assuming um I was reading the docs yesterday actually and I'm like wow I I wish that this wasn't so new because surely uh the the AI agents don't know about this yet and so they can't explain it to me.
SPEAKER_01But that was just an untested assumption I'm curious if I go to Claude and I say explain MCP does it know about this like did the Yeah a couple of months ago it didn't but they but the the last few times I've been asking about it it seems to be getting better at um when you say MC because there's another protocol out there called MCP I think it's a kind of gaming industry thing and and for a while I was getting answers about that most of the time rather than rather than their own but it it does seem to be um getting into the training data now I think.
SPEAKER_00Yeah yeah I just tested it and Claude does in fact know about MCP. And it seems like this can be bootstrapped you know um even if MCP is changing faster than the training data can be updated uh surely you could plug into an MCP server for MCP itself and say hey get the latest info on MCP from this MCP server and help me write my own MCP.
SPEAKER_01Yeah very very probably certainly there are these MCP gateways coming up that that are are you know themselves what much wider indexes to to other servers out there. I mean there's like Cursor and I think um uh in fact I think Microsoft fairly soon are going to be launching a an MCP server not exactly a marketplace but kind of a kind of a an index um and so on so there's there's some the the the discovery mechanisms are um are improving um whereas you know before it was a little bit DIY you have to go and figure out for yourself kind of what the server was and what and where it was um evaluate whether you feel like it's secure enough for your environment and then um and then kind of install it yourself but that seems to be seems to be changing um kind of over time.
SPEAKER_01Yeah one thing I will say sorry yeah slight plug but I've I've made a whole load of videos recently um showing different things you can do with our with our MCP server. So it's working with coding assistance plus wiremox mcp server. So if you go to wiremox YouTube channel um I think there's five or six videos on there now showing various use cases. Oh nice and I said that would give you a flavor I I I'm I'm about to do a kind of MC like an MCP 101 video because precisely because I've realized that um uh you know not that maybe a lot of people don't really know what the what what the protocol is and does and yeah there needs to be a bit of introductory stuff but so a lot of these videos start with like oh and by the way I've installed the MCP server and cursor and now we're gonna go and solve this problem with it.
SPEAKER_01But um uh that said you know it gives an it gives a flavor of kind of what's what's achievable.
SPEAKER_00Um yeah yeah I I hate to make very specific predictions about the future um but some predictions are very safe like uh you know the prediction that it's very clunky to switch between the editor and browser so probably that mode is going to go away I felt like that was a safe prediction and now I'm patting myself on the back for being right about that. And MCP specifically who knows but certainly this idea that I don't want to just get uh the capabilities that the AI tool has with its generic training data.
SPEAKER_00I don't want just that I want it to have the context that I want it to have that that concept um is obviously very compelling and maybe MCP will become like the the the prevalent standard for that and maybe it won't but seems like in I'd say we're already well on the way to to that being the case actually I mean I I think whereas previously people were building custom agents and having to do you know their own sort of rag integration in order to get their you know their own context into their agents.
SPEAKER_01Yeah they with with MCP it gives it gives you a way of plugging an existing agent you know like your clawed code or whatever into those um data sources or whatever it is that are specific to your environment.
SPEAKER_00Mm-hmm Yeah um so it's what what I was gonna say is like learning about how this stuff works seems like a very um durable investment in knowledge because even if the details end up changing um learning how to integrate an AI tool with specific contexts I I mean obviously that's that's just very uh a very useful skill and already my like my wish list for things I want to do is so much longer than my capability to cross things off that list um so so the opportunities are really boundless.
SPEAKER_01Yeah I think a lot of people feel that way these days they're just you know um yeah where do you even begin?
SPEAKER_00Right yeah the next five ten years are going to be super interesting. Well that's probably a good place to to leave it.
SPEAKER_01The final thing I want to ask you Tom is if there's anything specific, you know you shared the WireMock uh YouTube channel if there's anything else you want to share that we should put in the show notes that kind of thing yeah so um I I mean I I'm most active on LinkedIn so if you um if you want to you know come and discuss stuff with me on there about API mocking and API we know uh integration and APIs in general then uh more than happy to do that. Um uh the uh YMOK website is YMOK.io so obviously I'd I'd um suggest going and checking that out uh and we also have a a Slack community as well so if you uh if you want support or want to get really get into the details of of using either the open source or the commercial product then um you can show up there and and I am my engineering Team and our wider community tend to hang out there.
SPEAKER_00So we'll put that stuff in the show notes and thanks so much for coming on the show.
SPEAKER_01Thank you for having me. It's been a pleasure.
In this episode I talk with Joel Hawksley, Staff Software Engineer at GitHub and creator of ViewComponent, about whether AI is making us dumber, why blocking code reviews struggle under a flood of AI-generated code, and...
In this episode I talk with John Athayde, designer-developer and co-author of The Rails View, about what architecture school taught him about software design. John tells the story of a user test at LivingSocial that...
In this episode I talk with Riccardo Carlesso, Google developer advocate and co-founder of Rubycon, a Ruby conference held in Rimini, Italy in May 2026. Riccardo explains how a joke about PyCon became the conference's...
In this episode I talk with Travis Dockter, organizer of Blastoff Rails in Albuquerque. When Sin City Ruby and RailsConf both ended, Travis decided to start his own conference. We discuss how cheaply you can run a...
In this episode I talk with Julien Danjou, CEO of Mergify, about merge queues: why PRs that pass CI separately can still break main together, and how testing changes in a queue keeps main green. We discuss how Mergify...