A set of principles to enhance testing APIs created inside R, and the engine behind automatic reloading of rendered Shiny apps becomes available for new use cases. Episode Links This week's curator: Jon Calder - @jonmcalder@fosstodon.org (Mastodon) & @jonmcalder (X/Twitter) Testing Your Plumber APIs from R watcher 0.2.0: filesystem watching for R, and the engine behind Shiny auto-reload Entire issue available at rweekly.org/2026-W28 Supplement Resources {plumber2}: https://plumber2.posit.co/ {httr2}: https://httr2.r-lib.org/index.html {mirai}: https://mirai.r-lib.org/index.html {nanonext}: https://nanonext.r-lib.org/index.html roxygen2 @exampleIF tag https://roxygen2.r-lib.org/articles/rd-functions.html#examples {watcher}: https://watcher.r-lib.org/ fswatch https://emcrisostomo.github.io/fswatch/ Supporting the show Use the contact page at https://serve.podhome.fm/custompage/r-weekly-highlights/contact to send us your feedback R-Weekly Highlights on the Podcastindex.org - You can send a boost into the show directly in the Podcast Index. First, top-up with Alby, and then head over to the R-Weekly Highlights podcast entry on the index. A new way to think about value: https://value4value.info Get in touch with us on social media Eric Nantz: @rpodcast@podcastindex.social (Mastodon), @rpodcast.bsky.social (BlueSky) and @theRcast (X/Twitter) Mike Thomas: @mike_thomas@fosstodon.org (Mastodon), @mike-thomas.bsky.social (BlueSky), and @mike_ketchbrook (X/Twitter) Music credits powered by OCRemix Relm's Weepy Loop - Final Fantasy 6 - MkVaff - https://ocremix.org/remix/OCR00178
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
Hello friends, we are back of episode two thirty three of the Our Weekly Holidays podcast and we were actually off last week since yours truly was definitely away from a microphone for a week, but had a lot of good times away from things for a bit, but I'm back in the saddle as they say. And if you're new to the show, this is the weekly show, usually weekly show, where we talk about the awesome highlights and additional resources that have been shared on this week's issue of R Weekly, which you can find at rweekly.org.
My name is Eric Nance and I'm delighted you joined us from wherever you are around the world and yes, I did miss you all last week, but I can't say I missed it too much because when you're on a beach at Lake Michigan, that's kind of a good time. But nonetheless, it is good to be back in the normal side of things along with my awesome co host Mike Thomas. Mike, I'm not sure if you went to any beaches since I was gone, but how are you doing? Been to a few pools, not any beaches, saw some fireworks over the US Independence Day last week, and doing doing pretty well. Can't complain. Had our first,
POSIT speaker training meeting, so that means that POSIT Conf is coming up, fast approaching here. It sure is. You're even repping the POSIT shirt as we speak in our video feed here. And my orientation for the lightning talks, I think, is coming very soon as well. So, yes, that means I gotta start drafting some stuff together. Time is sneak it up on me. But, yeah, you and I are both very excited about that. It'll be great to connect in person once again and meet all of you out there. You know, last year we were there, we heard a lot of great fans of the show say hi to us. So we definitely welcome that feature.
But, so you're to welcome me, let's welcome this issue to talk about. We've got issue week 28 to talk about and our curator this week is John Calder and as always he had tremendous help from our fellow, our weekly team members and contributors like you around the world with your poll requests and other wonderful suggestions. We definitely have a bit of a in the weeds ish kind of tool, kind of highlights to talk about, but if you're building capabilities, especially at this intersection
of web front ends like Shiny and utilizing other web based resources, you'll definitely want to pay attention to what we're about to talk about today to make your life easier when you want to test that this stuff actually works in a repeatable and deterministic fashion. And what we're talking about here is effective testing strategies when you are dealing with web based APIs, and not just ones that you may be consuming off the shelf, but ones that you are building yourself
via the plumber or the more recent plumber two package. And so this first highlight comes to us from the Absolom Blanc, but it's a familiar author in terms of recent highlights here, Jacob Soboluski, who has been, he may be near the top now in 2026 or how many highlights he's had on our very show here, but he is writing to us as a software engineer on the Epsilon team, talking about some great patterns and reliable principles that you all can take when you are dealing with testing your next Plumber API.
The key part that I think also applies to a lot of other constructs when you develop, you know, an R package or other capability, is you do want to kind of what he deems separation of concerns, I. E. Separate key fundamental functionality into different compartments. And where this really fits into play here, oftentimes your plumber API or whatever API you're consuming is doing some kind of analysis or maybe data summarization
or data querying, and that is the kind of stuff that you could technically do outside of your API call. You could just have an R package that does that thing like a dplyr kind of filtering thing, or maybe running a quick model to generate a p value and statistics or a best fit, or maybe a machine learning model. That's where you want to separate, as you're building this, your business logic from your actual plumber API
kind of calling manifest. The reason you want to do this for, there's actually more than a few reasons I would say, but the reason we're talking about in this post here, is that you can rely on the tried and true frameworks such as test that to handle that business logic kind of event, that testing strategy and you don't need plumber for that stuff, you just need the tried and true test that, build your test scripts effectively, have good documentation and your function calls and you're going to be set up for success there.
And then the rest of the layer would be what he calls the API contract layer. This is making sure that to get to the back end, that business logic from your plumber API, that all the different calls you do via the endpoints you've built in, the various request types that you do, that those give you the right response, that you get the right, what they call in HTML world or API world, the status codes that say things work like a 200 or things are kind of broken or you might have a 400 error.
I see those every day for many reasons, let's not get into that, want to be positive today. But yes, you want to be able to account for that in your testing strategy. Maybe you have a situation where you want that error to occur, want to build that in, but you don't want to inter, you know, conflate that, if you will, with the business logic stuff. So how do you actually go about doing this? So you got a few options here and you can use the combination that he recommends here
of using the mirai package, which has been the subject of a few highlights here, to run those API like processes in background processes, so that you don't have to hog your current R session. With that type of testing, you can offload that in the background while you do other things, and that also lets you do asynchronous evaluation of these expressions. So you can, you know, run through these API layer kind of tests quite quickly
and deal with, you know, the constraints you might have to deal with there in a more seamless fashion. And then also a great package to use to run these requests that you don't necessarily need for Plumber API building itself, but more for consuming APIs is the hitter two, h t t r two, however you want to say it, which has been the staple of my recent packages that have been interacting with a service called the Podcast Index API,
but I also use that internally for quite a few projects too. So you'll definitely want to invest some time learning HTTr2 as you're building your testing strategy of the API layer. And then he also recommends from the pattern side of things that you how you go about actually performing this. You first, in this arrange step, you want to launch that background process in the a for the API, get your test data set to go, and set up whatever authentication you might need, especially if you're layering on top of another service in your Plumber API.
That, you want that setup stuff to be done in a repeated fashion so that you don't have to keep doing manual calls to maybe logging in to AWS or other things like that, which I've had to do every day as well. Another thing you want to think about in this next phase of it is to actually do the operation, that's the acting side of it, that's where again, HTTR two is your friend to perform those requests, and then lastly, looking at what you expect
out of those calls and building your assertions, again, using familiar frameworks like test that to make sure that you're not as concerned in this side of it, this API testing side of it, with what the business logic stuff's going to do. You're more concerned about maybe the shape of what you're getting out. Is it a rectangular set, not necessarily the contents inside it. Are there any ancillary information that you get in like response headers? Are no status codes working as expected?
That's what you want to build in with this, you know, optimized framework. He also recommends having a consistent naming convention for your testing files. Yes, please, to that because I have often seen that in my future look at a legacy package I built tests for, if I don't name them very well, yep, I am just opening up many of them trying to figure out where did I bury that particular test assertion, and I just waste time on that. So if you can have a logical prefix kind of strategy to your file names,
you'll be better off to kind of organize that and figure out if something's broken, where do you go immediately to find that. He does mention in the case where if you're building a Plumber API that's expected to be handling not just random requests here and there, but concurrent requests, maybe in a high demand fashion, you might look into leveraging the NanoNEXT package, also authored by the same author of Mirai, Charlie Gao, who will be featured in a little bit here.
That will let you run these API calls not just in the background, but also in a concurrent fashion if you have to think about how are you handling high demand in your API calls. I think that is a very awesome recommendation as well. And then he concludes here with a couple other principles of avoiding some pretty common pitfalls that you might get. This happens to me a lot, although I'm trying to get better with it. Sometimes you find yourself duplicating logic that you're kind of slowly migrating stuff from the business logic side accidentally
into this more API testing side of it. Yeah. Don't do that. Try to keep them separated. You'll you'll thank your future self. We'll thank you later for it. And, also, don't try to test too much about your implementation detail that you should, using the traditional paradigms of calling APIs, be well equipped even if, like, certain things change under the hood. It's more about the process, not so much the specific nooks and crannies
of the implementation. And, yeah, I've been through this last point I'm about to mention. You may have external services you have to rely on. You may not be able to avoid it. That's where mocking becomes a very helpful thing to invest into so that you don't always have to call that external service every single time you're testing your Plumber API. Use mocking. It helped me a lot in a previous project, and I am using that technique anytime I deal with an AWS API, an Azure API,
or anything like that. Very, very helpful. But what, what, Jacob, you know, hypothesizes at the end of this is that putting this upfront investment here, you will be able to get that time back by getting the feedback really quickly as things break, as things have regressions. You're gonna be able to find where those things happen very in a very optimal way, and we think that is where your future self will give you a lot of kudos if you implement this approach. So as always, Jacob comes with the awesome practical insights here that always make me feel like every time I think I got a somewhat bigger handle on things, he changes the questions as a famous wrestler would say. Yeah. I mean, I like the way that this blog post just starts in general talking about that separation of concerns,
philosophy philosophy in terms of, you know, separating your business logic from your API contracts. And I always, think of sort of when we're developing packages or Shiny apps or things like that to make sure you have sort of that mindset where you're putting your business logic in one place and other things in other places because I think it makes your life a lot easier when it comes to unit testing and you know, all those other downstream activities. I really also, you know, like all the little tidbits in here and the use of modern tools like MirrorEye,
as you mentioned, to to launch those background processes that allow you to sort of keep developing and have non blocking environments for your tests. And then hit or two, can't say enough good things about it. Just the quality of life improvements that we got with Hit or two over OG hitter are phenomenal. I actually really like writing hitter two code. I sort of also like the fact that Claude isn't none of the, not just Claude, but none of the big LLMs are are that great at generating
good hitter two code, in my opinion. There's just some functions in that package that are so useful that I think the LLMs, you know, don't know about just because there's not enough training data around it. But highly recommend diving into that package. There's all sorts of little things like throttling, request agents to tell the API who you are, the ability to just kind of build these URLs and just pass parameters. It's really actually enjoyable to work with for anyone that still actually likes writing code.
And then that that We're still out there, Mike. We're still out there. I'm so happy to hear that. That triple a pattern, arrange, act, assert, where arrange, you know, start the API and prep the data, act to execute that HTTP request, you know, using hitter two, and then assert to to verify the contract status codes. You know, is the JSON structure correct? I really like the way that that's laid out in the blog post. And, you know, the there's test that tips, you know, for naming conventions,
for running only specific tests. I think a lot of us that are package developers, maybe when you're, I mean, starting out or even when you've been doing it for years like me, I only recently realized that there's like a whole other giant library of functions within Testthat that you can run besides just running all of the tests. Right? There's ways to only run the tests in a particular file or only in a particular directory or only those that match a specific naming convention and it's super useful. So, if you haven't taken a look at some of the other sort of helpful helper functions and test that, it can be really handy,
especially when your test suite builds up larger and larger over time and when there's sort of an expense to your test suite as well, like, say, you're hitting a live API. And, you know, to that end, you know, mocking is a must when writing R packages that are API wrappers. It can take away some of that expense that I was talking about. There is a Roxygen tag that I also, you know, admittedly, just recently learned about. It's the example if tag where it'll only run your examples Yep. If, it meets a certain criteria that you sort of just put like a tiny little r function right next to that example if tag which is great because it saves us from like doing a whole, you know, old school. Don't run. If you've ever done that in curly braces and then have some if statement in there. It's this is way more concise. So check that one out if you're not familiar.
And then I I think sort of a at a high level as well, just an overall look into, like, the landscape of what we can do with Plumber, hitter two, you know, MirrorEye nowadays, we can put R based APIs in production. I think back to Heather and Jacqueline Nolis's website, PutR in Prod, where they were running heavy API workloads with R based APIs at oh, what was the T Mobile? T Mobile. Right? Yep. Yeah. And that we're handling, like, a ton of traffic. I think hundreds of thousands or millions of requests per day. And, you know, they were also hitting, I think, a neural network model and this is back, you know, before AI was what it is now. This is OG AI models
and I have to imagine that if they had the tooling that we have now to do that, one, their life would have been a lot easier, probably, for parallelizing this API with, you know, tools like MirrorEye. But they could also probably, you know, push the boundaries of what they were doing even further. So, hopefully, you know, that work has been able to expand with some of the new tooling that we have here that's that's laid out in this blog post. But it's an exciting time to be working with APIs in R.
Yeah, it is the, you know, the web lingua franca of interacting with different systems as they might say, and yes, certainly with the age of LLMs, you know, having robust APIs is a great way to put those into, say, tool calling or other, you know, MCP servers or who knows what you want to, you know, shake into that that way, but that's how all these are communicating too. It's all, you know, API calls one way or another under the hood here. So yeah, great thanks to Jacob for drafting yet another post that's taken us to school in the world of novel testing strategies.
And speaking of having kind of rapid feedback while you're building your, you know, testing and verification, well, as Mike and I build a lot of shiny apps in our daily work and we still do, we have not taken away from doing that yet, There always is a time when you are in rapid iteration mode and you're working on that module, you're working on that feature and you'll, you know, re save your code. And then a lot of times like I had to do in the old days, you'd have to rerun the app manually.
You just have to stop that process I was doing with the stop button in RStudio proper, launch it again. Well there have been recent enhancements to Shiny that may or may not have gone noticed where if you opt into a certain mode called development mode you may have realized that when you start making changes to your shiny app code and you already had it running it automatically picks it up. That's a nice quality of life feature, isn't it? Well, the tooling behind that now has become a little more public, as they say, where it started on the shiny side of it, now it's available to us for use in any context where we need to look at
what's changing somewhere in our system to re trigger a certain process automatically. That is the subject of our next highlight here, which is called the Watcher package, and this is authored by Charlie Gao, the author of Watcher and also a very talented software engineer at Posit, who's been featured many times here. And this blog post is about the recent version 0.2.0 of the Watcher package, which is tagged as the file system watching package for R,
and then we're also going to talk about how it's been used in the Shiny ecosystem for what we just talked about. So Watcher itself, available on CRAN, very lightweight dependencies, you can install via the usual means, but again, many of us out there, myself included, did not realize that in recent versions of Shiny, this was being used under the hood to power its more recent enhancements to reload the app
to make sure this is a seamless experience. Now before Watcher existed, the Shiny team had a different method of doing this, and that was through polling itself every few 100 milliseconds. It would literally run that to relist the directory of where the app code is, look at the timestamps, if anything was modified, then to reload the process. But that is repeated polling of this, right? It's just like when you do a reactive poll in Shiny,
you're saying run this kind of polling operation at a regular interval. Well, what if there's a better way? Well, absolutely yes there is, because what Watcher does is it looks at an operating system's own file system notification layer to determine if you have changed that application code. And so in this version 0.2, it's not the newest, it's not the original release of the package, but they have simplified the process
of how this can install from the source version. And just to be extra clear, this package itself is wrapping a library called the LibFS Watch, which has been authored by Enrico Cristodomo and a former POSIT engineer, Alan Dippert. So it's got, we'll put a link to that library in the show notes if you want to look at what's more under the hood of watch. With using that library,
it has, like I said, become much more efficient to watch the file system for these changes without running this kind of expensive polling operation. This package exposes the object you create to initialize the watching, which are example, snippets with the watcher function in the blog post here, it is an R6 compliant class. So that means that you get your familiar interfaces of stopping it with the stop method,
check that it's running, get what file paths it's watching, which you have to supply when you call the function, and also you can also augment additional arguments to the watcher initialization itself, such as where to watch stuff, one file, multiple files, a directory, you have all that available to you. You're gonna have a custom callback to run when things change, and also to determine what the latency it is for the bouncing events before you report them
and everything like that, but in essence, this package lets you use this kind of framework outside of Shiny for interesting use cases, which is the part I am putting my head around after reading this is where could this be useful. There could be cases where maybe you're running an R process kind of in a long term way, such as in your Shiny app, and maybe you're relying on a different area in the file system where that Shiny app is running from, that somebody might just drop a file in there. Maybe it's an Excel file, an XML file for specs, or things like that,
if you have this watcher package invoked inside, you could have your Shiny app immediately get those changes without necessarily doing a reactive pull, at least that's what I was thinking in my noggin here about trying it. But there could be other things outside of shining where you just want that reliability to know when things change and do something with it. It's a framework that's in a lot of operating systems these days that you just don't realize because it's all just kind of automatic, but now in the R side of things, you can tap into that same type of back end functionality.
So I'm intrigued by where I could take Watcher, and again, credit to Charlie and the team at POSIT for, again, open sourcing was started as a very specific feature in Shiny, and now making it available to all of us that consume in our R coding adventures. Yeah. That all Shiny auto reload, feature sort of just snuck up on some of us at least on our team for those that do a fair amount of shiny development work. And Shiny's had some pretty interesting and significant
updates in terms of the features in the latest versions of the packages that have been rolled out. And as you mentioned, it's been quietly powering Shiny behind the scenes now. And it's a general purpose tool on Krayon for all our developers, which is super exciting. And the old method, as you mentioned, was a more polling based method where you check file modification times every few 100 milliseconds maybe, which obviously is is computationally pretty expensive and and resource heavy for large projects. And maybe even on some of your large Shiny apps, you actually felt that yourself. And and this new method with Watcher,
you know, allows that package to subscribe to the operating system's native file system change notifications as you mentioned like that LibFS watch package and this is all probably above my pay grade but it's still super interesting stuff and just as like a a tangent as you know, some members of our team have, we have sort of a bring your own device policy and everybody has different architectures. And even if we have multiple folks on Windows, one might be running AMD,
and the other one might be running an ARM architecture. And it is wild how much code is required or out there to handle developing agnostic packages like this that can work across all these different architectures handle all of these little different things sort of behind the scenes. I think that's probably what drives some software engineers to take up art or something like that. They could go a little crazy. Yes. No. I can understand that. Become
a general Exactly. So, yeah, that the really cool thing about Watcher is, you know, object oriented based. It returns an an r six object, simple sort of start and stop methods, in that object oriented approach. The the three key arguments are path, which tells you what to watch. Callback tells you what function to run, on a change, and then latency to debounce rapid fire events, which is, you know, really powerful stuff there. And again, it runs, you know, on a background thread. Seems to be a theme of this issue
while your callback actually runs on R's main thread via the later package. Another one that I wasn't familiar with, but there's so many of these tiny little packages that underpin the big ones that we use all the time. So, you know, some creative use cases that they they lay out are automatically, you know, rebuilding reports or documents when data changes, processing new files as they land in a directory, or sort of hot reloading, you know, package code or configuration without actually having to restart services, which is
one of those pain points that I think that we've all had to deal with in terms of iterating on a project instead of running dev tools, load all every time you make a change. Or if you're not in a package setting, you know, source sourcing your script every time that you want to, sort of update the code that's changed behind it. These can be some use cases that can help mitigate that. So pretty cool stuff here. Yeah. There's a lot of other ideas that maybe I just haven't percolated yet in my brain of where maybe in the past, like I said, I was watching
or trying to do, you know, random polling type mechanisms myself when we have these internal functions that depend on a directory in the file system, but yet we didn't know how in R to be notified of when that stuff changes over the manually run, like a file. Info an EFS package, getting the file timestamps and then doing something with that. So I could definitely see some potential in this space. And yeah, there is,
tend, at least I do, I tend to take it for granted, just the engineering that happens with things like, or packages like Watcher and many of the other packages in the RWB kind of repo deposit maintains, where so many of them have interoperability between operating systems so that when you install it, you don't have to care if you're on Mac ARM or that Windows Intel or Linux Intel or Linux AMD. Everything
is just built in a way that you, that's all abstracted away, especially on the recent HPC packages like Mirai, like the future verse, all that very similar tooling here. So yeah, Watcher, I think is going to be an underrated hero in a lot of these, you know, robust automation pipelines where you have to deal with file system changes. And I'm definitely intrigued by where that can go. And again, I always like seeing what Charlie's cooking up and we will be seeing him at POSITCOMP in a couple months, so it'll be great to touch base with him in person on this.
And with that, that will conclude our highlights section. We had two highlights to talk about today, but as always, the fun never stops there with our weekly. You've got a whole collection of additional resources that have been available to you in the curated issue that John pushed out earlier this week. You've got, you know, we don't have time to dive into all of it, but some recent explorations and things like Claude Science, which kind of gives me the bullies a little bit, we'll say that for another time, as well as some other great packages that have been released in the ecosystem and some
upcoming calls to action and great tutorials out there too. So definitely make a bookmark as always to rweekly.org, it is refreshed every week and also if you want to help the project, which again is something I hope to talk about in my lightning talk, but one of the best ways to do that is to help keep the project going via your contributions. Send a poll request of that new package that either you author or some others that you found in the community, great blog posts, great resource,
great video tutorial, they're out there. I hope to get back into that sometime soon, but yeah, just send us a poll request with that information and the markdown supply template and our curator for the upcoming issue, we'll be glad to get that in and also you can get in touch with us on the social medias out there if you want to get in touch with Mike and myself directly. I am on blue sky where I am at our podcast at bsky dot social. I'm also on Mastodon where I am at our podcast at podcastindex.social
and I'm on LinkedIn. You can search my name and you'll find me there. And I'm happy to report that, it was right before I left for vacation, but not too much before, I did get out the, recordings for the recent Art Pharma Gen AI Day virtual event. And there are a lot of interesting insights there that don't always make me feel doom and gloom, and it makes me feel optimistic of the future of this intersection of life sciences, data science, and AI that we're all just kind of learning on the fly, so to speak. But, Mike, where can our listeners find you?
You can find me on blue sky at mike dash thomas dot b s k y dot social or on LinkedIn if you search catch Brooke Analytics, k e t c h b r o o k, you can see what I'm up to. Excellent stuff. Well, I know you've got some things to push out on production very soon, and I got some things to wrap up. So we'll close-up shop here on this episode of our weekly highlights. We thank you so much for joining us. And, yeah, it's great to be back in the swing of things, and we will be back with another edition of our weekly highlights
Our takes on the increasing removal of friction in good and not-so-good ways thanks to AI, a visual tour of hot air balloon accident data, and a case study of novel image recognition with the rplatipus package. Episode...
The flow of both data and analytical jobs just got smoother with mizu, opening the door for multiple agent harnesses in Positron, and an optimal statistical model tailored to complex experimental designs with a friendly...
What happens when that online data set isn't so easy to download anymore? We learn of two packages to ease the pain of obtaining these files and then some. Plus a visit to the visualization corner with a handy way to...
The celebration of rOpenSci's 15-year anniversary continues with the community sharing their perspectives, and a fun Shiny app that could be quite useful during celebrations on a weekend or the holidays. If you ever...
We learn about the variety of ways you can contribute back to the core of R, a roundup of the excellent choices available for package and environment reproducibility, and an amazing collection of Shiny applications and...