How the new logrittr package gives you a peek behind the tidy pipelines of dplyr, and a fascinating glimpse into the universe of seeds going far beyond a favorite 80's song. Episode Links This week's curator: Jonathan Kitt - @jonathankitt.bsky.social (Bluesky) logrittr: A Verbose Pipe Operator for Logging dplyr Pipelines Generating universes within universes with a single seed Working Smarter in R: Tips, Tricks & Real-World Lessons Entire issue available at rweekly.org/2026-W18 Supplement Resources tidylog https://github.com/elbersb/tidylog/ logrittr documentation https://guillaumepressiat.github.io/logrittr/ Fine-grained control of RNG seeds in R https://blog.djnavarro.net/posts/2023-12-27_seedcatcher Supporting the show Use the contact page at https://serve.podhome.fm/custompage/r-weekly-highlights/contact to send us your feedback R-Weekly Highlights on the Podcastindex.org - You can send a boost into the show directly in the Podcast Index. First, top-up with Alby, and then head over to the R-Weekly Highlights podcast entry on the index. A new way to think about value: https://value4value.info Get in touch with us on social media Eric Nantz: @rpodcast@podcastindex.social (Mastodon), @rpodcast.bsky.social (BlueSky) and @theRcast (X/Twitter) Mike Thomas: @mike_thomas@fosstodon.org (Mastodon), @mike-thomas.bsky.social (BlueSky), and @mike_ketchbrook (X/Twitter) Music credits powered by OCRemix Passing Breeze (Lounge Suite) - OutRun - Remorse - https://ocremix.org/remix/OCR00875
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
Hello friends, we are back of episode two twenty four of the Our Weekly Highlights Podcast and this is the usually weekly show where we talk about the great highlights and other resources shared at rweekly.org. And just a quick admin note, as much as I was saying we're going to try and be consistent with our week to week release, unfortunately we did not have an issue last week so we were off, but we got a new issue this week so we are back. My name is Eric Nance and I'm delighted that you joined us from wherever
you are around the world and joining me virtually as always is Mike Thomas. Mike, how are you doing today? Doing pretty well Eric, fighting the spring allergies, but got some Flonase going and got to power through this. Yeah, I've got feel like I got that midweek here. It's different than when I was sick like a month ago, but it's enough to be like, yep, the seasons change, the molds are out there and yeah, next time I have to mow the lawn, I'm probably going to sneeze 50 times. But it is what it is, first world problems I guess. It could be a lot worse, but we're going to talk about positive Yeah,
about positive things today. We got a great selection of highlights and our curator for this week's issue is Jonathan Kitt, and as always he had tremendous help from our fellow RWQY team members and contributors like all of you around the world with your poll requests and other suggestions. And we're going to lead off with a great utility in the land of the tidyverse and data processing and getting a glimpse into kind of what's happening maybe behind the scenes or at least through
a typical, you know, dplyr like pipeline. So I think most of you listening to this are familiar with the tidyverse and dplyr in particular, where you get your full gamut of being able to create new variables, do filtering, do other transformations, it's become a big staple in the world of data science and data munging in particular. Well, when you run dplyr off the shelf and you're doing a batch of mutate filter steps,
you typically won't see any output unless you ask for that resulting data frame that you get back out of that pipeline. That's typically by design, right? Most pipelines will, you know, only let you opt in to more verbosity if you want to. Well, in the world of other languages, especially one that is still fairly dominant in my industry, such as SAS, you do get in its PROC DATA step actually a pretty copious amount of messages
in SAS's log file when the data processing is being conducted, such as a number of observations and maybe some records that were deleted or added and whatnot. But up until now there wasn't really an equivalent version of that in the world of dplyr or tidyverse pipelines. Bowen, our first highlight today, we are going to talk about a new R package called LogReader. I'm going to take a venture that's the way to pronounce it given its similarity to the MagReader,
which is tagged as a verbose pipe operator for logging dplyr pipelines and this package and accompanying blog post is offered by, we're gonna try my best at this pronunciation, Guillem Precia, I believe that is French correct me if I'm wrong out there, but he has created, yep, like I said, this aforementioned package called Lagriter, and on the TIN what it gives you is a different operator
that you can use to replace your call to either the Magruder pipe or even the base r pipe, and the operator for this is surrounded by a percent sign with a greater than sign and an equal sign immediately afterwards. So much like the Magruder pipeline, which I believe also had its origins from France, but you just add an equal sign after that greater than sign in between the percent signs. So in the example on the blog post,
author does a simple filtering and mutating step with the iris data set, and what you see is the output is showing kind of these interesting messages in the terminal that at each step of the pipeline, whether it's just converting it to a tibble, doing the filtering, doing the mutating, it's going to indicate the number of rows, the number of columns, the time it took to complete it,
but as far as kind of like a diff is concerned, it will show next to the appropriate rows or column numbers how many were either added or subtracted at that particular step. So for instance, when it does the filter for the sepal length being less than five, it goes from 150 observations in the IRIS data set down to 22, and then it has next to that 22
the delta of that difference, -one hundred twenty eight. So this could be very helpful if you want a quick read in an audit, so to speak, of what's happening in terms of the number of observations and the number of variables that are being added or subtracted. So this one is again very utilitarian, the rest of the post does mention that the package does play nicely with another package that we may or may not have talked about maybe last year
called Lumberjack, which kind of has its own unique take on messaging and logging with respect to tracking changes in your data with typical dplyr like pipelines. So that might be interesting if you want to do a nice combo of those together, but this is a very interesting take on this. The post concludes with some limitations and that this is really only going to be supportive for data frames themselves in your R session,
not so much with databases themselves that you might interface with the dbplyr package as your front end to those kind of queries and whatnot. And there, and it sounds like join cardinalities are a little bit difficult to handle, but it looks like he is working on that, he is working on that. But yeah, he invites others to give it a test and there will be a link to the package documentation in the show notes, but it might be a useful tool if you want just a little bit of a glimpse into what your DPR pipeline's
doing to your precious data. This is a really cool, new new package out there, and I think as as Guillaume says himself in this blog post, it's really kind of pushing the boundaries or he was really exploring what you can do with CLI, the CLI package. And if you see kind of this this beautiful console printing, with this header at the top and then sort of information graphics and and things laid out in very structured rows neatly
in the console, it's it's fantastic. I mean, the information is just presented there so nicely. And I think we've seen probably, you know, in the last couple of years a lot of movement to kinda beautiful CLIs, not just the CLIR package, if you will. And I think this is a a great example of that as well. One other adjacent package that, Guillaume mentions is the tidy log package, which he said gave him some motivation to develop the loggerator package.
But tidylog, in a different way, kind of works by masking dplyr functions, instead of really working alongside dplyr instead. So I, like you, Eric, are probably a little too familiar with sasproc log, outputs and having to take a look at, not only like a, you know, a sas script, but also sort of the log script that goes alongside that. And I'll be honest, it is nice, to be able to kind of see those those logs and help keep track of of what's happening to the data behind the scenes, in a way that, you know, is often abstracted from us if you are, you know, working in R and not baking in a lot of CLI or or print statements or or logging or updates, things like that. So this is a really, I think, convenient way with this logRitter package to be able to kinda fill that gap. Yeah. And it's amazing how
the SAS logging paradigm has had such influence, especially in life sciences, that there are packages in R to try and replicate a lot of that functionality. This is certainly scratching a bit of that itch. Even in the PharmaVerse, there are certain packages like LogRX and the like that are trying to make it so that it's easy as an R user to have that log ready to go in case you need to, you know, audit it for more regulatory purposes. But yes, it's good to have because I'm going call it like an in between here because it's so easy to opt into this and it's just one pipe operator different. Everything else in your dplyr code can remain the same. There's no like hidden options you have to specify at the top of your script. This is a really elegant solution and I'm eager to see, you know, maybe what future development is in this great package.
And our next highlight today is very much feeling like we're going back to school, that on a concept that is easy, and I do mean really easy, to take for granted, but has so much going on under the hood that I did not realize, or maybe I just didn't think to look until this great highlight that came our eyes, so to speak, in this week's issue. And the author and the title of this highlight is Generating Universes
Within Universes With a Single Seed. There's a lot to unpack even in that message alone or that tagline and we'll unpack that here as best we can and the author of this post is the esteemed Andrew Heiss, very frequent contributor to our weekly highlights, always an entertaining read whenever he posts out there, and yes he is such a modern stack with what he's building in his educational materials and his online presence and there's some cool modern stuff in this post here. But let's kind of dive into this a bit. Mike and I will tag team on different parts of this post here,
but in essence seeds are a critical part, and this is speaking from my experience as well as a statistician, to ensure reproducibility in whatever random ish kind of process your analyses are employing. And this is especially true as we look at algorithms from say Bayesian methodologies and even just simulation of data itself.
We want to make sure that we can have that one ID that we can assign so that if we run this over and over again in a different R session, we will get that similar result. But and then Now, Andrew, of course, has been using seeds for as long as I have in R and in fact the post starts with an area they might not expect seeds to play in and that's with ggplot2 when you want to tackle
over plotting, in the case with the Palmer Penguin data set, where if you just do the geom point by default when you look at the facets of species on the X and I1 on the Y, you just get basically six or like five dots on the plot, but those dots have a lot going on because they're the same, but if you want to distinguish them, you want to introduce a jitter. So then you get that kind of scattered representation of all the data points instead of seeing just five data points, you now get the full set of data points. But if you run that jitter code again,
the pattern will look different because that jitter is actually a random ish process for how it's jittering all the points. So if you want that pattern to be consistent, you got to set yourself a seed there and that is again how to do this in R, it's a one liner folks, it is just set dot seed and then you give it a number. We'll get to how big that number is in a little bit, but even before that point, the post has an entertaining diversion that
seeds are also used in many different places outside of statistical programming, such as Minecraft, when it generates that world that you are spawned into, unless you tell it to with a seed, it will always be random each time, but you could set a seed and it will use that in its procedural generation of the different worlds out there. And there has been a whole like rabbit hole that many in the Minecraft community have gone down to try and kind of see interesting things that happen with specific seeds. So yes, there is a sub reddit just for that alone.
That would be an interesting time sink on the weekend. But back to seeds themselves. Andrew's had kind of a notion of how he thought this would work, but let's first kind of take a step back here, and what is a seed exactly in the big picture here? Well, the definition that he highlights here is that it's an arbitrary integer that R's internal random number generator or RNG for short,
whatever that algorithm needs to initialize that process. That will be a key we also get back to later on, And you could use, as I said, many, many different numbers for this, even to my 80s or 90s heart, puts in my favorite seed of 8675309. Love it. I will not quit my day job, it's my favorite one and I won't lie, I have used this for actual projects.
Goodness gracious. Okay, anyway, so Jenny, Jenny. Yep, Jenny, Jenny baby. So in his, in the rest of the posts in the next section, he talks about what kind of he's been telling his students based on his mental model of how seeds work and one interesting exercise is that when you use the runif function or r u n I f in r, this is the function that'll let you generate random numbers from the random uniform distribution,
another staple in statistical theory, and of course if you run this with say five to get five numbers repeatedly, they'll be different every time, but then when you set the seed and you run that the first time you will get a set of five numbers, and then he had the other students in his class set that same seed, run that same line, and they all get the same number that first time they run it. It's interesting that if you run that statement again, the run uif of five,
you get a different set of five numbers, and what's interesting about this is that these random number generators are in essence giving you a sequence of these numbers, and you just kind of, in essence, push them or pop them off a stack every time you call this different invocation with that same length. And so what is actually driving that process? It's not hocus pocus, this is based on mathematical
theory, just where does this grand list of numbers come from? So there is a relatively classic algorithm called linear congruential generator, or LCG for short, that actually has a set of key parameters in this sequence, and he has a nice notation of the equation in the blog post here, but some of the key inputs include things like the multiplier,
the increment, and the modulus operator that in essence help you determine what the next number will be based on what was the previous number. And hence, what's interesting is that you need something to start with because it's based on what was done previously, and that what you start with is, you guessed it right, is that seed that you define up front. That kicks off the entire sequence. And what's really cool about this post is that Andrew has a way for you to interactively play
with these different inputs to this LCG algorithm in both a downloadable excel spreadsheet, xosx file, as well as an embedded looks like a javascript widget that you can literally put in the numbers and it will update it on the fly with the sliders you can toggle. I'm doing it right now and you kind of see what the next numbers will be in the different sequence depending on the numbers you decide to generate. You could go anywhere from 12 up to much higher too.
This would be a fun way to play with this a bit. And here is the nugget that he's found in researching this algorithm, is that eventually you're going to reach a point where there is a cycle of numbers and they are repeated. That kind of took me a step back a bit, that these are not all unique numbers in this sequence. There is a time, and this is actually a limitation of the OCG algorithm, where you will have to you know account for
these different periods or cycles based on those inputs that you put into the equation. So hearing this limitation you may be wondering uh-oh are we in trouble here when we do random number generation in R? No, threat not, because R itself does not use the lcg algorithm as its default. It actually uses a more complex technique and then the default for R at the time of this recording is called the Mersenne twister algorithm.
This is, a very interesting one. There are links in the blog post that talk about this in more detail, but in essence you can be assured that it will take a long time for you to encounter this different cycle to repeat itself, and you might say how long will it take? Well, it takes about two to the power of 19,937 minuteus one, which you don't need me to tell you that's a really big number. So until you hit that number,
you're not going to see this repeated sequence of numbers in your random number generator. He also shouts out a great post from Danielle Navarro when she looked at in 2023 a blog post that she authored called cCatcher to try and dive into more of this Mersenne Twister algorithm. It's a cool read as well. Now in and of itself this is a great start right, but there are some interesting things that you can encounter
and in essence there were some hot takes from very esteemed members of the community on this and in fact the esteemed claws wilk has a great take of if your random seed is 42 I will come to your office and set your computer on fire, shades of Jenny Brian there with the workspace, line on the top of our scripts, and that is just kind of a dovetail into what are some of the pitfalls that might happen based on random number generator that you might want to watch out for. So Mike, why don't you take us through kind of what,
overtakes on this, hot topic that we're seeing here? Yeah, I mean I think the core idea that Andrew's trying to present here is that, you know, random isn't really random, because your seed is going to kind of create this fixed predetermined downstream sequence of numbers, which is great, right, for reproducibility, but also can hurt for other applications. You're kind of walking through a prewritten list and not
generating complete randomness on the fly. So, you know, kind of using a a seed is is like choosing a a universe where every random event is already sort of decided. And when obviously you use those common seeds, forty two, one, two, three, four, it means that you and everyone else are kind of exploring just that same, select universe and not trying other universes, if you will. You're just exploring that same sequence of numbers. So it creates sort of artificially limited
variation in in our analysis. There's infinite universes, everyone keeps picking the same one if you're all choosing the same seeds. And, you know, another thing to think about here is the possibility of of seed hacking. So you can search for a seed that gives the result that you want. And I would be lying if I said that in the past I haven't changed my seed or, you know, a few different times to try to make my plot look a little bit nicer, to explain the thing that I'm the point that I'm trying to get across.
I will say that I have never, because I don't think I could sleep over this, you know, just changed my seed to, like, change a coefficient from negative to positive Mhmm. Or something like that. But you could think that that's certainly possible and there is an area in the in the blog post, where I think that there were, Naimi Yu and and Bodnar. I'm not sure what their their first name is, but, you know, the the link in the citation is here. In the blog post, they used sort of several different machine learning techniques, random forests, neural nets,
which all, you know, incorporate some element of randomness. They analyzed the same data, and estimated sort of the same average treatment effect, and the estimated effect varied substantially based upon the seed that they chose. So, you know, that makes that makes it tricky and obviously leads to a lot of potential for seed hacking or, you know, not only seed hacking, but, you know, just only seeing things in your individual universe and not really baking in enough variation
to sort of take a look at all of the different possibilities. And I think when you get into like info sec and things like that beyond data science, that's where you start to have some real world consequences of this because people can exploit, this in ways, you know, that are fraudulent. And Andrew makes sort of the leads into sort of a next section about how you can potentially get around this because, you know, the conclusion here is that computers themselves just can't generate true randomness
on their own. So if you you kind of know the algorithm that's being used and you know that seed, that starting point, you can predict all future values. So pure algorithmic randomness is is kind of deterministic under the hood. And Andrew's argument for creating real randomness is that you need some sort of external, like, physical process or physical entropy in the world. And he has some great examples of this. One of my favorite here is Cloudflare. I don't know if you saw this Yes. Eric, but they're one of the largest, like, content delivery networks,
CDNs on the the Internet, and they handle a ton of, like, encryption related tasks, like SSL encryption, you know, h think HTTPS type of, stuff. And they have this wall in their office of, I don't know, a 100 lava lamps, something like that in their San Francisco office. And they have a camera that looks that just watches these this wall of lava lamps, takes a picture of, you know, the the state of the fluid moving around in these lava lamps, assigns a numeric value to the pixels in that picture,
and then generates, you know, a seed or a truly random, sort of process downstream based upon that physical picture of the lava lamps, which is a wild thing. You know, there's a couple other examples here where, there's a service called PGP. I think it stands for pretty good privacy. Right. It's like a terminal interface where it lets you move your mouse around and kind of type random stuff into your keyboard. And based upon your mouse movements and whatever you typed, it will create a, you know, truly kind of random,
pseudo random seed for you. And there's another example, using atmospheric noise is a company called random.org that has three radios that capture atmospheric noise and then uses the entropy of that noise to generate truly random numbers. And that's something that Andrew shows. He actually uses quite a bit. He'll go to random.org to, first, you know, type some random number into random.org, and then whatever that spits out, he'll use for his seed. So I thought that that was pretty interesting there as well. And, I you know, again, I think sort of the the big philosophical
takeaway here is, you know, every coin flip, every every shuffle or or simulation outcome is is already gonna be predetermined, the moment that you set your seed. So I think it's important to make sure that you're leveraging a lot of variation in the seeds that you choose from project to project. Yeah. It's really interesting to see not just the parts that are happening in our actual programming work, but then how these other industries are tapping into these other processes
to give it that additional random ish start that is easy to miss. So while I won't have the budget to buy all these lava lamps and take pictures like Cloudflare does, that was a really interesting find that Andrew put in here. I am intrigued by that random.org kind of integration. I think that could be really cool in my next project to at least have a script like he has here to get that seed started. And of course, I document that seed for prosperity in my simulations or whatnot, but that
probably will be a nicer thing too, as the post mentions, maybe step outside my comfortable universes of eight, six, seven, five, three, zero, nine and do something a little more robust than that. So there's a lot to learn here. I think it's fascinating for a concept that is, from implementation side, as easy as it gets. It's a one liner, but what goes into that line? Yeah, a lot of things to consider. Not to dissimilar to other parts of the data science process that you mentioned, yeah, when people kind of hack seeds, so to speak, that this post highlights.
I mean it is maybe not quite as egregious as p value hacking that often happens when you run inferences over and over again until you get to that right result that you want for that less than 0.05 level, but it can lead to consequences not too dissimilar to that. So it is worth thinking about in the long term, but yeah, there are definitely mechanisms you can take as a data scientist right now to give that seed a more gentle or more random ish start. So really cool stuff here.
The deepest dive on seeds I've ever seen. Yeah. And, if you didn't feel the need for seeding, well, you sure do now after that. But and then, rounding out our highlights today. They're actually going to be a bit abbreviated with this last one because yours truly did some sleuthing and tried to find the recording of the talk that this mentions, I could not find it. But we'll give a quick shout out here to Nicole O'Renee, another frequent contributor to the highlights,
where she amongst others in the R Ladies remote group were part of a, sounds like a really interesting panel discussion, that was tagged Working Smarter for R Tips, Tricks, and Real World Lessons. So without access to the talk, I can't give you like my Reader's Digest version, so to speak, on the points I highlighted, but what I can say is that the link we have in the show notes has a copious amount of additional links that admittedly I don't have the full
context around, but there are some interesting things you might look at as you're waiting for the talk recording to get out there. Covering the full gamut of R Markdown, Quarto, some of Nicola's recent work on visualization, and yes, the targets package, we're always a soft spot for that amongst others. So definitely additional links within our weekly, that's kind of what we're specialized in, that might give you a starting point to working smarter with R.
And with that we're going to put a bow on this episode of our weekly highlights. We got to get out of here kind of quick so we don't have time for additional finds, but as always we invite you to check out the issue and yeah, read at your leisure for all the great content that's shared on this and in the back catalog as well. And of course this project is built on the community, which will be a very key point that I highlight in my upcoming lightning talk at positconf about our weekly and one of the best ways to contribute is to share that great new resource
that blog post about a package or another great find We're just a poll request away folks. If you're comfortable with markdown and GitHub, you'll be right at home with this. A link is right at the top right corner of the rweekly.org page for you to send that poll request with the template already ready for you to complete. And also you can find us on the social medias causing you know some amount of trouble there or not, but I am also well what are my big three? I should have it memorized by now. I am on blue sky with r podcast
b s k social. I'm also on Mastodon where I'm rpodcastspodcastindex.social and I'm on LinkedIn you can search my name and you'll find me there, certain announcements and other fun things I'm working on. Mike, where can the listeners find you? You can find me on blue sky at mike dash thomas dot b s k y dot social, and a big shout out to John Harmon, who was super excited after our long absence, two weeks ago when we came back. So really appreciate the shout out from John on there and the excitement about us getting back on the mic. Last week was not our fault. There were no highlights since, but we hope to, be a little more consistent going forward. And you can also find me on LinkedIn if you search for Catchbrooke Analytics, k e t c h b r o o k, you can see what I'm up to. Yes.
Mike and I are just working out our scheduling for the future weeks. We think we got a good spot. We're gonna at least more a problem on yours truly side. Protect that calendar folks. Man, sometimes it just gets to be a wild west out there, but if you know, you know. Anyway, we're gonna close-up shop here on this episode of our weekly highlights. We again thank you so much for joining us and we hope to have you listen to next week's episode of our weekly highlights next week.
Our takes on the increasing removal of friction in good and not-so-good ways thanks to AI, a visual tour of hot air balloon accident data, and a case study of novel image recognition with the rplatipus package. Episode...
The flow of both data and analytical jobs just got smoother with mizu, opening the door for multiple agent harnesses in Positron, and an optimal statistical model tailored to complex experimental designs with a friendly...
What happens when that online data set isn't so easy to download anymore? We learn of two packages to ease the pain of obtaining these files and then some. Plus a visit to the visualization corner with a handy way to...
The celebration of rOpenSci's 15-year anniversary continues with the community sharing their perspectives, and a fun Shiny app that could be quite useful during celebrations on a weekend or the holidays. If you ever...
We learn about the variety of ways you can contribute back to the core of R, a roundup of the excellent choices available for package and environment reproducibility, and an amazing collection of Shiny applications and...