Skip to content
PodSubs
ShowsPeopleGuestsJobsNewsEvents
More
BlogPodCred
Menu
ShowsPeopleGuestsJobsNewsEventsBlogPodCredNetworksSign in
Join nowSign in
  • Shows
  • People
  • Jobs
  • News
  • More
    • Guests
    • Events
    • Blog
    • PodCred
    • Networks
    Join nowSign in

In the Interim...

In the Interim... artwork

Fairness in Soccer and Clinical Trials

Jul 13, 202635 minAudio

In this episode of "In the Interim...", Dr. Scott Berry investigates the practical meaning of fairness by connecting a controversial World Cup soccer ruling to foundational questions in clinical trial statistics. Scott scrutinizes FIFA’s unusual reversal of a red card suspension for US striker Folarin Balogun, referencing reports of US presidential influence, and draws explicit parallels between the enforcement of rules in international sport and the necessity for rigorously defined procedures in science. He references how systems thrive, or fail, on clear, consistently applied standards. Using Sherlock Holmes’ “Silver Blaze” and Abraham Wald’s WWII aircraft analysis, Scott revisits core statistical ideas about inference and missing data, survivorship bias, and the difference between prespecified versus post-hoc analyses. This episode affirms that adaptive and Bayesian approaches, when built on sound pre-specification and methodological discipline, represent scientific progress, offering a measured perspective on how standards and expectations of fairness continue to evolve. Key Highlights: FIFA’s red card reversal, reports of external influence, and ramifications for procedural legitimacy Analogies from soccer, golf, baseball, and wrestling on the societal role of rules and enforcement Classic statistics lessons on missing data, inference, and survivorship bias Discussion of post-hoc versus prespecified analysis and its implications in trial integrity Defense of adaptive and Bayesian methodology as scientifically valid through pre-specification and covariate adjustment Reflection on the ongoing evolution of fairness and rigor in sport and science For more, visit us at https://www.berryconsultants.com/

Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.

Transcript

From the show. Tap a time to play from there.

Judith: Welcome to Berry's In the Interim podcast, where we explore the cutting edge of innovative clinical trial design for the pharmaceutical and medical industries, and so much more. Let's dive in. Well, welcome back everyone to In The Interim. I'm your host, Scott Berry. And today I'm going to touch on multiple topics. Those of you who join into this, uh, podcast periodically know we like to delve into the world of sports and how that's related to science and clinical

trials and statistics and learning. So we're gonna, we're gonna dive back and forth from sports to clinical trials here, and, uh, with it. And, and part of, uh, the, the wonderful thing about this podcast is we've been doing this podcast, uh, weekly, only missed a couple weeks within a couple years now. And I always wonder, "Okay, what's-- what, what, what topics are coming next, uh, in the world of science and statistics?" And, you know, the, the world brings topics.

Uh, i- it's kind of amazing. And so I was, I was very struck by this particular story, and I thought it was related to who I am as a scientist and a statistician, and, um, see if you find it compelling. And the, the title of this is Fairness in Soccer and Science. So I am not… Uh, and I'll say soccer in a sense, um, uh, within this. So, uh, so let me get to the story of this.

Uh, those of you who can sort of see where, where I'm, I'm currently located, I, I spend summers in Minnesota, and beautiful woods, uh, surrounding me. You can sort of see the trees there. Gorgeous nature. Um, part of some of this gorgeous nature is, is, is a negative. Uh, flying pests, insects, ticks. Uh, uh, northern Minnesota is, is famous for its mosquitoes.

Uh, i- it's, uh, for, for those of you who have not been here, um, in the, in the summertime, it can be, it can be really bad. At dusk, uh, in the evening, it's, it's unbearable actually. Uh, horseflies. And so here we have in, in our cabin here, uh, outside of our laundry room in the window, there is a gorgeous, uh, spiderweb and, uh, a

gorgeous spider that's right outside the window and, and it lives there. And large spider. And by the way, they're, they're incredible. Um, they're incredible. Um, um, I, I, I should be careful here. I, I, I don't think they're an insect, but, but they're incredible beings and to watch them. So I enjoy watching them. And, um, I, I- I can help the spider by turning on the light in the laundry room.

So, uh, i- in the evening here, if you flip on the light, of course, that attracts many insects to the window and creates a, a huge benefit to that spider, uh, in terms of the flying things. I, I actually concern there are too many, uh, at times, uh, given Minnesota. So is it cheating that, that I turn the light on? And you know what, uh, now the fact that I don't like the flying

insects, um, within that, but in… A- and maybe that's me as a statistician that, uh, I turn that on, and is that sort of cheating? Um, there's a spider elsewhere, um, that doesn't have the light go on sort of thing in, in, in all of that. At least the thought crosses my mind. Yes, I, I do turn the light on, and no, I don't have any issues with, with, um, um, helping that spider, uh, do its job, uh, within it.

I think it's a productive thing around there. Now, it, it makes no difference to us as humans to go outside. We don't notice anything. But, but I feel like spiders are good things in that. But that, that thought goes through my mind, um, uh, as to, I, I don't know, almost fairness, uh, within that. So what's the story that came up here, uh, triggering the podcast topic? Um, I have been watching the World Cup. We are, we are in the throes of the World Cup.

I am not a, a, a football, I- I'll say it that way, the, the way the rest of the world says it. The US calls it soccer. Uh, I'm not a big football fan. I, I enjoy watching it, but I don't follow it tremendously. My kids played it, and I, I kind of understand the rules. As statisticians, we, we love rules. A- and I love rules, and I understand it. I'm trying to understand strategy and all that. And I was actually gonna do a podcast on part of, uh, part of this that

to a statistician and, and rules that I don't like about soccer that I think could be improved. And now people, "Well, this is tradition," and, you know, sort of thing. And an interesting thing is if, if you get a penalty inside the, the large goal box, it's a penalty kick, which is an enormous penalty. I don't know, 85% chance of a goal or something like that. That can happen well away from the goal, uh, right on the very corner of the

box, and if you're one foot away from that, it's a very different penalty. It's a much less severe penalty. There's dichotomization in soccer that I think is, is not on the continuum, and I don't think is… I, I don't think it's a good rule. But that's not the topic of today. Sort of related to, to maybe what's fair and all of that, and I, I think they don't call the same penalties. One foot difference, they would call it a penalty, but not because of the severity of it. So it's, it's strange to watch, to, to watch that.

So I think that could be improved But what is it? It, it is, it is the recent decision of FIFA, the governing body of the World Cup, to suspend the red card of Folarin Balogun, Balogun. I, I'm probably saying his wr- his name wrong. He's, uh, the leading scorer for the US, a striker for the US, and in the earlier game he got a, um, uh, a red card.

During the game he, he had to leave the game and they had to play actually one, one player short. And for about 25 minutes at the end, they actually scored while they were playing short and, uh, they, they gave up a goal but won the game. That player is then suspended for the next game. Now the US gets to have their full contingent of players, so they get to play all 11 players at this next one, but, but the rules are if you get a red card you're suspended the next game.

So FIFA announced that they're, they are, uh, changing that and they're allowing him to play in the next game against Belgium, and this, this, this podcast is, is going to come out after that game, um, in terms of it, but it's being recorded before that game. It's actually the morning of that game within it. Um, and it was announced the day before that he was no longer suspended, and it was, it was reported that there was contact with, um, United States President

Trump asking them to look into this and switched it, and the FIFA changed this. It's the first time in over 50 years that this has ever happened. It's never happened in the World Cup. So what, what… Yeah, it feels corrupt, and it feels corrupt from the standpoint that had that been a player on Belgium that would have happened, this would not have been changed. That player would not, um, be, be, the, the resi- They suspended the

decision, which is awkward because they suspended the suspension, but, uh, is, is the wording they gave. That feels like that wouldn't have happened. It feels like this is related to the relationship of Trump to the FIFA president. The, the, the, the… This is influence, uh, from that. Now, I, I understand the red card call was controversial within that, but it was the call and it was what happened, and it was a video review that created

that, uh, in it, and I think people would argue, and I think generally people think it was somewhat of a bad call. But it was the call, and to remove it feels unfair. Yeah, a little bit corrupt. It feels that this is not right. And it's interesting, much of the- many of the reactions are almost feeling sorry for the US now. They, they, they're in a no-win situation. Should he score a goal in this game, um, you know, it, it- we will

always, the, the, the US soccer team will always be felt like They had an unfair advantage. Uh, you can call it cheating, call it what you will, that, that, that scarlet letter will be there no matter what. Um, it, it, it bothers me in my, my watching of sports, and this is where this podcast goes, is this issue of fairness and, and me as a, a statistician and, and what it means.

So for example, I won't be watching the game. There, there's something about that is, is just so, so negative to me that this happens, that it feels like this is now not a fair competition within it. And I, I don't wanna root against the US, uh, in it, uh, but it… there's just so much of it that I'd, I'd rather not watch it. I don't watch WWE.

Um, this is wrestling. Um, all… I, you know, I… as a kid, I called it all-star wrestling. I, I don't even know where, where this is. It's scripted, uh, wrestling, and it's entertainment, and I… you know, people can enjoy that. It's fantastic. They're phenomenal athletes, by the way. Um, stunning athletes, but it is a performance. It's, it's Broadway. It's, it's scripted. I d- I, I just don't enjoy that. I, I don't care who wins, you know, and all that, and that's not really the point of it. It's, it's a scripted performance.

It's not, it's not sort of sport, and at the center of sport has to be this level of fairness. And it's interesting because there was an earlier game that I was watching, and I, I've been watching quite a bit of it. I, uh, it's been really, really entertaining. It's been incredibly good sport. I, uh, I've, I've gra- gained an appreciation for soccer. I'm not gonna call it football because that puts me at some level that I, I, I, I'm now a, a, a knowledgeable person on that, and I, I'm, I'm not.

I don't, I don't wanna be a pretender in that. But I've r- I've thoroughly enjoyed it to this point. There was a game I was watching and, and England was playing Croatia, and their superstar, uh, Harry Kane, had a penalty kick and, um, missed it. Uh, the goalie saved it. They did a video review of it, and the goalie left his spot too early, and that violation, and so on video review, they gave him a second kick and he made it.

And so he s- he scored And to me, that was completely fair. There's a rule, and, uh, as long as the rule is reinforced, and if it's reinforced by video, it, it got it right. And, you know, you could, you could argue that that seems, that seems detailed or something, but it's the rule. And I, I… And I think that's related to who I am as a statistician, that there are rules in the, uh, with- within that, and that's completely fine.

Um, and I thought I had no problem with that. I don't feel that that's unfair. There's a set of rules, and you follow the rules. And that's why this bothers me, that he's, the, the striker is not suspended because there are rules and you follow the rules, and we are not following the rules This is related to, to golf. If you watch a golf tournament, you'll see many times a golfer is in a, a, a sort of bad position, but they're standing on a sprinkler head, or their ball is

in a particular position, there's a camera in the way, and they are legally allowed to move their ball, and they generally end up in a better position. It's not cheating, it is the rules, and it's, it's important for every golfer to completely understand the rules, and there are officials there that make sure everybody follows the rules. Golf is incredibly, um, um, uh, rule-conscious in fairness. Cheating in golf, you're essentially done as a golfer.

I mean, you can't cheat in golf, and they go out of their way to make sure to protect the field, and everybody, your playing, uh, partner should be reinforcing the rules, and it's, it's a really important part of golf is, is sort of fairness, so that Scottie Scheffler, the number one ranked player in the world, doesn't get an advantage over the 500th best player. They play under the same rules. I'll watch that. I enjoy that. That's sport, and, and that's fantastic.

Um, American baseball now has video review of pitches. There's a strike zone, and you can, you can challenge that. I, I think it's great. Getting it right, having, having the rules, I think it's actually made the umpires, who are, who are fantastic at what they do, and they generally get it right, um, has made them, have made them better. But all of this has, uh, has been a very positive thing because everybody plays under the same rules within that.

There's a fairness to it which is critical in sports. Okay. So what, what, what does that mean? I've gone on for 15 minutes about World Cup soccer and my reaction to it. Um, I, I, I think as a statistician now, I don't know whether it's that I became a statistician because, uh, I, I saw the rules and, and, and, and part of it, I almost think it's, it's science. Or being a scientist and working in clinical trials make me understand and

appreciate the, the importance of it to Sports and clinical tri-- uh, to, to clinical trials that then relate back to sports, and I have such an incredibly negative reaction to, um, removing that suspension, um, uh, in it, and the lack of fairness to it. We as statisticians in clinical trials, fairness are everything in clinical trials, and it is, it is science.

We are, we are learning about particular therapies. Are they safe and effective? Is A better than B? There's a lot of parallels between sports and clinical trials. If there is undue cheating or unfairness in a clinical trial, it means we don't know the answer. Is A better than B? Is, is, is, is this drug safe and effective for a regulator to make a decision?

Um, and let's leave out of it perhaps influence in, in those decisions, which we all as clinical trialists, uh, you know, abhor, uh, within that, um, uh, within it. And so there's such a parallel in what we do in clinical trials to this question of fairness, and part of why I love it is you can't fake science, uh, within that. Let's get out of the aspect that you can fake data, and there's a great deal

of, of things done in clinical trials to make sure the data is honest data. It's reported as collected. So you could fake it by presenting data that's not right. Everybody on the treatment lived. Everybody on the placebo died. The drug's a, a miracle. But, but short of that, you can't fake it. The data and the results of it, the, the, the, the statistical analysis tied to it, demonstration, demonstrating the likelihood of benefit

of that treatment, uh, within it. There's this, uh, there, there's this incredible structure and importance to things we do in clinical trials to create this fairness, to answer these questions. Maybe that's why this sort of reacted-- I reacted so negatively, um, to that. It's, it's the, the, the same reaction that I had to the whole aspect of the acetaminophen and being the cause of autism.

It's because it was not right. I'm just, just… Blatantly, it was a violation of everything about science that we do. It was wanting it to be right rather than it being right within the rules of science inferences that we made, uh, uh, within it. It was not supported by the data within it. Uh, vaccines causing autism are not supported by the data.

There's a want to it. It is-- It's, it's corruption within it, and there's such a reaction. We should all react to that as clinical trialists because it lowers everything about other conclusions that we draw within that setting. And there is this battle, uh, within it, um, that, that we continually go on to, to be able to present this as fair, to learn the truth about the therapies

out there that we're looking at. Now We, we, the, the, this adjudication of a clinical trial, um, within it is A better than B. We run these contests, and there is this sort of analogy to me of sports, and maybe it's why I, I, I love what I do. I love sports. I love clinical trials. I, I love seeing the results of these trials.

I love seeing the inferences on these trials. What, what conclusions are we making? Are we understanding sort of what, what's fair and how to make these conclusions? A little bit I'll go back, when I'm asked to present to non-science people, students, a- anybody about statistics, I generally don't go in and present quantitative things. My draw… Two, two exam- two examples I go back to that I think are just so,

so awesome examples, and they're very, very simple examples. One is Sherlock Holmes, and it's-- He, he, uh, he wr- Arthur Conan Doyle wrote about Sherlock Holmes. It's a sor- story, Silver Blaze, 1892 short story, where there's a disappearance of a valuable racehorse within it. And, um, the Scotland Yard detective asks, "Is there any other point

to wish to, to which you would wish to draw my attention?" And Sherlock Holmes says, "To the curious incident of the dog in the nighttime." And the Scotland Yard detective says, "The dog did nothing in the nighttime." And Sherlock Holmes says, "That was the curious incident." And it was the lack of barking by a dog that's out there that

would bark if somebody was there. That w- the lack of information was the incredible information there, and that's something as statisticians that it's so important in what we do. It's not just the data, it's how did we get the data? What data did we not see? How, uh, you know, was this prospectively defined? Was this one of 1,000 things done and presented to us?

That has very different meaning. The data are the same, very different meaning to us making inferences about something than if it was the primary analysis in the trial. These are so critically important within it, but it can be presented differently to us. And so that example of making inferences from data, I, I feel like is what is so attractive to me about science, about what I do as a statistician.

The other example is the… And this is also a famous example. Um, by the way, I don't know if this example, you know what I mean, the whole sense of this is whether this is a, a tall tale of, of it, but it's such a brilliant example, uh, I'll assume it- it's true. It's a story of Abraham Wald making, um, looking to reinforce fighter j- fighter planes in World War II, and the data being collected were the

location of bullet holes in planes when they went out and during a battle. Within it, you, they, the location, you, they got data on where the holes were on the plane And the, the fascinating thing was his conclusion was not to reinforce where the bullet holes were, but to reinforce where they're not. Why is that? Because of the survivorship bias that you only see the planes that

survive and come back, and they have bullet holes in certain locations. What you don't see are the planes that go down. You'd love to see… The best data would be, let me see where the location is of the bullet holes that took planes down. That's where you wanna reinforce. You don't see that data. You see planes that survive. There's a survivorship bias. And so it's almost as though where those bullet holes are didn't take the plane down, but assuming a uniformity of bullet holes over the plane, and

there, there's an assumption there, it's where they're not, uh, within it. Such a cool example of inference from data and science, uh, wi- within that scenario. And I believe these are the kinds of things in clinical trials. By the way, this survivorship shows up as incredibly important thing in making me- in medical decision-making within that. And I think as scientists in clinical trial, not, not… I, I mean, this is, this is also, it's not just statisticians.

We, we… This is something we very much deal with, but clinical trial scientists, clinicians, everybody involved in the enterprise of clinical trials and medical decision-making, th- this is such an important part, uh, is making inferences from the data Within that. So that's-- It's the joy of, of, for me, clinical trials and science and the importance of it. And there are a lot of parts of, of clinical trials that, um, i-in some

sense, people maybe think are un- uh, uh, you know, not appropriate. Uh, many times, uh, running clinical trials, sponsors will look at subsets, look at other endpoints, and see data that looks really good and believe the result of that, not understanding the process of looking at many things is a real, uh, a part about drawing those conclusions.

You're very likely to find something positive which lowers the inferential strength of that. Doesn't mean it's wrong, but it lowers the inferential strength of that, uh, in that. So these post-hoc analyses, i-it's, it's part of the science of these conclusion. Survivorship bias absolutely shows up in clinical trials. If you're comparing to a natural history s- database and you have a clinical trial where people enter the clinical trial, there are people

in the natural history study that wouldn't get in the clinical trial. Maybe they die before they would get the drug. And if somebody has the disease and has a period of time before they are randomized in the trial, and now they take the drug, it creates a survivorship bias for people that took the drug that somebody in the natural history that had the disease that died before they'd get the drug would be in the natural history study, but

they wouldn't be in the clinical trial. And then the clinical trial, uh, uh, people, uh, patients do better in comparison to the natural history study. Such a clear absolute survivorship bias. It shows up in many times about making inferences about data. It's about fairness i-i-in, in all of this, and it's-- We, we are the officers of, of this fairness. Now interestingly, uh, I, I and, and, uh, Berry Consultants many times are involved

in non-standard clinical trial stuff. We enjoy being involved in hard clinical trials. In the most sort of standard clinical trial that you enroll two arms to a fixed sample size, and you do an analysis, no interim analysis. In some sense, the, the, the, the cleanest data you can have. Um, and it's awesome.

The, the, these trials are awesome, but we're typically involved in adaptive designs, Bayesian analyses. The pod-- The last episode of, of the-- In the Interim talked about bias in the estimate of the treatment effect when you have futility stopping or superiority stopping and, and, and the biased estimate of that. And by the way, it's, it's very different. It doesn't mean it's wrong. It has-- This has influence, and when we get the result,

we understand the influence of that, uh, completely within that. Borrowing from subgroups. If you borrow from an adult to a pediatric, uh, patient, it creates bias in that. Um, if you borrow from group A to group B, you use controls from a different era, uh, in a platform trial within that. Is there potential bias in that? Now, all of these things, you know, maybe it feels strange that I started this off

with this very high level, um, that this feels unfair that this player is allowed to play in the game W-within the setting, but yet now talking about adaptive designs and Bayesian and all that. It, it's, it, it's incredibly, um, important that we can't keep running those trials. We miss shots on goal. There are drugs that are not explored.

We have to be efficient in patient and time and resources. Uh, otherwise, we get three really good answers, and we didn't ask 100 questions within that. There's an efficiency to this. Now, this can be, and it is science. Bayesian is science. Adaptive designs is science. Pre-specification in these settings is so incredibly important. Covariate adjustment is science, and it's incredibly important in that.

It's not cheating to adjust for covariates within the setting. You might look at the data as the raw data of A and B, but a covariate adjustment gives you a different answer. Is that cheating? No. We understand that as science. We can have very clean science. Let me give you another example. Suppose you were collecting data, and you were comparing to natural history. We understand, and I just presented how there's a survivorship bias in comparing

those natural history patients to patients in a clinical trial setting. Does that mean we can't use it, or does that mean we could potentially adjust and estimate the survivorship bias? That can be done in very clean scientific ways that are honest and, and is science, and that's where w-we, we kind of live on this, this border and,

and, and the hard part is that's not cheating, uh, uh, within the setting. And so, um, adjusting for it, understanding the bias and early stopping and understand making conclusions from that is still incredibly important, and it's an incredibly positive thing for clinical trial science, and that's kind of the s- the space we live in, and I think the importance of us sharing these ideas

and presenting this, uh, uh, within it Now, it's, it's the evolution of the standard trial that's 300 patients, five years, look at the data, make a conclusion within that. This can all be done and is fair. It's a little bit like the advances of adding video review to it, of, of new rules within the setting. It's different than it used to be, but it can all be done in a, in a fair way.

There could be an avenue where a red card is reversed as long as it goes in the rules of how this is done. And within this particular setting, it, it went outside the boundaries. There isn't even really a setting for that, uh, within it, which hence that feels unfair. So we all as statisticians, clinicians, clinical trial scientists, regulators, sites, uh, all of this, we're all officers of science, uh,

in this, and science is evolving. We are making this machinery better and better and better, and it, it improves the human condition to understand the truth about different treatments, the, the efficacy and safety of it. It's, it's all such an important thing, and maybe that's why I reacted to that particular case in, in such a way. Or maybe it's because, uh, I became a statistician because I reacted to that. Uh, I, I, I don't know which one of those it is, but I, I did

have such a reaction to that. So everybody out there, keep up the incredible important work you do, um, uh, within this. And until the next time, we will be here in the interim

Open the transcript file

Hosts and Guests

Hosts

  • Scott Berry Host Feed-stated

Credit the people on this episode

Be the first to rate this episode

More episodes

All episodes
  1. Can Borrowing Ever Increase Power?

    Nov 16, 202625 minAudio

    In this episode of "In the Interim…", Dr. Kert Viele, Director of Research at Berry Consultants, takes up a requested topic: published results showing that Bayesian borrowing of historical control data cannot increase...

    Credit the people on this episode

  2. Mailbag: Learning Healthcare, Multiplicities, and the Pace of Innovation

    Nov 9, 202647 minAudio

    In this mailbag episode of "In the Interim…", Dr. Scott Berry answers two listener questions and weighs in on the pace of innovation in clinical trial science. Asked what infrastructure a learning healthcare system...

    Credit the people on this episode

  3. Reflecting on EMA Reflections on Platform Trials

    Nov 2, 202644 minAudio

    In this episode of "In the Interim…", Dr. Scott Berry and Dr. Kert Viele, Director of Research at Berry Consultants, review the European Medicines Agency's draft reflection paper on methodological aspects in...

    Credit the people on this episode

  4. Statistical Methodology Groups in the Pharmaceutical Industry

    Oct 19, 202656 minAudio

    In this episode of "In the Interim…", Scott Berry is joined by Jenny Devenport and Kaspar Rufibach of Merck Group, co-authors of the EFSPI Statistical Methodology Leaders paper "Statistical Methodology Groups in the...

    Credit the people on this episode

  5. A Discussion on Virtual Controls

    Oct 12, 202647 minAudio

    In this episode of "In the Interim…", Dr. Scott Berry shares the framework he presented at a CSRC regulatory science hackathon on digital twins and synthetic controls in cardiovascular clinical trials. He separates...

    Credit the people on this episode