On the latest episode of "In the Interim…", Dr. Scott Berry provides an empirical examination of two recent JAMA trials: TRACK (low-dose rivaroxaban in advanced kidney disease) and VICTORY (IV vitamin C in severe burn injury). The TRACK trial lacked any pre-specified futility criteria, with a DSMB-initiated stop based on conditional power calculations. Scott argues that conditional power, especially in this interim context, is a poor, misleading tool - contrasting it against a Bayesian predictive probability calculation that produced a much lower and more realistic estimate of success. In VICTORY, a pre-specified risk ratio threshold for futility was incorporated, with simulation confirming minimal effect on bias and statistical power. Scott underscores the practical and ethical importance of rigorously pre-specified, simulation-based futility rules and operationalizes the case for Bayesian predictive probability as a decision metric in interim monitoring. He reiterates that responsibility for defining futility belongs to trial designers, not left to ad hoc DSMB judgment, and calls for precise statistical planning in adaptive trial protocols. Key Highlights TRACK: No pre-specified futility rule; DSMB stopped for futility using conditional power post hoc. Technical critique of conditional power as misguided at interim, supporting Bayesian predictive probability instead. VICTORY: Pre-specified futility threshold, with simulation confirming minimal operational bias and power reduction. Emphasizes pre-specified, simulation-based futility planning and predictive probability monitoring as standards for all trials. For more, visit us at https://www.berryconsultants.com/
Audio and video play straight from the show's own hosting. PodSubs stores and serves no media.
Transcript
From the show. Tap a time to play from there.
Judith: Welcome to Berry's In the Interim podcast, where we explore the cutting edge of innovative clinical trial design for the pharmaceutical and medical industries, and so much more. Let's dive in. Well, welcome everybody back to "In The Interim." I'm your host, Scott Barry. And for today's topic, I feel a little bit like, uh, years ago, I was on faculty at Texas A&M.
I was assistant professor of statistics at Texas A&M, and, uh, just starting at A&M, I had come from graduate school, came from Carnegie Mellon. And, uh, and I don't know if they still do it, but Texas A&M had a, at the time, a policy that every, um, graduate student, everybody pursuing a dissertation, would get a random faculty member on their dissertation committee
So if you were a statistics graduate student doing a dissertation, you might get a faculty member from, uh, meat sciences or, uh, from something to be on your dissertation committee, a voting member of the dissertation committee. And so when I got there, I was assigned to a student's committee, and it was when I walked in, it was sort of like, "Oh, no."
The, the-- generally, throughout all of the graduate programs, the worst thing that could happen to you was getting a statistician randomly assigned to your dissertation committee. So I think the first one I did was forestry and crop sciences, and they're actually doing experiments. It was, it was fascinating. And the second one I think was, was wildlife and fishery science. It was a fascinating project where they were following wolves, and they were
tracking them, and they were reintroducing wolves into the wild, and they were trying to figure out their behaviors. It was dynamic data. It was absolutely fascinating, but they, they were fearful of a statistician walking in like, "Oh no, I now have a statistician on my committee." Now what, what does that have to do with today's topic? Today's topic, uh, I don't think it's the same sort of scenario because statisticians are rampant in the clinical trial, uh, uh, clinical trial world.
But I am gonna talk about today a look at a recent in, i-issue of JAMA. So a wonderful thing about clinical trial design is you can go see the results, the published results of other trials and see those trials' design, see how they carry out the analyses, the interpretation of those, and of course, JAMA, New England Journal, Lancet, all of these provide this really nice opportunity.
So I make sure whenever possible to open up these trials, to check out the design, see what they say, what's being done in the world of trial design, what, what would I have done differently in them, and do I agree with the interpretation from the medical journal? So I get, I get to jump in a statistician review, and I'm gonna talk about them on this show. So I always-- I, I've done, I've done one other. You can go back to the episode where I, I sort of randomly fell upon a really,
really interesting trial of using, um iron supplements in heart failure where the, the patients are deficient in iron. Fascinating, fascinating trial. So you should go back to that episode. These trials, uh, I'm gonna look at two trials from read-- The email comes in from JAMA and I, I click on them and, you know, these are, these are interesting enough I, I, I wanna talk about these.
Okay, so a little bit is doing this, doing this a bit publicly, and I hope people don't say, "Oh no, a, a statistician is reviewing my trials," uh, in this setting. By the way, an interesting thing, one of the m- more interesting things I got to do when I was at Texas A&M, I was a columnist for Chance Magazine. It's a publication of the ASA. And each quarter I got to write an article that was a statistician
reads the sports pages. So I got to go in and find topics that were coming out, and I did this for 10 years. It was a quarterly magazine, so I wrote 40, uh, articles about reading the sports pages, things that come up in sports, uh, statistician's interpretation of them. So this is a little bit like that. This is a statistician reads JAMA and, and you know, what, what, what is, what, what do I look at when I read these trials?
Okay, probably a bit of a bias in the trials that, that I look at, but this is a re- recent, uh, episode of what showed up in my mail. So the first trial was published June 4th, 2026, and it's called the TRACK Randomized Clinical Trial. It is a trial of, and the title of it is "Low-Dose Rivaroxaban in Cardiovascular Events in Advanced Kidney Disease."
So a little bit about the trial, and I, I am not a clinician, so I, I'm, I'm reading the article of this. So rivaroxaban is a anticoagulant and apparently anticoagulations in some form of kidney disease with aspirin have demonstrated good clinical outcomes. So this is a trial looking at low-dose rivaroxaban in that looking at does it
alter cardiovascular outcomes for patients with advanced chronic kidney disease Now these are, uh, CKD, chronic kidney disease stage four or five, including patients on dialysis, and they're randomized to, uh, a rivaroxaban low dose, uh, or not in the trial, and the primary outcome is a composite of cardiovascular
death, non-fatal MI, stroke, uh, or peripheral artery disease event. And the primary safety outcome is major bleeding, which is typical of trials of anticoagulation. The risk, of course, are bleeding events, uh, in that. So that's the trial, the, the major part of what the trial is trying to address. Now, a little bit about the trial design.
It's an event-driven trial. It, it's a little bit⦠I, I found it a little bit confusing in the trial design. It's an event-driven trial, but it talks about, uh, uh, the trial is designed to enroll nineteen hundred participants over three years with an expected follow-up of five years. So the nineteen hundred patients would provide statistical power to detect a
reduction, so a hazard ratio of point seven five, would have ninety percent power of, to detect a difference. They're gonna do a log-rank test. They're gonna look at the hazard ratio in a Cox model for, um, the, the cardiovascular events, time to first event. Now it does-- And then it says this, the nineteen hundred is a bit, they're gonna enroll that, but they're gonna wait for the events. It's an event-driven trial, five hundred and fifty primary, uh, outcome
events, assuming a log-rank test. Uh, ninety percent power for a point seven five hazard ratio. And it also says the number of events provides eighty percent pow-power for a twenty-two percent risk reduction, point seven eight. It's, we'll come back to that interesting point seven eight has eighty percent power, uh, in the trial. Now, the first thing I, I try to diagnose from the trial was this
adaptive i-in any way in the trial, or is it a trial and we're gonna wait five years and look at the data? Well, it says two formal interim analyses are planned to assess efficacy when approximately one-third, which is pretty early in event-driven trials, and two-thirds of the primary endpoint outcomes have been observed They use Hay-Bittle-Peto stopping rules. It says, uh, using three standard deviations at either
of the first two interims. So that's a, a, a nominal alpha of, of 0.27, um, uh, within that. So the, uh, you know, the typical two point five, this is, this is 0.27, uh, which has it, it describes minimal impact on the nominal type one error rate by the end of the trial. So four point eight two percent is the alpha. So, uh, uh, at the end of the trial, so it pre- preserves a good bit of it.
Now, it says nothing about futility analyses in here. It says a detailed DSMB charter will be developed before starting the study in consultation with DSMB members. That's essentially the extent of it. I looked through the SAP, uh, that which was a, uh, which was an appendix to this, a supplement to it. Looked through the SAP, and this is really the wording it gives there. I think in hindsight, the answer is there were no futility analyses in the trial. So it's a superiority trial for rivaroxaban on cardiovascular events.
Now, what happened in the trial? So it says, and this is in the, in the paper, in the report, on July 8th, 2025, during a pre-specified review of interim data, and it's kind of interesting because the, the-- it says two hundred and fifty-four primary outcome events, which is fifty percent of the planned events. Fourteen hundred and thirty-two patients randomized.
Remember they said nineteen hundred, so fourteen hundred and thirty-two randomized, and it, it's at two hundred and fifty-four events. They said that they're gonna do a superiority at a third of events and two-thirds of events. So it says on July 8th, uh, fifty percent of the events, two hundred and fifty-four. The Data and Safety Monitoring Board recommended early termination of the trial to concerns regarding net harm and a low probability of demonstrating efficacy
Okay, so the DSMB made this recommendation, and then it says the recommendation was not based on pre-specified stopping rules. So from all my read, there were no pre-specified futility, but the DSMB makes this recommendation. Net harm and low probability of demonstrating efficacy. And then it says it gives additional detail. It says, "Rather, the decision was based on a post hoc analysis
estimating a conditional power of 16% for a hazard ratio of 0.78, indicating a low likelihood of demonstrating efficacy even if the trial had continued until the planned 515 primary outcome events." Okay, so a lot to unpack here. So no pre-specified rules.
They do a post-hoc. I think with no pre-specified rules, everything is post-hoc. So I, I, I found that wording somewhat weird, uh, within that setting as though that's super relevant. But⦠And they do conditional power, and they say a low likelihood of demonstrating efficacy. So first of all, what is conditional power? 16% probability of demonstrating s- uh, superiority at the final
analysis, and it says 16%. Now, conditional power is given the current data at the interim at 254 events. It's a calculation of what's the probability of seeing superiority at the end of the trial. Now, conditional power assumes a single value for the hazard ratio, and it can be done in many ways. Sometimes conditional power looks at the current estimate of the hazard ratio.
Sometimes it uses a preset hazard ratio like it did here. So it assumes a hazard ratio of 0.78 for the rest of the trial, uh, with variability of the rest of the data. Suppose the truth is 0.78. What's the probability that at 550-- 15 events we see superiority? That's the calculation that's made, and that's 16%.
My first reaction is that's not that low. That's, that, that, that likelihood, I d- I don't find that low at all. Uh, it was interesting. My, my son, Cooper, who I've talked about on this show before, he's, um, he was in seventh grade, and we h- would not let him play tackle football. And he came to us with an argument that he should be allowed to play tackle football.
And as part of his argument, he found, uh, uh, data that said the likelihood of a neurological event For a kid playing tackle football, a serious injury was only 13%. And he presented that in his PowerPoint presentation to us. Yes, we have a weird family, um, in this which, uh, that was only 13%, so he
should be allowed to play tackle football. And our reaction was, "Oh my God, 13% is enormous." You know, the 13% chance of, of serious injury in that setting is very large. So 16% probability of demonstrating superiority by the end of the trial doesn't strike me as that low, actually, that unlikely. Now, there's multiple parts to this.
It uses this .78 to make this calculation. Now, I went back and figured out if the, uh, you can find the, the formula for conditional power, uh, in this. There's a beautiful normal approximation to the log hazard ratio. If you assume .78 and the conditional power for 16% by the end of the trial, at the time, it
means that the hazard ratio was 1.03. So it was worse than one at the time, and the, the data was doing worse. The 95% credible interval for that 1.03 hazard ratio at the time of 254 events goes down to about .81.
So somebody's calculating this conditional power for what's the chance this trial is successful, and they're using .78 The 95% confidence interval goes down to 0.81 as the lower bound of that. So 0.78 is not even in the 95% confidence interval of when they're making this calculation. It's a very, very unlikely value.
So what is the relevance of this calculation? It's a super odd calculation, and I think it's, it provides no information for the DSMB to make the decision. It's not a low number. You sh- probably shouldn't be stopping trials for futility at 16%. It's not that unusual of a number. But it's calculated using-- Remember when we talked about the way they powered the trial, they said it has 80% power, assuming 0.78.
So halfway through, through the trial, if you still assume that 0.78, it's gone from 80% to 16%. Doesn't seem that super odd at the time. I, I-- That I feel like it's not a very informative number at all. You won't be surprised to find out I'm not really a fan of conditional power. Now, conditional power assumes you know the answer and what's the probability that that, that happens. And it uses a number that's incredibly unlikely given the data we've observed.
And so it's, it's, it's a bad analysis in that. Now, what would I do? I would do a Bayesian predictive probability. I would use the current posterior distribution of given 254 events with a current hazard ratio of 1.03 What is the posterior distribution of the hazard ratio? And then given that estimate of the hazard ratio, what's the probability the trial's going to demonstrate superiority?
Okay, I did that with this data, and it comes out to be about 1.4 So given the current data, the, and the current posterior distribution of the hazard ratio at the interim, integrating over that, the probability of seeing statistical significance at the end of the trial is 1.4%. Now, I think that's a really, really useful number. And by the way, it's a number we use.
It's, it's a calculation we use very frequently in s- for stopping for futility because it incorporates the uncertainty at the interim, the variability of the data to come of demonstrating superiority or the goal of the trial. It's, it's a beautiful number. And you can use, you can use prior distributions for that that are informative. You could use fairly weak priors.
Uh, that, that, uh, analysis prior that goes into that, that's a word that the FDA draft guidance used, the analysis prior that goes in to calculate that could be based on, uh, an optimist's view. And when an optimist thinks it's unlikely, you could stop the trial. Now, if your optimist believes 0.78 with no variability, that's the conditional power. That's equivalent to what they calculated.
You have a prior probability that the hazard ratio is 0.78 with zero variability. You think the chance of winning this trial is 16%. I don't think that Optimus would stop the trial, but I think it's, it, it's not the right calculation at that particular time. Okay, so a lot went into that. Now, the other component to it is that at the time, they saw increased major
bleeding, which was the safety endpoint, which is what anticoagulations do. They n- you know the bleeding's gonna be higher, but you're hoping it's offset by reduction in cardiovascular events. Now, the final data, I don't know what the data were at the interim on bleeding. The final data on bleeding was for the rivaroxaban arm, it was 8.8% major bleed. That was 64 events, uh, on that arm, 44 events on placebo for a
s- a 6% rate of major bleeding. So that went from six to 8.8, which surely went into the DSMB's decision to recommend stopping the trial. The probability of showing superiority, and I think they used an awkward number for that, but with this elevated rate. Now, we know major bleeding's gonna be higher. That's what the drug does. But the perception of this 2.8%
increase with a hazard ratio of 1.3 at the time had them recommend stopping for futility. Now, the final result of the trial was a hazard ratio on cardiovascular events that was 1.09. There was additional follow-up. They, they, um, at this July event, they stopped the trial a month later, so there were deliberations on this, but they stopped it.
There was additional follow-up, probably additional events going through adjudication. With the final events, it ended with a hazard ratio of 1.09 in that, um, 13, uh, uh, events per 100 years on rivaroxaban, 11.8 on placebo. It was doing worse than placebo, uh, at this particular time, and stopped with 1,463 patients randomized, about five- 400 and some plus patients less
than what they planned to enroll in it. Okay. So overall, what does this mean? So I think the DSMB got it right. Um, knowing what I know of the results, uh, I would have recommended stopping. But I think the problem is, I don't think that's the group to make this decision Uh, the thing that bothers me here is there were no prospective futility
rules So now you turn over this trial. It is an incredible amount of effort by a ton of people, 1400 patients agreeing to enter this trial, and there's no rules for stopping the trial for futility. You leave it up to smart people on the DSMB. But those are really hard decisions for a DSMB. I think this is something that the PIs in the trial should define.
They should shet- set up what, what is futility? What is the be- should it be a balance of, of bleeding to, to that? Are they happy that the trial was stopped at this particular time? Took about a month before they, they eventually stopped it. Did they review the data? Did they agree with this? Uh, I feel like it has to be part of the design. It's frustrating as a statistician to read trials like this that don't have futility rules set up. It's a five-year trial with nineteen hundred patients.
Cardiovascular death is part of this endpoint, major bleeding. So I think the DSMB did right, but they're in a really hard spot to make a decision for futility. Is sixteen percent right? Is that conditional power number the right thing in it? Now, this is a trial looking at a readily available treatment, so this is kind of comparative effectiveness, if you will. I don't think this is gonna result-- Maybe it results in some level of,
of guideline saying, "Oh, patients with CKD stage four or five should take low-dose rivaroxaban," if the data were in a particular situation. But it's not regulatory approval. It's not a sponsor going after this. I don't, I don't believe, uh, in the setting. Um, but especially in sponsor-designed trials, the sponsor should make the decision about futility. When is enough is enough? When should we stop the trial?
It's a business decision. It's a money decision. There's an ethical component to it about asking patients to be randomized into a trial with a very low likelihood of demonstrating a, a, a result that changes practice. And, uh, so are they contributing to science if the probability of success is one point four four percent? All these questions should go into the design of the trial, not humans that have to sit around and make this decision post-hoc in it.
The last part about it is I, I think conditional power is just a bad tool for that. It's a bad tool. Integrate over the uncertainty of what you know at the time, reflective that point seven eight was very, very unlikely, so sixteen percent is just the wrong measure. Whether it's big or small, it- it's the wrong measure So I'd love to see them use a much more realistic, relevant calculation in that. Okay, so that was article number one that shows up in my email box.
Uh, article number two is the VICTORY randomized clinical trial. Love the names. And this one caught my attention right off the bat. This is the first one I looked at. It was-- This came out June 10th, published online in JAMA, June 10th, 2026. And this is the VICTORY trial, and it is high-dose intravenous vitamin C, uh, studying mortality and organ dysfunction in severe burn injury.
I don't, I don't know much. We've done a little bit in burn, but I don't, I don't know much about it. Um, uh, the d- the, the syndrome, uh, within it, you can imagine the clinical syndrome of this. Uh, but I don't know much about treatment in this or endpoints. So, but it struck me was the vitamin C part of it. Why did vitamin C strike me? Well, um, in, in our REMAP-CAP trial, we investigated, um, REMAP-CAP and LOVIT.
LOVIT was a trial investigating vitamin C in sepsis at the time, and COVID, the pandemic broke out. The two trials combined their data together to investigate and randomize patients with COVID and investigating does high-dose vitamin C improve outcomes in severe COVID
And our trial came out with-- and the endpoint was organ support-free days, in the back of my mind, I remember this result. It came out, it estimated essentially harm on the ordinal endpoint of mortality and then organ support-free days, it came out with a adjusted odds ratio of .88, less than one is harm, with a ninety-one point four percent probability of harm. It was doing worse on mortality, uh, so on survival was fifty-seven percent in
the vitamin C group and sixty percent on the non-vitamin C, the control. So it did worse. The trial was stopped for futility, uh, uh, in, uh, in that case. So I knew this vitamin C did not do well in COVID. And then I saw that LOVEIT read out in sepsis. And so this, the paper came out, um, uh, came out in two thousand twenty-two reporting the results of vitamin C compared to
placebo in patients with sepsis. And the, um, the primary endpoint in that trial was persistent organ dysfunction, uh, or mortality. So they died or I believe at day twenty-eight they still had persistent organ dysfunction. They couldn't get off organ support, a bad outcome. It was a dichotomous outcome of that.
And in the vitamin C group, the proportion of patients that met that condition, died or organ dysfunction was forty-four point five percent. In placebo, it was thirty-eight point five percent. So a six percent increase in that primary outcome in sepsis, uh, statistically significant, uh, harm in that trial. So it sort of-- I, I remembered vitamin C was not doing well, uh, in, in those two trials.
So here vitamin C is being given to patients with severe burn injury So I said, "Oh, I've, I've got to look at, see this. Does vitamin C work here?" So the trial design, one-to-one randomized high-dose vitamin C given by IV versus placebo. The primary outcome was a composite of 28-day mortality, so clearly severe burn, and persistent organ dysfunction, uh, defined as dependence on mechanical
ventilation, kidney replacement, or needing vasopressors inotrope. So, uh, still, uh, needing, you know, low blood pressure vasopressors. Uh, you'd have that support at 28 days. So this, uh, this composite outcome of bad things at day 28 is the primary endpoint, similar to the LOVIT trial actually, for the Burn trial. So the total sample size was planned three-333 patients per group.
So 666-patient trial, which would provide 78% power, uh, for demonstrating a decrease in that outcome at 27% for placebo to 18%. So that, that reduction, it would be 78% powered. So still relatively rare, 27% on placebo was thought to be the rate of this outcome.
My first thought as a statistician, I'm just jumping in, is when I read this is, boy, there's got to be a better endpoint here. A dichotomous outcome of these bad events, but there's got to be more to this to, to look at this outcome where you learn something from the 73% of patients that aren't dead or aren't in this really bad state. Now, death is worse than being on vasopressors on day 28 and, and likewise. So I don't like the endpoint.
I'd like to see something better. Now, that isn't so much the point of this, but as a statistician reads through this, I cringe a little bit at that. Now, it says, while allowing for two interim analyses for futility/safety The, and it says, "The operating characteristics of the trial under the pre-specified futility rule." So they have a pre-specified futility rule here in this vitamin C may
be triggered by the poor results of these other trials, the harm, the reasonable likelihood of harm. In one case, 92%, the other one was statistically significantly harmful. So the pre-specified futility rule is an adjusted risk ratio of greater than 1.1. So less than one is good, favors vitamin C. So at these two pre-specified interims, if they saw a relative risk above 1.1
favoring control, it-- they would s- they would recommend futility. That was the pre-specified rule, and these w- these interims were to take place at one-third and two-third of the events that if it was above 1.1, they would stop for futility. And it says it was determined through simulation. So in the SAP, they actually go through and they look at the operating characteristics of this futility rule, and they talk about, uh,
it reduces power from 79 to 78%. Futility rules reduce power because every once in a while, the trial w- hits those futility rules, depending on what it is, and that would've gone on to be successful. Depending on that rule, you can see reduction in power. So they, they report what it is, and it says minimal effect on bias, which, um, uh, futility rules, uh, any stopping rule has bias. Bias is not bad, and in this case, this bias is not bad.
So they say it's minimal, so that's good. But they investigated, they created a pre-specified futility rule in this trial as opposed to the track trial where there were no, uh, futility rules. Now, it doesn't mention anything about efficacy, so it looks like they did two futility rules and, and not for efficacy. Fantastic, uh, in the trial. Now, I might throw out a little bit of, you know, superiority at the time, but, uh, maybe they wanted enough data to demonstrate and, and,
and change practice in that Okay. So two interim analyses were pre-planned at 222 patients, 444 patients of the 666 total sample size, uh, in it. Uh, by the way, I wonder, again, statistician reading this, they report the sample size in the trial as 333 per group, and then later they talk about the interims happening at 222 total and 444 total.
Did they not wanna write the sample size of 666? I throw that out there as a possibility. So they re- said it was 333 per group. Um, is 1.1, and so what happened in the trial? This is where they report that. This threshold was crossed at the first interim analysis, prompting the DSMB to recommend termination of the trial. By the way, I think that's a, a very reasonable thing to do, that you have
pre-specified rules, but you have a human group that's looking at the data and they say, "Yes, we think it's appropriate." But you've predefine what that is in the setting So predetermined rules hit the trigger. DSMB said, "Yes, you should stop the trial," and the trial was stopped in this setting. Okay, what were the data? The data were on this endpoint were, it was at 120 patients
randomized on vitamin C, 41%. Remember they talked about making 27% on control go to 18. There were⦠I, I, I'll do placebo first. 30% of placebo patients met this endpoint, very close to the 27 that they said, which was a, a pretty good design estimate of that. It was 41% on vitamin C, an increase from 30 to 41%.
The Fisher exact test p-value at the time was .08, of course, on the way to harm, um, uh, in that setting. Uh, I don't know why you give two-sided p-values in a superiority trial, but we, we, we, we can interpret that, uh, Fisher, Fisher exact p-value, of course, going the wrong way in the setting. The mortality rate on placebo was 8%, so 9 of 118.
It was 15%, 18 out of 120 on⦠I'm sorry, I, I may have said that wrong. On placebo it was 8%, nine of 118. On vitamin C it was 15%, so it went from 8 to 15%, almost a doubling of the mortality rate, uh, within the setting. And it hit the, the relative risk, the final relative risk was 1.38, which hit the 1.1
and the recommendation, recommendation to stop, uh, for futility Okay. So overall within this, uh, and by the way, the, the result, I, I, I love reading the conclusion. So I imagine, uh, clinicians don't have time to read all these, these results, but they wanna get the highlights of this. And that's why I think this little cartoon they do in JAMA, and I think New England and other journals do that, is a summary of the trial results.
And many times I disagree with the conclusion. I strikingly disagree with it. I think it's, it's bad. But here it says the conclusion is high-dose intravenous vitamin C did not reduce mortality or organ dysfunction. So again, that's, it, you know, didn't reach statistical significance and may be associated with harm in patients with severe burn injury.
I think that's pretty reasonable in this setting. It wasn't statistically significantly harmful. You know, this P value, if we, if we do a Bayesian interpretation of the P value, there's a ninety-six percent chance of harm. So I, I would've been upset had they just said, "Did not reduce mortality of an or- or organ dysfunction." I think there's reasonable evidence that it is harmful. And of course, I just in my results with vitamin C, vitamin C seems to be
harmful for people with severe disease. This is a third result now where it does worse and seems to be harmful. Uh, so it says may be associated with harm. I thought very reasonable conclusion. Okay. So I liked this. I liked this, that they put this futility rule in there. By the way, I think it could have stopped earlier than that, um, uh, in the trial. I think had they seen it even earlier, they might have stopped for futility, but they had it set up.
Now, I don't li-- a- and by the way, it stopped at thirty-three percent of the sample size. So it stopped at roughly two hundred and forty-ish patients rather than six hundred and sixty-six patients and likely doing harm to those patients, certainly not contributing to, uh, uh, scientific results that are gonna change care, uh, in this setting of that vitamin C is a good thing to add to the, the, the, the
way to care for patients with severe burn. I don't love the rule of relative risk greater than one point one. At a third of the way through the trial, one point one is a very different predictive probability than two-thirds of the way of trial, a one point one And now this isn't where the, you know the number of events. Uh, this could have been an event-driven trial, but it's rarely done in sort
of a binary yes, no at day 28 kind of thing, so they do number of patients. So had the events gone up and down, one point one is a very different statistical conclusion as to the likelihood of success of the trial if the event on control would have been 15% or 40%. It means very, very different things a third of the way through. Now, they simulated the rule and presumably under a number of different scenarios, different control rates, but I'd much prefer
predictive probability in that. I think I'd even prefer conditional power, but using the MLE, I don't love it, and there's better ways to do it, but then just an absolute value. But when you do sufficient, um, simulations of that and you, you time it and carry out the analyses at the appropriate time, you know that, that's a, that's a reasonable summary of the data. Now here, you know, there could have been less events, and it's a different kind of thing.
So I don't love the rule, but I love the fact that they did futility, was pre-specified, they carried out the futility analyses. Kind of a no-brainer that this trial, very low likelihood of success and high likelihood of harm. Uh, and by, by the way, should have calculated the Bayesian predictive probability at the time. Uh, much less than 1%. No chance of success in this trial, uh, on it. So I, I commend the designers on it. I would have done it differently.
Lots of flavors to that. So two papers in JAMA. Interestingly, you know, a lot of people talk about these medical journals as, you know, they only publish positive results, publication bias in the setting. Here are two papers published where futility was the result of both, both trials, and they got published in JAMA. I think really important results I, I suspect. I don't know whether vitamin C is commonly done in burns, um, uh, in that,
but I imagine anticoagulation is, is used and not that uncommon in patients with chronic kidney disease in that. So very, very important results in it All right. Uh, I, I leave you with when you're designing trials, design futility. It, it-- really, it's a really important part of the, the trial design. Uh, don't leave it to the DSMB to make hard decisions, uh, in this.
Help make those decisions ahead of time. I appreciate you joining me here. I'll look at several papers. Again, we can go back to the design and think about them from the results. But I appreciate you joining me here and, and joining again. And until next time, we will be here in the interim
In this episode of "In the Interim…", Dr. Kert Viele, Director of Research at Berry Consultants, takes up a requested topic: published results showing that Bayesian borrowing of historical control data cannot increase...
In this mailbag episode of "In the Interim…", Dr. Scott Berry answers two listener questions and weighs in on the pace of innovation in clinical trial science. Asked what infrastructure a learning healthcare system...
In this episode of "In the Interim…", Dr. Scott Berry and Dr. Kert Viele, Director of Research at Berry Consultants, review the European Medicines Agency's draft reflection paper on methodological aspects in...
In this episode of "In the Interim…", Scott Berry is joined by Jenny Devenport and Kaspar Rufibach of Merck Group, co-authors of the EFSPI Statistical Methodology Leaders paper "Statistical Methodology Groups in the...
In this episode of "In the Interim…", Dr. Scott Berry shares the framework he presented at a CSRC regulatory science hackathon on digital twins and synthetic controls in cardiovascular clinical trials. He separates...