In clinical trials, you can have it both ways

 

"Were you in the vax group or the placebo?" It sounds like a simple question, that should have a simple answer, right? And usually it does. Unless it doesn’t. Welcome to the world of the cross-over trial!

The garden variety randomized trial is a parallel or concurrent trial: people get randomized to one of 2 or more groups, and they continue on their parallel tracks, at the same time. At the end of it, if all goes well, you have solid answers to the main question or questions you set out to resolve.

In a cross-over trial, on the other hand, people start off in one group, then along the way, each group of people swaps over with those in another group. Everyone gets the same options, they are just randomized to going through them in a different order – intervention A then B, or intervention B then A. That’s how the guy in the cartoon can be in both the vaccine and the placebo group.

Let's start with the advantage of doing trials like this. A crossover trial means each person is their own control. With that one move, you have removed a common reason for differences in outcomes – individuals' differences. And that means you need fewer people to get an answer.

Every health intervention question can't be answered this way of course – think surgery versus antibiotics for appendicitis, for example, or a drug that isn't going to leave your body to revert back to your usual state during the break between interventions ("wash-out period").

But what about our guy in a vaccine trial, though? They don't fit this picture, do they? Vaccines may wash out, but the benefits to your immune system of recognizing its enemy sure isn't supposed to!

Crossover trials for vaccines are in the spotlight because they're being used for Covid-19 vaccine trials. I discuss this in depth over at Absolutely Maybe – and for more technical discussion on this, see this preprint on the thinking behind the proposal, and Steve Goodman's slides for the US Food and Drug Administration's deliberations.

The crossover extensions of the Covid vaccine trials can't do everything a randomized controlled trial can do, but they can provide valuable data on some issues, especially if the people stay blinded. High amongst those is how long immunity lasts. That's because you now have one group that was vaccinated early, and one group who had deferred vaccination. After the crossover, if the infection rate between the groups stays the same, you know the early-vaccinated group's immunity isn't waning.

Back to the average crossover trial, though, which will be of treatments. What should look out for with those?

One problem is if the groups before the crossover are treated as though they are parallel trials. That's risky. Randomizing enough people to a parallel trial means you don't have to worry about differences between the individuals skewing the results – you don't have that when you're randomizing the order of interventions, not the people.

You also have to keep in mind what possible influence could the previous intervention have had. If the trial goes on for a while, then you have to consider whether the different time periods are now a factor – and more people might have dropped out before they had the second intervention, too.

And 2 final bonus points: "N of 1" trials are cross-overs. That's when you are trying out treatments in a formally structured way, though like all cross-over trials, it only works in some situations. (A quick look at those here at Statistically Funny.) And there's another kind of trial where people are controls for themselves: (Here's my quick look at those.)

Hilda Bastian

March 2021

To learn more about crossover trials, check out Stephen Senn's book, Cross-Over Trials in Clinical Research. This link will help you find it in a library near you.

Clinical Trials - More Blinding, Less Worry!

She's right to be worried! There are so many possible cracks that bias can seep through, nudging clinical trial results off course. Some of the biggest come from people knowing which comparison group a participant will be, or has been, in. Allocation concealment and blinding are strategies to reduce this risk.

Before we get to that, let's look at the source of the problems we're aiming at here: people! They bring subjectivity to the mix, even if they are committed to the trial - and not everyone who plays a role will be supportive, anyway. On top of that, randomizing people - leaving their fate to pure chance - can be the rational and absolutely vital thing to do. But it's also "anathema to the human spirit", so it can be awfully hard to play totally by the rules.

And we're counting on a lot of people here, aren't we? There are the ones who enter an individual into one of the comparison groups in the trial. There are those individual participants themselves, and the ones dealing with them during the trial - healthcare practitioners who treat them, for example. And then there are the people measuring outcomes - like looking at an x-ray and deciding if it's showing improvement or not.

What could possibly go wrong?!

Plenty, it turns out. Trials that don't have good guard rails for concealing group allocation and then blinding it are likely to exaggerate the benefits of health treatments (meta-research on this here and here).

Let's start with allocation concealment. It's critical to successfully randomizing would-be trial participants. When it's done properly, the person adding a participant to a trial has no idea which comparison group that particular person will end up in. So they can't tip the scales out of whack by, say, skipping patients they think wouldn't do well on a treatment, when that treatment is the next slot to allocate.

Some allocation methods make it easy to succumb to the temptation to crack the system. When allocation is done using sealed envelopes, people have admitted to opening the envelopes till they get the one they want - and even going to the radiology department to use a special lamp to see through an opaque envelope, and breaking into a researcher's office to hunt for info! Others have kept logs to try to detect patterns and predict what the next allocation is going to be.

This happens more often than you might think. A study in 2017 compared sealed envelopes with a system where you have to ring the trial coordinating center to get the allocation. There were 28 clinicians - all surgeons - allocating their patients in this trial. The result:

With the sealed envelopes, the randomisation process was corrupted for patients recruited from three clinicians.

But there was an overall difference in the ages of people allocated in the whole "sealed envelope" period, too - so some of the others must have peeked now and then, too.

Messing with allocation was one of the problems that led to a famous trial of the Mediterranean diet being retracted recently. (I wrote about this at Absolutely Maybe and for the BMJ.) Here's what happened, via a report from Gina Kolata (New York Times):

A researcher at one of the 11 clinical centers in the trial worked in small villages. Participants there complained that some neighbors were receiving free olive oil, while they got only nuts or inexpensive gifts.

So the investigator decided to give everyone in the same village the same diet. He never told the leaders of the study what he had done.

"He did not think it was important"....  

But it was: it was obvious on statistical analysis that the groups couldn't have been properly randomized.

The opportunities to mess up the objectivity of a trial by knowing the allocated group don't end with the randomization. Clinicians could treat people differently, thinking extra care and additional interventions are necessary for people in some groups, or being quicker to encourage people in one group to pull out of the trial. They might be more or less eager to diagnose problems, or judge an outcome measure differently.

Participants can do the equivalent of all this, too, when they know what group they are in - seek other additional treatments, be more alert to adverse effects, and so on. Ken Schulz lists potential ways clinicians and participants could change the course of a trial here, in Panel 1.

There's no way of completely preventing bias in a trial, of course. And you can't always blind people to participants' allocation when there's no good placebo, for example. But here are 3 relevant pillars of bias minimization to always look for when you want to judge the reliability of a trial's outcomes:

• Adequate concealment of allocation at the front end;
• Blinding of participants and others dealing with them during the trial; and
• Blinding of outcome assessors - the people measuring or judging outcomes.

Pro tip: Go past the words people use (like "double blind") to see who was being blinded, and what they actually did to try to achieve it. You need to know "Who knew what and when?", not just what label the researchers put on it.


More on blinding here at Statistically Funny

6 Tips for Deciphering Outcomes in Health Studies at Absolutely Maybe.

Interested in learning more detail about these practices and their history? There's a great essay about the evolution of "allocation concealment" at the James Lind Library.

More than one kind of self-control

If you like reading randomized trials about skin and oral health treatments - and who doesn't? - you come across a few split-face and split-mouth ones. Instead of randomizing groups of people to different interventions so that a group of people can be a control group (parallel trials), sections of a person are randomized.

It's not only done with faces and teeth. Pairs of body parts can be randomized too, like arms or legs. These studies are sometimes called "within-person" trials. This kind of randomization means that you need fewer people in the trial, because you don't have to account for all the variations between human beings.

It has to be a treatment that affects only the specific area of the body treated, though. Anything that could have an influence on the "control" part is called a spill-over effect. There are still inevitably things that happen that affect the whole person, and those have to be accounted for with this kind of trial. Body part randomization is one of several ways a person can be their own control: the n of 1 trial is another way.

Randomizing sections didn't start in trials with people: it began with split-plot experiments in agricultural research. The idea was developed by the pioneer statistician, Sir Ronald Aylmer Fisher, who had done breeding experiments. He explained the technique in his classic 1925 text, "Statistical Methods for Research Workers."

It's great to see that neither blackheads nor treatment effects are hampering the Twilling sisters' style! They do seem to be at risk of susceptibility to the skincare industry's hard sells, though. Those issues are the subject of my post Blemish: The Truth About Blackheads.

Drugs go head-to-head at the Pharma Olympics

At the Olympics, humans try to go "higher, faster, stronger" - and achieve their personal best. The bar is constantly raised. Drugs don't have to be better to cross the line, though: they can get by on what's called non-inferiority or equivalence trials. "No worse" (more or less) can be good enough.

Some drugs are now only loosely possibly non-inferior to other non-inferior drugs - several degrees removed from proven superior to doing nothing. Add the increasing reliance on shortcut measures of what works, and there's a real worry that for drugs, the performance bar is being lowered.

If you want to read about the differences between traditional randomized controlled trials that can show superiority and their non-inferiority and equivalence cousins, click on the PDF here at the CONSORT website.

Blind luck: in praise of control groups

People often don't like the idea of "drawing the short straw" in a randomized trial. But being in the control group could turn out to be a very lucky break!

Ideally, trials are blinded so you don't know which group you're in. Knowing you're in the control group could affect your behavior and your opinions about whether or not you're benefiting (or being harmed). But even when it's not possible, it's not always fail-proof. (A placebo wasn't going to do THAT to Lisa's eyebrows!)

In theory, a trial is being done because it's genuinely not known whether the interventions being tested are better than alternatives (including doing nothing). And people who participate in clinical trials, on average, don't seem to be any worse off than people who don't - whether or not they were in a treatment or control group.

For studies that addressed this question, it was possible for researchers to get an average on experimental versus established treatments: only around half of new experimental treatments turned out to be better, and very few turned out to be a lot better. Those studies only covered about 1% of trials. Still, it's reassuring to know that people who participate in trials and end up in control groups aren't necessarily losing out.


If you'd like to read a quick introduction about control groups, go to the short sections 8.11 and 8.12 in Part 2 of the Cochrane Handbook. And here's research on blinded allocation to trials and on subjective assessment in trials.