鶹ý

What I Learned When My Research Project Got 'Scooped'

— This past week was probably the toughest of my research career

MedpageToday
A computer rendering of binders in a trash or recycling bin.
  • author['full_name']

    Jeremy Faust is editor-in-chief of 鶹ý, an emergency medicine physician at Brigham and Women's Hospital in Boston, and a public health researcher. He is author of the Substack column Inside Medicine.

I've always vowed to take you "inside" my professional life. And I'm conscious of the fact that I often highlight my successes with you. But I also have defeats, just like anyone else. So -- exhale -- here we go.

This past week was probably the toughest of my research (non-clinical) career.

I want to open up and share a genuine setback with you; actually, "setback" would be a euphemism. This felt like a true failure. So, I want to pull the curtain back, talk with you about how I've been processing it, and share the positive things I've already taken from this experience.

Long story short, we got "scooped."

Majorly.

What I mean is that research very similar to work that my colleagues and I were in the process of getting published was published first by other researchers in another major medical journal. It felt like watching months of work vanish into thin air. This happens to researchers, but it was especially soul crushing because the findings were novel and of great public interest. It was about the terrible effects of the Texas abortion ban, called SB8.

Stage 1: Executing an Idea

Sometime last year, a colleague of mine at 鶹ý and I were talking about the post-Roe landscape. At some point, she asked whether my research team could determine whether states that had banned abortions after the Dobbs decision might have had an increase in infant mortality.

I started to look through some data, and quickly realized that the situation in Texas (which in September 2021 had banned abortions after the detection of fetal heart activity, generally around 5 or 6 weeks into a pregnancy) was uniquely bad. The reason for this, it seems, is that unlike any other state that banned abortions after Dobbs, the distance to the closest legal abortion care for Texans was now almost always more than a day's drive. For those who want (or need) to end a pregnancy, Texas is a desert.

My colleagues and I decided to take tools we'd developed during the COVID-19 pandemic and use them to measure mortality in Texas neonates and elsewhere. When we did this, we saw something disturbing, if somewhat expected. After Texas enacted its abortion ban, there was a sudden increase in neonatal mortality there. A long-standing decrease in neonatal mortality had reversed, seemingly overnight.

We knew this was a massive finding that had medical and political implications.

But we hold ourselves to the highest standards of science that we can. That meant we could not just publish the finding until we were absolutely certain that we were right. Put another way, good science ends not when you have an answer that fits your own views, but rather, when you've convinced yourself that there's no way you could be wrong.

So, we subjected the data to another type of statistical test. And we tried various time periods and other demographic groupings to make sure our models held up. Once we were sure, we submitted our paper to an important medical journal.

Stage 2: Revisions

After a few weeks, the journal reviewed our work. They were interested but wanted us to use yet a different statistical test -- one we had not used before. The data scientists I collaborate with were able to do that too. It confirmed the qualitative findings from our previous analyses, with a couple of differences. First, the new test calculated fewer excess neonatal deaths. But, second, the "testing" for statistical significance was stronger than our previous analyses. So, basically, the new way of doing things found a somewhat smaller increase in neonatal deaths in Texas, but the statistical significance of the findings was a lot stronger. We felt like this was a reasonable result.

Stage 3: The Scoop

I keep up with medical journals, in case you haven't noticed. So, when I saw the title of in JAMA Pediatrics, "Infant Deaths After Texas' 2021 Ban on Abortion in Early Pregnancy," I got a pit in my stomach.

I opened it and started reading. It was great. (Damn it.) The paper, written by researchers at Johns Hopkins University, had just about everything we had found. The methods were excellent. I knew immediately that my team and I were toast.

Like us, the Johns Hopkins team had found that the increase in infant mortality in Texas was mostly due to spikes in deaths due to anatomical conditions that have very low survival probabilities ("congenital malformations, deformations, and chromosomal abnormalities"). Most of these conditions -- severe "birth defects," being the layman's term -- are detectable during pregnancy. When faced with having to carry to term and deliver a baby whose life span might be minutes, hours, days, or weeks, many would-be parents make the understandable, if sad, choice to end the pregnancy. Usually this information is determined sometime during the second trimester, and often after 18-20 weeks. So, elective terminations that happen at this relatively later point in pregnancy are often due to situations like this. In other words, banning abortions after 18-20 weeks (without exceptions for likely lethal anatomical conditions) forces people who had otherwise wanted to be parents, to endure physical risks and emotional burdens that I can scarcely imagine. It makes no sense. To me, it is cruel.

That is what I had wanted to tell the world. That is what our competitors got to do instead.

Stage 4: The Fallout

When the JAMA Pediatrics paper came out, the first thing I had to do was notify my co-authors. Our exciting (if troubling) work -- which we had thought would get a lot of attention and make a lasting impact when we released it -- had suddenly become a lot less important. Indeed, the major medical journal where our work was being considered (we were in a "revision" stage with them), quickly notified us that they were no longer interested in publishing our study. I couldn't blame them. There are times when one research group scoops another out of a finding or two. In this case, though, the overlap in the important findings across the two papers was almost complete. There were a couple of things left in our paper that the other one didn't have, but it wasn't enough to rescue it from oblivion.

I felt sorry for myself, but I felt guilt as well. This had been an unfunded "volunteer" project that I had roped my colleagues into doing with the promise of a high-impact paper at the end. So, we had gone for it, working after hours and on weekends.

But getting the paper across the finish line was my job, and I had failed to do so in time. For what it's worth, both our competitors and we had submitted our respective papers in the first week of March (a coincidence). But in all likelihood, we were still a couple weeks away from a "yes" from the journal we were working with, in part because we had to redo our analyses twice. Meanwhile, it's likely our competitors got it closer to "right" the first time; that is, the methods they used had been accepted by the journal for another project in the past, so they didn't have the same hurdles to clear. So, even though both teams submitted our papers to journals around the same time, our competitors' initial submission was probably nearer to "ready" than ours, even though both were rigorous and reached the same conclusions.

Meanwhile, as I had anticipated, there was a lot of media attention in the days following the JAMA Pediatrics publication. The study was covered in all the major print, TV, and online outlets. Seeing that play out admittedly felt like getting hit in the stomach a bunch of times.

Stage 5: Acceptance

In the Five Stages of Grief, acceptance is the last one. So, here we are. Reader, I'll admit that I spent a lot of this week feeling rather sorry for myself. But I also realized that this was self-indulgent and pointless. I'm feeling better every day, and I'd like to share some insights that I've already gained from this humbling experience.

First, the most important thing is that the information is out there. It would be a real tragedy if neither paper were published, and the truth about the cruelty of Texas's abortion ban were not revealed to the public. Research is about learning and making progress. While there may be some occasional "glory" in getting work like this done, this study wasn't about us. It was about the people affected by the misguided law.

Second, taking on this project was a known gamble. Often, gambles do not pay off. While we thought that this work was a little too complicated for just anyone to tackle (making it less likely that we'd be scooped), we should have known that the other group would be interested in this topic. Therefore, I should and could have been faster in getting this all done. But as I said, we had to be so certain we were not wrong before submitting the work for publication. Ultimately, getting scooped had always been a realistic possibility. Even knowing that, I'm still glad we tried.

Should I have "pre-printed" our study (i.e., posted the manuscript on a public server prior to peer review publication)? Probably; and not just so we would have "planted the flag" and have been first on this highly important finding. Rather, pre-printing would have meant that the important information we uncovered would have been available to the public weeks-to-months sooner. So, while getting scooped in the medical journals would have still stung, having already pre-printed our study would have made it less of a blow.

Third, the journal that ultimately slowed us down by requiring multiple revisions did indeed make our paper better. While the process led to us finishing in second place, it does leave us better than we started. We are now equipped with a new and powerful tool. We will use this to analyze other data and hopefully answer important public health questions in the future.

Ultimately, I learned something both philosophical and technical through this process -- which I'm no longer seeing as a complete failure. While it wasn't a particularly fun week, I have already reached the "bargaining" and "acceptance" stages of grief, I'm happy to report. I'm thinking about how much I've learned, and the ways I may apply those lessons in the near future. This endeavor is beginning, once again, to feel like it was a genuinely productive one.

This article was originally published in .