Full Reviews of Rejected STAP Paper Point to Early Signs of Big Trouble

Before the two STAP cell papers were published in Nature in January of 2014, much of the same data was reportedly submitted as single papers to other high-profile journals including Science.

In these cases, the proto-STAP papers as we might call them were rejected.

But why?

Until now we largely could only speculate.

However, the reviews of the 2012 proto-STAP manuscript at Science can now be read at Retraction Watch.

Retraction Watch, STAP cell papers
Retraction Watch, on STAP cell papers and reviews.

As a result of reading the Science reviews, today we know what the reviewers at Science thought in 2012 of this proto-STAP paper and this sheds much light on what went so terribly wrong with STAP overall. There were many big red flags. Keep in mind that the Nature reviewers would not know about the Science reviews unless by chance one or more of the reviewers for Nature had also participated in the review process at Science.

This early generation STAP paper was entitled “Stress altered somatic cells capable of forming an embryo”.

There was no “STAP” acronym at that point. Instead, the stress-produced stem cells were called “SACs”, an acronym presumably standing for “stress-activated somatic cells” or “stress-altered stem cells”. Therefore, let’s call this proto-STAP paper, the SAC paper.

All three Science reviewers had serious doubts about the SAC paper and pointed out numerous specific concerns.

For example, Reviewer 1 right away early in their review pointed out that the SAC phenomenon was probably not real and was instead explainable by two simple experimental problems: stress-associated GFP reporter activation and cell culture cross contamination.

Crucially, this same reviewer noticed the gel splice, later present in the accepted Nature STAP article Figure 1. However, apparently the STAP/SAC team did not take that concern or most of the other reviewer issues to heart.

Reviewer 2 was extremely skeptical of SAC as well, listing twenty-one specific problems/issues to be addressed. Unfortunately, it seems that most of these concerns also remained unaddressed in the later accepted Nature STAP papers. It is fair to say that although 21 issues seems like a lot that these concerns seem reasonable and not overly harsh.

What else did the reviewers say?

Both Reviewers 1 and 2 had the shared concern that pluripotency-related gene expression seemed abnormally high in the SAC cells. Way way too high.

Reviewer 2 wanted much more data before being convinced. For example, they wrote:

Given the novelty of the claims, a thorough characterization of the SACs is warranted, as is some probing of the mechanisms. This would necessitate a more sophisticated genomic analysis of SACs, through microarray or RNA-seq, and genome-wide DNA methylation analysis — analyses that other pluripotent stem cell lines have been routinely subjected to and for which methods for smaller cell numbers have been developed.

Reviewer 3 was not as detailed with their concerns, but more generally identified some areas of concern such as those articulated in this paragraph:

the methods and cell protocols used must be described in far more detail. For example, the section on Oct4 should state how many cells were sorted and describe the appearance of the cells. Is it possible that rare populations of cells pre-exist or are already apparent on day 1 (thus, what are the “dots” of Fig. 1?). The authors will argue that, indeed, under certain circumstances, they were able to reprogram terminally differentiated cells, and that this was attributable to TCR recombination. I think, ideally, that the cells should be experimentally tagged and traced. This would unequivocally clarify the source of the cells and, further, would exclude the possibility that some cells pre-existed in a pluripotent state.

Critically, it is necessary to determine whether SAC cells can propagate stably in culture and whether such cells can be passaged.

 

Experimental tagging and tracing of the cells would have been a major step forward for clarifying whether the SAC/STAP phenomenon was real. STAP/SAC cells should have been made in parallel to iPS cells as well for direct comparison.

One has to wonder how the Nature reviewers and editors could not have picked up on so many problems that were apparent to the Science reviewers. Every review at a different journal of the identical paper will be distinct of course, but this data seemed inherently flawed in a systematic way. This was no ordinary paper either. It was a no-brainer that this kind of paper with revolutionary claims required extraordinary, very meticulous editorial oversight. It is therefore not an unreasonable expectation that the Nature review process of the STAP papers should have picked up on some of these serious problems.

Nobody likes to get a harsh review of a submitted manuscript, but it’s crucial after you calm down in that situation to consider that some of the comments by the reviewers likely raised legitimate, important issues to address before resubmission. This way you can avoid problems and improve your paper. Apparently to a large extent that didn’t happen between the SAC paper and STAP paper stages.

In the end these Science reviews of the rejected SAC paper indicate that the STAP manuscript and data were problematic in fundamental ways back in 2012.

5 thoughts on “Full Reviews of Rejected STAP Paper Point to Early Signs of Big Trouble”

  1. It’s really baffling to me how they can get such thoughtful, detailed, and–in a way– devastating review, and yet still submit essentially the same flawed paper to Nature. Without reading these reviews I was always willing to give all the authors the benefit of the doubt. Maybe they simply did not see the problems of their own work.

    But now it is clear: the problems were clearly pointed out to them and NONE of the authors that were on the flawed Science submission have any excuse left.

    Also, this makes Nature’s defense highly incredible. I guess it is possible that Science ended up with three star reviewers while Nature only got duds. Only if we see Nature’s reviews can we know for sure. But no matter what: Nature got either the review or the editorial process completely wrong.

  2. As for the former manuscripts of the (retracted) STAP papers, you might want to check the documents of the patent applications:

    US 61/637,631 (24.04.2012): http://mirror.patentscope.wipo.int/patentscope/docservicepdf_pct_mirror/id00000022851022.pdf

    US 61/779,533 (13.03.2013): http://mirror.patentscope.wipo.int/patentscope/docservicepdf_pct_mirror/id00000022881386.pdf

    PCT/US2013/037996 (24.04.2013): http://mirror.patentscope.wipo.int/patentscope/docservicepdf_pct_mirror/id00000022883817.pdf

  3. The flaws in the STAP cell papers were so obvious that my first reading of them immediately led me to doubt them and thus I posted a blog article on it on the very same day these papers appeared online in Nature. I was wondering why such a sloppy research could be seriously valued by Nature. I was even amazed that Nature delayed the retraction of these problematic papers even when all authors had agreed to retract them. In the end Nature still think the “science” of the STAP cell papers has not been questioned.
    Nature should open its reviewing documents on the STAP cell papers to show how it handled extraordinary claims associated with unbelievable discoveries. Otherwise, Nature should not call it a responsible scientific journal any more.

  4. The STAP flap is the gift that keeps on giving. The idea of stimulus or stress triggering as we have isolated population of stress selected stem cell from cultures of our UCB derived hMSCs. Maybe this should this idea should be used stating with iPSCs, eSCs and adult SCs to isolate populations of more potent and predictable cell populations?

Comments are closed.