PubPeer “Stem Cell Shoot Out” & STAP 2.0 Clarification

A heated series of discussions on stem cell publications is ongoing on the Post-publication review (post-pub) website, PubPeerwhich has been called by a PubPeer userThe stem cell shoot out“. This “shoot out” goes well beyond the one page on PubPeer.

It is notable to see that the authors of the papers that are the focus of these interactions, often prominent stem cell scientists, are now sometimes participating in the discussions by leaving comments themselves on PubPeer. For instance, Austin smith has commented on PubPeer regarding ground state stem cells and some of his lab’s work.

Jacob Hanna has also recently responded to PubPeer commenters in discussions on the website. This is a different case though as PubPeer commenters have raised a series of specific concerns regarding apparent re-use of data in multiple high-profile papers from Hanna’s doctoral work. Update: Retraction Watch also mentioned allegations of potential data manipulation in some of these papers as well.

As to the data re-use, an example is a Blood paper entitled “CXCL12 expression by invasive trophoblasts induces the specific migration of CD16- human natural killer cells” on which Hanna was first author. Commenters on PubPeer pointed out data re-use in this paper. Hanna responded that he didn’t see a problem with it (see screenshot image below). A commenter “Peer 1” disputed Hanna’s statement.

Hanna pubpeer

When I polled readers on scientists re-using their own data a few months ago, the results were notable and most respondents were OK with data re-use, but only as long as authors clearly acknowledged in the paper in question that they were re-using their own data from an earlier paper and cited it.

What do you think?

A number of other Hanna papers are being discussed on PubPeer including other instances of alleged data re-use and it’s still too early to say what more if anything will come of it. Since this is a situation that is rapidly evolving and very complicated, I’d recommend a cautious approach to it until more facts are available and perhaps Dr. Hanna will have more to say to clear things up. Again, it is not just Hanna involved in this “stem cell shootout”, but also other top scientists and papers. Also, to clarify this situation is not what I was referring to when I did a blog post on STAP 2.0. Stay tuned for more on that in the coming months as I think this other situation will come to light in 2015.

More broadly bioethics issues such as different kinds of image re-use or other data re-use and other issues warrant more open discussion. Because these issues only become apparent after publication, it follows that post-pub review fills an important needed role.

Two main areas of post-pub review are evident in the “stem cell shoot out” and elsewhere in post-pub review more generally: (1) conceptual and philosophical discussions and debates as well as (2) identification and discussion of potentially problematic issues in specific data sets (e.g. images) in publications.

Most readers of this blog favored post-pub review, but those who felt negatively about it were concerned about a “gotcha” mentality in a recent poll.  Some of the authors responding to concerns about their papers on PubPeer have voiced concerns or mixed feelings about the anonymity of most commenters and reviewers. It is notable that post-pub review on PubMed is non-anonymous and post-pub review on blogs (e.g. this one you are reading now) can be either anonymous or non-anonymous.

Anonymity in science is a complicated issue.

Without anonymity, many commenters simply would not participate in traditional pre-publication review or post-pub review because of legitimate fear of retaliation. With anonymity, clearly some commenters cross the line to non-constructive actions.

So what’s the best system of post-pub review?

I’m not sure there’s one “right” answer to this, but careful moderation of anonymous comments can address some of the concerns. Such moderation can become a complex, onerous task though, especially if one is disinclined to block comments but is also not a fan of comments that cross the line.

Post-pub review is having a rapidly growing influence in science and it’s so new that a number of these elements are evolving right before out eyes. Let’s see what develops. The examples on PubPeer are illustrative of the power of post-pub review as well as its growing pains. As to the “stem cell shootout”, it is likely to continue and spark discussion.

16 thoughts on “PubPeer “Stem Cell Shoot Out” & STAP 2.0 Clarification”

  1. A new paper just published in Nature by Marius Wernig (Stanford), independently confirmed the validity and authenticity of reprogramming obtained in Mbd3flox/- system.
    http://www.nature.com/nature/journal/vaop/ncurrent/pdf/nature14274.pdf

    Selected quote:
    “We then wondered whether a similar intermediate population arises in high-efficiency reprogramming systems (12, 13). Published expression analysis of two high-efficiency systems showed transient CD73 up-regulation, suggesting the presence of a similar intermediate (Extended Data Fig. 8a, b). We then characterized one of these systems, the Mbd3fl/− secondary MEF system, in greater detail (12). After confirming reported reprogramming efficiencies, we analysed this system by mass cytometry (Extended Data Figs 8, 9, 10). By day 3, fibroblast marker repression was evident, and CD73 was upregulated within this population (Extended Data Figs 8e and 9b). Within the CD73high/MEFSK4low population, CD49d (Itga4) up-regulation was not apparent, but we noticed the emergence of a separate integrin, CD104 (Itgb4). By day 4, the major CD73high branch clearly overlapped with the CD104high branch and persisted into day 5. SSEA1high and CD326high expression was present on day 4, but clear co-expression was not seen until day 5. By day 9, CD73 and CD104 expression was dramatically reduced while CD326 and SSEA1 expression remained high. These data demonstrate a transient CD73high/CD104high population arises after donor cell program repression and before ESC marker acquisition, even in a highly efficient reprogramming system. Similar to CD49d and CD73, CD104 is not highly expressed in ESCs (Fig. 1a). And similar to viral reprogramming, adenosine treatment abolished reprogramming in the Mbd3 reprogramming system, albeit only at late stages, whereas compounds affecting CD49d function had little effect (Extended Data Fig. 4f).”

  2. On another note, I don’t think moderation of comments is feasible, it will be too much work. How about a system of “upvotes”? Hopefully insightful comments will soon be at the top of the comments section and unsubstantiated bashing would be far down the list.

    1. There is an up and down vote active, but the comments do not sort by that (rather by date). This is a good idea to put “top’ comments at the top. I’m looking into it.

    2. Person who failed to replicate MBD3 experiment

      Comments are a little bit edgy, but I don’t think they are unsubstantiated.

      Factually correct statement: Rais et al. 100% reprogramming efficiency paper was published in 2013, it is currently 2015 and there is no lab that replicated this finding using independently derived reagents AND there were claims that MBD3 also boosted human iPSC generation.

      The implications are certainly negative, but not unsubstantiated.

      Now I’m not saying that deterministic reprogramming does not exist — I’m only saying that Mbd3 doesn’t seem to be the key for this effect.

      If you compare this to the 2006 Yamanaka mouse iPS cell paper, by 2008 several groups had progressed to generating human iPS cells… you get what I mean.

    1. Hanna has issued a very detailed response to these misleading and desperate allegations:

      “I had already placed a comment on Nature website 6 months ago explaining this and in the GEO file submission website. (http://www.nature.com/nature/journal/v502/n7469/full/nature12587.html)

      In any case, we have already issued a very detailed response to the misleading presentation of findings by Bertone and colleagues: https://www.dropbox.com/sh/7yuxmqwaupblfjp/AAACUroOJkRDbBZd5LYq9-yfa?dl=0

      Direct summarizing quote from Hanna response:

      “In summary, we completely disagree with Bertone and colleagues in their blatantly misleading report, and highlight that all efficiency sets presented in our cells are matched and comparable, and live up to cutting edge and applied standards in the reprogramming field. Trying to intentionally confuse readers
      between efficiency experiments and genomics experiments for which reporters and selection were not applied, is greatly misleading.

      Mbd3/NuRD is a major pathway that inhibits the maintenance and induction of pluripotency and its controlled manipulation should be an integral pathway for inducing and maintaining naive pluripotency in a variety of species.

  3. The source of all data in a paper has to be acknowledged. So if you use data from another paper, you should state so. Not explicitly stating this misleads the reviewers and readers into believing the the experiments were done alongside (same time, same batch of reagents, etc) each other. In this case, with some years apart, not bothering to repeat the control smacks of laziness or cutting corners to get a “glam” publication. There are three results.
    1. The corner cutters benefit in the short term – grants, tenure, promotion.
    2. The corner cutters train young scientists in cutting corners, and after one or two generations we do not have any science.
    3. “Glam” publications lose their gloss.
    The conclusion can only be that this is a self-defeating strategy – it does remind me somewhat of Prisoner’s Dilemma, but I am no game theorist.

    1. Concerned Reader

      Exactly.

      Let’s just think about this for a second.

      Given the issues with the PhD papers, do you think these “issues” would have disappeared during the postdoc and independent investigator stages?

      Quite terrifying, I might say.

  4. Concerned reviewer

    The issues with Hanna PhD papers go far beyond simple “re-use” of data; to simply question whether it is okay to re-use data would be misleading as to what has been uncovered.

    While not “officially” confirmed, there is compelling evidence of data manipulation in multiple instances, which was not found in others’ papers.

    This is in addition to the difficulty in replicating the MBD3 100% deterministic reprogramming and “evolving” protocols for making human naive cells.

    1. The Hanna lab has done an outstanding job in making protocols and reagents available (e.g. plasmids on Addgene, protocol updates on their website: http://hannalabweb.weizmann.ac.il/), sending cell lines etc..

      The original naive protocol is still on their website as published, and they are already sharing their improvements with the community (feeder free, Xeno-free, FGF free etc). Nothing wrong with providing additional “evolving” protocols. Mouse growth conditions evolved over 20 years, and for sure the first description of human naive cells is going to dictate different growth conditions and is not going to be entirely solved in one single paper.

      This greatly misleading “Jacob Hanna bashing” campaign is starting to look pathetic, and it is very clear who are the individuals standing behind it. This has gone too far and too low. Further, it almost seems as though the stem cell field is begging for scandals, and Pubpeer and iPS.com are hyping this way too much (and way too early), so that they can promote themselves further (classic tabloid-like journalism).

      However, I am not concerned, as Jacob Hanna responds to his critics and advances his research, by publishing great and insightful follow-up papers that many others follow closely and keep looking forward to reading.

      1. To IPS Researcher,
        There’s an important difference between tabloid-like “bashing” campaigns and responsible handling of difficult situations where important issues are raised that need to be discussed in the stem cell and biomedical community. Of course I greatly favor the latter approach and that’s what I had in mind with this blog post. I was very careful in how I discussed this “stem cell shootout” situation and hopefully managed a relatively balanced overview of the situation. I’m also going to reach out to Dr. Hanna and give him the opportunity to provide his views on this.

      2. Concerned reviewer

        “The original naive protocol is still on their website as published, and they are already sharing their improvements with the community (feeder free, Xeno-free, FGF free etc). Nothing wrong with providing additional “evolving” protocols. Mouse growth conditions evolved over 20 years, and for sure the first description of human naive cells is going to dictate different growth conditions and is not going to be entirely solved in one single paper.”

        This is an evasive and misleading explanation and it should be called out.

        On the PubPeer website, one can clearly see that Jacob Hanna published two papers on naive human PSCs, so the “evolution” has been underway since 2010.

        The first paper was essentially not reproducible, which is why there is no single second group that uses the culture conditions (2iL + FSK) described in this study. It seems that Forskolin, which was working in 2010, appeared to “stop working” in the 2013 paper.

        The second paper had slightly different problems. Perhaps the cells can be grown and exist, but as shown by at least three other studies, the cells were not exactly as “naive” as advertised.

        Finally, if you actually take a look at the protocol listed on Hanna’s website, the concentrations of the factors used are already different from what’s actually published in the original Gafni Nature paper!

        I don’t even want to get into the MBD3 because obviously, no one is getting 100% reprogramming efficiencies using independently derived systems and it is 2015 already! And there was already a fantastic paper from Jose Silva’s and Brian Hendrich’s group showing that there is something wrong with the MBD3 Nature paper! Some of these claims are just getting ridiculous.

        1. It is clear from Pubpeer and discussions in the field over the last year, that the 2009 Nanog “Gateway” and 2007 Mbd3 “Gateway” papers by Silva and Hendrich do not support their conclusions anymore, and thus many think they should be retracted (including myself).

          1) https://pubpeer.com/publications/DD9B0F7E3070410638B19E7560BA21

          2)https://pubpeer.com/publications/01475E1935F54B71CC24C9DF6F4CEC

          The Cell Stem Cell papers is very poor and incoherent according to some people:
          https://pubpeer.com/publications/70792BE7C58EB218C689B7A7F29FD3

          Finally, it is misleading to ignore the following important papers that are all based and supportive to Hanna’s work:
          1) Mbd3 paper by Ingrid Grummt: http://www.ncbi.nlm.nih.gov/pubmed/23533168

          2) Thomas Graf has shown up-to 100% iPSC by OKSM + C/EBPa controlled delivery (and 2i/LIF)(http://www.ncbi.nlm.nih.gov/pubmed/24336202). Jacob Hanna had identified this already in 2008 (http://www.ncbi.nlm.nih.gov/pubmed/18423197).

Comments are closed.