Refining & revising research on the public record

179024690The life of a research project often doesn’t end when a publication appears in a journal. Experiments continue. Errors are found. Supporting or refuting data is published. Alternative explanations emerge… So PubMed Commons members are posting updated and complementary information to keep work current.

Honing interpretations

Consider, for example, Daniel Haft’s reflection on his publication from 2009. He identified some peptides in two Bacillus strains that he referred to as putative “bacteriocins”. The term describes bacterial peptides that are toxic to related strains. Use of the term was “overconfident”, he notes, “…given the lack of evidence then that these heterocycle-containing natural products were toxins rather than, say, peptide pheromones.” But now results from another group suggest he was on the right track after all.

Andrew Sharp’s group studies how genome structure relates to human disease including X-chromosome inactivation. He considers how a new publication describing DNA methylation in development might be relevant to their own work.

Meta-analysis can show patterns and discrepancies between results from different studies. The data can be calculated in different ways. Valeria Fadda provides alternative data visualizations by posting forest plots for risk differences from published meta-analyses.

Dopamine is a central player in addiction in rodents. Many factors, including where it acts, influence the effects of this neurotransmitter. Roy Wise notes how his group’s work supports recent findings on the consequences of location and what this might mean for the field.

“The mechanisms for establishing, maintaining, and reinstatement of cocaine self-administration have been studied extensively, but have not yet led to a proven medication for cocaine addiction,” Wise remarks. “Perhaps it is time to turn attention to the endogenous mechanisms for what appears to be a state of drug satiety.”

Amending prior work

Some authors highlight work that might strengthen or refine interpretations. Others note how new or existing data might change conclusions.

With co-author Raphael Silberzahn, Eric Uhlmann documents their new work on names and career outcomes. They had previously reported, “Germans with noble-sounding surnames, such as Kaiser (“emperor”), König (“king”), and Fürst (“prince”), more frequently hold managerial positions…” Their new analysis seems to overturn their previous conclusions.

Like many researchers trying to identify new antitumor agents, Miguel Lopez-Lazaro and his colleagues began by testing compounds in a cancer cell line. Since publishing their work, they followed up in other cell lines. Now he considers whether these in vitro effects will really translate to therapeutic potential.

Teasing apart the functions of specific proteins in cells or animals can get complicated. Using inhibitors or genetic manipulation doesn’t always provide a clear picture.

For example, Jens Staal explored the interplay of the protease MALT1 and the deubiquitnase CYLD in immune cells. After further experiments, he updates his interpretation.

Meanwhile, Jim Woodgett notes that care should be taken with assigning specific functions to a single protein isoform. He highlights the trouble with glycogen synthase kinase-3 inhibitors. After a few comments on the subject, he remarks, “…no small molecule inhibitors are isoform selective – I am a scratched record.”

Bridging published updates

When substantive errors are found, updates are not always clearly connected to original publications. PubMed Commons gives authors a chance to link pieces together in PubMed.

Sometimes errors are reported in follow-up studies. Consider a mutation in an HIV protein that was correlated with survival. Stuart Ray notes that a subsequent report of an alignment error invalidates the findings.

Other times, flaws in methods or results lead to correction or retraction of a report.

Randy Blakely finds that a paper about the human dopamine transporter continues to be improperly cited, despite a correction to the results. He says, “I am hoping the PubMed Commons forum will provide a suitable opportunity to redirect readers attention to the, as yet, unknown properties of the E602G mutation…”

Ivan Oransky uses PubMed Commons to annotate recent retractions. He includes links with more about the history of the publication and why it was withdrawn. For instance, here he points to where co-blogger Adam Marcus notes, “Failure to cite leads to ignoble end for xenon paper, and a correction.”

If you’re an author of a PubMed-indexed publication and ready to get involved, check out this page to learn how you can get started.

The PubMed Commons Team


Unveiling a new look – and more – for PubMed Commons

PubMed Commons set the stage for commenting on any publication in PubMed, the world’s largest searchable database of biomedical literature. Lately we’ve been tackling infrastructure and design to improve the user experience and support the PubMed Commons community. Those developments are now live on PubMed and PubMed Commons. Here’s what you can expect from the PubMed Commons update.

Center stage

Some changes are likely to jump out for frequent PubMed Commons users.

@PubMedCommonsWe’ve adopted new artwork for our blog, Twitter account, and homepage. We’re going for a clear, unified identity across platforms, one that we hope will be recognizable wherever you see us.

We’ve made some modifications to streamline our homepage. We’ve consolidated information about joining and using PubMed Commons in a single page to help you get started. You’ll also find a synopsis of our most recent blog post at the top of our homepage to help you stay up-to-date on PubMed Commons.

For several months, comment rating has given members the chance to weigh in on what comments they find useful. Visitors to PubMed can see these ratings alongside comments. Ratings are a key element in calculating the comment and commenter scores that determine the appearance of comments in the “Selected comments” stream on our homepage.

Some new site modifications will highlight your contributions to PubMed Commons. On our homepage, “Top comments now will feature the top three recent comments. On PubMed records, “Selected comments” (from our homepage stream) prompt the appearance of an icon above abstracts, directing readers to comments below.

This new icon will appear above some abstracts in PubMed.

This new icon will appear above some abstracts in PubMed.

For a while, we’ve selected highly-rated comments to post to our Twitter stream. Starting today, the most recent tweet about a PubMed Commons comment appears on the homepage for PubMed searches. Check it out!

Behind the scenes

Some key changes in the PubMed Commons development won’t be visible on the PubMed Commons site. We’ve improved our “Invite an author” function. It looks the same as before, but we hope you’ll encounter fewer errors when inviting authors to join or comment. (If you do encounter errors, please let us know by using the “Write to the Help Desk” link, found at the bottom of every NCBI page.)

inviteIn response to community feedback, we are also notifying corresponding authors of comments on their publications and inviting them to join PubMed Commons. We’ve been at it for two months, and we’re encouraged by the increase in author responses. We will continue our notification process, but we still encourage members to notify a publication’s author(s) when commenting.

More to come

We’re not done yet. In May, an external working group for PubMed Commons was established. The members offered great feedback and ideas on where PubMed Commons is and where it’s going. But more on that later…

The cornerstone for the continued growth of PubMed Commons is you! Here’s how you can get involved:

  • Learn more about the PubMed Commons pilot.
  • Are you an author of a PubMed-indexed publication? Join PubMed Commons!
  • Already a PubMed Commons member?
    • Post a comment! Log in to My NCBI, find a PubMed citation, and start typing. Not sure what to post? Check out these examples of how authors have been using PubMed Commons.
    • Rate comments. It only takes a moment, and as we discussed above, you can influence what’s highlighted on PubMed Commons – and PubMed.
    • Invite your colleagues!
  • Finally, help us spread the word about PubMed Commons!

Thanks for your support and contributions to PubMed Commons. We look forward to seeing where this community takes us next.

The PubMed Commons Team

Spotlight on… Amanda Capes-Davis, setting the cell line record straight

450755747You think you’ve found what you need – a cell model for a specific type of cancer you’re studying. But the label on those cells may have you fooled. In labs around the world, many cell lines are mistaken for characters they are not. Dr. Amanda Capes-Davis has seen it happen, time and again. She received her medical training and PhD in cancer genetics from the University of Sydney. After six years as a research officer at the Children’s Medical Research Institute (CMRI), she helped establish CellBank Australia, a non-profit cell line repository. Now working as a cell culture consultant and chair of the International Cell Line Authentication Committee (ICLAC), Capes-Davis is on a mission to bring the long-standing problem of cell line misidentification to light – and she’s using PubMed Commons as a tool in this fight.

Mistaken identities

Cell culture is a part of everyday life for many life science and biomedical researchers. To address biological questions, scientists often work with populations of cells grown in incubators. Typically, cells isolated from a human or other animal will only survive and multiply for a few days or weeks, even under optimal conditions. “Immortalized cell lines,” on the other hand, can be grown in flasks or dishes for months, even years. Often these lines are derived from cancers that allow cells to bypass checkpoints that would normally stop them from dividing.

The advent of immortalized cell culture 60 years ago opened doors for new studies, but there are also persistent problems. Many cell lines bear mistaken identities. So far, ICLAC has identified 472 cross-contaminated or misidentified cell lines, based on 89 publications. The cell type or tissue origin of cells grown is sometimes mislabeled at the outset. Other times, a cell line cross-contaminates another, overtaking the original line. “Perhaps 10-15% of all cell lines are cross-contaminated,” Capes-Davis reports.

HeLa cells exemplify the potential and the pitfalls of cell culture. Henrietta Lacks was a woman who unknowingly pushed biomedical science forward. While she was under treatment for aggressive cervical cancer, a sample of her tumor was taken. Shortly before her death in 1951, those cells were used to establish the first reported immortalized human cell line. Within a few years, the cells were widely used, including in development of the polio vaccine. However, HeLa cells grow so robustly that, if a few cells mistakenly get mixed in with another cell type, HeLa can quickly overtake the others. In their survey of the literature, ICLAC has noticed an abundance of HeLa contamination. Capes-Davis notes, “There are 135 different cell line contaminants, but HeLa is by far the commonest. We list 113 misidentified cell lines where HeLa is the contaminant.”

HeLa cross-contamination of cell lines was first reported in 1967. Yet many new publications continue to misidentify cross-contaminated cell lines. “It’s very understandable, I think, for a scientist,” Capes-Davis says. “You’re doing research in that field. You’re seeing everyone else using this cell line. You think this must be appropriate because all of your colleagues use it.” Reports of cross-contamination disappear under the mountain of other publications using the cell line. In doing as they were trained – building on work published by others – many scientists are actually perpetuating errors. Capes-Davis notes, “They say, research is meant to be self-correcting, but with these cell lines, that doesn’t appear to be the case.”

It’s impossible to fully assess the impact that cell line contamination has had – and continues to have – on research and development. “You only really get hints here or there,” Capes-Davis says. “I first learned about cell line contamination as a PhD student… The lab I was working in started to require testing as I was writing up my PhD.  That was quite a stressful experience for me – my PhD was entirely cell culture-based, and it was frightening to think that all my hard work might be wasted if those cell lines were contaminated.” In some cases, experimental results from misidentified cell lines supported patenting and testing of compounds in people. “These days, I think, with the level of regulation that comes in, problems with cross-contaminated cell lines will be picked up before an agent is trialed, but you have to wonder about the waste of time and waste of money required to get to that point.Amanda Capes-Davis

For Capes-Davis, though, solving cell line misidentification is about more than scientists’ time and research funding. “I am also a medical doctor and would go in to operating theatres to collect tissue samples from individuals who consented to their tissue being used in research.  Many cell lines represent a legacy from donors who have died as a result of that disease.  To me, testing of cell lines is part of our responsibility to the donor, and makes sure that we put the donor’s gift to the best use we can in finding future treatments.”

Correcting the record and the future course

In 2009, Capes-Davis joined two initiatives to address cell line misidentification. The American Type Culture Collection (ATCC) Standards Development Organization established a workgroup (ASN-0002) to develop standards for human cell line authentication. Capes-Davis recalls, “Some members wanted to have an ongoing group on cross-contamination.” During this time, she also collaborated with Ian Freshney to develop a database of cross-contaminated cell lines. “We could really see the value of having involvement of a larger group of people, the advantages of people being able to contribute data or having additional expert opinions on whether a cell line was cross-contaminated, whether or not it’s possible to find authentic stocks.” In 2012, she and other scientists  with a shared interest in the problem of cell line misidentification founded ICLAC.

“I think publicity and awareness is always a challenge. It’s always a bit of a surprise to people, even people who’ve been in the field a number of years,” Capes-Davis notes. “A lot of what we’re involved with is making people aware of the need for quality, even when they’re doing preclinical research. It’s not a series of steps that are going to be difficult to do, but it’s something that should be part of good lab practice. It’s part of good research.” ICLAC is also seeking to establish infrastructure for researchers, such as maintaining the database she and Freshney created, developing guidelines for best practices, and making resources available online.

Capes-Davis has taken to PubMed Commons to annotate publications that misreport identities of known cross-contaminated cell lines. “Ideally we want these things picked up as part of peer review, but it’s not as easy sometimes as you might think,” she notes. “Correcting the scientific record is a really important thing because even if you take the assumption that a signaling pathway isn’t going to be affected, that’s not necessarily going to be the case for another paper that cites that work. The assumption is that if you describe a cell line as oral carcinoma, that’s what it is. It might not matter for one person’s work, but it could well matter for the person who reads that paper.”

“I use PubMed pretty much everyday… it’s an area where a lot of people come to look at abstracts and hunt down papers,” Capes-Davis notes. “So [PubMed Commons] seemed like a great opportunity to raise awareness of our database… We’re hoping that if we do the best we can to bring all the known publications that relate to cross-contaminated cell lines into our database, that will be a useful resource for people. It’s not enough…but it’s certainly a step in the right direction.”

Learn more about PubMed Commons. If you’re an author in PubMed, join and make your own contribution to scientific discourse.



Blogs and their links with PubMed Commons

Photo of blog entry in a dictionary

Blogs grew quickly into a dynamic and substantial element in the way we discuss, criticize and share information about scientific publications. Science blogs are one of the filters many now use for the science literature – and one of the ways members of the public learn about research results and efforts.

Anchoring these blog posts to the publications they discuss is one of the ways members are putting PubMed Commons to good use. Here are examples from the spectrum covered by dozens of links to blog posts so far in the Commons.

Dorothy Bishop responded to Michael Farthing’s lecture on the challenge of research conduct, including links to her blog posts on perverse incentives for academics and registering proposed research in psychology.

Daniel Simons writes post-publication peer reviews on his blog. He’s posted one on Joaquin Anguera and colleagues’ study of video game training for cognitive control,  and one on a recent study of inattentional blindness to African Americans by Jazmin Brown-Iannuzzi at the Commons.

Coming across a media report of a study draws the attention of many academic bloggers, spurring both posts on their blogs and links to them on PubMed Commons. This happened when Andrew Kniss saw a newspaper article on Carsten Bruehl’s conclusions about pesticide use and frog population decline.

In other areas where there’s a lot of debate about a publication, or they read a post they found helpful, Commons members draw readers’ attention to detailed coverage in another person’s blog. George McNamara, for example, points to a post by Michael Eisen at the PubMed record for John Bohannon’s publication about peer review at open-access journals.

Important blog posts also unwrap the history of misleading use of studies in public debates. An example now in the Commons is about an autism prevalence study by Simon Baron-Cohen and colleagues. It was added by Jamie Horder, pointing to Ben Goldacre’s post on “The story behind this paper, and the remarkable media coverage it received.” It’s a Whodunit? about an MMR vaccine scare in the media.

Retractions of articles are being noted with links to blog posts about the background, for example by Allison Stelling and Ivan Oransky.

Attacks on research lead defenders to their blogs, too – and now to the Commons afterwards. James Coyne responded to critics of a randomized trial by William Hollingworth and colleagues of screening for distress in cancer patients.

Authors of publications who discuss their work on their own blogs draw readers’ attention to these posts at PubMed Commons. For example, Anne-Marie Cunningham has posted links to blog posts on her work with colleagues incorporating a pre-publication version with one article and an audio-visual presentation with another.

Seth Borderstein links to a post explaining the background to a publication on maternal microbe transmission. And Jonathan Eisen links to his post telling the story behind his team’s work on phylogenetic trees.

Discussing implications for future research of a study’s findings are another part of blog posts brought into the Commons. Graham Coop posted a link to his blog post on Sriram Sankararaman and colleagues’ work on genetic variants on the possible offspring of Neandertal and modern human parents.

The Commons is growing: from 2,000 members in January to over 3,000 today, who have made just over 1,000 comments so far. Want to keep up with comments that are getting good ranking scores in PubMed Commons? We tweet them @PubMedCommons.

That’s one more important reason for PubMed Commons members to add their “helpful? yes or no” clicks to the system: when you read it, please rate it. Help shape the Commons into a valuable place for discussions about biomedical publications – and drawing together the valuable discussions from outside. If you write or read a blog with a valuable discussion about a publication, think about sharing it at PubMed Commons.

The PubMed Commons team

More information

Membership of PubMed Commons is open to all authors in PubMed.

Join PubMed Commons

“You found this helpful:” What people like in comments on PubMed

Comment search and alert: A PubMed Commons guide


“You found this helpful:” What people like in comments on PubMed

Commented rated "You found this helpful"

Rating is a critical function for a commenting system. It’s a key way a forum’s members can encourage the kind of discussion they want to see – and discourage what they don’t find helpful.

Increasingly, ratings are influencing PubMed Commons, the new commenting system in PubMed. There’s a selected comment stream on the PubMed Commons landing page. We’ve also started sending out tweets when new comments start ranking well on several factors.

The rating options are only available for comments – and you can’t rate your own comment. PubMed authors who have joined PubMed Commons can click on “yes” or “no” to the question, “Was this helpful?”

Although it’s early days, ratings are already sending some messages. One of those is what people don’t find helpful: the kind of comment that praises an article without saying why – or that’s only communicating something that is already clear in the abstract.

The message seems to be: use the comment function when you have something specific to say about, or add to, the publication. Making a similar comment repeatedly also tends to be unpopular, as is self-promotion.

Which comment has been the most popular so far? It’s a comment by Gonzalo Otazu in December. Otazu calls into question the validity of statistics in a paper that got a lot of media attention: a study suggesting that mice can inherit fear of smells associated with traumatization of their parents.

As well as good critiques, people have rated discussions positively – especially when the publication’s authors respond or encourage discussion about their paper. Some recent examples:

Integrating outside discussions in journal clubs and blog posts with PubMed Commons comments gets positive ratings, even when the authors don’t participate.

You may be able to invite yourself with our “Get started” wizard. If your email isn’t there, one of your colleagues should be able to join and then invite you. Any member of PubMed Commons can invite others – including an author of a paper that’s been commented on.

You can check out the selected comments at the PubMed Commons landing page – one click away from PubMed’s home page – or follow us on Twitter. And you can find out how to set up alerts on topics, articles or authors you’re interested in via our “Comment search and alert” guide.

From now on, the quality of PubMed Commons will rely on both the comments and the ratings. You need to be a member of PubMed Commons to rate articles. If you’re an author of a publication in PubMed, you’re eligible.

So join today – to discuss the biomedical literature or, just as importantly, to rate the comments.

The PubMed Commons team

More information

Meet PubMed Commons: The new comments forum in PubMed

Join PubMed Commons

Comment search and alert: A PubMed Commons guide


Meet PubMed Commons: The new comments forum in PubMed

Image of comments icon

If you are one of the millions of people who visit PubMed today, be on the look-out for something different. On each abstract page, there’s now a section called PubMed Commons. It’s a forum for scientific discussion on publications open to any authors in the world’s largest biomedical literature database.

Several hundred comments have been made during a closed pilot in the last few months. But there are over 23 million articles in PubMed, with thousands more pouring in every day, from Tuesday to Saturday. So the chance of coming across an article with comments is still very low.

We’ll show you some interesting ones shortly, though – and you can learn how to look for articles with comments and set up alerts in one of last week’s blog posts. Or you can check out the stream of selected new comments – as well as articles that are trending in PubMed – at the PubMed Commons home page.

If you happen onto an article that has comments, the first sign will be in your search results. There will be a little icon letting you know an article has comments, and how many there are – like this:

Comment icon with number 7

Anyone can read the comments. Members may also have rated their helpfulness, which looks like this on the comment:

8 out of 8 found this helpful

Each comment comes with a “Permalink.” That’s a permanent link you can use to share the comment or cite it. You can find more about how to cite a comment in the PubMed Commons FAQs.

If a comment has cited another article in PubMed, and used particular PubMed Commons formatting to do it, you will see a note letting you know it is mentioned in a PubMed comment.

If you are the author of any publication in PubMed, you are eligible to join PubMed Commons now. PubMed Commons is a pilot, beginning an evaluation phase. Through that time, we’ll be exploring possible ways to widen participation.

Registered members of the Commons can invite others to join. We have developed a wizard to help you and your colleagues work out who can get the ball rolling. Check out the new instructions and get started today.

Then you could rate whether a comment was helpful or not, join in a discussion, or make a new comment on an article. For members of PubMed Commons who are also signed into their NCBI account, the PubMed Commons section looks like this:

Screenshot of member view of PubMed Commons showing name and commenting box

Members have already used PubMed Commons in many ways:

There are more examples in our recent post on how authors are using PubMed Commons. We will be blogging in detail about other ways people are using the Commons, as well as talking about new features under development and offering tips about making comments and using the system.

We’re looking forward to our first Twitter chat soon, too, so keep your eye out on @PubMedCommons and this blog for the announcement.

If you’re an author we hope you will join and contribute by rating comments – and adding your thoughts about scientific publications, too. Thanks again to the hundreds of people who contributed to the development of PubMed Commons this year, and the discussion about it. And welcome to everyone who is new to the Commons: we hope you will visit the home page and find out more!

The PubMed Commons team

More information:

How to join PubMed Commons

PubMed Commons FAQs

PubMed Commons Guidelines (Updated version on 19 December 2013)

Expanding and updating the record: Authors using PubMed Commons

Image of people talking

Soon, the comments being made in PubMed’s pilot scientific discussion forum will be visible to all PubMed users. You can read about that in a previous post. In the meantime, let’s look at how authors are using the Commons with their own scientific publications – and authors’ responses to comments by others.

Last week, there had been just shy of 400 comments live in PubMed. Almost one in five were made by an author of the publication concerned (73/394). The most common contribution – half of the authors’ comments – was adding study data, a link to full free text, further materials or updates related specifically to that publication.

Some typical examples:

For just over another quarter of the comments, authors took the opportunity to link the record of one study with other work they had done previously or subsequently, or information from others they thought provides useful context. Here are a couple of examples:

Authors almost universally did not use the comment function purely to say on their own record that the publication was good (only two did). Just under 10% used it to notify errata, or to notify readers that they now believed the conclusions of their publication are not valid. Here’s an excerpt from one: it’s from Andrew Kniss, an author of a 2006 paper on the effect of an herbicide on sugarbeet bred to be resistant to it.

“This paper (on which I am a co-author) was an early report based solely on greenhouse and laboratory studies. Glyphosate applied to glyphosate-resistant sugarbeet increased disease under greenhouse conditions in this work. Our conclusions with respect to field management of the disease went beyond what the limited data could support. Subsequent research has shown over five field seasons, two growing regions, and 6 sugarbeet cultivars that the effect presented here is unlikely to occur in the field…”

What about responding to comments made by others? With relatively few people currently using PubMed Commons, a lot of responses can’t be expected yet. There have been some interesting exchanges, though, with the seven authors who have engaged with commenters.

These include a discussion with Huber about a method of statistical analysis for sequencing data, and an answer by Meissner to questions by Ward on a 2013 paper from his lab on the cell invasion of a protozoan parasite responsible for several diseases in humans and animals (apicomplexan parasites).

Eyre-Walker and Stoletzki provided a graceful response to a comment by Cherry disagreeing with their interpretation of data and the conclusion that followed: “We thank the author for his insightful comments. Unfortunately Dr. Cherry is correct.”

If you make a comment, you can invite an author of the publication to respond using the “Invite an author to comment” link near the comment box. With that invitation, you both let them know you have commented and enable them to join PubMed Commons at the same time.

If you’re keen to be sure you see any further commenting on that article – or you’re an author wanting to see if there are comments on your articles – we have a blog post showing you how to search for comments and set up alerts with My NCBI.

Follow us here or on Twitter (@PubMedCommons) for more news, and the public release of the comments coming soon.

The PubMed Commons team

More information

Comment search and alert: A PubMed Commons guide

Early developments in the PubMed Commons pilot

How to join  PubMed Commons  (Note – if you are an author of a publication in PubMed Commons, your email may be in the list explained in this post)