PubMed comments & their continuing conversations

bubbles_3aWe have many options for communication. We can choose platforms that fit our style, approach, and time constraints. From pop culture to current events, information and opinions are shared and discussed across multiple channels. And scientific publications are no exception.

PubMed Commons was established to enable commenting in PubMed, the largest biomedical literature database. In the past year, commenters posted to more than 1,400 publications. Of those publications, 80% have a single comment today, and 12% have comments from multiple members. The conversation carries forward in other venues.

Sometimes comments pull in discussion from other locations or spark exchanges elsewhere.Here are a few examples where social media prompted PubMed Commons posts or continued the commentary on publications.

Debating disease association

On June 3, 2016, Daniel MacArthur took to Twitter to express his skepticism of a report describing an association between a gene mutation and familial multiple sclerosis published in the journal Neuron. His critique stirred a bit of interest. A few days later, he posted a comment, co-written with Eric Minikel, to PubMed Commons. MacArthur and Minikel highlighted, “Enrichment in cases over controls is one important criterion for establishing pathogenicity of sequence variants.” The comment prompted more discussion on Twitter.

Over the following days, author Carles Vilariño-Güell responded, and MacArthur and Minikel replied. Shortly Chris Cotsapas posted a comment on behalf of the International Multiple Sclerosis Genetics Consortium. His comment summarized an attempt to validate the findings, linking to results posted in a bioRχiv preprint. With Simon Heath, Daniel Weeks noted further concerns in an August comment on the journal’s website, which he linked from PubMed Commons.

As the critiques unfolded, some readers commented on blog posts highlighting the results, (such as here and here) to point to the comments on PubMed. In September, STAT published a story reviewing the concerns that had been raised on PubMed Commons and elsewhere. In October, Neuron published letters from the International Multiple Sclerosis Genetics Consortium and Minikel and MacArthur, as well as a response from the authors. In an accompanying editorial note, the editors remark that the peer-reviewed letters offer “an important complement to other forms of commentary” including social media, PubMed Commons, and the journal’s online comments section.

Comments also sparked discussion of topics beyond the specific gene variant in question.

Self-correcting statements

In July 2016, a publication co-authored by Thomas Nichols reported on an artifact that might give rise to high false-positive rates in functional magnetic resonance imaging (fMRI) analyses. Across blogs and professional publications, there was consideration of what the findings meant for neuroscience research. But some in the community thought results were being too broadly extrapolated to all fMRI studies, not just the specific issues examined.

As the publication was discussed online, the authors recognized that some wording was being interpreted in ways they had not anticipated. So they asked to publish an erratum. That was initially rejected by the journal, since there was no change to the results or conclusions. Nichols published the note on his blog. Following an exchange on Twitter, he subsequently posted a comment on PubMed Commons to make a more circumspect significance statement.

At least one blogger updated a post to reflect the authors’ statement.

Although the journal ultimately published a correction a month later, PubMed Commons enabled authors to rapidly communicate a reframed interpretation of their work.

Replicating and reviewing search strategies

Comments can initiate discussion of specific results and interpretations. But they can also serve as a jumping off point to evaluate approaches and highlight practices.

Literature search strategies lie at the core of systematic reviews and meta-analyses. Melissa Rethlefsen posted a comment describing an attempt to replicate the search strategy reported in a meta-analysis. She noted key missing information such as date ranges. She concluded: “This study highlights the need for more accurate and comprehensive reporting needed for search strategies in systematic reviews and other literature search-based research syntheses, and the need for better peer review of search strategies by information specialists/medical librarians.”

One library used this example to encourage the use of structured reporting guidelines for systematic reviews and meta-analyses.

Another library used the comment to illustrate the importance of reviewing search strategies. And medical librarians and researchers chimed in on Twitter.

Extending the reach of scientific discourse

As you browse the web, you might just run across a mention of a comment on PubMed. Blog authors and readers might mention comments, as they have about a genetic variant associated with body mass index , ‘bad luck’ and cancer, or the occurrence of amphetamines in water systems. They might even appear in the references list, such as a roundup of publications on cancer risk or a look at psychological debriefing after traumatic events. Perhaps the most talked-about comments were those from Jennifer Doudna and Emmanuelle Charpentier on a perspective of the history of CRISPR. The comments were shared and discussed on social media. They were also mentioned in several blog posts and articles, including ones from news outlets such as the Washington Post.

PubMed Commons offers a key place to quickly anchor critical points for future readers to see­­—in one of biomedical science’s most heavily used resources. On a typical day in 2016, 1.6 million users ran 2.5 million web searches on PubMed and viewed more 8 million records.

Through October 31, 2016, PubMed Commons had 10,632 members. They’ve posted 5,739 comments to 4,595 publications. Want to join in? Check out our Get Started page for more information!

The PubMed Commons Team

“You found this helpful:” What people like in comments on PubMed

Commented rated "You found this helpful"

Rating is a critical function for a commenting system. It’s a key way a forum’s members can encourage the kind of discussion they want to see – and discourage what they don’t find helpful.

Increasingly, ratings are influencing PubMed Commons, the new commenting system in PubMed. There’s a selected comment stream on the PubMed Commons landing page. We’ve also started sending out tweets when new comments start ranking well on several factors.

The rating options are only available for comments – and you can’t rate your own comment. PubMed authors who have joined PubMed Commons can click on “yes” or “no” to the question, “Was this helpful?”

Although it’s early days, ratings are already sending some messages. One of those is what people don’t find helpful: the kind of comment that praises an article without saying why – or that’s only communicating something that is already clear in the abstract.

The message seems to be: use the comment function when you have something specific to say about, or add to, the publication. Making a similar comment repeatedly also tends to be unpopular, as is self-promotion.

Which comment has been the most popular so far? It’s a comment by Gonzalo Otazu in December. Otazu calls into question the validity of statistics in a paper that got a lot of media attention: a study suggesting that mice can inherit fear of smells associated with traumatization of their parents.

As well as good critiques, people have rated discussions positively – especially when the publication’s authors respond or encourage discussion about their paper. Some recent examples:

Integrating outside discussions in journal clubs and blog posts with PubMed Commons comments gets positive ratings, even when the authors don’t participate.

You may be able to invite yourself with our “Get started” wizard. If your email isn’t there, one of your colleagues should be able to join and then invite you. Any member of PubMed Commons can invite others – including an author of a paper that’s been commented on.

You can check out the selected comments at the PubMed Commons landing page – one click away from PubMed’s home page – or follow us on Twitter. And you can find out how to set up alerts on topics, articles or authors you’re interested in via our “Comment search and alert” guide.

From now on, the quality of PubMed Commons will rely on both the comments and the ratings. You need to be a member of PubMed Commons to rate articles. If you’re an author of a publication in PubMed, you’re eligible.

So join today – to discuss the biomedical literature or, just as importantly, to rate the comments.

The PubMed Commons team

More information

Meet PubMed Commons: The new comments forum in PubMed

Join PubMed Commons

Comment search and alert: A PubMed Commons guide