Commenting on PubMed: A Successful Pilot

endpilot_blogimgWe are pleased to announce that PubMed Commons is here to stay! After developing and piloting the core commenting system for PubMed, a pilot of journal clubs was added. And we have completed a major internal evaluation of the use of the Commons. We aim to publish that soon, so stay tuned to this blog or Twitter for news on that.

PubMed Commons provides a forum for scientific discourse that is integrated with PubMed, a major database of citations to the biomedical literature. Any author of a publication in PubMed is eligible to join and post comments to any citation.

More than 9,500 authors have joined PubMed Commons – and they have posted over 4,000 comments to more than 3,300 publications, mostly on recent publications. Commenting has plateaued, so the volume is low. But the value of comments has remained high. And comments often attract a lot of attention.

About half the comments are on clinical or health-related publications. Members have been using PubMed Commons to:

  • Update and expand the public record, for instance by pointing to new data, relevant publications, or alternative interpretations
  • Note corrections and retractions to publications
  • Post discussion and critique, either directly or via links to blog posts and other platforms
  • Provide links to datasets, code, or publicly accessible versions of publications
  • Call attention to issues affecting reproducibility, such as cell line misidentification

Authors posting to their own publications contribute about one in five comments. About one-third of these have been replies to questions or discussion from others. Since the PubMed Commons Team began notifying authors of comments on their publications, the proportion of comments with author replies has increased. However, the rate of reply remains below 10%. We will keep working on ways to encourage more author response.

Just a year ago, we introduced a new mechanism to capture the synthesis of journal club discussions of scientific publications. PubMed Commons Journal Clubs have full commenting privileges and profile pages to provide background information about the club. To date, 20 journal clubs have joined. These institutional, virtual, and hybrid journal clubs represent a range of clinical and biomedical disciplines. They have become a critical and vibrant part of PubMed, and we are planning more support for this initiative.

PubMed will shortly turn 20. It has become a major resource for finding biomedical and health-related literature. There are now more than 25 million citations. And there were more than 2.7 billion searches in the last year – that’s more than 7 million searches a day.

That means that comments have a large potential audience, and the interest in them is growing. Visits to the PubMed Commons homepage have nearly doubled, from 1.2 million in the first half of 2014 to 2.3 million in the first half of 2015.

We believe the commenting function addresses a critical need, for PubMed and for the development of biomedical research. So a big “thank you” from us to everyone who has contributed their time and energy to supporting the Commons and commenting at PubMed.

Just because the pilot has ended, doesn’t mean PubMed Commons will stop evolving. With the pilot over, we’re working on an application program interface (API) that will enable hosting of PubMed comments on third-party sites. And other new features are in the pipeline. Meanwhile, anyone can submit suggestions and feedback by using the “Write to the Help Desk” link at the bottom of NCBI pages.

 

Ready to get involved? Visit our Getting Started page to learn more about how to join and participate in PubMed Commons – or start here if you would like your Journal Club to join in.

 

The PubMed Commons Team

Introducing PubMed Commons Journal Clubs

Around the world, the journal club is a cornerstone engagement with the scholarly literature. Whether in face-to-face meetings or on social media platforms, researchers, physicians, and trainees gather to debate and converse about publications. Participants share their views on methods and interpretations of results. They discuss how publications fit into a broader context or might inform their own research or practice.

In short, the journal club can represent a major intellectual investment – and a long-standing form of post-publication evaluation.

Yet often, the analyses and ideas don’t travel far beyond core participants. Digital records and virtual journal clubs can help deliver the discourse to others. Still, wouldn’t it be fantastic if more of us could see what these groups have to say?

Today we’re excited to announce the launch of PubMed Commons Journal Clubs. These accounts will allow groups to establish their own identity on PubMed Commons. Journal clubs will be able to share key points, questions, and summaries from their discussions – right below citations in PubMed.

Bringing local discussion to the global Commons

Gary Ward is a professor in the Department of Microbiology and Molecular Genetics at the University of Vermont. (He is also a member of the external working group providing feedback on PubMed Commons.) His lab studies Toxoplasma gondii, a protozoan parasite. It’s widespread among humans and other mammals and can cause serious illness for those who are pregnant or have weakened immune systems.

UVM Toxo Journal Club covers work on parasites like Toxoplasma gondi (Image courtesy of Aoife Heaslip)

UVM Toxo Journal Club covers work on parasites like Toxoplasma gondii.

Ward also facilitates the University of Vermont (UVM) Toxoplasma Journal Club, a group of grad students, postdocs, technicians and faculty who do research on T. gondii. “We try to review both classic papers (why is this a classic in our field?) and very recent findings in the world of parasite cell biology.”

“We each take turns picking a paper and leading the discussion,” he explains. Last year, the group added a new step. “Immediately after the journal club, the discussion leader is responsible for drafting a PubMed Commons comment that summarizes the key points of the discussion. The comment is revised based on feedback from the group and then posted.”

Ward notes the direct benefit of this process for participants. “Having to summarize our meeting in the form of a comment forces us to distill the many things that were discussed into the two or three most important points. The ability to focus one’s critique/comments in this way is a great skill for grad students and postdocs to learn, and for the rest of us to practice.”

He also thinks that journal clubs have something more to offer to the scientific community at large. “Other than the journal club setting, how often does a paper get read critically from beginning to end by 10-12 informed readers who then discuss it at length as a group?  This kind of collective discussion is a great way to surface the strengths and weaknesses of a study and to identify connections to other work.”

“Posting journal club comments in PubMed Commons adds depth to the literature and may give the reader a different perspective on the work,” Ward explains. “They will be particularly useful when they stimulate the authors to engage in a PubMed Commons dialog. If our journal club had a particular question about the paper, it is likely that other readers will as well.”

The UVM Toxoplasma Journal Club has a great example of just how that can happen. 

1000х1000

Expediting lab-to-lab communications

Three thousand miles away from Burlington, Vermont, Markus Meissner’s group at the University of Glasgow had worked out a method to target genes in T. gondii for conditional deletion. They applied the approach to look at how the parasite infected host cells. Meissner’s group found that actin was essential to T. gondii survival – but not because the parasites couldn’t invade host cells. Rather, they argued, the parasites die because they lose a specialized part of the cell called the apicoplast.

“In our discussion of this paper,” Ward notes, “a new graduate student in the group suggested a great idea on how to test this hypothesis.”

The apicoplast is essential for survival of Plasmodium falciparum, the parasite that causes malaria. However, blood-stage P. falciparum can live without an apicoplast if supplied with isopentenyl pyrophosphate (IPP), which is normally produced in the apicoplast.

The journal club asked in their comment: Could T. gondii lacking actin survive if given IPP?

Meissner replied. His lab had considered the experiment but scrapped the idea after learning from other experts that IPP treatment doesn’t have the same effect in T. gondii as it does P. falciparum.

This instance illustrates how PubMed Commons can initiate useful exchanges. “Now anyone wondering if IPP rescues an apicoplast defect in T. gondii can discover that it doesn’t,” says Ward. “That information had not previously been captured, but now it is in the form of a PubMed Commons comment.”

Calling journal clubs to join the discourse

NephJC brings discussions from the nephrology (& related specialties) Twitter community.

NephJC brings discussions from the nephrology (& related specialties) Twitter community.

With PubMed Commons Journal Clubs, we’re hoping to see groups and individuals engaging on PubMed Commons and beyond. We’re pleased to welcome the UVM Toxo Journal Club, NephJC, and CREBP Journal Club as our first PubMed Commons Journal Clubs.

To encourage connections, PubMed Commons Journal Clubs will have profile pages on PubMed Commons. These pages will provide descriptions of the groups and ways to connect with them outside PubMed Commons (click the Journal Club images in this post to see their pages). We’re also starting a Facebook page to offer a space for group members to start sharing their ideas (link coming soon). We’ll be exploring other ways to help groups network, as we build and develop the PubMed Commons Journal Clubs community.

CREBP Journal Club hails from the Centre for Research in Evidence-Based Practice at Bond University.

CREBP Journal Club at Bond University looks at the gaps between evidence and current clinical practice.

PubMed Commons Journal Club accounts are currently open to journal clubs discussing literature for research, graduate and postgraduate education, or continuing professional education. Applications will need to be supported by PubMed Commons members who participate in the group’s discussions. For more information or to apply for a Journal Club account, email pubmed.commons@ncbi.nlm.nih.gov.

The PubMed Commons Team

Unveiling a new look – and more – for PubMed Commons

PubMed Commons set the stage for commenting on any publication in PubMed, the world’s largest searchable database of biomedical literature. Lately we’ve been tackling infrastructure and design to improve the user experience and support the PubMed Commons community. Those developments are now live on PubMed and PubMed Commons. Here’s what you can expect from the PubMed Commons update.

Center stage

Some changes are likely to jump out for frequent PubMed Commons users.

@PubMedCommonsWe’ve adopted new artwork for our blog, Twitter account, and homepage. We’re going for a clear, unified identity across platforms, one that we hope will be recognizable wherever you see us.

We’ve made some modifications to streamline our homepage. We’ve consolidated information about joining and using PubMed Commons in a single page to help you get started. You’ll also find a synopsis of our most recent blog post at the top of our homepage to help you stay up-to-date on PubMed Commons.

For several months, comment rating has given members the chance to weigh in on what comments they find useful. Visitors to PubMed can see these ratings alongside comments. Ratings are a key element in calculating the comment and commenter scores that determine the appearance of comments in the “Selected comments” stream on our homepage.

Some new site modifications will highlight your contributions to PubMed Commons. On our homepage, “Top comments now will feature the top three recent comments. On PubMed records, “Selected comments” (from our homepage stream) prompt the appearance of an icon above abstracts, directing readers to comments below.

This new icon will appear above some abstracts in PubMed.

This new icon will appear above some abstracts in PubMed.

For a while, we’ve selected highly-rated comments to post to our Twitter stream. Starting today, the most recent tweet about a PubMed Commons comment appears on the homepage for PubMed searches. Check it out!

Behind the scenes

Some key changes in the PubMed Commons development won’t be visible on the PubMed Commons site. We’ve improved our “Invite an author” function. It looks the same as before, but we hope you’ll encounter fewer errors when inviting authors to join or comment. (If you do encounter errors, please let us know by using the “Write to the Help Desk” link, found at the bottom of every NCBI page.)

inviteIn response to community feedback, we are also notifying corresponding authors of comments on their publications and inviting them to join PubMed Commons. We’ve been at it for two months, and we’re encouraged by the increase in author responses. We will continue our notification process, but we still encourage members to notify a publication’s author(s) when commenting.

More to come

We’re not done yet. In May, an external working group for PubMed Commons was established. The members offered great feedback and ideas on where PubMed Commons is and where it’s going. But more on that later…

The cornerstone for the continued growth of PubMed Commons is you! Here’s how you can get involved:

  • Learn more about the PubMed Commons pilot.
  • Are you an author of a PubMed-indexed publication? Join PubMed Commons!
  • Already a PubMed Commons member?
    • Post a comment! Log in to My NCBI, find a PubMed citation, and start typing. Not sure what to post? Check out these examples of how authors have been using PubMed Commons.
    • Rate comments. It only takes a moment, and as we discussed above, you can influence what’s highlighted on PubMed Commons – and PubMed.
    • Invite your colleagues!
  • Finally, help us spread the word about PubMed Commons!

Thanks for your support and contributions to PubMed Commons. We look forward to seeing where this community takes us next.

The PubMed Commons Team

Meet PubMed Commons: The new comments forum in PubMed

Image of comments icon

If you are one of the millions of people who visit PubMed today, be on the look-out for something different. On each abstract page, there’s now a section called PubMed Commons. It’s a forum for scientific discussion on publications open to any authors in the world’s largest biomedical literature database.

Several hundred comments have been made during a closed pilot in the last few months. But there are over 23 million articles in PubMed, with thousands more pouring in every day, from Tuesday to Saturday. So the chance of coming across an article with comments is still very low.

We’ll show you some interesting ones shortly, though – and you can learn how to look for articles with comments and set up alerts in one of last week’s blog posts. Or you can check out the stream of selected new comments – as well as articles that are trending in PubMed – at the PubMed Commons home page.

If you happen onto an article that has comments, the first sign will be in your search results. There will be a little icon letting you know an article has comments, and how many there are – like this:

Comment icon with number 7

Anyone can read the comments. Members may also have rated their helpfulness, which looks like this on the comment:

8 out of 8 found this helpful

Each comment comes with a “Permalink.” That’s a permanent link you can use to share the comment or cite it. You can find more about how to cite a comment in the PubMed Commons FAQs.

If a comment has cited another article in PubMed, and used particular PubMed Commons formatting to do it, you will see a note letting you know it is mentioned in a PubMed comment.

If you are the author of any publication in PubMed, you are eligible to join PubMed Commons now. PubMed Commons is a pilot, beginning an evaluation phase. Through that time, we’ll be exploring possible ways to widen participation.

Registered members of the Commons can invite others to join. We have developed a wizard to help you and your colleagues work out who can get the ball rolling. Check out the new instructions and get started today.

Then you could rate whether a comment was helpful or not, join in a discussion, or make a new comment on an article. For members of PubMed Commons who are also signed into their NCBI account, the PubMed Commons section looks like this:

Screenshot of member view of PubMed Commons showing name and commenting box

Members have already used PubMed Commons in many ways:

There are more examples in our recent post on how authors are using PubMed Commons. We will be blogging in detail about other ways people are using the Commons, as well as talking about new features under development and offering tips about making comments and using the system.

We’re looking forward to our first Twitter chat soon, too, so keep your eye out on @PubMedCommons and this blog for the announcement.

If you’re an author we hope you will join and contribute by rating comments – and adding your thoughts about scientific publications, too. Thanks again to the hundreds of people who contributed to the development of PubMed Commons this year, and the discussion about it. And welcome to everyone who is new to the Commons: we hope you will visit the home page and find out more!

The PubMed Commons team

More information:

How to join PubMed Commons

PubMed Commons FAQs

PubMed Commons Guidelines (Updated version on 19 December 2013)

Expanding and updating the record: Authors using PubMed Commons

Image of people talking

Soon, the comments being made in PubMed’s pilot scientific discussion forum will be visible to all PubMed users. You can read about that in a previous post. In the meantime, let’s look at how authors are using the Commons with their own scientific publications – and authors’ responses to comments by others.

Last week, there had been just shy of 400 comments live in PubMed. Almost one in five were made by an author of the publication concerned (73/394). The most common contribution – half of the authors’ comments – was adding study data, a link to full free text, further materials or updates related specifically to that publication.

Some typical examples:

For just over another quarter of the comments, authors took the opportunity to link the record of one study with other work they had done previously or subsequently, or information from others they thought provides useful context. Here are a couple of examples:

Authors almost universally did not use the comment function purely to say on their own record that the publication was good (only two did). Just under 10% used it to notify errata, or to notify readers that they now believed the conclusions of their publication are not valid. Here’s an excerpt from one: it’s from Andrew Kniss, an author of a 2006 paper on the effect of an herbicide on sugarbeet bred to be resistant to it.

“This paper (on which I am a co-author) was an early report based solely on greenhouse and laboratory studies. Glyphosate applied to glyphosate-resistant sugarbeet increased disease under greenhouse conditions in this work. Our conclusions with respect to field management of the disease went beyond what the limited data could support. Subsequent research has shown over five field seasons, two growing regions, and 6 sugarbeet cultivars that the effect presented here is unlikely to occur in the field…”

What about responding to comments made by others? With relatively few people currently using PubMed Commons, a lot of responses can’t be expected yet. There have been some interesting exchanges, though, with the seven authors who have engaged with commenters.

These include a discussion with Huber about a method of statistical analysis for sequencing data, and an answer by Meissner to questions by Ward on a 2013 paper from his lab on the cell invasion of a protozoan parasite responsible for several diseases in humans and animals (apicomplexan parasites).

Eyre-Walker and Stoletzki provided a graceful response to a comment by Cherry disagreeing with their interpretation of data and the conclusion that followed: “We thank the author for his insightful comments. Unfortunately Dr. Cherry is correct.”

If you make a comment, you can invite an author of the publication to respond using the “Invite an author to comment” link near the comment box. With that invitation, you both let them know you have commented and enable them to join PubMed Commons at the same time.

If you’re keen to be sure you see any further commenting on that article – or you’re an author wanting to see if there are comments on your articles – we have a blog post showing you how to search for comments and set up alerts with My NCBI.

Follow us here or on Twitter (@PubMedCommons) for more news, and the public release of the comments coming soon.

The PubMed Commons team

More information

Comment search and alert: A PubMed Commons guide

Early developments in the PubMed Commons pilot

How to join  PubMed Commons  (Note – if you are an author of a publication in PubMed Commons, your email may be in the list explained in this post)

PubMed Commons going public soon

Image

It’s been a month since the beta launch of PubMed Commons, the pilot system that enables authors’ discussion and sharing of information about publications in PubMed.

The first public version of the PubMed Commons pilot will be released in the coming weeks. All users of PubMed will be able to see and cite comments.

We’re grateful to the hundreds of you who joined the closed phase of testing – especially for your patience with the inevitable bugs in a beta system. Your activity and feedback have made the system better in several ways:

  • There will be a simplified way for eligible authors to join – including all those with current author email addresses in PubMed and PubMed Central;
  • A permanent citable link will be available;
  • We have increased the space in individual comment boxes (up to 8000 characters), and the new release will have warnings if you’re getting close;
  • Article helpfulness ratings are influencing the comment stream on the home page;
  • New specific guidelines have been released to address concerns reported by members.

More features are in the pipeline. There will be an increased use of data from the helpfulness ratings to make the display of comments more helpful – and we will support sharing on social media. Also in development is an application programing interface (API) to integrate comments from PubMed Commons into other websites.

Exploring options to ensure a vibrant and useful forum for discussion of scientific publications will be a key focus of the next stage of the pilot. We will be exploring ways to expand people’s access to commenting and rating helpfulness, for example through group accounts. Enhancing the value of PubMed for users is critical to the success of PubMed Commons, and we are relying on the community to help shape the conduct and system it wants to see.

We are establishing a working group to advise us during the next stages of the pilot and its evaluation. And we look forward to community discussion, too.

The upcoming release marks the start of evaluation of PubMed Commons. Evaluation results will be considered at 3 months and 6 months, with the final report on the pilot anticipated after 9 months. Three key areas will be our focus: uptake and reputation, quality and impact of comments and discussion, and sustainability.

We will be blogging more about comments being made in the Commons, what we’re learning, and explaining more about aspects of the system. We will be trying out a Twitter chat too, so keep your eye out on @PubMedCommons for the announcement.

Thanks again to everyone who has contributed to PubMed Commons and the discussion about it. We look forward to an even wider discussion soon. Stay tuned to this blog or @PubMedCommons for news of the Commons going public.

The PubMed Commons team

More information

How to join PubMed Commons (Note – if you are an author of a publication in PubMed Commons, your email may be in the list explained in this post)

Early developments in the PubMed Commons pilot