Pre-prints as a speedup to scientific communication

Tomorrow, I’ll sit on a panel about Open Data and Open Science as part of Reed’s Digital Scholarship Week.  I am somewhat familiar with these topics in computer science, but I decided to read up on the progress with Open Access in Biology.

As a junior professor trying to get a foothold in a research program, I’ll admit that I haven’t spent a lot of time thinking about Open Science.  In fact, the first thing I did was look up what it meant:

Open science is the movement to make scientific research, data and dissemination accessible to all levels of an inquiring society.                       – Foster Project Website

Ok, this seems obvious,  especially since so much research is funded by taxpayer dollars.  Surprisingly, Open Science is not yet a reality.  In this post, I’ll focus on the speed of dissemination – the idea that once you have a scientific finding, you want to communicate it to the community in a timely manner.

Biology findings are often shared in the form of peer-reviewed journal publications, where experts in the field comment on drafts before they are deemed acceptable for publication.  Peer-review may be controversial and even compromised (just read a few RetractionWatch posts), but in theory it’s a good idea for others to rigorously “check” your work.  However, the peer-review process can be slow. Painfully slow.  Findings are often published months to even years after the fact.

In computer science, my “home” research discipline, it’s a different story.  Computer science research is communicated largely through conferences, which often includes paper deadlines, quick peer-review turnaround times, and a chance to explain your research to colleagues.  Manuscripts that haven’t undergone peer-review yet may be posted to arXiv.org, a server dedicated to over one million papers in physics, mathematics, and other quantitative fields.  Manuscripts submitted to arXiv are freely available to anyone with an internet connection, targeting “all levels of an inquiring society.”

A biology version of the site, BioRxiv.org, was created in 2013 — more than 20 years after arXiv was established.   It only contains about three thousand manuscripts.  What is the discrepancy here?  Why is the field reluctant to change?

Last February, a meeting was held at the Howard Hughes Medical Institute (HHMI) Headquarters to discuss the state of publishing in the biological sciences. The meeting, Accelerating Science and Publication in Biology (appropriately shortened to ASAPbio), considered how “pre-prints” may accelerate and improve research.  Pre-prints are manuscript drafts that have not yet been peer-reviewed but are freely available to the scientific community.  ASAPBio posted a great video overview about pre-prints, for those unfamiliar with the idea.  While the general consensus was that publishing needs to change, there are still some major factors that make biologists reluctant to post pre-prints (see the infographic below).

This is an excellent time to talk open science in Biology.  It has become a hot topic in the last few months (though some in the field have been pushing for open science for years). The New York Times recently wrote about the Nobel Laureates who are posting pre-prints, and The Economist picked up a story about Zika virus experiment results that were released in real time in an effort to help stop the Zika epidemic.

Open Science has the potential to lead to more scientific impact than any journal or conference publication.  The obstacles are now determining what pre-prints mean to an academic’s career – in publishing the manuscripts, determining priority of discovery (meaning “I found this first”), and obtaining grants.  I rely on freely-available data and findings in my own research, yet I’ve never published a pre-print.  After writing this post, I think  I may start doing so.

preprint-opinions-graphicAdditional Sources:

Mick Watson’s 2/22/2016 post about generational change on his blog Opiniomics.

Michael Eisen’s  2/18/2016 post about pre-print posting on his blog it is NOT junk.

Handful of Biologists Went Rogue and Published Directly to Internet, New York Times, 3/15/2016.

Taking the online medicine, The Economist, 3/19/2016.

Networks in Biology (it’s not what you think)

I am currently designing my upper-level undergraduate class I will teach next fall.  The proposed course description* begins with:

Computational Systems Biology

A survey of network models used to gain a systems-level understanding of biological processes.  Topics include computational models of gene regulation, signal transduction pathways, protein-protein interactions, and metabolic pathways…

As a result, I’ve been keeping my eye out for networks (or, mathematically-speaking, graphs) in biology.  I found a fascinating network in this recently-published paper:

Males Under-Estimate Academic Performance of Their Female Peers in Undergraduate Biology Classrooms
Grunspan DZ, Eddy SL, Brownell SE, Wiggins BL, Crowe AJ, et al. (2016) Males Under-Estimate Academic Performance of Their Female Peers in Undergraduate Biology Classrooms. PLoS ONE 11(2): e0148405. doi: 10.1371/journal.pone.0148405

I often see reports on gender bias in computer science, but I somehow thought that biology would be the least gender biased of the STEM disciplines.  I was surprised that this type of bias has been uncovered in biology, and in classes with more female students than male students.  The paper has already been highlighted on sources such as Science Daily, The Atlantic, and the Huffington Post, among others.  The wealth of information in the paper — from the experimental design to the study setting to the final results — warrants an important, broad discussion.

In this post, however, I’ll focus on the networks.

The authors conducted multiple surveys where students nominated the “best performers” in their introductory biology courses at a large American university.  These surveys were given at different parts of the course, and they were conducted across three different iterations of the same undergraduate biology class.  Figure 1 of the paper shows two networks displaying two surveys from the same class, six weeks apart.

 

 

journal.pone.0148405.g001

Figure 1. Unequal distribution of peer perception of mastery of content among genders grows over the term.  Grunspan et al., PLOS ONE 2016.

These networks show the students (represented as nodes in the graph) in a particular class, and “votes” as directed edges from nominators to nominees.  Male students are shown in green, and female students are shown in orange.  The size of nodes indicates the number of nominations received by each student.  The structure of these networks is striking.  There are many students who do not nominate anyone and are not nominated by anyone, resulting in “singleton” nodes.  In both networks, there is a general cohort of students that receive nominations; however the distribution of these nominations are much more skewed in the second survey.

The intuitive trend that we see in these graphs is that “the green nodes tend to get bigger” corresponding to a larger proportion of nominations go to male students.  However we see female students also receive more nominations in the second survey compared to the first.  The authors quantify these aspects using exponential-family random graph models (ERGMs) to assign coefficients on model statistics relating to gender, outspokenness, and grade.  They found a specific gender bias, that male students tend to nominate other male students, after controlling for grade and outspokenness.  Female students, on the other hand, do not exhibit a gender bias toward nominating males (or females for that matter), after controlling for these factors.

There are many, many other factors that may contribute to these observations, and some are noted in the paper.  The courses were taught (and in some cases co-taught) by four male instructors and only one female instructor, the classes ranged in size from 196 to 760 students, one class employed “random call” lists rather than calling on raised hands.  Besides outspokenness, interactions in lab sections and outside class would undoubtedly affect students’ perceptions.  This paper opens a tremendously important conversation about implicit gender bias in the classroom, even in majors with more female students than male students.  As the paper concludes,

This gender biased pattern in celebrity was experienced by over 1,500 students in our analyses.  This number is striking, but less worrisome than the millions of students who attend college STEM classes that may perpetuate the same biases described here.

Grunspan et al., PLOS ONE 2016.

* Pending approval of various college committees – it may change

How many authors is too many?

Nature recently published a quick blurb about a paper on fruit fly genetics that has sent social media abuzz.  Why?  Because the paper, published in G3: Genes Genomes Genetics, lists over 1,000 authors.  Further, more than 900 of these authors are undergraduates and members of the Genomics Education Partnership, an organization that has posted a record of the commentary on the author number. The author list, which spans the first three pages of the PDF, is shown below.

    fly2     fly3

The paper has sparked a larger debate about the role of training and education in research, particularly when it comes to undergraduate involvement.  Alongside the paper, the authors also released a blog post about undergraduate-empowered research in the Genetics Society of America’s Genes to Genomes blog.  This is the first paper I’ve seen that lists a blog post as supporting information.

I can see arguments on both sides. On one hand, crowd-sourcing allows us accomplish tasks impossible for a single person to execute. The computer scientist in me loves this aspect of the story.  Here, “the crowd” is the sea of undergraduates that edited and annotated a DNA sequence (the Muller F element, or the “dot” chromosome) in fruit flies by analyzing and integrating different types of data.  Unlike papers that use Mechanical Turk to collect data, where the crowd is typically non-experts, this particular crowd has learned a set of specialized skills that facilitated the research.  Undergrads dirtied their hands with real data and learned valuable insights about how to conduct research.  The educator in me finds the endeavor incredibly impactful for the scientists-in-training.  On the other hand, being buried in the author list makes one’s contributions look meaningless.  What does it mean to be a co-author on such a paper?  If the Genomics Education Partnership consisted of only a few dozen undergraduates, would it be better?  Some of these questions are discussed in the Neuro DoJo blog post.  The academic “I-want-to-get-tenure” in me cringes at the thought that good research may be down-weighted because the author contributions were unclear if I were in the middle of the author list.

I think that the undergraduates from the Genomics Education Partnership did conduct research that contributed to the paper, and they should be credited in some way.  It seems that in the age of crowdsourcing, perhaps there needs to be an intermediate between authorship and acknowledgement that indicates a collective contribution from a group of people (e.g. students in a class or members of a consortium).

Responses to the sexist review by PLOS One

There has recently been a lot of attention on the journal PLOS One and their handling of a heavily gender-biased review received by an evolutionary biologist on a manuscript about gender differences in the Ph.D. to Postdoc academic transition.  PLOS One has taken a few actions, including asking the academic editor who handled the manuscript to step down from the editorial board and removing the offending reviewer from their database.  Dr. Michael Eisen, one of the founders of PLOS, has provided interesting commentary on the subject.

What’s just as troubling is that the reviewer clearly used his personal assessment of not only the authors’ gender but also their “junior” academic status in his criticism.  A blog that focuses on manuscript retractions has another summary of the issue.  Dr. Fiona Ingleby, one of the authors of the manuscript, tells the authors,

Megan and Fiona are pretty unambiguous names when it comes to guessing gender.  But in fact, the reviewer acknowledged that they had looked up our websites prior to reading the MS (they said so in their review). They used the personal assessment they made from this throughout their review – not just gender, but also patronising comments throughout that suggested the reviewer considered us rather junior.  – Fiona Ingleby

This has raised some major issues about the peer-review process, including whether a reviewer’s identity should ever be revealed in a single-blind or double-blind review.  Dr. Eisen addresses this in his blog posted above.  Dr. Zuleyka Zevallos wrote about removing publishing bias in science, here in the form of sexism. In response to Dr. Eisen’s post, she explicitly addresses the accountability that institutions and publishers need to have in place.

It will be interesting to see how PLOS One and other publishers address this now-viral issue, especially by the changes to the peer-review process.  Some have noted that the PLOS One editor failed in his/her duty by returning this gender-biased review to the authors instead of disregarding it, and it wasn’t necessarily a problem with the procedures.  But more and more journals are moving to different review styles, including Nature’s experiment with double-blind peer review.  If the PLOS One review had been double-blind, the reviewer may have been able to guess the gender of the authors but would not have been able to verify that, and especially not verify their current academic positions.