bims-skolko Biomed News
on Scholarly communication
Issue of 2019‒06‒02
23 papers selected by
Thomas Krichel
Open Library Society


  1. Gigascience. 2019 Jun 01. pii: giz053. [Epub ahead of print]8(6):
      BACKGROUND: The academic publishing world is changing significantly, with ever-growing numbers of publications each year and shifting publishing patterns. However, the metrics used to measure academic success, such as the number of publications, citation number, and impact factor, have not changed for decades. Moreover, recent studies indicate that these metrics have become targets and follow Goodhart's Law, according to which, "when a measure becomes a target, it ceases to be a good measure."RESULTS: In this study, we analyzed >120 million papers to examine how the academic publishing world has evolved over the last century, with a deeper look into the specific field of biology. Our study shows that the validity of citation-based measures is being compromised and their usefulness is lessening. In particular, the number of publications has ceased to be a good metric as a result of longer author lists, shorter papers, and surging publication numbers. Citation-based metrics, such citation number and h-index, are likewise affected by the flood of papers, self-citations, and lengthy reference lists. Measures such as a journal's impact factor have also ceased to be good metrics due to the soaring numbers of papers that are published in top journals, particularly from the same pool of authors. Moreover, by analyzing properties of >2,600 research fields, we observed that citation-based metrics are not beneficial for comparing researchers in different fields, or even in the same department.
    CONCLUSIONS: Academic publishing has changed considerably; now we need to reconsider how we measure success.
    Keywords:  Goodhart’s Law; academic publishing metrics; big data; data science; science of science; scientometrics
    DOI:  https://doi.org/10.1093/gigascience/giz053
  2. F1000Res. 2018 ;7 1863
      Background: Several meta-research studies and benchmarking activities have assessed how comprehensively and timely, academic institutions and private companies publish their clinical studies. These current "clinical trial tracking" activities differ substantially in how they sample relevant studies, and how they follow up on their publication. Methods: To allow informed policy and decision making on future publication assessment and benchmarking of institutions and companies, this paper outlines and discusses 10 variables that influence the tracking of timely publications. Tracking variables were initially selected by experts and by the authors through discussion. To validate the completeness of our set of variables, we conducted i) an explorative review of tracking studies and ii) an explorative tracking of registered clinical trials of three leading German university medical centres. Results: We identified the following 10 relevant variables impacting the tracking of clinical studies: 1) responsibility for clinical studies, 2) type and characteristics of clinical studies, 3) status of clinical studies, 4) source for sampling, 5) timing of registration, 6) determination of completion date, 7) timeliness of dissemination, 8) format of dissemination, 9) source for tracking, and 10) inter-rater reliability. Based on the description of these tracking variables and their influence, we discuss which variables could serve in what ways as a standard assessment of "timely publication". Conclusions: To facilitate the tracking and consequent benchmarking of how often and how timely academic institutions and private companies publish clinical study results, we have two core recommendations. First, the improvement in the link between registration and publication, for example via institutional policies for academic institutions and private companies. Second, the comprehensive and transparent reporting of tracking studies according to the 10 variables presented in this paper.
    Keywords:  clinical studies; follow-up; private companies; registries; trial tracking; trials; university medical centers
    DOI:  https://doi.org/10.12688/f1000research.17022.1
  3. Obstet Gynecol. 2019 Jun;133(6): 1081-1083
      Peer review is the major method used by the scientific community to evaluate manuscripts and decide what is suitable for publication. However, this process in its current design is not bulletproof and is prone to reviewer and editorial bias. Its lack of objectivity and transparency raise concerns that manuscripts might be judged based on interests irrelevant to the content itself and not on merit alone. This commentary reviews some of the most common biases that could potentially affect objective evaluation of a manuscript and proposes alternatives to the current single-blind peer review process that is being used by most scientific journals, including Obstetrics & Gynecology. By rethinking and tackling the shortcomings of the current methodology for peer review, we hope to create a discussion that will eventually lead to improving research and, ultimately, patient care.
    DOI:  https://doi.org/10.1097/AOG.0000000000003260
  4. Acad Med. 2019 May 21.
      Traditional peer-review remains the gold standard for assessing the merit of scientific scholarship for publication. Challenges to this model include reliance on volunteer contributions of individuals with self-reported expertise, lack of sufficient mentoring and training of new reviewers, and the isolated, non-collaborative nature of individual reviewer processes.The authors participated in a AAMC peer-review workshop in November 2015, and were intrigued by the process of group peer-review. Subsequent discussions led to shared excitement about exploring this model further. The authors worked with the staff and editors of Academic Medicine to perform a group review of four submitted manuscripts, documenting their iterative process and analysis of outcomes, in order to define an optimal approach to performing group peer-review.Individual recommendations for each manuscript changed as a result of the group review process. The group process led to more comprehensive reviews than each individual reviewer would have submitted independently. The time spent on group reviews decreased as the process became more refined. Recommendations aligned with journal editor findings. Shared operating principles were identified as well as clear benefits of group peer-review for reviewers, authors and journal editors.The authors plan to continue to refine and codify an effective process for group peer-review. They also aim to more formally evaluate the model, with inclusion of feedback from journal editors and authors, and compare feedback from group peer reviews to individual reviewer feedback. Finally, models for expansion of the group peer-review process are proposed.
    DOI:  https://doi.org/10.1097/ACM.0000000000002804
  5. J Pathol Inform. 2019 ;10 16
      Background: Professionals and trainees in the medical and scientific fields may receive high e-mail volumes for conferences and journals. In this report, we analyze the amount and characteristics of unsolicited e-mails for journals, conferences, and webinars received by faculty and trainees in a pathology department at an academic medical center.Methods: With informed consent, we analyzed 7 consecutive days of e-mails from faculty and trainees who voluntarily participated in the study and saved unsolicited e-mails from their institutional e-mail address (including junk e-mail folder) for medical/scientific journals, conferences, and webinars. All e-mails were examined for characteristics such as reply receipts, domain name, and spam likelihood. Journal e-mails were specifically analyzed for claims in the message body (for example, peer review, indexing in databases/resources, rapid publication) and actual inclusion in recognized journal databases/resources.
    Results: A total of 17 faculty (4 assistant, 4 associate, and 9 full professors) and 9 trainees (5 medical students, 2 pathology residents, and 2 pathology fellows) completed the study. A total of 755 e-mails met study criteria (417 e-mails from 328 unique journals, 244 for conferences, and 94 for webinars). Overall, 44.4% of e-mails were flagged as potential spam by the institutional default settings, and 13.8% requested reply receipts. The highest burden of e-mails in 7 days was by associate and full professors (maximum 158 or approximately 8200 per year), although some trainees and assistant professors had over 30 e-mails in 7 days (approximately 1560 per year). Common characteristics of journal e-mails were mention of "peer review" in the message body and low rates of inclusion in recognized journal databases/resources, with 76.4% not found in any of 9 journal databases/resources. The location for conferences in e-mails included 31 different countries, with the most common being the United States (33.2%), Italy (9.8%), China (4.9%), United Kingdom (4.9%), and Canada (4.5%).
    Conclusions: The present study in an academic pathology department shows a high burden of unsolicited e-mails for medical/scientific journals, conferences, and webinars, especially to associate and full professors. We also demonstrate that some pathology trainees and junior faculty are receiving an estimated 1500 unsolicited e-mails per year.
    Keywords:  Electronic mail; open access publishing; peer review; publishing; scholarly communication; scientific conferences; webinars
    DOI:  https://doi.org/10.4103/jpi.jpi_12_19
  6. BMC Med Res Methodol. 2019 May 31. 19(1): 112
      BACKGROUND: From 2005 to 2010, we conducted 2 randomized studies on a journal (Medicina Clínica), where we took manuscripts received for publication and randomly assigned them to either the standard editorial process or to additional processes. Both studies were based on the use of methodological reviewers and reporting guidelines (RG). Those interventions slightly improved the items reported on the Manuscript Quality Assessment Instrument (MQAI), which assesses the quality of the research report. However, masked evaluators were able to guess the allocated group in 62% (56/90) of the papers, thus presenting a risk of detection bias. In this post-hoc study, we analyse whether those interventions that were originally designed for improving the completeness of manuscript reporting may have had an effect on the number of citations, which is the measured outcome that we used.METHODS: Masked to the intervention group, one of us used the Web of Science (WoS) to quantify the number of citations that the participating manuscripts received up December 2016. We calculated the mean citation ratio between intervention arms and then quantified the uncertainty of it by means of the Jackknife method, which avoids assumptions about the distribution shape.
    RESULTS: Our study included 191 articles (99 and 92, respectively) from the two previous studies, which all together received 1336 citations. In both studies, the groups subjected to additional processes showed higher averages, standard deviations and annual rates. The intervention effect was similar in both studies, with a combined estimate of a 43% (95% CI: 3 to 98%) increase in the number of citations.
    CONCLUSIONS: We interpret that those effects are driven mainly by introducing into the editorial process a senior methodologist to find missing RG items. Those results are promising, but not definitive due to the exploratory nature of the study and some important caveats such as: the limitations of using the number of citations as a measure of scientific impact; and the fact that our study is based on a single journal. We invite journals to perform their own studies to ascertain whether or not scientific repercussion is increased by adhering to reporting guidelines and further involving statisticians in the editorial process.
    Keywords:  Number of citations; Peer-review; Reporting guidelines; Reproducibility; Transparency
    DOI:  https://doi.org/10.1186/s12874-019-0746-4
  7. Actas Dermosifiliogr. 2019 May 25. pii: S0001-7310(19)30069-9. [Epub ahead of print]
      
    DOI:  https://doi.org/10.1016/j.ad.2019.01.007
  8. J Pediatr. 2019 Jun;pii: S0022-3476(19)30433-0. [Epub ahead of print]209 1
      
    DOI:  https://doi.org/10.1016/j.jpeds.2019.04.013
  9. JAMA Netw Open. 2019 May 03. 2(5): e195000
      Importance: Underrepresentation of female authors in research publications is prevalent, but it is unclear whether this is attributable to sex disparities in research conduct or authorship practices. Case reports are a poorly understood component of the biomedical corpus, and the production of anecdotal observations is not confounded by factors associated with disparities in female representation in research publications. Whether female authorship disparities exist in nonresearch publications of clinical information is unknown.Objectives: To examine the authorship of case reports and elucidate factors associated with sex disparity.
    Design and Setting: Cross-sectional study of all case reports published by US authors in 2014 and 2015 indexed in PubMed performed from July 2015 to July 2018.
    Main Outcomes and Measures: The primary outcome measure was the proportion of female first authors. The secondary outcome measures were the proportion of female last authors and female authorship representation among different clinical specialties.
    Results: Bibliometric data was abstracted from 20 427 case reports published across 2538 journals. A total of 7252 (36%) and 4825 (25%) case reports had a female first and last author, respectively. In comparison, 44% and 34% of US trainees and physicians, respectively, were female in 2015. Among adult case reports, female authorship was more prevalent in academic environments compared with community settings (34.0% vs 28.2% for female first authors and 23.4% vs 19.7% for female last authors). Across states, the proportions of female first authors and last authors were universally less than the proportions of female trainees and active female physicians, respectively. Female first authorship was associated with larger author teams (odds ratio [OR], 1.02; 95% CI, 1.01-1.03), an academic affiliation (OR, 1.16; 95% CI, 1.06-1.27), and a female last author (OR, 1.58; 95% CI, 1.47-1.70). Relative to general internal medicine, specialties dominated by male clinicians were less frequently associated with female first authors. Several exceptions displaying a relatively equivalent tendency for male and female first authorship included oncology (OR, 0.97; 95% CI, 0.81-1.16), ophthalmology (OR, 0.87; 95% CI, 0.72-1.05), and radiation oncology (OR, 0.94 95% CI, 0.56-1.56).
    Conclusions and Relevance: The underrepresentation of women among first and last authors in publications of case reports underscores the pervasiveness of sex disparities in medicine. Collaboration and female mentors may be critical instruments in upsetting longstanding practices associated with sex bias. Not all clinical specialties were associated with lower-than-expected female authorship, and further exploration of specialty-specific norms in publication and mentorship may elucidate specific barriers to female authorship.
    DOI:  https://doi.org/10.1001/jamanetworkopen.2019.5000
  10. Front Psychol. 2019 ;10 1007
      
    Keywords:  mentoring; psychology; publishing; research; social science; undergraduate
    DOI:  https://doi.org/10.3389/fpsyg.2019.01007
  11. Eur J Psychotraumatol. 2019 ;10(1): 1614823
      Adopting Registered Reports is an important step for the European Journal of Psychotraumatology to promote open science practices in the field of psychotrauma research. However, adopting these practices requires us as individual researchers to change our perspective fundamentally. We need to put fears of being scooped aside, adopt a permissive stance towards making mistakes and accept that null-results should be part of the scientific record. This is difficult because the culture in academia is competitive. Incentives are on publishing novel and positive results in high impact journals. A change in journal policies, such that openness and transparency are reinforced, can facilitate an attitude change in individual researchers.
    Keywords:  Registered reports; open science; replication crisis
    DOI:  https://doi.org/10.1080/20008198.2019.1614823
  12. Clin Spine Surg. 2019 May 28.
      Developing a well-written research paper is an important step in completing a scientific study. This paper is where the principle investigator and co-authors report the purpose, methods, findings, and conclusions of the study. A key element of writing a research paper is to clearly and objectively report the study's findings in the Results section. The Results section is where the authors inform the readers about the findings from the statistical analysis of the data collected to operationalize the study hypothesis, optimally adding novel information to the collective knowledge on the subject matter. By utilizing clear, concise, and well-organized writing techniques and visual aids in the reporting of the data, the author is able to construct a case for the research question at hand even without interpreting the data.
    DOI:  https://doi.org/10.1097/BSD.0000000000000845
  13. Early Hum Dev. 2019 May 25. pii: S0378-3782(19)30298-1. [Epub ahead of print]
      Several guidelines have been created in an attempt to raise the quality of different kinds of studies. These also attempt to ensure that research is reported accurately and transparently. Six such guidelines (CONSORT, MOOSE, PRISMA, SPIRIT, STARD, and STROBE) will be briefly discussed in this paper. The purpose of these guidelines is "to create a manual for the authors to follow, which should lead to total transparency, accurate reporting, and easier assessment of the validity of reported research findings." This goal has not been completely reached and it remains uncertain how it can as mandatory and full implementation at the journal end would entail significant and costly man-hours.
    DOI:  https://doi.org/10.1016/j.earlhumdev.2019.05.014
  14. Early Hum Dev. 2019 May 24. pii: S0378-3782(19)30296-8. [Epub ahead of print]
      The possibility of assembling this series of Best Practice Guidelines papers pertaining to WASP (Write a Scientific Paper) arose from the international WASP course, and the realisation that the Statistics at Square One papers initially published in the British Medical Journal in the 1970s, dealing with medical statistics using a hand calculator were now completely superseded by spreadsheets. Early Human Development has now published a complete set of guidelines on virtually all matters pertaining to WASP. All of these were written by academics from the University of Malta. This final set wraps up by elaborating on the techniques that best create effective graphs and tables, as well as the medical guidelines that are mandated by modern research. This coda should be viewed by WASP novices as a tipping point from where effective papers may be created and published. Like Superman's father, we hope that we have given you "an ideal to strive towards. They will race behind you, they will stumble, they will fall. But in time, they will join you in the sun… In time, you will help them accomplish wonders." Our WASP revels are therefore not ended, but just starting as authors of high quality papers.
    DOI:  https://doi.org/10.1016/j.earlhumdev.2019.05.012
  15. Early Hum Dev. 2019 May 23. pii: S0378-3782(19)30297-X. [Epub ahead of print]
      Readers are drawn towards tables and figures (such as graphs) because these are perceived as easy reading, when compared to attempting to interpret a verbose account of the same content. Previous publications in these Best Practice Guidelines series have already given some basic instruction in Excel chart creation and in Excel automatic histogram creation. This paper will outline some more advanced instructions with regard to graphs and tables, along with suggestions pertaining to the general formatting of these items. Specific recommendations pertaining to Microsoft Excel will also be highlighted since this is the most commonly used charting software. At all times, graphs "should avoid prolixity, complexity and gaucheness and strive instead for simplicity, brevity, cogency and clarity."
    DOI:  https://doi.org/10.1016/j.earlhumdev.2019.05.013