RSS

My Worries About the Peer-Reviewed Publishing Process and Are Open Access Journals Becoming the Fallback?

20 May

I blogged recently about the Publishing Process and when the publishers get it right. That was my blessing of their work. This posting has a slightly different flavor…it’s about my concerns regarding the editorial and peer-review process. This posting is not meant to be generally applicable to all journals nor all editors. It is simply a review of recent experiences. It’s NOT about ChemSpider per se, just a general rant about my experiences of publishing and reviewing science, something we will be doing in the future about our experiences with ChemSpider.

In my line of work I get to work with some excellent scientists. Similarly, a number of the people I have developed respect for now make up the ChemSpider Advisory Group. As a result of scientific collaborations I author/co-author about 8 peer-reviewed articles a year. I also review about 6 articles a year for various journals. Of late the processes involved have me concerned.

Some of my recent experiences having articles I am involved with reviewed are discussed below.

1) One article had multiple “Publish As-is” with minor grammatical corrections. One reviewer clearly didn’t have an understanding of the science and got stuck on “it’s too long”. After a long discussion with the editor, despite the multiple “Publish as-is” we were left with no choice…reduce the length of the article or retract. We reduced the length of the article. I suspect that there are page limits being put on editors to the disservice  of good science.

2) A recent submission to one of the most popular journals in the world was sat on for many months. This work resulted from work with a pharmaceutical company and the submission therefore was from a chemistry software company and a pharmaceutical company.The manner in which this work was treated suggests unfair bias to non-academic submissions as if the fact that the work was included into a commercial product was problematic. As a result of the treatment received we have committed to not return to that journal to publish.

3) I was asked to review a publication including a comparison of performance of one of our prediction algorithms. A table contained a comparison of numerous algorithms including ours. The article in question highlighted that the authors algorithm outperformed any of the other algorithms under comparison. A close examination showed that for our algorithm four numbers had been mixed up in the table and arranging them properly showed that actually our algorithm had the best performance. This feedback was given to the editors. The paper was then published with MORE errors than originally identified and our algorithm was way down the list. A request to the editor for an Erratum or publication of a rebuttal by us was turned down…specifically because they did not feel it was appropriate to get in the middle of such a comparison with a “commercial organization.” It took two years but the original author was kind enough to publish a retraction with us but by that time the damage was done.

My recent experiences with articles I have reviewed include two in particular (no names or topics mentioned)

1) I was asked to review an article that was categorically “bad science”. I strongly suggested that the article be retracted..not rewritten. The science behind the article was simply wrong. There was nothing to rescue. The article was withdrawn but published very shortly after with 70% of the content intact in an Open Access journal (no title mentioned). It remains bad, but published, science.

2) I was asked to review an article by an ACS journal editor. This was another one comparing the performance of a series of algorithms, one of ours included. The work made our algorithms look terrible. It did the same for many others. Some of the data used to build the correlation were extracted from other publications. Some of the data were not even what they were correlating. A close examination of the data showed that when bad data were removed and the remaining data were treated appropriately that our algorithms gave excellent performance. I provided detailed feedback to the editor and suggested retraction. The paper was eventually retracted and published later in an open access journal with the SAME conclusions and comparisons. This was all the more upsetting since we’d been in discussions with the author(s) about reworking their data in collaboration.

I have two primary concerns and requests:

1) There appears to be a flavor in the air of “commercial chemistry software” is a bad thing. This is certainly true on many blogs discussing open source solutions. It appears to permeate into the publishing process based on some of my experiences (NOT all I should emphasize!!!!). This is my chosen career….I’ve done the PhD, a postdoc with a government laboratory, worked at a University and even in a Fortune 500 company. I eventually ended up at a chemistry software company. This is how I pay my bills, clothe my kids, live. I work with 140 other people producing chemistry software. It is a respectable career. We do science for a living. We get paid for our science by selling our products. No grants, no large company above us to carry us…our efforts are all we have to produce our living. We do EXCELLENT science and this should be the measure by which we are measured, not the fact that our science is eventually commercialized.

2) There IS good science in Open Access journals of that I have no doubt. I hope that part of the Open Access Journal process is that they check for the refusal of publication of submitted works from other journals prior to acceptance. Then they only have the ethics of the submitters to deal with. Peer-review is still necessary in Open Access Journals. My concern is with the burden of scientists who are generally overwhelmed. I get too many publications to review as it is. I have colleagues who get one a month. These are people who are already overwhelmed with work. I have no solution to the problem….just acknowledging it.

These are just observations and I welcome feedback. If and when we publish about ChemSpider where will we publish? We already have had invitations to publish in an Open Access journal…a decision is yet to be made (and a manuscript written).

 

About tony

Antony (Tony) J. Williams received his BSc in 1985 from the University of Liverpool (UK) and PhD in 1988 from the University of London (UK). His PhD research interests were in studying the effects of high pressure on molecular motions within lubricant related systems using Nuclear Magnetic Resonance. He moved to Ottawa, Canada to work for the National Research Council performing fundamental research on the electron paramagnetic resonance of radicals trapped in single crystals. Following his postdoctoral position he became the NMR Facility Manager for Ottawa University. Tony joined the Eastman Kodak Company in Rochester, New York as their NMR Technology Leader. He led the laboratory to develop quality control across multiple spectroscopy labs and helped establish walk-up laboratories providing NMR, LC-MS and other forms of spectroscopy to hundreds of chemists across multiple sites. This included the delivery of spectroscopic data to the desktop, automated processing and his initial interests in computer-assisted structure elucidation (CASE) systems. He also worked with a team to develop the worlds’ first web-based LIMS system, WIMS, capable of allowing chemical structure searching and spectral display. With his developing cheminformatic skills and passion for data management he left corporate America to join a small start-up company working out of Toronto, Canada. He joined ACD/Labs as their NMR Product Manager and various roles, including Chief Science Officer, during his 10 years with the company. His responsibilities included managing over 50 products at one time prior to developing a product management team, managing sales, marketing, technical support and technical services. ACD/Labs was one of Canada’s Fast 50 Tech Companies, and Forbes Fast 500 companies in 2001. His primary passions during his tenure with ACD/Labs was the continued adoption of web-based technologies and developing automated structure verification and elucidation platforms. While at ACD/Labs he suggested the possibility of developing a public resource for chemists attempting to integrate internet available chemical data. He finally pursued this vision with some close friends as a hobby project in the evenings and the result was the ChemSpider database (www.chemspider.com). Even while running out of a basement on hand built servers the website developed a large community following that eventually culminated in the acquisition of the website by the Royal Society of Chemistry (RSC) based in Cambridge, United Kingdom. Tony joined the organization, together with some of the other ChemSpider team, and became their Vice President of Strategic Development. At RSC he continued to develop cheminformatics tools, specifically ChemSpider, and was the technical lead for the chemistry aspects of the Open PHACTS project (http://www.openphacts.org), a project focused on the delivery of open data, open source and open systems to support the pharmaceutical sciences. He was also the technical lead for the UK National Chemical Database Service (http://cds.rsc.org/) and the RSC lead for the PharmaSea project (http://www.pharma-sea.eu/) attempting to identify novel natural products from the ocean. He left RSC in 2015 to become a Computational Chemist in the National Center of Computational Toxicology at the Environmental Protection Agency where he is bringing his skills to bear working with a team on the delivery of a new software architecture for the management and delivery of data, algorithms and visualization tools. The “Chemistry Dashboard” was released on April 1st, no fooling, at https://comptox.epa.gov, and provides access to over 700,000 chemicals, experimental and predicted properties and a developing link network to support the environmental sciences. Tony remains passionate about computer-assisted structure elucidation and verification approaches and continues to publish in this area. He is also passionate about teaching scientists to benefit from the developing array of social networking tools for scientists and is known as the ChemConnector on the networks. Over the years he has had adjunct roles at a number of institutions and presently enjoys working with scientists at both UNC Chapel Hill and NC State University. He is widely published with over 200 papers and book chapters and was the recipient of the Jim Gray Award for eScience in 2012. In 2016 he was awarded the North Carolina ACS Distinguished Speaker Award.
Leave a comment

Posted by on May 20, 2007 in Uncategorized

 

0 Responses to My Worries About the Peer-Reviewed Publishing Process and Are Open Access Journals Becoming the Fallback?

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

 
Stop SOPA