RSS

Tag Archives: AltMetrics

The confusing world of Altmetrics – A personal opinion, with confusing data

I am a fan of Altmetrics. At least in concept. But I starting to get very concerned with both the tools used to measure them and what the “numbers” are expected to indicate. We would expect that a high “number” in an Altmetric.com “donut” would be indicative, in some way, of the relative importance or “impact” of that article. One would hope it at least points to how well read the article is, whether the readers like the science and the potential for the article to, for example, move forward understanding or proliferate data into further usage. I am not sure this is true…at least for some of the articles I am involved with.

Let’s take for example the recent Zika Virus article that Sean Ekins led. The F1000 site gives us some stats in regards to Views and Downloads and the Metrics shows the Altmetric stats. I would assume that 48 DOWNLOADers would have at least some of them reading the article. Some of the VIEWers are likely to have read it and maybe printed it. For the Altmetric stats the 33 tweets are likely people pointing to the article and because of the way I use Twitter I am going to suggest that Tweets are less indicative of the number of readers of the article. There is a definition on the Altmetric site regarding how Twitter stats are compiled.

Metrics on the F1000 Zika Article Page

If we use the Altmetric Bookmarklet we can navigate to the page with a score

Altmetric page with score

The score of “41” is essentially the sum of bloggers, tweets, Facebook posts etc. summarized below (1+1+1+33+1+3+1 for being on Altmetric.com???)

Sum all networking posts to get 41

When I asked F1000Research via Twitter why they don’t show the “number” I appreciated their answer. I AGREE with their sentiment.

F1000 feedback on Twitter

 

Yesterday I received an email about our Journal of Cheminformatics article “Ambiguity of non-systematic chemical identifiers within and between small-molecule databases“, part of which is shown below.

Journal of Cheminformatics email

On the actual Journal of Cheminformatics page it says there have been 1444 accesses (not 2216 as cited in the email).

Journal of Cheminformatics Accesses

Also the Altmetric score is 8. So somewhere between 1400-2200 accesses (and it is safe to assume some proportion actual read it!). But it has a low Altmetric score of 8. This is versus an Altmetric score of >40 for the Zika Virus paper and a lot less accesses and probably a lot of the altmetrics for that article don’t necessarily indicate reads of the article as they are Tweets, many of them from the authors out to the world.

Using PlumX I am extremely disappointed regarding what it reflects about the JChemInf article! Only 10 HTML Views versus the 1400-2200 accesses reported above, and only 7 readers and 1 save! UGH. But 13 Tweets are noted so it seems so I would expect at least an Altmetric.com score of 13 or 14, instead of the 8 marked on the article?

PlumX stats

I also tried to sign into ImpactStory to check stats but got a “Uh oh, looks like we’ve got a system error…feel free to let us know, and we’ll fix it.” message so will report back on that.

Altmetrics should be maturing now to a point where the metrics of reads, accesses, downloads should be fed into some overall metric. I think that reads/accesses/downloads should carry more weight than a Tweet in terms of impact of an article? At least if someone read it, whether they agree with it or not they are MORE aware of the content than if someone simply shared the link to an article, that then didn’t get read? The platforms themselves are so desync’ed in terms of the various numbers themselves that we must wonder how are things so badly broken? I would imagine that stats gathered in someway through CrossRef or ORCID will ultimately help this to mature but until then treat them all with a level of suspicion. I believe that AltMetrics will be an important part of helping to define impact for an article. But there is still a long way to go I’m afraid….

 
2 Comments

Posted by on February 15, 2016 in AltMetrics, Kudos, ORCID

 

Tags: , , ,

The future of scientific information & communication presented at the SUNY Potsdam Academic Festival

This is a LONG presentation….I talk about the “It’s All About Me” attitude that can positively feed science….we want to share OUR science, we want people to know about our opinions, our activities, our collaborators, we want to get funding, recognition and attribution. And why not…it can all be to the benefit of science.

This presentation was given at the SUNY Potsdam Academic Festival

The future of scientific information & communication

Our access to scientific information has changed in ways that were hardly imagined even by the early pioneers of the internet. The immense quantities of data and the array of tools available to search and analyze online content continues to expand while the pace of change does not appear to be slowing. While scientists now have access to the enormous capacities and capability of the internet the vast majority of scientific communication continues to be through peer-reviewed scientific journals. The measure of a scientist’s contribution is primarily represented by their publication profile and the citations to their published works and offers an incomplete view of their activities. However, we are at the beginning of a new revolution where the ability to communicate offers the opportunity to embrace new forms of publishing and where scientific participation and influence will be measured in new ways. This presentation will provide an overview of our new generation of “openness” in which open source, open standards, open access and open data are proliferating. The future of scientific information and communication will be underpinned by these efforts, influenced by increasing participation from the scientific community and facilitated collaboration and ultimately accelerate scientific progress.

 

Tags: , , , ,

Honored to be a Recipient of the Jim Gray eScience Award from Microsoft Research

Last night in Chicago I was awarded the Jim Gray eScience Award. I didn’t know Jim personally but I know I benefit from the fruits of his work. Before Tony Hey gave me the award he played a video about the previous award winners. To be recognized for my contributions and to join scientists of the caliber of the previous winners was, to say the least, very emotional. My entire career has been focused on doing what I thought was the right thing for the role I was charged with. And when I didn’t want the role I was in I would move on. That’s migrated me through various roles in science from lab manager in academia, in industry, to start-up cheminformatics company product manager, through marketing, through sales, to community website for chemistry, to where I am today at RSC, a publisher. If I had been asked to map out my career path there is no way I would get to here…but which of us would be able to really?

Last night I presented on “The Possibilities and Pitfalls of Internet-Based Chemical Data”. I talked about how much data I have generated in the lab over the years that is now lost. And how we can change this moving forward for the existing generation of scientists. I talked about the history of ChemSpider from hobby-project to present day as one of the web’s primary sites for chemists. I talked about how scientists should PARTICIPATE in annotating and curating data online…how data sites specifically should enable commenting to capture issues. I talked about the measure of scientists and how efforts including ORCID and ImpactStory will be important to deal with the impact and notability of scientists. I hope I was able to share my view that while technology will continue to improve in terms of allowing us to contribute that it is personal choice to make a difference that is crucial in terms of correcting errors, annotating data and continuing the journey of creating improved resources for the chemistry community (and of course other branches of science).

I also announced our intention for RSC to create a Global Chemistry Hub (a topic for a separate post) and to “data enable the RSC archive”…extracting chemicals, reactions, data etc from our archive going back to the 1840s. We do not have all of the technologies, the processes or the approaches yet defined. But we have the intent and the courage to go for it, learning as we go and producing beneficial outcomes in an iterative manner. It’s an exciting time for the RSC cheminformatics team and it is my privilege to work alongside a great team of individuals to create a step change in terms of how we manage and deliver chemistry data to the community.

I have had a lot of trusted advisers over the years and last night I acknowledged a list of those closest to me in recent years. They include: Jean-Claude Bradley, Sean Ekins, Lee Harland, Gary Martin and Martin Walker. The closest to me however is Valery Tkachenko. I was happy that Valery was able to be at the conference with me. So much of what has been achieved to data with ChemSpider (as well as MANY projects we worked on together while at ACD/Labs) rests squarely on his shoulders. The future technical implementation of the cheminformatics projects we are undertaking at RSC is under his guiding hand. I am glad to have such a great “partner in crime”….

My thanks to Microsoft Research, to the judges for selecting me for the award and to the community who has chosen to embrace some of the fruits of my work. I am leaving Chicago proud, tired and looking forward to making an ever bigger impact with some of our new projects.

 

 

 
 

Tags: , , , , , , ,