By Dennis D. McDonald

In Addicted to the brand: The hypocrisy of a publishing academic author Philip Moriarty laments how difficult it is to tear people away from using unreliable metrics such as  "journal impact factors" to guide the evaluation of individual researcher performance:

Despite bemoaning the statistical illiteracy of academia’s reliance on nonsensical metrics like impact factors, and despite regularly venting my spleen during talks at conferences about the too-slow evolution of academic publishing towards a more open and honest system, I nonetheless continue to contribute to the problem. (And I take little comfort in knowing that I’m not alone in this.)

Getting people to change from a current business model is a problem in a lot of fields, not just academic journal publishing. "Rocking the boat" in any community is always accompanied by some kind of pain and suffering even for those who stand to benefit eventually from the change.

I left academic life decades ago after several years of researching how scientists use the journal system. I am not surprised at the institutional resistance to change that is described by Moriarty regarding journals. Having also seen how much resistance to changes in collaboration and communication behavior in the corporate world regarding use of technology, I see two major factors to consider when thinking about change.

One factor has to do with whether or not the change is actually disrupting the current institution or just making it more efficient. The latter type of change is much easier to digest. An example is the use of technology in the corporate world to automate operations that were previously performed manually. In academia, speeding up the peer review process with better technology doesn't fundamentally change the system. Making a change from how journal metrics are used, however, is much more disruptive, as Moriarty attests, since much more is at stake in terms of career advancement, hiring, tenure, and prestige.

Another factor to consider is the role of leadership. In academia leadership is much more diffuse than in the corporate world. It is difficult to rally multiple leaders from multiple institutions to overcome a system where reputation and leadership is so highly decentralized. Journals and impact factors are one response since they centralize visible creative behaviors and are countable thus providing an aura of respectability and objectivity. (What I learned early in my career was that, just because you can count something, doesn't make it real.)

What really underpins the journal system are people and how they create, communicate, and collaborate about data and ideas. These are activities that are much harder to observe, count, and assess than citations, rankings, and patents.

Rather than attack the journal system, given the many good things it does, perhaps it would make more sense to acknowledge its limitations and to educate creators and non-creators what its values and limitations are.

Copyright (c) 2017 by Dennis D. McDonald