On Research Quality and Impact: What Five Decades in Academia Has Taught Me
Abstract
Research quality and impact need to be assessed for various purposes, including promotions or salary raises at universities, advancements at research labs, awarding of grants by research-funding organizations, and bestowing scientific/technical honors. Research must ultimately be at service to the society that supports it, but there is more to evaluating research than assessment of its immediate social impact. Research programs within and between various disciplines are interconnected, with the chain-link from a particular piece of research often going through multiple projects (and sometimes human generations) before it connects to a practical application. In scientific fields, research is expected to validate or challenge proposed theories of how our world works, and therein lies its impact. In technical fields, research is at the service of building better devices, systems, and processes, through the derivation of innovative designs or more accurate modeling and evaluation methods. In this paper, I present some thoughts on factors used to assess science and technology research efforts and their impact.
Keywords
Assessment, Citation, Conference, Impact, Journal, Paper, Productivity, Promotion, Scholarship
References
- [1] Baldwin, M. “Origins of the Journal Impact Factor”: https://physicstoday.scitation.org/do/10.1063/PT.5.9082/full/, 2017.
- [2] Beall’s List “Potential Predatory Scholarly Open-Access Publishers”: https://beallslist.net/, 2020.
- [3] F. C. Fang, and A. Casadevall, “Retracted Science and the Retraction Index,” Infection and Immunology, vol. 10, pp. 3855- 3859, 2011.
- [4] E. Garfield, “Citation Indexes for Science”, Science, vol. 122, no. 3159, pp. 108-111, 1995.
- [5]Garfield, E. “Citation Analysis as a Tool in Journal Evaluation,” Science, vol. 178, no. 4060, pp. 471-479,1972.
- [6] Glasson, V. 2020. “6 Ways to Spot a Predatory Journal”: https://www.rxcomms.com/blog/6-ways-spot-predatory-journal/
- [7] Lerchenmueller, M. J., O. Sorenson, and A. B. Jena, “Gender Differences in How Scientists Present the Importance of Their Research: Observational Study,” BMJ, vol. 367, l6573, pp. 1-8, 2019.
- [8] Ng, I. C. L. (undated. “Guidelines on How to Evaluate a Research Article,” On-line document. https://www.academia.edu/1744631/Guidelines_on_how_to_evalu ate_a_research_article
- [9] Parhami, B., “Low Acceptance Rate of Conference Papers Considered Harmful,” IEEE Computer, vol. 49, no. 4, pp. 70-73, 2016.
- [10] Prater, C. “Eight Ways to Identify Questionable Open Access Journals,” American Journal Experts Web site, https://www.aje.com/arc/8-ways-identify-questionable-open- access-journal/, 2020.
- [11] Smith, A. J. “The Task of the Referee,” IEEE Computer, vol. 23, no. 4, pp. 65-71, 1990.
- [12] Sokal, A. D., The Sokal Hoax: The Sham that Shook the Academy, University of Nebraska Press, 271 pp, 2000.
- [13] Stop Predatory Journals 2020. “List of Predatory Journals”: https://predatoryjournals.com/journals/
- [14] Sugimoto, C. R. et al., “Global Gender Disparities in Science,” Nature, vol. 504, pp. 211-213, 2013.
- [15] Van Noorden, R., “Publishers Withdraw More than 120 Gibberish Papers,” Nature, News & Comment, 2014.
- [16] Web of Science, “History of Citation Indexing”: https://clarivate.com/webofsciencegroup/essays/history-of- citation-indexing/, 2020.
- [17] WikiHow “How to Write a Letter of Recommendation”: https://www.wikihow.com/Write-a-Letter-of-Recommendation, 2019.
- [18]Wikipedia, “Impact Factor”: https://en.wikipedia.org/wiki/Impact_factor, 2020a.
- [19]Wikipedia, “Predatory Publishing”: https://en.wikipedia.org/wiki/Predatory_publishing, 2020b.
- [20] Zhou, Z. Q., Tse, T. H., and Witheridge, M. “Metamorphic Robustness Testing: Exposing Hidden Defects in Citation Statistics and Journal Impact Factors,” IEEE Trans. Software Engineering, early access, pp. 1-22, May 2019.
