Research assessment and analytics

From HLWIKI Canada
Jump to: navigation, search
Are you interested in contributing to HLWIKI International? contact:

To browse other articles on a range of HSL topics, see the A-Z index.


Last Update

  • Updated.jpg 14 March 2017


See also Altmetrics for librarians: pros and cons | Bibliometrics | Impact factors | FigShare | Scopus vs. Web of Science | ImpactStory | Utopia Documents | Webometrics

Sir Paul Nurse, President of the Royal Society, said at a 2015 meeting on the Future of Scholarly Scientific Communication:

  • "...[the] measurement of research quality is very important to science and it’s important for career advancement – but the emphasis on impact factors as often the only metric has its issues. I get so cross when I’m in a reviewing committee, and I read letters or hear people saying, ‘X has published in Nature, Science’ or whatever – as if this, by itself, means something. The laziness of senior colleagues in looking at this is completely extraordinary."
  • Research in the 21st century: data, analytics and impact

What is research assessment?

Research assessment may be broadly defined as a way to measure the return on investment (ROI) in scientific and scholarly research. Research assessment includes the evaluation of research quality and measurements of research inputs, outputs and impacts, and embraces qualitative and quantitative methodologies, including the application of bibliometrics and citation mapping, and peer review. Research performance is increasingly regarded as a key factor in economic performance and societal welfare. Research assessment is a major issue for a range of stakeholders in academia and there is increased focus on research quality and excellence, transparency, accountability, comparability and competition in the academy.

Some countries, such as the United Kingdom, undertake a review of research every five years to assess the quality of research undertaken by higher education institutions. See Research Excellence Framework.

Academic librarians in RA&A

The profession of librarianship is constantly evolving. As the information landscape changes, librarians of all stripes must adapt and expand the range of services provided to ensure we remain relevant and valuable to our user communities. One recent change in this regard is the growing demand for bibliometric support as well as research assessment and analytics at research institutions. These analytics-based activities (such as altmetrics and bibliometrics) seek to understand who produces what research at any given institution, who they collaborate with to produce this knowledge, what exactly their research is about, and what impacts their research has in fields, and generally within the academy. As a response, librarians at academic institutions offer research assessment services. New roles are available to academic librarians in analyzing research within their organizations and to facilitate communication among those who want to provide aspects of research assessment services in their liaison work. The need for research assessment services must be determined in conjunction with librarians and information professionals who already provide these services, and opportunities should be identified to expand these services at other institutions.

Major trends in research assessment

  • Research institutions and universities operate in the context of global markets. International comparisons or rankings of institutions are published regularly with the aim to inform students and knowledge-seeking groups about quality. Research managers use this information to benchmark institutions against competitors.
  • Research institutions are setting up internal research assessment processes, and building research management information systems. These are based on relevant inputs and output measures of the performance of individual research units within an institution, enabling managers to allocate funds within the institution according to the past performance of the research groups.
  • Trends in publishing have had an impact on assessing research outputs. Publishers make their content electronically available and researchers report that their access to the literature has never been more important or more valuable. Disciplinary and institutionally oriented publication repositories are being built, along with implementation of institutional research management systems, which include metadata on an institution’s publication output.
  • Currently, three large multidisciplinary citation indexes are available: Elsevier’s Scopus, Thomson Reuters’ Web of Science, and Google Scholar.
  • With increased access to journals and literature, more indicators of research quality and impact are available. Bibliographical databases implement bibliometric features such as author h-indexes, as well as publication and citation charts. More specialized institutes produce other indicators, often based on raw data from the large, multi-disciplinary citation indexes. Today, the calculation of indicators is not merely the province of experts in bibliometrics, and the concept of “desktop bibliometrics” is increasing becoming a reality.

More on research impact

  • Many universities are pushing faculty into having a greater impact in their fields so there's been considerable interest in these topics, particularly at the Provost level.
  • Academic librarians present about metrics, peer networks, increasing reputation to faculty during university-held faculty research/teaching conferences, workshops on high impact research, and to new tenure-track faculty during orientations
  • Librarians follow up with subject guides as general introductions; workshops on bibliometrics and altmetrics; several workshops for faculty

What is your relationship with research?

  • Conducting research can be rewarding in your career development, and in understanding the needs and pressures of library users.
  • Academic librarians and information professionals have a dual role with respect to research: leading their own research/participating as part of research teams and advising researchers on achieving research impact

Valuable links on research impact

Personal tools