Performance Measurement and Metrics

Nitecki, Danuta A ; Wiggins, John ; Turner, Nancy B

2015 Performance Measurement and Metrics

doi: 10.1108/PMM-10-2015-0032

Purpose – This essay is offered in response to an invitation to share reflections on a topic of current concern. The concern is how to position the library profession to not just gain appreciation and support of academic libraries, but to develop librarians as leaders continuing the values of higher education as essential to maintain and improve a democratic society. The paper aims to discuss these issues. Design/methodology/approach – This is a viewpoint piece. Findings – The paper speculates that assessment alone may be an inadequate objective of an organizational culture for academic libraries; assessment is not universally accepted throughout higher education as a primary focus, perhaps with the exception when meeting accreditation requirements. Practical implications – An informed citizenry, freedom of information, open and equitable access to knowledge, and intellectual diversity are important values to preserve. This concern overlaps with the authors’ shared and independent work to engage in assessment efforts and developing a library culture of assessment. Social implications – Preparation of academic librarians may benefit from more strongly advancing the core mission of their institutions rather than their service relationship to its customers as an objective of the library culture. Originality/value – This is the first work to examine why assessment is not enough for libraries to be valued.

Matthews, Joseph

2015 Performance Measurement and Metrics

doi: 10.1108/PMM-10-2015-0034

Purpose – The purpose of this paper is to explore a wide range of performance measures, while acknowledging that of late, outcome measures have become increasingly important. Several initiatives are underway to develop a broad framework and suggest approaches to developing outcome measures for all types of libraries. However, while outcomes may be increasingly used, determining the value of a specific outcome is complicated by the reality that value is determined by a combination of perspective and each individual experiencing a specific library service. Design/methodology/approach – This paper reviews the literature about outcomes and determining the value of outcomes as a foundation for exploring these two inter-related issues – outcomes and value. Findings – The findings of a number of reports and studies are reported. Originality/value – Separating the question of identifying the outcomes associated with library services and the value of these outcomes are shaped by the perspective of value and determining value will assist libraries in attempting to measure the impact of the services they provide.

Town, Stephen

2015 Performance Measurement and Metrics

doi: 10.1108/PMM-10-2015-0033

Purpose – The purpose of this paper is to present a case study of the implementation and use of the Value Scorecard in a university library. The Value Scorecard seeks to articulate the full value of a library through a four dimensional matrix populated with data, evidence and narrative. Design/methodology/approach – The study covers two years of collection of data and evidence to populate the Value Scorecard at the University of York, UK. This is action research and development. Findings – The paper describes the success of the implementation of the framework across a broad university service including library, archives and IT services. The reporting template is outlined and the availability of relevant measures for populating each dimension are discussed, together with developments in the concepts of each dimension since the original paper on the scorecard. The paper reflects on the advances in the understanding and practice of performance measurement and assessment in libraries that the Value Scorecard offers. The strengths and omissions of other pre-existing frameworks, including the Balanced Scorecard, are discussed and absorbed into the value framework. The application of the Value Scorecard offers a practical and successful framework for library performance measurement and advocacy in a dynamic and changing landscape. Research limitations/implications – The limitations of the research are those generally applying to a single case experience. Practical implications – Nothing arises from the study to suggest that other libraries could not apply this framework, as it encompasses other previous frameworks and allows for local variations and circumstances. Some elements of the framework lack full measurement methods, and this is discussed. Originality/value – The originality and value of the paper is that it provides a unique framework for measurement of all dimensions of activity and value in an academic research library, and one that can be tailored to local requirements.

Bakkalbasi, Nisa ; Goertzen, Melissa

2015 Performance Measurement and Metrics

doi: 10.1108/PMM-10-2015-0035

Purpose – Over the past decade, as the electronic book (e-book) collection continues to grow, Columbia University Libraries has been gathering information to develop policies related to e-book acquisition, discovery, and access. The purpose of this paper is to investigate users’ e-book search behavior and information needs across different disciplines. Design/methodology/approach – The research method utilizes text data from two sources: users’ e-book search queries that were entered into the libraries discovery tool called CLIO and e-book title words provided by the Counting Online Usage of Networked Electronic Resources (COUNTER) usage reports. The analysis involves identifying and quantifying certain words from users’ search queries with the purpose of examining the contexts within which these words were used. Findings – The prominence of topical words such as “history,” “social,” and “politics” in the list was an interesting reflection on the kinds of works users were looking for, as were the terms “handbook,” “guide,” and “manual.” The high frequency of these words imply that users were searching for broad topics, reference works, or other collections of instructions, all of which are intended to provide ready reference. Originality/value – Running search queries and e-book title words through a text analysis tool revealed new ideas related to what types of materials users search for and use. Text analysis of search terms and title words provided insight into the nature of e-book use, including broad topic (e.g. history), academic level of use (e.g. introductory), and genre/type (e.g. reference). While it is challenging to deduce reader intent from word frequency analysis, as text data remain widely open for interpretation, the methodology has significant strengths that drive us to continue to use in future studies.

Ali, Shameem

2015 Performance Measurement and Metrics

doi: 10.1108/PMM-08-2014-0028

Purpose – Students at higher education institutions in under-developed countries face many study and learning-related impediments and challenges. The purpose of this paper is to ascertain the library usage behaviour and experiences of students as well as to explore reliance on campus-based private study options due largely to the disadvantaged living conditions of many students. Design/methodology/approach – A survey of students was conducted to identify how the institutional library is used, what perception students have of the library services and what the service gaps are. Perceptions were measured across 20 indicators adapted from previous studies. The survey sought information on the rate of library usage, the main reason for the most recent library visit and reasons that had ever prevented them from visiting the library. Post survey, three focus group sessions were conducted to discuss issues relating to home study environments and preliminary findings from the survey stage. Findings – Based on the item groupings five service dimensions were identified. These were student services/facilities, management, collection, study spaces and access. Despite indicating a high level of dissatisfaction with some aspects of the library services, students preferred the library because home environments are not conducive to study and learning. The service dimensions identified by the study relate to the areas of greatest need from the student perspective. Practical implications – The implementation of many of the virtual characteristics of university libraries should be based on a clear understanding of the local information needs, searching behaviours as well as being cognizant of how and why libraries are accessed. In this regard consideration must be given to the lack of study facilities at home. Originality/value – In the context of developed countries the notion of virtual libraries is a reality. However, in the African context such a notion seems “remote” from the perspectives of many students, especially those from the lower socio-economic backgrounds. Many of these students come from poor living conditions and have no study and learning opportunities outside of campus. This paper raises these issues for consideration in library planning and budget allocations for universities in poor communities.

Calvert, Philip ; Goulding, Anne

2015 Performance Measurement and Metrics

doi: 10.1108/PMM-05-2015-0016

Purpose – The purpose of this paper is to report the results of a study exploring New Zealand library managers’ views on the purpose, outcomes and approaches to measuring library economic value and social impact. Design/methodology/approach – Semi-structured interviews with library managers drawn from a purposive sample were undertaken in four main centres in New Zealand. Findings – Analysis of the interviews suggest that library managers do not feel under immediate direct pressure to provide evidence of the value and impact of their services but feel that it is important to have data available if required. There was some scepticism about the validity of quantitative data collected and some uncertainty about how to interpret and communicate it for the best outcome for the service. Strong support for qualitative “stories” or narratives was a feature of the interviews. Research limitations/implications – This was a small scale study, limited to just four centres in the North Island of New Zealand. Practical implications – Library managers may want to consider systematic methods of collecting narrative data to support quantitative data. They could also reflect on how to communicate value and impact data and information proactively, using alternative communication strategies and tools such as social media. Originality/value – There have been no previous studies of library managers’ views of measuring the value and impact of libraries in New Zealand. The focus on the purpose and outcomes of measurement in the New Zealand context is also original.

de Jager, Karin

2015 Performance Measurement and Metrics

doi: 10.1108/PMM-08-2015-0021

Purpose – The purpose of this paper is to explore from a number of points of view the perceptions of mainly undergraduate students at the University of Cape Town (UCT) about the value they derive from visiting the physical library. Design/methodology/approach – Data from different investigations were assembled to derive a composite view of undergraduates’ perceptions of the value of the library. LibQUAL+® data reflecting the “Library as Place” dimension of the survey which was completed in 2014 were scrutinized; data from two surveys conducted in the information commons and the 24/7 venue of the main library at UCT were compiled and data from gate counts during the past three years showed different aspects of undergraduates’ opinions and behaviour. Findings – The combination of data from difference sources provided convincing evidence that undergraduate students value the library as a physical space and that they believe their working in the library enables them to get better marks for their university work. Originality/value – The originality of this paper lies in the combination of different data sets to focus on one particular issue, the value of the physical library.

Adedayo, Adeleke Victor

2015 Performance Measurement and Metrics

doi: 10.1108/PMM-01-2015-0001

Purpose – The purpose of this paper is to suggest that citations made in the introduction and literature review sections of academic writings should not count in the analysis of citations to measure the quality of research papers. Design/methodology/approach – Elucidatory expositions are made on the purposes of the introduction and literature review sections. Findings – The nature of citations to knowledge to establish these purposes is identified and used to suggest that citations made in these sections should not count in citation analysis that are used to determine quality of publications. Introduction sections are written to identify the importance and justification for the subject of study, while literature reviews are written to identify gaps, opposing views, strengths and weaknesses in the status quo knowledge. Originality/value – This paper will provide insight and awareness to new methodologies to cull and curate appropriate citation count in the computation of quality of publications.