“There are three types of lies — lies, damn lies, and statistics.”
― Benjamin Disraeli (1)

In today’s library, we may feel as though we are constantly looking for better ways to capture data that truly describes the services and programming we provide, the relevance of our collections, and the individual and societal impact of libraries. But this is not a new dilemma; using statistics in assessment has been a hot topic for many years, as described in a brief history of library statistics in the United States:

The systematic compilation and publication of library statistics is a phenomenon of the nineteenth and twentieth centuries. (2)  In his ALA Bulletin article of 1966, James Krikelas noted the following, which still holds true today:

Numerous articles have been written about the usefulness of library statistics, but in general, the uses may be grouped under one of two categories: 1) to support administrative decisions and 2) to describe various types of library activities. A third category, seldom encountered in the literature but frequently implied (and perhaps of more concern in the future) is the use of statistical data in library research, that is, in attempting to establish general principles and relationships concerning library organization, administration, and use. (3)

Krikelas noted that concern with giving more meaning to library statistics arose as early as 1877 (4) with calls for modifications to survey instruments in years following, and a clear division between public and academic libraries (and subsequently school and regional libraries) occurring by vote in 1914.

In the eighty years since 1877, there has been constant criticism of the meaninglessness of reported statistics, inadequacy of the data, ambiguities of the categories and internal inconsistencies of the published data. (5) Sound familiar?

While Krikelas points out the shortcomings of data gathering in 1966, such as the lack of evidence to demonstrate that size correlates to quality or service, remain relevant today. (6) He suggests that other considerations are missing in the data collection and use of statistics such as the predictive nature of effect on the individual user, the competency of staff, and the adoption of qualitative inference. (7) We recognize these as things that should be accounted for in today’s statistical pursuits.

Five Questions

The IFLA Statistics and Evaluation Section asked an Assessment Librarian to provide their thoughts on five questions regarding statistics and assessment in libraries today, and what may be the future of data gathering.

Justine Wheeler is the Assessment Librarian at the University of Calgary, Libraries & Cultural Resources in Calgary, Canada and member of the standing committee of the Statistics and Evaluation Section, IFLA.  In her work, Dr. Wheeler conducts assessment projects and research and assists her colleagues in formulating their own assessment inquires.

Thank you, Justine for taking the time to answer some questions today. 

Question 1: Edward R. Tufte said “Above all else show the data.” (8)  Why should we gather data?

There are many reasons to collect data.  Traditionally, data has been collected to measure, for benchmarking purposes, and to make administrative decisions.   Of course, this comes back to making the case for funding and showing value. Data can also be used to identify stakeholders who may need new programming, services or resources.

Markku Laitinen wrote that library statistics can be considered an essential tool in evidence-based librarianship. (9)  There is hardly any quarter that would bring the value of the library into question but in a hardening economic climate, there is still more pressure to show the value, impact and efficiency of the library operations. (10)

Statistics and data tell stories of input output, activity, and impact; they help us to define successes as well as craft goals and objectives on both micro and macro levels. This information feeds the future of projections and budgets. Data informs decision making and policy creation, as demonstrated in one example below developed by Laitinen: (11)

Many governments and funding agencies require more granular data and demonstrated outcome measures and indicators.

Question 2: Mr. Tufte also said, “If the statistics are boring, then you’ve got the wrong numbers.” (12)  So, what the heck is it with all the numbers?

Number collecting, or ‘quantitative’ data, has been the primary way of gathering information about libraries for a very long time. Prior to delivering information online, we relied on these numbers to inform service in the physical sense – a measure of success as ‘hash’ marks benchmarked patron and librarian activity over time – not to mention number of volumes held. Nowadays, tracking activity and collections by simply counting is still relevant – think number of items downloaded or an organization’s digitization efforts – so while we recognize that aggregated data and descriptive statistics are ubiquitous in libraries, there has also been a lot of questions about what they tell us. Notably, libraries had a huge impact during COVID and yet many of the statistics we traditionally track (number of students in the building, number of books signed out) were not useful during the pandemic, so we have to be open to counting and describing other types of activities to more accurately reflect impact.

For me, aggregate data and descriptive statistics are great indicators of where we may need to dig more.  In other words, they can assist us in identifying where we need to do further work in order to make visible impact and insight. Take the COVID example; we all experienced that libraries were busier than ever, and occupancy while might have been an indicator of success at one time, it was not the best statistical measure of actual activity during the pandemic. This will be true moving forward, as many patrons have become use to getting deliverables electronically, as well as having self-service options, and so we have to adapt our data gathering and assessment accordingly.

Many libraries are now working on collecting statistics that reflect impact that is not captured with traditional counting.  These ‘qualitative’ statistics list qualities or characteristics and are descriptive in nature. This data is collected using questionnaires, interviews, or observation, and often appears in narrative form, such as telling a story that details an experience.

Utilizing both quantitative and qualitative statistics paints the most complete picture of the activity of a library and its impact for constituents and the community. See Seismonaut and Roskilde Central Library’s “The impact of public libraries in Denmark: A haven in our community” for a great example of how to use both kinds of data in our impact stories and pathways from Denmark

Question 3: Ron DeLegge II said that “99 percent of all statistics only tell 49 percent of the story.” (13)  I feel like there is so much more we can do to effectively present what we collect; can I describe these things statistically?

As mentioned, the inclusion and use of qualitative statistics to tell a library story can be very effective. Demonstrating value by describing the impact or activities of our users when engaged with our services – in their words – is an example of one effective method.

Another very simple thing we can do to increase the impact of our data is to present our data visually. This can be especially useful in multilingual environments, or in reaching audiences less familiar with library vernacular – like saying ‘bookshelf’ instead of ‘stacks’. Additionally, good data visualizations can be used to quickly convey large amounts of information, for example word clouds for descriptive statistics or illustrative charts and graphs that can have immediate impact for the reader, as we saw in the previous public libraries report from Denmark above.

Question 4: What are the ‘future trends’ we should be aware of?

  • Increased focus on Equity, Diversity, Inclusion, Accessibility and ‘Well-being’ statistics.
  • More emphasis on balancing quantitative, qualitative, and User Experience (UX) methods
  • Increase emphasis on statistics and assessment that reflect the culture, location and mission of individual institutions
  • Evaluation or Assessment as action research (from the work of Starr Hoffman)
  • Space Usage – virtual and real
  • Measuring Engagement with and Empowerment of Users

Question 5: What advice do you have for someone new to this field?

Get involved with community.  Find a mentor. Volunteer with the IFLA Statistics & Evaluation section. Attend our upcoming satellite workshop.  Go to conferences like the Library Assessment conference or Qualitative and Quantitative Measures in Libraries (QQML).  Take courses on statistics and evaluation for libraries.

Bonus Question: Dogs or cats?

Dogs!  My family and I have a bichon shih tzu (Zeus) and shih tzu (Xena). I grew up with a German Shepherd and Black lab, so I love all sizes of dogs!

Justine Wheeler, PhD is a Member of the Statistics and Evaluation Section Standing Committee. Bella Karr Gerlich, PhD, our interviewer, is Chair of the Statistics and Evaluation Section Standing Committee.