Just when we get used to one new thing, another crops up that we have to deal with. As if learning new technologies wasn’t hard enough, now we need to figure out how to answer funding decision makers like Ron Carlee, COO of the International City/County Management Association, who made the statement at ALA Conference 2011 that; “If your brand is a book, it might as well be a buggy whip,” Carlee said. “The product is not the book, it is the consequence of the book…knowledge is a core service.” The big question is: How do we prove that the library is accomplishing its core services – that it is valuable?
Like the character in Rita Mae Brown’s novel Sudden Death says; “Insanity is doing the same thing over and over again but expecting different results.” Describing librarians as insane would be a mistake, because that would imply that they actually expect certain outcomes from their activities. Traditionally, librarians don’t expect ANY results from their daily tasks, so they can’t be insane – right?
Simply working harder is not the answer to proving that libraries are valuable. Outcome-based evaluation (OBE) has become the new metrics for just about everything, and certainly has application in library operations. But, it may be so foreign to most librarians that it requires another new mind-set.
Last April I wrote about Answers for Library Leader based on the previously spotlighted, exceptionally thought provoking article by Scott Corwin, Elisabeth Hartley & Harry Hawkes – “The Library Rebooted” published at Booz & Company website strategy+business. One of their seven propositions included;
6. Expand the metrics.
“… Keeping track of the number of monthly and annual physical visitors … monitoring the number of books … in circulation” must give way to “online-specific metrics … especially as libraries invest more resources in digital initiatives and put bigger parts of their collections online. And it will be important … for the measurements to move beyond the strictly countable … into attitudinal areas like level of engagement and customer satisfaction. … [I]n the bigger context of changes, this resistance to [measure staff performance] should be easy to surmount. Institutions that proactively measure performance, embrace change, and look for ways to serve users will have an easier time getting financial support in an era of reduced public resources and private donations.” [Emphasis added.]
EXACTLY WHAT MR. CARLEE IMPLIED! AGAIN – “Institutions that proactively measure performance, embrace change, and look for ways to serve users will have an easier time getting financial support in an era of reduced public resources and private donations.”
The Institute of Museum and Library Services (IMLS) defines outcome-based evaluation (OBE), and provides an excellent summary of what it looks like.
What is outcome evaluation?
IMLS defines outcomes as benefits to people: specifically, achievements or changes in skill, knowledge, attitude, behavior, condition, or life status for program participants…. Any project intended to create these kinds of benefits has outcome goals. Outcome-based evaluation, “OBE,” is the measurement of results. It identifies observations that can credibly demonstrate change or desirable conditions…. It systematically collects information about these indicators, and uses that information to show the extent to which a program achieved its goals. Outcome measurement differs in some ways from traditional methods of evaluating and reporting the many activities of … libraries, but we believe grantees will find that it helps communicate the value and quality of their work to many audiences beyond IMLS [like funding decision makers].
…
How does a library … do outcome evaluation?
Outcome-based evaluation defines a program as a series of services or activities that lead towards observable, intended changes for participants …. Programs usually have a concrete beginning and a distinct end. The loan of a book … might constitute a program, since these have a beginning and an end, and increased knowledge is often a goal. An individual might complete those programs in the course of a single visit. Outcome measurements may be taken as each individual or group completes a set of services (a workshop series on art history, an after-school history field trip) or at the end of a project as a whole. Information about participants’ relevant skill, knowledge, or other characteristic is usually collected at both the program beginning and end, so that changes will be evident. If a program wants to measure longer-term outcomes, of course, information can be collected long after the end of the program.
… [Example]
PROGRAM: COLUMBIA COUNTY READ TOGETHER PROGRAMProgram Purpose: The Columbia County Public Library, Columbia Regional High School, Columbia County Head Start, and Columbia County Literacy Volunteers cooperate to provide story hours, literacy information, materials, and other resources to increase the time parents and other caretakers spend reading to children.
Program Services:
1. Make information visits to neighborhood community centers, County Head Start programs, and Columbia High School parenting classes
2. Provide daily story hours for parents and other caretakers and children at library and other sites
2. Provide library cards
3. Provide literacy counseling
4. Connect learners with literacy tutors
5. Provide children’s and basic reader materials to meet individual needs
6. Provide a participant readers’ support networkIntended Outcomes: Adults will read to children more often.
Indicators: Number and percent of parents or other caretakers who read to children 5 times/week or more.
Data Source(s): Participant interviews.
Target for Change: At the end of year one, 75% of participating parents and other caretakers will read to children in their care 5 times per week or more.
Based on the example cited above, it seems simple enough. All it takes is thoughtful creation of a Strategic Plan that includes OBE. 21st Century Library Strategic Plan – Measures and Outcomes outlines in considerable detail the How To for creating a Strategic Plan that includes OBE.
If you aren’t doing it – WHY NOT? Don’t you want to demonstrate to your funding decision makers what your library is accomplishing – that it is valuable?
P.S. University of North Carolina SLIS has a nice list of OBE resources at their website.
Thank you Steve,
OBE’s are another example of practices we need to emulate that are common in the for-profit sector and among innovative not-for-profits. I do believe the impact of indicators such as numbers of visitors, circulation, programs sponsored and even anecdotal evidence of customer satisfaction are still essential in some circumstances. For example, we were able to leverage enough money from our municipal government to make a third year of staff furloughs and two week summer closings unnecessary. I want to make a point of telling our city council that those extra hours mean that x patrons are served, x computer hours made available and x programs sponsored. Those are numbers that will resonate with council members-especially in an election year! All that said, Mr. Carlee is correct: we have to do a better job of defining and promoting our core services and relating resources, be they books, computer use or programs to the provision of those services and the achievement of measurable outcomes. .
Thank you Bob. Good observation that we have to give funding decision makers positive feedback of our successes whenever they have given us what we ask for. Nothing is more damaging to future funding than to NOT demonstrate what we accomplished with what we were given. In spite of feeling like “Oliver”, libraries MUST recognize that funding can be WITH HELD MORE EASILY than it can be granted.