Library workshop evaluation

From HLWIKI Canada
Jump to: navigation, search
Student perceptions of a library workshop
Are you interested in contributing to HLWIKI International? contact: dean.giustini@ubc.ca

To browse other articles on a range of HSL topics, see the A-Z index.

Contents

Last Update

  • Updated.jpg 20 May 2017

Introduction

See also ADDIE model | Assessment | BOPPPS Model | Instructional courses & theory for librarians | Presentation skills | Teaching library users

"...The very best teaching, like the very best writing, is often transparent..."

Library workshop evaluation (see assessment of information literacy) has received increased attention in the library profession in recent years. (The Association of Research Libraries has an extensive bibliography of papers, presentations, and articles that review the use of LibQual+™ for library workshop evaluation.) Often, evaluation and/or assessment will focus on the librarian's teaching or simply how well students (or participants) respond to the teaching session in question (i.e., whether they found it to be of value). Some academic librarians regularly evaluate their workshops as part of their quality improvement (QI) practices, but others have begun to select formal, increasingly sophisticated assessment methods to do so. In 2013, both information literacy programming and library assessment trends place a renewed emphasis on academic librarians' roles within the academy.

Entire information literacy programs (or even one-off workshops or library "in-services") can be incrementally improved simply by engaging other librarians in a peer review process, and building feedback into teaching for the instructor-librarian. This is very important for the novice or junior members of the library profession. The problem for many academic librarians is knowing where and how to begin, and which evaluation methods and tools are most effective and suitable. The aim in library workshop evaluation is to begin the process of evaluating our delivery but also to incorporate new knowledge, skills and approaches into future delivery of workshops. By adopting the most current best practices, it may be necessary to read the case literature especially as revealed by other librarians in the field (and even the many blogs available on the topic).

Before undertaking an assessment of your teaching, it may be worth considering why you want to evaluate yourself, your process or pedagogical methods - or whether it's simply about the content of your workshops. Further, what are the reasons for evaluating your pedagogical methods? Is it to individualize learning for your students and differentiate how you deliver instruction to improve learning? By acquainting yourself with current teaching practices, and building library assessment into your development of courses, the process may also be easier. In the final analysis, assessment/evaluation is recursive and should entail periods of practice, adopting new techniques and conducting empirical evaluation in continual loops of refinement.

See also the Association of Research Library (ARL)'s Library Assessment blog.

Why engage in assessment? Part I

An evaluation of a workshop, its value, usefulness ...based on a Likert scale of 1-5

Instructional librarians need to evaluate their goals and objectives as one part of their assessment cycle. Assessment can be a useful tool for aligning librarian priorities and pre-existing student skills. Formative and summative assessment methods can be deployed to gather empirical evidence about librarian-led workshops.

Are your goals in assessment to...?

  • Determine whether your teaching program, and its components, makes a difference in student learning
  • Assess library performance individually (e.g., each librarian) and collectively (i.e., entire program, staff)
  • Assess whether teaching students is a good use of precious professional resources
  • Demonstrate a need for funding or redistributed funding in the university
  • Improve teaching and reference services to make them more responsive to student needs
  • Redesign teaching materials and methods, and use them more effectively
  • Practice evidence-based librarianship throughout the process, and ground activities in pedagogies

Source: Principles of Good Practice for Assessing Student Learning. American Association for Higher Education

Why engage in assessment? Part II

  • What do you hope to gain from assessment? Assessment is most likely to lead to improvement when it is part of a larger set of conditions that promote change.
  • Assessment is most effective when it reflects an understanding of learning as multidimensional, lifelong and often social.
  • Theory: assessment for learning, assessment as learning, and assessment as learning to teach are the three assessment theories underpinning assessment practices at most libraries. What theory will be most useful for you and your library programs?
  • Links to strategic documents: connect assessment to the institution’s mission, vision and general learning outcomes.
  • Structures: Are there assessment committees or a coordinator who might shape the assessment practices? What organizational structures will facilitate assessment?
  • Resources: Assessment requires staff time and materials at the very least. You may also wish to hire a consultant or provide professional development opportunities for staff involved in assessment efforts. The resource section of an assessment plan will describe these needs.
  • Data Policies: Include policies for data gathering, storage, access and reporting that will protect the rights and privacy of students and librarians.
  • Goals & Outcomes: A list of agreed-upon overarching goals and specific, measurable learning outcomes is a necessary element of any assessment plan.
  • Timeline for Continuous Assessment: Describe the schedule for assessing and reassessing individual outcomes. This should articulate realistic plans and recognize that a “one at a time” approach to outcome assessment is best.

Source: Instruction Services. Plan for Information Literacy at the University of Rhode Island.

Methods of assessment

There are as many methods of assessing a library workshop or class as there are learning theories. A variety of assessment tools can be used to gather information about student learning and information literacy instruction: pre- and post-surveys, student feedback surveys, faculty feedback to librarians, librarian self-reflection, library worksheets, student research journals, and citation analysis of students' final research paper bibliographies. The quality, amount and level of instruction (including shared instruction) drives the assessment process. Is the class a one-off workshop and unlikely to be offered again? Or is the workshop embedded within a course? Faculty assessment of a librarian is common, although specific suggestions about how improve teaching can sometimes be lacking from a faculty member. Peer assessment is important for librarians at all levels, competencies and years of experience.

A common method of evaluation is to administer a simple questionnaire after a class. A questionnaire invites participant feedback and assesses satisfaction with the session, and "how effective" a librarian was at teaching. Academic librarians can use pre- and post-test measures for ongoing, quality improvement of teaching. Questionnaires provide some view of how much users learn or retain after our sessions. In the literature, studies suggest that participants view their abilities after library workshops differently from their actual skill levels. By measuring skills post-hoc, some librarians use search quizzes to test comprehension of what the librarian taught. This approach is suitable for academic institutions where librarians liaise with faculty.

How to choose the best assessment tool for student learning

In selecting an assessment tool, you may want to ask yourself some of the following questions:

  1. Does the assessment tool adequately evaluate academic performance relevant to a desired outcome? (validity)
  2. Does this assessment tool enable students with different learning styles to show you what they have learned?
  3. Does the content examined by the assessment align with content from the course? (content validity)
  4. Does assessment adequately address the knowledge, skills, abilities, behaviours and values associated with the intended outcome? (Domain validity)
  5. Will assessment provide information at a level appropriate to the outcome? (Bloom’s Taxonomy)
  6. Will the data accurately represent what students can do in authentic or real life situations? (authentic assessment)
  7. Is the grading scheme consistent; would a student receive the same grade for the same work on multiple evaluations? (reliability)
  8. Can multiple people use a scoring tool, and come up with the same general score? (reliability)
  9. Does assessment tool provide data that is specific enough for the desired outcomes? (alignment with outcome)
  10. Is the assessment summative or formative - if formative does it generate diagnostic feedback to improve learning?
  11. Is the assessment summative or formative - if summative, is the final evaluation built on multiple sources of data?
  12. If this is a summative assessment, have students had opportunity for formative feedback and practice displaying what they know?
  13. Is the assessment unbiased or value-neutral, minimizing an attempt to give desirable responses and reducing any cultural misinterpretations?
  14. Are the intended uses for assessment clear? (grading, program review, both)
  15. Have other faculty provided feedback? Has the assessment been pilot-tested?
  16. Has the evaluation instrument been validated?
  17. Will the information derived from the assessment help to improve teaching and learning? (AAHE Good Practice)
  18. Will you provide the students with a copy of the rubric or assignment grading criteria?
  19. Will you provide students examples of model work?

Peer-to-peer observation

  1. After observing the class, how well do you think the class went?
  2. Did the librarian connect with the students?
  3. Did the students accomplish the goals planned for the class?
  4. Were any visuals used to amplify the goals?
  5. What were the teacher's strengths today?
  6. Did you notice anything especially engaging or useful?
  7. What did not work well? Is this a common problem?
  8. Did the librarian provide a summary? How did the librarian conclude the class?
  9. Can you provide any suggestions or ideas for improvement?

Teaching style evaluation

  1. What steps did the librarian take to create a friendly/open attitude for the class?
  2. How did the librarian employ his/her voice throughout the class? Was it clear/audible?
  3. How did the librarian engage with students? Did they ask questions? Make eye contact? Move through the classroom?
  4. Did the librarian use a scripted presentation or conversational? How did this work with the goals of the session?
  5. What active learning techniques were used? What seemed to work? What didn’t?
  6. How did the instruction encompass different learning styles?
  7. How was student learning evaluated during the class? By asking questions? Exercises?
  8. What examples did the librarian use to clarify concepts? Were they effective?
  9. Was any library jargon employed? If so, was it explained adequately?
  10. What three things did you find most effective in this presentation? areas for improvement?

Key websites & video


The purpose of these presentations is to explore the use of authentic assessment to evaluate learning within the 50 - 90 minute library instruction one-shot classroom.

References

Personal tools
Namespaces

Variants
Actions
Navigation
Toolbox