Report on Usability Process and Assessment of Yufind

1. Report on Usability Process and
Assessment of Yufind
Nelinet
May 27, 2009
Usability and Assessment at Yale University Library
http://www.library.yale.edu/usability

2. Collecting evidence from Orbis

log files. Assess current behavior
2

3. Log file line
3
12/2/07 20:06

20071202200646
Title Index
TALL JOHN DOGGET JNR
0

4. Log file sample descriptive data
4
…
Most common search: title (41.8%) followed by
keyword (31.4%)
…
Average search phrase is 2.5 words long
…
Most common phrase searched is united states
…
Most common number of hits: 0 (21.4% )

5. Examples of title searches
5
…
BOOKS ON PHOTOGRAPHY?
…
ALABAMA 2004?
…
PATHOPHYSIOLOGY CHILDREN?
…
JOHNSON, JAMES?
…
INTRODUCTION TO AFRICAN AMERICAN STUDIES:
A READER?
Topic
Name
Close

6. Classification of Reasons for 0 Hits
6
N=506

7. Declining search types
7

8. Some conclusions from log analysis
8

9. Next Gen OPAC --

Yufind
9

10. 10
Search for Geriatrics

11. After deciding to implement Yufind, use data
from log files to help design usability tests Conduct usability tests
11

12. Creating usability tests for Yufind
12
…
Objective: Would the participant see and choose
facets? (more than 26% of the time)
„
Priority = High
„
Success = See and try to use the facets
…
Objective: Would the participant use facets
successfully?
„
Priority = High
„
Click on the appropriate face t and reach desired outcome
within two clicks

13. E-books only service
13

14. Alternative interface for catalog
14

15. Questions ‰
Find books within a set by a particular author
(John Adams).
‰
In a set of books about healthcare, find some
that are about healthcare reform.
‰
Within a subject-based display of ebooks
find a topic-based subset.
15

16. 16

17. Facets: key findings
17
…
Were seen and sometimes used but †
Subsets didn’t always make sense
†
Hard to navigate (not alphabetical)
…
Little use of call
number facet
…
Some preferred search refinement to facets
…
Participants noted potential usefulness: format,
language
…
Some users are a target market, others are not

18. Did facets increase metadata use?
Opportunities for
facet use
Facet
Use|Success
Search Refinement
with 2
nd

search
Use|Success
5 (question 2a
undergraduate)
3|1
3|1
5 (question 5a undergraduate)
2|2
1|0
8 (question 5 medical)
4|4
4|1
All opportunities=18
9|7
50.0% | 38.9%
8|1
44.4% | 16.7%
18
YES

19. Relevance ranking and facets
19

20. Best practices for facets …
Implement a more narrowly focused search
…
Present a limited number of facets based on user
behavior
…
Good facets require good data
…
Well defined, mutually exclusive and have clear
labels
…
Facets work best when they are tied to the level of
the search of selection process
20

21. 21

22. 22

23. Limit facets based on user behavior
…
Author
…
Journal/book/video
…
Language
…
Classification
…
Availability
…
Location
…
Fulltext
First Level
Second Level
23

24. Implement changes and re-assess
24

25. Survey (83 respondents) …
79.3% found the results they expected and 75.6%
found what they needed
…
41 respondents or 50% tried facets. Of these,
85.0% were positive about the results.
…
57.8% of respondents preferred Yufindto Orbis
…
System Usability Score = 66
25

26. First place you would look to find a
book?
Answer Options Response Count Percent
Orbis33 40.2%
Yufind29 35.4%
Google14 17.1%
Amazon4 4.9%
Worldcat2 2.4%
26

27. Standards & priority functionality
27

28. Patron empowerment
28

29. Error handling
29

30. Effective and efficient search and
delivery …
Provide alternative navigation through facets
…
Visual shelf list
…
Integration with circulation/ordering/ILL
30

31. Usability process …
Assess current behavior
…
Conduct usability protocol testing
…
Implement changes and re-assess
…
Determine standards & priority functionality
…
Need more cooperative efforts
31

32. Results available
32
Usability and Assessment at Yale University Library
http://www.library.yale.edu/usability
Send questions to [email protected] http://www.library.yale.edu/libepub/usability/yulstudies.html