- Purpose: Assess the usability of the redesigned Library home page
- Stakeholders: Students, staff, faculty using the Library home page
- Survey dates: July 8-13, 2022
- Participants:
- Rising 3rd year undergraduate, History
- Rising 2nd year undergraduate, African American Studies
- Rising 4th year undergraduate, Environmental Science
- Rising 3rd year undergraduate, Political Science and Psychology
- Rising 3rd year undergraduate, Economics
- Methodology: Participants were recruited from our existing research queue. A protocol was developed based on known and anticipated issues, and common tasks were identified. Testing was conducted synchronously on Zoom. URLs and a PDF of task slips were “pushed” to participants in Zoom chat, and participants shared their screen. Sessions were recorded and live captions were captured by Otter.ai to provide a reasonable transcript of each session. During the session participants answered questions and completed tasks to determine usability of the prototype. Participants were compensated with a $25 Amazon e-card upon completion of the session. See full protocol.
- Objectives
- Assess clarity, intuitiveness, and usability of new design
- Assess visibility and use of main navigation and utility navigation
- Assess tone of new design and alignment with Design Principles
- Determine if tasks previously accomplished on current home page can be successfully completed with the new design
- Determine if common tasks can be successfully completed
Summary of Findings
- Virgo is heavily used by the testing participants and the prominence of the search bar is seen as an improvement by all.
- Today’s Hours are very visible and all 5 participants used the hyperlinked library names (T1).
- All participants used Equipment and Technology to find information about 3D printers (T5). An impromptu poll about the division by physical versus digital media had mixed results: 2 thought all the listings could be put together under one heading, the other 3 thought two shorter lists would be easier to read than one list (T5).
- 4 out of 5 participants had no trouble finding the “Visit Databases A-Z” link within Search, Borrow, Request (T4).
- Main nav was heavily used by participants, but utility nav was not.
- 3 participants used Today’s Hours and a hyperlinked library name to look for a map of a library location (Q4). 2 participants first used Using the Library to look for maps.
- Participants were usually able to find the links they use often in the new design (Q5). These included information about printing, specific LibGuides, My Account, Today’s Hours, and Research.
- 4 out of 5 participants did not identify the About the Collections link as the place they would locate a summary of the types of materials in the Library. Participants did not seem to connect the terms “materials” and “collections.” Several thought About was the right region and 2 thought Search, Borrow, Request would hold the answer (Q6).
- The term, Interlibrary Loan, is not universally understood by undergraduates, although 4 out of 5 participants found their way to the Make a Request heading on the Using the Library page, and knew they were in the right place (T3).
- Library Happenings adds prominence to events and was viewed positively by all participants.
- Participants used the terms “bright,” “white,” or “clean” to describe the new design, although sometimes also used the terms “plain” and “empty” (Q7). Other terms used were “user-focused,” “inviting,” “convenient,” and “navigable.”
- 4 out of 5 participants found Explore Study Spaces under Library Spaces. The 5th went to Today’s Hours and stopped looking after she found the Room Reservation button (T7).
- 2 out of 5 participants found Subject Guides within Search, Borrow, Request, and a 3rd eventually found it there. Using Virgo and Get Help were the other strategies (T8).
- All participants used Equipment and Technology to look for tutorials. Some would opt for Workshops and Training while others would click on the specific technology (“3D printers”) and expect to find tutorials there (T9). None looked under Get Help or considered “How Do I…”
- Final impressions (Q8) were mostly positive:
- “Unlike the other library site where you sort of, were expected to come in with exactly what you wanted…These allow for more exploration.”
- “Organized very well… it was very easy to navigate and very easy to pick up on”
- “These dropdowns are pretty organized and I don’t think I could get lost in any kind of wormholes.”
- “I feel it’s very straightforward.”
- Likes the directness of clicking on Research in the current design, doesn’t like having to browse under Search, Borrow, Request (considers it less convenient). Doesn’t like the subtlety of the colors, likes the UVA theme on the current design. This participant offered that she would rank the current page a 4, and this new interface a 3 or 2.
- Other 4 participants gave rankings of 5 or 4 (Q9).
Recommendations
- Virgo search bar and Today’s Hours with hyperlinked libraries were viewed very positively. Do not change!
- Under Make a Request heading, add text to ILL link to identify what service it provides.
- As determined in previous testing, many undergraduates want to find a link to a specific library, then drill down to what services or equipment they offer. Others want to find the service, then see which libraries have what they need. The hyperlinked libraries under Today’s Hours should continue to be an important navigation feature.
- Information about printing is findable but could be more prominent.
- Link relevant “How Do I…” tutorials in equipment-specific areas; for instance, 3D Printing should hold everything related to both reserving and learning how to use 3D printers.
- The division between physical and digital media is correct but most important is that the items listed there are easy to scan.
Full Protocol
[Push https://www.library.virginia.edu/ to participant]
Q1: This is the current Library home page. In what ways have you used this page?
Rationale: Assess familiarity with Library website, which may inform how well they complete tasks; ascertain important tasks; Note one or two tasks to use in Q5
Goal: Introduce the new home page design; Ask questions that don’t require clicking on links
[Push prototype design URL to participant]
Q2: This is a prototype of the new Library home page. Remember, many links won’t work! What are your first impressions of this page?
Rationale: Impressions of design should indicate tone, alignment with Design Principles
Q3: Where would you look for more information about the Fine Arts Library? Outline your first steps.
Rationale: Do they head for Today’s Hours or use main nav (Using the Library/Library Spaces or About)?
Q4: Where would you go to find a map to a library location? [If not successful]: What would you try next?
Rationale: Will they find “Plan a visit” under About or use “See all hours and locations”; or will “Using the Library” translate to “Visiting the Library”
Q5: You mentioned [this specific task] as something you have done on the current Library home page. Where would you accomplish that task on this new page?
Rationale: Success at completing a task important to the participant is a benchmark for effectiveness of new IA and navigation
Q6: Where would you find a summary of the types of materials held by the Library?
Rationale: Be able to find About Our Collections page under About in main nav; does “materials” translate to “collections”
Q7: Describe the new Library home page in three words. What page elements make you think of those words?
Rationale: Choosing three terms requires thought about tone; do terms align with Design Principles?
Goal: Complete common tasks
TASK1: You will need to go to the Music Library tomorrow morning, but don’t know if it will be open.
Rationale: Do they use utility nav or “See all hours and locations” (links won’t work)
TASK2: You need a book, “Gender and Body Language in Roman Art,” for a paper you’re writing.
Rationale: Can they locate the search box, interpret Virgo as library catalog as the tool to use to find a book in Lib collection
TASK3: Good grief! UVA Library doesn’t have an article you really need for your paper. How can the Library get a copy for you?
Rationale: Find LEO/ILL request, which has moved from home page to Search, Borrow, Request; Does link, “Request Interlibrary Loan” have meaning; is it seen under Make a Request heading; could also search Virgo, find it in WorldCat, and request ILL from there, or can find request ILL in one’s Virgo “account”
TASK4: You want to start another research project by finding an article in the Web of Science database.
Rationale: “Popular Links” moved from Research page; new path is to Search, Borrow, Request to link, “Visit Databases A-Z”
TASK5: You need to use a 3D printer for an assignment. Outline your steps to gather information.
Rationale: Will they look for it on Equipment & Technology, is it visible under Physical Media subheading under Workshops and Training
TASK6: Your professor mentioned a lecture sponsored by the library that she thinks may be occurring soon. Find out if she’s right.
Rationale: Find Library Happenings below News, Blog, & Events
TASK7: Are there any places to study in the libraries that you DON’T have to reserve?
Rationale: Can they find “Explore study spaces” under Using Library Spaces (should it be above Reserve Library Spaces?)
TASK8: You’ve used an online guide of Psychology resources put together by librarians. If only you’d bookmarked it! Can you find it now?
Rationale: Find a subject guide under Search, Borrow, Request; site search will also work in the future
TASK9: It’s summer! And a good time to learn more about 3D printers through online tutorials. The Library has some but where do you start?
Rationale: Find “How Do I” link under Using the Library / Getting Help
Goal: Gather final assessments
Q8: What are your final impressions of the web pages you’ve worked with today? I’m interested in the organization of information and your ability to navigate to complete tasks.
Rationale: Final opinions on usability and possible improvements; determine if any opinion has altered since start
Q9: On a scale of 1 to 5, where 1 is Very Hard and 5 is Very Easy, how would you rank the ease with which you can find information and complete tasks on these web pages?
Rationale: Provide a quantified measure on ability to complete tasks