User Experience Project ID: UX-1922, Library homepage development
Purpose: Assess usability and clarity of proposed website designs
Stakeholders: UVA Library staff and users

Test dates

• 2/22/17 – 2/24/17 ; 9/14/17 – 9/19/17 ; 11/15/17 – 11/21/17 (usability testing with UVA undergraduate and graduate students)
• 4/5/17 – 4/14/17 (online surveys of UVA students, staff, and website visitors)

Methodologies: Usability tests (3) and online surveys (3) to present iterations of Library home page designs to test usability and elicit feedback. Efforts were focused on search box location, carousel location and navigation, Hours page, Research page, mobile and tablet navigation, and link naming.

Summary of findings

• Hours are easily found and testers liked its new prominence
• Mobile version of Hours is easily navigated and was praised as “well-formatted”
• Improved visibility of search, which is a primary or secondary task for website visitors
• The “Ask a Librarian” brand is firmly established and the “Ask a subject specialist” link seems to be understood as another place to get help
• Iterations of new design were usually viewed as more attractive than the previous versions
• Mobile hamburger menu is more easily found when labeled “MENU”
• The Research page redesign was described as clean, efficient, logical, and organized
• Thumbnail images under the carousel image helped testers navigate content

Project status: Newly designed website theme rolled out 3/7/2018

Final reports (UVA only): https://virginia.box.com/v/2017WebResearch

User Experience Project ID: UX-2368, Facilitate two focus groups with graduate students

Purpose: Investigate sources of dissatisfaction among graduate students
Stakeholders: Academic Engagement, Administration & Planning, Collections, Access & Discovery and other Library areas
Focus group dates: 11/29/17, 11/30/17
Participants: 11 graduate students (including one session with five international graduate students)

Methodology
Low ratings by a small subset of graduate students in the 2016 user survey were a cause for concern, but root causes were elusive. Two focus groups were offered to assist with identifying issues. One focus group was limited to international students to learn of their specific needs. Most students were recruited from usual avenues (web banner on the Library home page; contact cards left at print stations). Others had participated in the 2016 user survey and agreed to future participation in user research, and several were invited based on past feedback they’d given. UX research staff created two focus group protocols probing on problems and resolutions, confusion or difficulties, type of communication preferred, and how the Library can better contribute to graduate student work and to their overall positive experience. The international students were also asked about specific challenges they face.

Major findings

• Communication and connectivity are important elements to a positive graduate student experience. Graduate students want us to market to them early and often, and want communications and workshops that are geared toward them.
• Graduate students want document delivery service and they want us to be proactive, try harder, and care more about getting the paper and electronic materials they need.
• Library systems and policies are confusing and frustrating, especially to incoming students. Many navigate with the help of their peers. They want DIY short online video tutorials for online tools that can be found and viewed at point-of-need, but like in-person assistance with subject specialists for in-depth research.
• Graduate students want space for work and storage in libraries. Physical comforts impact willingness to use and stay in library spaces.
• Students with mobility issues can be negatively impacted by collection decisions that split call number runs between multiple libraries.
• International students matriculate with few expectations for library services beyond checking out books and comfortable study space; there were no notable differences in library experience with their peers from the U.S.

Recommendations

• Create a communication plan, to include physical materials, for incoming and other graduate students to convey services, policies, and contacts.
• Create workshops to engage graduate students in library services and research resources.
• Create short online video tutorials for services and processes.
• Consider providing scanning and electronic delivery of print journal articles to all graduate students.
• Explore possibilities to expand dedicated graduate student space in all libraries.
• Increase efforts to convey information about critical systems, services, and policies to graduate students, particularly incoming. Include basic information about online catalog, Interlibrary Loan, policies (borrowing, recall, course reserve), off campus access to online research materials, printing, obtaining a carrel, how to reserve a classroom, and stacks navigation.
• Increase in-person communication with graduate students about subject liaison program.
• Review tight stacks areas that may provide poor accessibility.
• Consider revising collections policies to keep all materials in the same subject together to enhance accessibility for students with mobility issues.
• Do further research to gather more complete data on the following:
  • assess how the Library should communicate with graduate students;
  • gain a deeper understanding of the specific needs of international students;
  • determine graduate student priorities for document delivery, paging from other libraries, and other delivery services;
  • assess what current graduate students wish they’d known as incoming students.

Final Report and protocols (UVA only): https://virginia.box.com/v/gradfocusgroups

User Experience Project ID UX-1687: Create, Launch, and Evaluate WhatUsersDo.com Online Test

• Purpose: Assess new home page design by non-UVA users
• Stakeholders: UX team
• Test date: 12/7/16
• Test participants: 3 online testers in the United States, age 18+, each using a desktop or laptop computer
• Methodology: I received three credits to run a remote test with WhatUsersDo.com. I wrote the test (see project files) and typed it into the online form. I was able to designate what device I wanted testers to use (desktop, smartphone, tablet), plus choose the testers’ country (UK, US, France, Germany, Netherlands), age range, and socioeconomic status. Testers were asked to view https://uvalib.github.io/pr
  

  Prototype screen (partial)

  ototype/ and use it to complete six tasks and rate their ease at completing each task. Testers were also asked for their opinions about the site. The testers used screen cast software which recorded their voice and keystrokes, and they were told to think out loud as they worked through the test. Links to the resulting .mp4 files and a summary of the test were made available for download.

Findings

• 3/3 testers had trouble locating subject specialist help from a Virgo screen. After a minute of clicking and scrolling 2/3 testers found the answer.
• From the Library Hours page 2/3 testers clicked on Collections when asked to look for a book, possibly because there is no search box on the Hours page (see fig. 1). One tester used the search box at the top of the Collections page to search Virgo, and the second tester used a Virgo link under the heading, “Ways to Explore” on the Collections page (see fig. 2). The third tester used his browser back button until he found a search box on a library page. All testers were able to successfully complete the task.

Figure 1 Library Hours page

Figure 2 Library Collections page

• 3/3 testers had some difficulty finding a library lab with reservable space, although each was eventually successful (in different ways). The first tester went to Reserve a Room under Spaces and Equipment, then clicked on Group Study Rooms (a link to the booking software). The second tester also selected Reserve a Room (but via the Services page). She scanned for the term “lab” but went back to the Services page when she didn’t find it. She then clicked on Digital Labs, Study Space Info, and again on Digital Labs before finding text about reserving rooms within the Scholars Lab. The third tester also went into Reserve a Room under Spaces and Equipment, then backed out to the main page and selected Explore Libraries and Labs, where he found text about reservable rooms in the Library Data Commons@Curry. It is reassuring that each tester found the answer, and it supports that we need to maintain some reciprocal links and different navigational paths.
• All testers began new tasks without navigating back to the main page, which reinforces the need for good navigation and design from every page.
• One tester confessed to being a retired librarian, so was quite familiar with terminology like “online catalog” and “journal finder”. Not my intended audience!

Project files: WhatUsersDo_Report

Project Details

• User Experience Project IDs: UX-1609 and UX-1414, Plan and Conduct First-Year focus groups
• Purpose: To understand changes in Library usage, attitudes, and perceptions over time; and to assess the feasibility of using a panel of committed First-Year students for this purpose.
• Stakeholders:UX team, First Year and other AE Instructors
• Dates: 9/29/16 and 11/11/16
• Participants: 9 undergraduate students
• Methodology: An open session was offered to library staff to generate questions for First-Year student focus groups. Participants determined categories for the questions and UX research staff created a focus group protocol of 10 questions. Seven FY students participated. Due to difficulty in recruiting enough students for a second focus group we instead interviewed two additional FY students with a protocol revised based on the results of the focus group.

Findings

• FY students use libraries for quiet study, meeting friends, finding resources.
• Librarians are useful for helping to navigate stacks and find books, help with printers, and make recommendations.
• Librarians go out of their way to help. Tours with librarians have been very helpful. Some students don’t have much contact with librarians.
• FY students believe that no one will touch their stuff in the library because of the Honor Code.
• Early impressions of walking into a UVA library were that there were lots of study spaces and not many books in sight.
• Navigating in the libraries is difficult for new students. It’s not clear how to look up a book or find a book in the stacks. Stacks are “confusing” and finding a book “takes way too long.”
• Things FY students like about the Library:
  • The late/all-night hours
  • Can always find a place to study
  • Egg chairs
  • “Librarians are always very friendly and they just want to help.”
  • Meeting rooms
  • Areas with tables and white boards
  • The absolute silence of the Harry Potter room
• FY students in their first semester have done virtually no research in the library
• FY students trust information that comes from professors, subject experts, upperclassmen, librarians, parents, peers.
• FY students frequently pack for the day and bring books, laptops, pencils, food/drink, chargers, and phones with them.
• Most take handwritten notes in class because electronics are a distraction; easier to format and organize; “can’t do math problems on a laptop”; helps to focus better; “I’m thinking as I’m writing.”

Next Steps

• Create survey themes and questions based on above findings.
  • Possible themes to include Spaces, Navigating, Searching, Help.
• Seek IRB approval in order to retain the option to publish results.
• Recruit First-Year students during summer orientation to create a panel of up to 10 students committed to participating for at least one semester.
• Plan for 8-10 online surveys, and possibly an in-person interview, during the 2017-2018 school year.

First-Year Panel Project: https://confluence.lib.virginia.edu/display/UX/First-Year+Research+Panel+Pilot

Project Files (UVA only):  https://virginia.box.com/v/1stYrPanel