Refreshing the RMC web pages

  • Purpose: Assess the usability of the refreshed Robertson Media Center pages Stakeholders: Students looking for and using media services on the web
  • Survey dates: February 1-3, 2022
  • Participants:
    • 2nd year undergraduate, Nursing
    • 2nd year undergraduate, Commerce
    • Graduate student, Religious Studies
    • 4th year undergraduate, Economics and Middle Eastern Studies
    • 2nd year undergraduate, BACS
  • Methodology: Participants were recruited from our existing research queue. A protocol was developed based on feedback of the previous design, and common tasks were identified. Testing was conducted synchronously on Zoom. URLs and a PDF of task slips were “pushed” to participants in Zoom chat, and participants shared their screen. Sessions were recorded and live captions were captured by Otter.ai to provide a reasonable transcript of each session. During the session participants answered questions and completed tasks to determine usability of the web site. Participants were compensated with a $25 Amazon e-card upon completion of the session. See full protocol.
  • Objectives
    • Determine what keywords are commonly used to search for the RMC website
    • Determine how students navigate from the Library home page to the RMC pages
    • Assess clarity and intuitiveness of Spaces, Equipment, and Learning categories
    • Assess visibility of the new camera comparison and 3D printing studio pages
    • Assess visibility of new subnav on RMC child pages
    • Determine if common tasks can be successfully completed

Summary of Findings

  • All five students had heard of Robertson Media Center and some had been to the space in Clemons but only one had actually used services for a class.
  • Participants were asked to start on a web search page and to look for a studio in the Library that they could reserve to record a podcast. Four out of five students followed instructions and successfully found their way to an RMC page where they could reserve an audio studio. Search strings used were CLEMONS LIBRARY RESERVE; LIBRARY PODCAST UVA STUDIO; ROBERTSON MEDIA CENTER; and UVA LIBRARY RECORDING ROOM.
  • Participants were then asked to navigate from the Library home page to the RMC page. Four out of five students followed instructions and eventually located the RMC page, but several commented that it was not prominently linked. Two participants saw RMC listed in Today’s Hours and clicked on “All Hours & Locations” while the other two clicked on “Explore Libraries & Labs” under the “Spaces and Equipment” heading. Shortly before testing began we changed the “All Hours” link to read “All Hours & Locations” which may have had a positive effect on findability.
  • Terms used to describe the RMC main page were as follows:
    • interactive, visually appealing, informational
    • helpful, neat, cohesive
    • organized, concise, general [leading to subpages with more detailed info]
    • digital, accessible, innovative
    • links to resources, calendar, general information
  • The three categories on the RMC page are intuitive and clear. Two participants thought the Learning page was more geared toward instructors because of the headings “Consultations” and “Classroom support for instructors.” “Walk-up Equipment” was understood to mean no reservations are needed. One participant expected to find RMC classes linked to this page.
  • Most participants struggled when asked to confirm if there was an RMC student advisory group, which didn’t neatly fall into the Spaces, Equipment, and Learning categories. The answer was in a block of text and was not a hyperlink, and only two out of five found it. Generally, the observed behavior is that they quickly scan for headings and prominent links and make quick decisions as to whether they’re on the right track, and back out if they think they’re headed down the wrong path.
  • Participants had to bounce back and forth between RMC pages and calendar pages to complete tasks which led one to comment, “Going back [to the RMC page] isn’t as easy as I would like.” There are known limitations in using the calendar product, but we should work to integrate the reservation process into the RMC web pages for ease of use and a better user experience.
  • None of the five participants found the “camera comparison” page to help them determine if a camera had a zoom lens or could do still photography. Some handled this task by doing Google searches and skimming descriptions on the reservations page, but only one participant found the answer and was confident that they’d successfully completed the task.
  • When faced with a task to find and reserve a 3D printer, all participants headed for the Equipment or Learning pages. None found the 3D printing studio page. When asked specifically about finding a studio, participants used the Spaces page. All were eventually able to complete the task.
  • There was minimal use of the subnav in favor of the breadcrumbs at the top of the subpages. There was common understanding that the Library logo would lead to the Library home page.
  • On a scale of 1 to 5 where 5 is ‘very easy’, on average the participants ranked the ease with which they could find information and complete tasks as 4.

Recommendations

  • Add a link to RMC classes on the Learning page. Rename heading to “Help and consultations” to be more inviting to undergraduates. Provide anchors for each section to make training section more prominent.
  • Links to the camera comparison page should be added in the description for individual camera reservation pages and added on the Equipment page in the following two places:
  • RMC digital camera RMC high-end digital cameraAdd prominent links on every RMC-related calendar page to the RMC equipment and space web pages.
  • We can’t currently put reservable cameras on same page as walk-up cameras and related equipment, but we want one calendar page for all cameras and related equipment. Also want to be able to show 3D printers (on Equipment calendar) with 3D printer studios (on Spaces calendar). Equipment and Spaces calendars are currently completely separate.
  • If calendars can’t be combined as requested above, navigation would be much easier with a subnav menu to link spaces and equipment calendars and related web pages (RMC equipment and space pages, Library home page, Library Services page).
  • Trainings and instruction materials get lost in a semi-hidden tab “instructions” when viewing equipment calendar pages. Would like to hyperlink the work “instructions” in the description tab.
  • Spaces calendars have a “week view” in addition to the default “day view.” If possible, add “week view” to all equipment availability calendars for consistent user experience.
  • Fix reservation calendars so user can select multiple hours. Studios can be booked for a maximum of three hours but a user can’t click on a second and third hour block. We should at least mitigate the confusion it causes by prominently listing the maximum number of hours a studio can be booked and include instructions on how to do so in the space description above the reservation calendar.
  • Longer term, consider if we should integrate the whole reservation process into the RMC equipment and spaces pages for a better use experience.

Full Protocol

Start: https://www.google.com/ (or anywhere not on Library website)
Target: https://www.library.virginia.edu/rmc and child pages


Goal: Warm-up

Q: Are you familiar with the Robertson Media Center? Have you used the RMC website? Have you physically been to the RMC in Clemons Library?

Rationale: Assess familiarity with RMC space and website, which may inform how well they complete tasks


Goal: Find RMC services by searching keywords or navigating from Library home page

Q: [Open your favorite web browser to any search page that is not a Library web page, such as google.com.] You are looking for a studio where you can record a podcast. Does the Library have one that you can reserve?
What makes you think you have or have not found what you are looking for?
Can you reserve a studio for next Friday? Book it for the maximum number of hours allowed.

Rationale: Assess ability to find RMC page, Spaces subpage, or audio studio reservation page. Note terms used to search. Note path navigated to RMC. Direct participant to the Spaces page if needed. Assess ability to reserve a studio for a future date.

Q:  How would you find out what other services the RMC offers to you? [If appropriate: Can you do get there without using the browser back button?]

Rationale: Assess ability to navigate from calendar to Library home page to RMC. Note path if navigating or keywords used if searching.

Q: [If needed]: Go to the Library home page. How would you navigate to the Robertson Media Center page from here?


Goal: Three categories on RMC page should be intuitive
Start here: https://www.library.virginia.edu/rmc

Q:  [before clicking] What do you think you’ll find under “Spaces”? [after clicking and reviewing page] How would you describe this category in your own words?

Rationale: Assess clarity and understanding of “Spaces” and two subcategories for equipment; Gain insight into user-centric terminology

Q:  [before clicking] What do you think you’ll find under “Equipment”? [after clicking and reviewing page] How would you describe “Walk-up Equipment” in your own words? How would you describe “Reservable equipment” in your own words?

Rationale: Assess clarity and understanding of “Equipment”; Gain insight into user-centric terminology

Q:  [before clicking] What do you think you’ll find under “Learning”? Can you name some possibilities for me? [after clicking and reviewing page] How would you describe this category in your own words?

Rationale: Assess clarity and understanding of “Learning”; Gain insight into user-centric terminology

Q:  Describe the Robertson Media Center page in three words. What page elements make you think of those words?

Rationale: Impressions of design; visibility of intro text, subpages, lower sections


Goal: Complete common tasks
Go back to Task Slips PDF document and scroll down to Task 1.

TASK1: You’ve heard there is a student advisory group that helps to run the Robertson Media Center. Can you confirm if this is true?

Rationale: Assess efficacy of the Mission and FAQ pages.

TASK2:  You’ve got a project to complete that requires a 3D printer, and you know you can use one in the Robertson Media Center. How would you find out what you need to know before you head over to Clemons Library?

Rationale: Find Spaces through subnav or from RMC main page, or navigate from Equipment to calendar pages and back to Spaces; Assess if 3D printing task leads to Equipment or Spaces, should FAQ be linked in either place, and make recommendations; Assess if the Equipment, Spaces, and/or FAQ pages answer common questions:

  • Who can use a 3D printer
  • How much does it cost to use a 3D printer
  • Can I reserve it
  • What training do I need

TASK3: You’re about to go to the RMC for your 3D printer appointment but wonder if there will be help nearby when you get there. What are your options going to be?

Rationale: Determine if help is readily available; Find links to “Learn more about RMC” and “read our FAQ”; correctly interpret information

TASK4: You need to use a video camera with a zoom lens to complete an assignment, but you’ll need to plan ahead if training is required. Outline your steps.

Rationale: Use subnav to get to “Learn about RMC equipment” or “Camera comparison information”; discern difference between “Camera equipment” and “High-end cameras” links; Assess visibility and effectiveness of sidebar nav; Assess clarity of instructions to get required training and reserving a video camera

TASK5: Now you’re thinking you might need a camera that can do both video and still photography. Does that mean you’ll need to do training and make reservations for more than one camera?

Rationale: Assess ability to navigate to the correct calendar page and tab for instructions; Assess visibility and effectiveness of sidebar nav; Assess clarity of instructions to get required training and reserving a multi-use camera

TASK6:  Your friend told you she’d used a space in Clemons that is set up for using virtual reality headsets, but what you really want to do is create your own VR content. What is available to you?

Rationale: Find information pertinent to reserving and using VR studios; ask the following questions if not addressed:

  • Which equipment can be checked out?
  • What equipment/studios need to be reserved?
  • How do I reserve a VR station?
  • Can I log into my own Steam account on the VR machines

Goal: Gather final assessments

Q: What are your final impressions of the web pages you’ve worked with today? I’m interested in organization and clarity of information, as well as design.

Rationale: Final opinions on usability and possible improvements; determine if any opinion has altered since start

Q: On a scale of 1 to 5, where 1 is Very Hard and 5 is Very Easy, how would you rank the ease with which you can find information and complete tasks on these web pages?

Rationale: Provide a quantified measure on ability to complete tasks

 

What We Learned from the Learning Hub Redesign

  • Purpose: Assess the usability of the redesigned Learning Hub area of the Library website
  • Stakeholders: Students and instructors looking for self-guided tutorials on how to use the Library
  • Survey dates: October 12-15, 2021
  • Participants:
    • 1st year undergraduate, Chemical engineering
    • 2nd year undergraduate, Government and Media Studies
    • Graduate student, Astronomy
    • 4th year undergraduate, Accounting
    • 2nd year undergraduate, Undecided
  • Methodology: Participants were recruited from our existing research queue. A protocol was developed based on feedback of the previous design, and common tasks were identified. Testing was conducted synchronously on Zoom. URLs and a PDF of task slips were “pushed” to participants in Zoom chat, and participants shared their screen. Sessions were recorded and live captions were captured by Otter.ai to provide a reasonable transcript of each session. During the session participants answered questions and completed tasks to determine usability of the web site. Participants were compensated with a $25 Amazon e-card upon completion of the session.See full protocol.
  • Objectives
    • Assess ability to locate online tutorials on Learning Hub site
    • Assess clarity and intuitiveness of Learning Hub categories
    • Determine if redesigned area is clear and inviting
    • Determine if tasks can be successfully completed

Summary of Findings

  • Students struggled to answer the question: “If you were looking for some tutorials to help you learn how to use the Library, where might you start?” Participants were directed to start on the Library home page. All 5 participants looked under About (a link on the menu bar). All were told that there was a link on the home page to help them complete this task. Each participant eventually found Learning Hub under Services, clicked through, and decided they were in the right place.
  • One participant offered this insight: “I don’t think students who are looking for help will realize that there’s actually a resource that gives you all of the links for learning how to use the library… They might look for individual sections or contact a librarian, but they won’t expect to have like a full resource on everything that the library can provide for them.”
  • Generally students approached tasks by quickly scanning and clicking on the first link they thought might be helpful. They assess what’s there and if it’s not what they want, they return to look for something else. The first choice they make will probably be something in sight. Even if “Learning Hub” is perfectly-named, students won’t find it until they’ve investigated more prominent links.
  • When asked to describe “Learning Hub” in their own words, responses were varied:
    • “Learn to Use the Library.”
    • “Not sure how to describe, but not Learning Hub!”
    • “QuickStart Tutorials, something with the word ‘tutorial’ in it. ‘Learning Hub’ evokes actual learning of subjects or material, doesn’t make me think of learning to use the library.”
    • “The name, Learning Hub, is good.”
    • “There’s really nothing else you could call it that encompasses these four different categories.”
  • The four categories on the Learning Hub page are intuitive and clear. With assistance from the subheading links and descriptions, each participant was able to accurately describe what types of information they’d find on the child pages.
  • Ask A Librarian is well-used and appreciated! Participants frequently mentioned Ask A Librarian as a way to answer questions and complete tasks, and all 5 participants mentioned Ask A Librarian when tasked with finding a way to “talk to a real person.”
  • Searching for Chemistry resources from the Learning Hub page was challenging for participants. One found the LibGuides link in the Learning Hub sub navigation menu, and two completed the task by returning to the home page and clicking on Research on the menu bar. Two participants were unable to complete the task.
  • Participants had an easier time finding contact information for the Art History subject librarian, although only one looked for a link on the Learning Hub page. Two participants went back to the home page; one went to About, having remembered seeing a link there; and one went to the staff directory from the footer. But all wound up in the right place.
  • Participants had minimal difficulty completing other tasks in the protocol (see Appendix).
  • Terms used to describe the Learning Hub site were similar: comprehensive, clear, organized, straightforward, concise, informative.
  • Participants recognized that links with images are videos, but commented that the images should reflect the topic: “To me [the image] doesn’t seem relevant, it looks like someone creating a podcast. Audacity image looks relevant, because I’ve used it before, but a better image could be used.”
  • Final impressions of the Learning Hub highlighted the need for more detailed descriptions under each category and the subpage links. There were positive comments about the design (“sparse” and “not cluttered with graphic elements”). The overall organization of the Learning Hub site is “quite clear and intuitive” and “everything seems to fit well under the categories.”

Recommendations

  • Consider renaming to Learning Hub Tutorials. The term “tutorials” would be recognized as a place for instruction during a quick scan of the page (but may not be a broad enough for the content as the site grows over time).
  • Add link to Learning Hub Tutorials under About/Find Library Help, and in footer under Help & Assistance.
  • Replace “guides” with “subject help” in the description under Do Research/Help with Research because students don’t seem to associate “guides” or “LibGuides” with subject content.
  • Improve descriptions by adding terms that can be quickly scanned and understood and that fully reflects the content of the subpages (“Virgo,” “podcasting,” “Audacity,” “policies”).
  • Thumbnails should be more relevant at-a-glance to the video topic.

Full Protocol

Start: https://www.library.virginia.edu/
Target: https://www.library.virginia.edu/services/learning/ and child pages


Goal: Find tutorials under “Learning Hub: link; general impressions of content and design

Q: If you were looking for some tutorials to help you learn how to use the Library, where might you start? What makes you think you’ve found what you were looking for?

  • Rationale: Assess ability to find/interpret Learning Hub link under Services. Direct participant to the link as needed.

Q:  Is this what you expected under a link, “Learning Hub”? [How would you describe this area in your own words?] What are some of the things you can do here?

  • Rationale: Assess initial visibility and understanding of categories, “More ways to learn” sidebar nav

Q:  Describe the Learning Hub page in three words. What elements make you think of those words?

  • Rationale: Initial impressions of design; goal is to be inviting, approachable

Goal: Four categories should be more intuitive

Q:  [before clicking] What do you think you’ll find under “Use the Library”? [after clicking] How would you describe this category in your own words?

  • Rationale: Assess clarity and understanding of “Use the Library”; Gain insight into user-centric terminology

Q:  [before clicking] What do you think you’ll find under “Do Research”? Can you name some types of ‘research’? [after clicking] How would you describe this category in your own words?

  • R: Assess clarity and understanding of “Do Research”; Gain insight into user-centric terminology

Q:  [before clicking] What do you think you’ll find under “Conduct Academic Instruction”? Can you name some possibilities for me? [after clicking] How would you describe this category in your own words?

  • R: Assess clarity and understanding of “Conduct Academic Instruction”; Gain insight into user-centric terminology

Q:  [before clicking] What do you think you’ll find under “Create and Use Multimedia”? Can you name some examples of ‘multimedia’? [after clicking] How would you describe this category in your own words?

  • R: Assess clarity and understanding of “Create and Use Multimedia”; Gain insight into user-centric terminology

Goal: Complete common tasks

  • Go back to Task Slips PDF document and scroll down to Task 1.

TASK1:  Your Art History professor told you about a librarian who can help you do the research you need to write your final paper, but you don’t have her contact information.

  • R: Find Contact a Subject Specialist under Do Research/Help with research OR under More ways to learn subnav

TASK2: You haven’t used Virgo, the Library online catalog, and would like to read some brief instructions for beginners.

  • R: Find PDF, “Getting Started with Virgo” under Use the Library/Accessing materials

TASK3:  You would like some basic online instruction on how to begin research for a project.

  • R: Find general videos and tutorials to begin research under Do Research/Getting Started

TASK4:  The videos about getting started with research are helpful but you’d still like to talk to a real person.

  • R: Get general help from Ask A Librarian under Get Library Help subnav or CHAT tab or Use the Library /Help Using the Library

TASK5:  Your professor mentioned something in class about “databases” you need to use, but you’re not sure why you would need them. How would you learn more?

  • R: Find help on databases under Do Research/Searching for articles & using databases

TASK6:  You know there’s a list of Library resources broken down by subject that a nice librarian showed you in class a few weeks ago. If only you’d written it down! Where would you find resources about Chemistry?

  • R: Find the video about subject guides under Do Research/Getting Started OR Do Research/Help with research OR More ways to learn subnav

TASK7:  You need a tool to help you keep track of the articles you’re using to write your paper.

  • R: Find information on how to manage citations under Do Research/Citations or Conduct Academic Instruction/Tools for learning

Q: Let’s take a look at the links about playlists within Create & Use Multimedia/More Software Tutorials. What do the images mean to you? Go ahead and start a video. (If needed: What would you expect to happen when you click on the image?)

  • R: Assess if a thumbnail provides a visual cue to identify video content; assess understanding of thumbnail as link to content

Goal: Gather final assessments

Q: Now that you have spent time working in the Learning Hub site, what are your final impressions of this site? I’m interested in organization and clarity of information, as well as design.

  • R: Final opinions on usability and possible improvements; determine if any opinion has altered since start

Q: Which of the four categories on this site are most useful to you?

  • R: Provide more insight on category headings

Q: On a scale of 1 to 5, where 1 is Very Hard and 5 is Very Easy, how would you rank the ease with which you can find resources on this site?

  • R: Provide a quantified measure on ability to complete tasks

One Search Bar To Rule Them All

  • Purpose: Understand use of and expectations about search from the UVA Library homepage
  • Stakeholders: UVA Library homepage users and staff
  • Survey dates: April 19-25 (students) and June 29-30 (faculty)
  • Participants: 33 UVA undergraduate students, 19 graduate students, and 9 UVA faculty
  • Methodology: Participants were recruited from our existing research queue and through emails forwarded to academic departments. Students were compensated with a $10 Amazon card for completing the survey. The online Qualtrics surveys presented examples of site search bars and search results using screenshots from Harvard and Stanford libraries. Participants were given tasks to determine clarity of designs and were asked for opinions about the preferred display of search results. See Survey for a full copy of the survey questions.

Summary of Recommendations

  • Keep the Virgo search bar prominently displayed on the Library home page. Faculty and graduate students in particular use it frequently.
  • The magnifying glass icon that represents site search isn’t sufficiently identifiable on its own. If used, the icon needs a label.
  • There is some evidence that patrons are not expecting separate search bars for the catalog and library site search, and also evidence that patrons prefer to have a comprehensive page of search results. This strongly suggests that we should work toward having a combined search for Virgo and the Library website, at minimum. Preferably we should also include LibGuides and other major content sites within the Library domain.
  • The search results page should be categorized in a familiar way (“catalog,” “articles,” “library information,” “LibGuides”) and should have a clean, clear, and modern design. Breadcrumbs will help patrons navigate this complex search results page.

A Closer Look: Survey Question #4

“Here is a screenshot of another library homepage.  Click or tap once on the area where you would perform a search for information about this library’s food and drink policies.”

Crimson banner with white search bar at center

Harvard home page screenshot

Only 10/52 of students and 3/7 faculty correctly identified the magnifying glass icon as the place to find site search. Almost half of the students (22/52) misidentified the Hollis catalog search bar as site search. Participants may have interpreted “perform a search” to be synonymous with clicking on links. Student comments indicate that many chose the Hollis search bar because of its size and central location. Students may have expected the main search bar to provide this information or they weren’t expecting a second search bar. “Hollis” may not be identifiable as an online catalog, leading to incorrect use. Or there may be an expectation that a search bar will search everything Library-related.

When asked to explain why they chose the area they did:

  • “Food and drink policies wouldn’t be in the catalog, it would be on the library site”
  • “That’s where I would normally search in other databases”
  • “It is a search box”
  • “Very in my face; I don’t have to search for the search box”
  • “It is at the center of the page”
  • “It looks like a regular search bar”
  • “It may be easiest just to ask someone”
  • “No good choices”
  • “Unsure where to search”

A Closer Look: Survey Question #9

“Which option do you prefer for the display of search results?”

One column with headings Library, Tool, Service, Collection

Option 1 [Harvard] search results screenshot

Option 1 [Harvard]

Two columns with headings Catalog, Articles, Guides, Library Website

Option 2 (Stanford) search results screenshot

Option 2 [Stanford]

Answer Undergrads Grads Faculty TOTAL
OPTION 1 [Harvard] 19 3 2 24
OPTION 2 [Stanford] 13 13 2 28
No preference 1 1 0 2

Combined totals are roughly split between Options 1 and 2, but the majority of graduate students (13/17) preferred Option 2 (Stanford). Faculty were evenly split between Option 1 and Option 2. Even when considering which students indicated that they currently use UVA Library site search at least once a month, the preference between Option 1 and Option 2 is evenly split.

The comments give us a good idea of what we should take away from each design. The Harvard design (option 1) was thought to be sleek, clean, and easy to read and navigate. The Stanford design (option 2) received high marks for breaking down search results into familiar categories like articles, catalog, and library information. The Stanford design was considered organized and complete in its presentation of results; but also criticized for being “outdated” and “cluttered.”

“Do you have any further comments about the display of search results?”

Faculty:

  • “I like how in [Option 2] it clearly distinguishes between logistics & research.”
  • “Option 2 is categorized in a slightly more clear way but the design looks hideous and outdated + confused about catalog vs. articles options. Option 1 would be better if it had clearer categories.”

Sampling of comments from graduate students:

  • “Option 1 is easier to read, but Option 2 provides the most information for more specific searches.”
  • “I like how [Option 2 ] breaks down the information into categories like articles and library website.”

Sampling of comments from undergraduate students:

  • “I like the subcategorization [of Option 2]. Helps me quickly eliminate places I need to look.”
  • “I think the descriptiveness of option 2 is great but option 1 layout is cleaner.”

Survey

Faculty and students were given surveys with only slight differences. This is the survey as presented to students. Some questions were removed from the final analysis due to problematic wording.


Please help the Library evaluate our website.

The Library is reviewing the search bar function on our web pages and we would like your help. You will be asked to view several screenshots from library pages at other institutions and asked questions about them. This survey should take no more than 10 minutes of your time. It is necessary to use a desktop or a laptop or a tablet in landscape view to complete this survey.

If you have any questions about this survey please contact Melinda Baumann, User Experience Librarian, at baumann@virginia.edu.

Thank you!

Q1. Which best describes your relationship with the University of Virginia?

  • UVA Faculty or Staff
  • UVA Undergraduate Student
  • UVA Graduate Student
  • UVA Alum
  • Other

Please tell us about your experiences and expectations when searching for information on the Library’s web page.

Q2. How many times a month do you start a Virgo (Library catalog) search from the Library home page?

blue header with logo and search bar

UVA Library header screenshot

____ 0 ____ 1 – 5  ____ More than 5


Q3. How many times a month do you search the Library website using the search bar shown here?

white search box with heading, Search Our Site

Library site search screenshot

____ 0 ____ 1 – 5  ____ More than 5


Here is a screenshot of another library homepage.

Q4. Click or tap once on the area where you would perform a search for information about this library’s food and drink policies.

Crimson banner with white search bar at center

Harvard home page screenshot

What prompted you to choose this area?

______________________________________________________


The remaining survey questions relate to the display of search results.

Here is a screenshot of another library’s search results page.

Q5. Click or tap once on the area of this screenshot where you might expect to find information about that library’s food and drink policies.

One column with headings Library, Tool, Service, Collection

Search results screenshot from another library

Please rate your confidence that you might find information on food and drink policies here, where 1 would be not confident, 3 would be neutral, and 5 would be very confident.

5 4 3 2 1
Very confident neutral Not confident

Q6. Click or tap once in the area where you might search the library catalog to find resources about different food and drink policies. [Question removed from final analysis]

One column with headings Library, Tool, Service, Collection

Search results screenshot from another library

Please rate your reaction to these search results, with 1 would be confusing or frustrating, 3 would be neutral, and 5 would be clear and easy to read. [Question removed from final analysis]

5 4 3 2 1
Clear and easy to read neutral Confusing or frustrating

Do you have other comments about this search results page? [Question removed from final analysis]

______________________________________________________


Here is a screenshot of another library’s search results page.

Q7. Click or tap once in the area of this screenshot where you might expect to find information about that library’s food and drink policies.

Two columns with headings Catalog, Articles, Guides, Library Website

Search results screenshot from another library

Please rate your confidence that you might find information on food and drink policies here, where 1 would be not confident, 3 would be neutral, and 5 would be very confident.

5 4 3 2 1
Very confident neutral Not confident

Q8. Click or tap once in the area where you might search the library catalog to find resources about different food and drink policies. [Question removed from final analysis]

Two columns with headings Catalog, Articles, Guides, Library Website

Search results screenshot from another library

Please rate your reaction to these search results, with 1 would be confusing or frustrating, 3 would be neutral, and 5 would be clear and easy to read. [Question removed from final analysis]

5 4 3 2 1
Clear and easy to read neutral Confusing or frustrating

Do you have other comments about this search results page? [Question removed from final analysis]

______________________________________________________


Q9. Which option do you prefer for the display of search results?

One column with headings Library, Tool, Service, Collection

Option 1 [Harvard] search results screenshot

____ Option 1 [Harvard]

Two columns with headings Catalog, Articles, Guides, Library Website

Option 2 (Stanford) search results screenshot

____ Option 2 [Stanford]

____ No preference

Do you have other comments about this search results page?

______________________________________________________


Thank you for helping to improve the Library web site!


 

Research Results: Library homepage survey of students and faculty

Web page with blue header and Happy New Year image, Today's Hours, and more links.

Screenshot of UVA Library homepage, 1/25/21

  • Purpose:  Update research on use of and expectations about UVA Library homepage
  • Stakeholders:  UVA Library homepage users and staff
  • Survey dates:  March 15-17 (students) and April 5-19 (faculty)
  • Participants:  58 UVA undergraduate and graduate students and 21 UVA faculty
  • Methodology: Participants were recruited from our existing research queue, homepage banners, and emails to Library liaisons forwarded to academic departments. Two online Qualtrics surveys were written based on data gathered four years ago when the homepage was newly-designed, and based on current research questions. Students were remunerated with a $10 gift card.

Highlights of Findings

Survey participants were asked about the three links or areas were most important to them (Q1). Faculty selected Virgo or Advanced Search Virgo as their first choice by a more than 2:1 margin, and 85% of faculty chose Virgo as one of their top three links. LEO/ILL and Research (tied at 40%), Databases and Services (tied at 35%), and Hours (25%) rounded out faculty’s top link choices.

bar chart with tan bars and yellow lines

Final tally of most important links (without regard to whether the choice was first, second, or third)

Students also selected Virgo as their first choice by a more than 2:1 margin. Other top choice links were Research (18%), Hours (13%), Spaces (11%), and Databases and Subject Guides (tied at 5%). Hours were more important to faculty than students, which may be an anomaly caused by the closing of most libraries due to the COVID pandemic; analytics from March and April 2021 show that the Hours page is still a top performer (second only to the Research page).

Faculty described the Library homepage as “informative,” “organized,” and “comprehensive” (Q2). Third terms provided by faculty were more likely to be negative

Prominent words are clean, informative, easy, organized, boxy, busy

Wordcloud combining faculty and student top descriptive terms

(“busy,” “cluttered, “overwhelming”). Students used descriptors such as “clean,” “informative,” “organized,” and “helpful.”

Survey participants were asked to rank the importance of nine types of information available on the About the Library page (Q5). Both students and faculty gave Library Collections top importance (86% of faculty and 77% of students ranked it 1, 2, or 3). Then opinions diverged for students, who chose Directions and Maps next, followed by Plan a Visit, Mission and Policies, and Staff Directories. Faculty ranked Staff Directories more highly than Directions and Maps, followed by Mission and Policies and Plan a Visit. Both faculty and students ranked Tours, Library Assessment and Statistics, Jobs, and Ways to Financially Support the Library as having less importance to them.

Survey participants were asked to evaluate the Cornell University Library Hours page (Q8). About a third of the students liked the green and red colors representing Open and Closed: “The color coding makes it easier to find libraries that are open and closed when quickly scrolling.” Almost as many students liked the contact information (email, phone numbers). Seven students liked that services such as making a research appointment and picking up materials were listed: “I like that it provides links immediately for further steps.” Five students appreciated the thumbnail photos of libraries, and one commented that they might provide a necessary visual cue: “I like how it shows pictures for some of the libraries. If possible, that would be great because we don’t always call the libraries by their proper name so it can be confusing.” Faculty had similar responses: They approved of the color-coding, contact information, and links to services. Neither group mentioned the Libraries map button in the upper right corner.

Recommendations

  • Virgo search bar, ILL/LEO, Research, and Hours should all be prominent on the new homepage design.
  • Terms used to describe the homepage include “informative” and “organized” that reflect the UVA Library brand. Students first described the homepage as “clean” which, while not a negative term, may not be what we envision. Refresh this data with more testing after we’ve determined what terms we’d like to represent the Library brand.
  • Faculty and students expect to find Collections on the About page; we’ve already put it there in the new navigation. The remaining top areas will either be on the About the Library page or a subpage. Plan a Visit ranked in the middle with faculty and students but should prove to be more important to other types of users, meriting its place of prominence in the new About navigation.
  • When viewing another institution’s Hours page, the color-coded Open and Closed buttons were favored by students and faculty, as were the contact information and the links to primary services to make appointments and retrieve materials. Some respondents cautioned about adding more information to our already-complex Hours page. Until we take on a complete redesign of the Hours page we should heed the latter advice.

Survey

Faculty and students were given surveys that differed slightly in question order. Faculty were not asked two questions about Hours as marked below. This is the survey as presented to students.


Please help the Library evaluate our website!

The Library is refreshing the look of its webpages and we would like your help. You will be asked to review several current pages of the Library website and a screenshot from a library website at another institution, and asked questions about them. This survey should take no more than 10 minutes of your time.

If you have any questions about this survey please contact Melinda Baumann, User Experience Librarian, at baumann@virginia.edu.

Thank you!

Which best describes your relationship with the University of Virginia?

  • UVA Faculty or Staff
  • UVA Undergraduate Student
  • UVA Graduate Student
  • UVA Alum
  • Community Member or Other

Please open the Library homepage (https://www.library.virginia.edu) in a web browser, and then answer the following questions.

Q1. As you think about using the Library homepage, what are the three links or areas on this page that are most important to you?

1st link or area ________________________________________________

2nd link or area ________________________________________________

3rd link or area ________________________________________________

Q2. What are three words or phrases you would use to describe this homepage?

1st descriptive word or phrase _______________________________________________

2nd descriptive word or phrase ______________________________________________

3rd descriptive word or phrase _______________________________________________

Please tell us more about your expectations when using the Library homepage.

Q3. If you were to click or tap on the About link in the Library website header, what types of information would you expect to find on the resulting page?

blue header with About circled in red

_______________________________________________

Q4. Now please open the Library About page (https://www.library.virginia.edu/about-uva-library) in a web browser.

Is this what you expected to find? Check the box that best describes your answer.

5 4 3 2 1
very much what I expected to find neutral not at all what I expected to find

Q5. Here are the types of information about the Library that you might find on an About page. Please rank them in order of importance to you, with 1 being most important and 9 being least important.

______ Information to plan a visit

______ Directions and maps

______ Tours

______ Library collections

______ Library mission and policies

______ Library assessment and statistics

______ Jobs

______ Staff directories

______ Ways to financially support the Library

Q6. (Students only) If you were to click or tap on the Hours link in the Library website header, what types of information would you expect to find on the resulting page?

blue header with Hours circled in red

_______________________________________________

Q7. (Students only) Go to the Library Hours page (https://www.library.virginia.edu/hours) in another browser tab.

Is this what you expected to find? Check the box that best describes your answer.

5 4 3 2 1
very much what I expected to find neutral not at all what I expected to find

Q8. Please compare the UVA Library Hours page to the screenshot from another library hours web page, below. Are there any features on this hours page that you like?

Library hours listed with closed button in red or open button in green, plus photo of library and other links

Please tell us about your experiences and expectations when searching for information on the Library’s web page.

Q9. If you were to use the search bar in the Library website header above, what type of search results would you expect to see? This question refers to https://www.library.virginia.edu/

  • Results from Virgo, the Library’s online catalog
  • Results from the Library website (site search)
  • Results from both Virgo and site search
  • None of these results

Thank you for helping to improve the Library homepage!

 

 

 

 

 

 

 

Research Results: Learning Resources and Mental Models

OBJECTIVE: Identify and assess mental models for Learning Resources web page via card sort research

Stakeholders: UVA students; Teaching & Learning staff
Testing dates: 10/7-10/9/20
Participants: 8 graduate and 24 undergraduate students

Methodology: Enrolled UVA graduate and undergraduate students from our research queue were solicited by email to take a fifteen minute online test for which they received a $15 VISA card. We used a card sort tool by Optimal Workshop to map how students think about organizing online resources. T&L staff created a list of 40 learning resources (or “cards”). The 32 participants were asked to sort the cards into groups that made sense to them (see Full Test Results, Analysis tab). The data were then analyzed to determine patterns in mental models to inform the organization of the Learning Resources web page. For more information see Nielsen Norman Group’s overview of the card sort methodology.


Summary of findings

  • There was considerable agreement on the grouping of cards relating to multimedia, technology, and training. Nine cards were grouped together from 84% to 100% of the time, indicating that participants saw a strong relationship between these cards.
  • A second grouping occurred with cards that related to conducting research, research methods, and definitions of scholarly terms.
  • A third grouping related to discrete tasks one would accomplish in the Library and things a new Library user might find helpful to know.
  • Cards that related to getting help were more often put into other topic-specific categories; there were few mental models that included a generic “Help” grouping.
  • Although the term “Virgo” was used on four cards there was little attempt to create a specific “Virgo” grouping (more likely to be a librarian’s mental model of categorizing by specific research tools). These and other cards relating to acquiring books and journal articles were distributed in categories about doing research and about accomplishing Library-related tasks.
  • After completing the card sort participants were asked to give a name to the web page the resources might appear on. Although suggestions were provided to steer people away from the terms “learning” and “resources,” the majority of participants (21/32) still used at least one of those terms, and of those, 11 used both terms.

Recommendations

  • The learning resources can be grouped into three task-based categories that align with students’ mental models.
  • Each category should have subcategories for Getting Started; Help; and other subcategories that will allow students to drill down to more specific tasks and resources.
  • Further usability testing on these categories may result in some fine-tuning of these terms. At this writing, the following categories and subcategories are recommended:
    • Doing Research
      • Getting Started
      • Help
      • Citations
      • Searching for Books and Articles
    • Training and Multimedia
      • Getting Started
      • Help
      • Tutorials
    • Using the Library
      • Getting Started
      • Help
      • Accessing Materials

40 Learning Resources cards

  • 3D Printing Studio Training-MakerBot Playlist
  • Accessing a Book Through the Hathi Trust Emergency Temporary Access Service
  • Audacity Basic Training Playlist
  • Booking a Room at a UVA Library
  • Boolean Search Tips
  • Canon C200 Camera Training Playlist
  • DaVinci Resolve 16: Basic Training Playlist
  • DaVinci Resolve 16: Beginner’s Guide
  • Deeper dive into databases
  • Finding a Book in the UVA Library
  • Getting Help
  • Getting Started with Virgo, the Library’s catalog
  • How Do I Find What I Need? – searching UVA resources
  • How Do I Start My Research?
  • How Do I Use Information Correctly?
  • How Is Information Created? – about peer review and scholarship
  • How to Make a Request in Virgo
  • How to Make Video Tutorials Playlist
  • iMovie Essentials Training Playlist
  • Information Ethics & Citations
  • Interlibrary Loan – when UVA Library doesn’t own it
  • Introduction to databases
  • Navigating the Library Website
  • Peer Review in 3 Minutes
  • Placing an Interlibrary Loan Request
  • Podcasting in Audacity
  • Recalling a Book
  • Scholarly vs. Popular Sources Checklist
  • Searching Virgo
  • Thinking Tool
  • Using Academic Search Complete for articles
  • Using UVA Library Subject Guides
  • Virgo Help
  • Walking Tour of Grounds
  • What is an Academic Library?
  • What is Authority? – how to determine an authoritative source
  • What Type of Source Do I Need?
  • Who is a Scholar and How Do I Become One? – foundations of scholarship
  • Word Press Training
  • Zotero Walk-Through – citation manager

Renovations, Remote Meetings, and the Promise of an Accessible Workplace

Alderman Library at the University of Virginia is starting its first major renovation since new stacks were added onto the back of the building in the 1960s. This renovation necessitates that the building be closed until 2023, which right now seems like a very long time…But already I’m noting a silver lining for the Library staff, and it all has to do with inclusivity during staff meetings.

Most of the Alderman staff moved in late 2019 to outposts on Old Ivy Road and up 29N to the University Research Park; a few others are cozied into other libraries (Clemons, Brown, Harrison Small) or embedded in Kerchof and Zehmer Hall. We had our first Library-wide staff meeting on January 14, 2020, an event that occurs monthly, in the Harrison/Small auditorium. There were maybe 75 present of our staff of 225. Many others attended remotely through Zoom, a streaming video conferencing platform. Remote attendees individually participated from their desks or collectively from conference rooms. More than passive observers, the remote participants were able to submit questions and comments via an integrated Zoom chat tool.

To facilitate a smooth inaugural remote staff meeting there were a few rules: always use a microphone, whether presenting from the podium or asking a question from the audience. No one was allowed to say “I have a loud voice and don’t need a mic.” Remote participants were asked to mute their microphones so Zoom wouldn’t pick up any distracting throat-clearing or ringing phones. Presenters were asked to repeat questions into the podium microphone (until it was confirmed that the audience microphones were, in fact, broadcasting audio to the remote posts). One person in the auditorium watched the chat screen and relayed technical issues and questions from afar. All were asked to be clear to whom they were addressing questions, since there would not always be a visual cue such as making eye contact with the person at the podium.

The meeting went pretty well for a first effort, in part because a little awareness and patience goes a long way. The people in the auditorium understood that they really needed to use the microphones so the remote people could hear everything. In order to be sure every comment was heard things were repeated, sometimes more than once, but that assured that everyone could fully participate whether they were in Harrison/Small or miles away. That was the goal: to be sure everyone was included.

The silver lining I see is this: in taking this step we actually made staff meetings a little more accessible. We just made life a little easier not only for staff who are hard of hearing, but also for anyone with an ear infection or sitting near a noisy air handling unit. These are conditions, whether permanent or temporary or situational, that might affect one’s ability to hear, and we are all susceptible.

That’s the thing about accessibility: we all need, or will need, this consideration at one time or another. There is no “normal” condition, only the human condition, which means constant growth and change. A broken arm may hinder you only until it heals, or you may need help opening a door only because you are holding a bag of groceries. No matter the reason, if you can hit that panel with your elbow to automatically open a door, the room is more accessible to all. And when we’ve improved accessibility for all, we are more inclusive.

There is a new UVA initiative called Inclusive Excellence that promotes the “active, intentional, ongoing process to build community well-being and belonging.” When we take time to assure that others can hear everything that is said at a staff meeting, we are being more inclusive. The Library is now one step closer to having an accessible and inclusive workplace for all.

Read more about accessibility

Research Results: Focus Group with Graduate Students

Objectives

  • Gather feedback from graduate student population on overhauled online catalog tool in development
  • Explore overall impressions of the design, utility, and clarity of the new interface
  • Obtain high-level guidance on how to approach future design/development
  • Expose areas of design/development consideration we may not have explored or that may elaborate on user stories
  • Identify potential usability issues and/or ideas we should explore in upcoming testing
  • Surface and discuss design/branding questions and ideas

Method

  • Focus group with 6 UVA graduate students on October 25, 2019
  • Set ground rules:
    • We want you to do the talking
    • No right/wrong
    • All answers are confidential (but we are recording and transcribing for a limited number of project participants)
  • What we’re interested in: Reactions to this work-in-progress
    • Look-and-feel
    • Organization of info
    • Functionality
    • Presentation/understandability of results
    • Ideas for improvement: Generating ideas and figuring out mental models. Answers the question: “How would that solve a problem for you?”
    • Paper for sketching, jotting notes, plus white board and markers provided

online catalog tool in development

Themes

  • Provide more information sooner (full screen display of records; provide more details on records)
    • Want to get to info in fewer clicks; don’t add unnecessary steps
    • Prefer list display of search results to see more at once
  • Groupings, “see all”, availability, filters/facets/values all caused initial confusion but were learnable to grads
  • Very much liked that it was possible to search multiple sources at once and find things they weren’t looking for
    • Also want WorldCat, special collections, digital humanities, Google Scholar pools
  • Grads would use customization if available to set default search options (sources, filters) and display views
    • Really like bookmarks that extend over sessions (unlike current catalog’s ‘starring’ option)
  • Important to be able to limit by date, language, availability before searching
  • Subject headings should be hyperlinks to aid research
  • Missed having facets on basic search
  • Comment: “This looks like a blog”
  • Biggest problem with VIRGO currently is the search algorithm that produces results that are not always relevant (books and articles)

Research Results: Guerrilla Testing with Learning Resources

By Dave Griles

Purpose and Method

The Learning Resources webpage is new and in production. It serves as a gateway to research and teaching resources for students and faculty, so our purpose was to confirm that the design, terminology, and navigation made sense. Students, primarily undergraduate, were recruited at Clemons Library on October 1, 2019 and asked to review the website. Each user session lasted from four to ten minutes.

Summary of Results

Overall, the page performed well for the four testers, with most tasks readily accomplished by testers without additional prompting. However, testers did seem to expect to find citation engines and databases under the Research Basics category. Tester suggestions for improving the page centered on more specificity in category descriptions and including more information in the sidebar.

Questions Learning Resources

  1. What is this page for? What can you do here?
    • Testers identified that this was a place for searching for library resources.
    • Testers characterized the page varyingly as “FAQish” to an “abstract tool for research planning.”
    • One noted the instructional content.
  2. How would you find out about the ACRL Sandbox? [If they search:] How can you find it another way?
    • Testers primarily used A-Z list.
    • Testers found search box when asked about alternative methods to find target.
    • One tester attempted search from category view, which resulted in no apparent results.
    • Researchers noted page default of category view was inconsistently applied, sometimes leading the testers to start with A-Z list.
  3. What does the down-arrow mean?
    • While all testers identified that the down arrow would provide more information, there were differences in the nuances:
      • “all the links”
      • “more specific resources”
      • “more about items”
      • “drop down menu”.
  1. What would you expect to find under Research Basics?
    • Two testers expected to find databases, with one indicating “Citation Machine”
    • One tester expected easier resources, ones with a daily need to access
    • One tester did not know what to expect, identifying it as a very broad category
  2. What would you expect to find under Beyond the Basics?
    • Testers provided a wide array of responses
      • “More about research processes”
      • “More researching methods and other resources”
      • “More in-depth and using resources”
      • “Tools for research planning”
  1. What would you expect to find under Using the UVA Library?
    • Testers provided a wide array of responses, with two focusing on using the library
      • “How to use the UVA Library”
      • “Using the Library, rooms in the Library, about the Library”
    • Two testers focused on finding resources:
      • Find books and articles
      • Virgo and room reservations
  1. What would you expect to find under Teaching Resources?
    • Two testers responded that this area was for faculty or faculty/staff use and how to teach.
    • Third tester responded “resources for class”
    • This question was not presented to the fourth tester
  2. Which of these categories are useful to you? Is there anything missing?
    • Research Basics: 2
    • Beyond the Basics: 0
    • Using the UVA Library: 3
    • Teaching Resources: 1
  3. How would you make this page easier to use?
    • Categories are vague, need to click deeper to know what is in it
    • Do not know what you can find in the page. Would like info in a sidebar.
    • Confused as to why recalling a book is in Research Basics

Recommendations

  • Confirm search box works in either category or A-Z views
  • Page should by default open in category view
  • Description of Research Basics category should emphasize that these are videos and tutorials about using different types of resources, not the resources themselves
  • Consider reordering the content with Using the UVA Library as first category

Research Results: Intranet Testing with Library Staff

UX-4517, Gather feedback from advisory group and staff about proposed Confluence Information Architecture

Purpose: Assess usability and clarity of staff website migrated from wordpress to confluence (Wiki)

Stakeholders: Library staff
Testing dates: 6/14/19
Participants: 6 Library staff

Methodology: library staff were recruited to spend 10 minutes reviewing a new interface and arrangement of intranet content and asked to perform tasks designed to assess usability and clarity of key areas.


Objectives were to assess first impressions and to obtain feedback about what is missing or confusing with the new organization of content, and to determine what improvements are needed. Major categories of information were informed by analytics and other data. Tasks were to find organizational information; contact Library Staff Council; update one’s own directory information; receive reimbursement for work-related travel; and subscribe to a library listserv.

screenshot of new interface

Intranet as tested with library staff on 6/14/19

Findings

  • Top of the page needs something visually interesting.
  • The term “workflow,” used in the first task, confused some people which may have reduced successful completion.
  • Add an alpha list of departments with a “More…” link to complete list.
  • “Our Organization” was either overlooked or not fully understood. Clarify with more links or teaser info about what’s in this area.
  • “Space Shortcuts” should be relabeled as “Forms,” but the wiki software won’t allow. Need to make it clearer that these are all forms.
  • Add Library Staff Council link on main page for higher visibility.
  • “Listserv” is not a universally-understood term, so add names of listservs for identification: “Library listservs: Libtalk, Lib-reftalk, Libstaff…”
  • The purpose and function of the left menu caused some confusion but proved to be learnable.
  • Add a “Communication” category to include info about signing up for listservs and linking to committee and project meeting minutes to answer the question: “What is this group/project up to?”
  • With the creation of a “Communication” category the “Working at the Library” category could more closely map to Human Resources and professional development-related items to narrow focus.
  • Highlight the search bar to make more visually prominent.
  • Review all Quick Links and group similar links together.
  • Link to Confluence Help under Get Help.

Protocol

Question:
What are your first impressions of this page?
Task1:
You need to look up a detail about a workflow posted in Scholarly Resources & Content Strategy. Where would you start?
Task2:
You’ve just moved to a new office. Where can you put in a request to update your directory listing?
Task3:
You want to sign up for some work-related training. How much will the Library pay for?
Task4:
You’d like to bring up an issue to Library Staff Council. How would you do so?
Task5:
You’ve realized you’re missing too much good information by not being subscribed to libtalk@virginia.edu. How do you sign on?
Questions:
What’s missing? What’s confusing?
Question:
What would you suggest to improve this page?

Research Results: Special Collections Request System (Aeon)

UX-4614, Perform Usability Testing on Special Collections Request System (Aeon)

Purpose: Assess usability and clarity of request process and dashboard
Stakeholders: Special Collections Library users and staff
Testing dates: 4/24/19-4/29/19
Participants: 1 non-UVA user (from Monticello); 1 UVA staff; 2 UVA graduate students; 1 UVA undergraduate student

Methodology: Participants were recruited from postcards in the Special Collections Reading Room, direct contact with researchers, online web banner, and by solicitation of our existing research queue. A targeted recruitment web form was created: https://virginia.libsurveys.com/ImproveSpecialCollections. Each participant was to have experience with either online or in-person Special Collections services and collections. A queue of UVA and non-UVA researchers was developed and solicited. Two very similar protocols were developed for UVA (Netbadge) and non-UVA users to account for minor differences in registration and logging in.


Summary of Findings

  • No participants had any significant problem with either registration procedure.
  • All participants sometimes struggled to complete tasks, but most felt confident of their ability to navigate and complete tasks after having gone through it once. On a scale of 1-5 where 1 was very easy and 5 is very difficult, the average rating of ease in completing tasks was 2.
  • One participant, who rated her ease a 3, had more trouble navigating and more confusion when completing tasks.
  • A few buttons and messages should be changed for clarity and to avoid confusion: “Anytime that the language can be particularly precise about what a button means is more helpful.” When editing a request 2 participants expressed concern that they might be duplicating requests rather than simply modifying them. They wanted to see a button that said Modify Request” rather than “Submit Request”.
  • Participants needed more cues as to where they were in the process to avoid confusion and mistakes (“I thought I ordered it, but apparently I didn’t”). More visual cues would also reduce memory load: “I just did it but it’s already vanished from my brain how it was that I did that.” Some participants expressed that they sometimes felt “caught in a loop”: “’You currently have 2 active requests from your available limit of 5.’ So why then can I not seem to request them? Here, you’ve two active requests, but then it’s saying, to submit your request, please select the request, indicate the date and click Submit Information…I seem to be in a circle here.”
  • Participants wanted confirmation of what happens next in the request process.
  • Participants were unclear about the differences between the various Request options on the left menu. Some questioned why Reproduction was separate from Requests since they were all things that could be requested.
  • All participants were confused by the location and function of the search bar. None understood that the search was for requests (some thought Virgo).
  • All participants understood what fields were required in the forms, and when asked, most could identify the “red star” or “red asterisk” as meaning that the field was required. Error messages were clear when a mistake was made in filling out the registration form.
  • The “Save for Later” function was understood (eventually) by 4 participants, but 2 struggled at first to understand what they were saving and then why they wouldn’t want to immediately submit their requests.
  • Participants were confused or annoyed by the two modal screens when requesting from Virgo: “This page is unnecessary. Because I just did that. I had the same information on the– this screen is unnecessary.” A similar complaint about what seemed like unnecessary steps when requesting from Archives at UVA: “I definitely felt like when I was looking at Poe records [in Archives at UVA] trying to figure out how to request it I kind of got into a loop there, clicking on Request, then it would take me back to the record, then back to the request thing.”
  • Final comments include:
    • “Pretty straightforward…pretty intuitive.”
    • “It’s just not a great user interface. It’s an easy process. It’s pretty straightforward. It’s just a lot of form-filling.”
    • “[Archives at UVA] was a little bit confusing, wasn’t sure how to get where I needed to go, lots of text in a form that was a little bit hard to decipher because it was in big paragraphs… but was pretty easy to eventually figure out and navigate.”

Recommendations

  • No button or link should open in a new tab or window. This is a usability best practice that allows visually-impaired researchers to remain oriented and able to navigate backwards. Read more about the experience for visually-impaired and keyboard-only researchers. At minimum, request buttons that open in a new tab or window should have a tooltip (and corresponding hidden labels for screen readers) that indicates that action. At this writing Virgo has a tooltip on the actionable request button but Archives at UVA does not.
  • Logoff and Main Menu buttons should be clickable everywhere, not just on the text. This is a usability best practice.
  • Add a visual cue on every entry in the Outstanding Requests table that the line is clickable. All text in the row should be a visible hyperlink and the cursor should turn to a pointing finger on hover. (Alternatively, put an Edit button alongside each entry.) This is a usability best practice.
  • The Outstanding Requests table should communicate effectively to researchers. Headings “Status” and “Order Status” aren’t significantly different to the researcher and should be changed to something more meaningful. “Awaiting User Review” should be changed to “Unfinished Request” or similar to emphasize to the researcher that they need to take action. Entries should be distinguishable from each other, therefore need more data displayed. One participant suggested “a timestamp or details about the request (in-person or digital). That would be helpful.”
  • The “Clone to Copy” button was not understood as the place to clone a record for a duplication request. 4 participants eventually found the button through trial and error (“Edit? No. Cancel? No. Clone? No. Clone to Copy? Okay.”) and were able to complete the task. I recommend changing the button name to something more meaningful to researchers, but I don’t have a good suggestion. Would either “Request a PDF/JPG” or “Request a Duplication” work in this context?
  • When coming into the system from Virgo the request form is pre-populated but participants still expressed confusion about where they were in the request process: “The title is misleading. If I’m actually in the process of making a request, then remind me I’m in the process.” Rather than “New Book and Printed Material Request” or “New Manuscript/Archives Request” use the same text as when coming into the system from Archives at UVA, as edited in this screenshot:

screenshot with corrections made to improve clarity

This change will help orient the researcher that they are in the middle of the process and will prompt them to do the next step to complete the request or save for later.

  • After selecting a date and submitting a request this screen appears as in this screenshot:

screenshot displaying conflicting language

There is a conflict between “Your request is almost complete” and “You do not currently have any requests in review,” and the messages in the blue bars were not consistently seen. I recommend replacing everything in the gray box with a heading, “Your request is complete” and text below that indicates next steps: “Your materials will be available to you on [scheduled date] in the Special Collections Reading Room. Please request them at the desk. This information will also be emailed to you.” Include a link to hours and a map to parking and building location. If possible, send all this information in an email to the researcher and include the TN number and other identifying information. Multiple participants voiced concern about what happens next in the request process, and it is best practice to clearly communicate success and next steps when forms are filled out.

  • Add placeholder text in the search bar, “Search Your Requests” and add a corresponding hidden label for screen readers.
  • Add tooltips (and corresponding hidden labels for screen readers) to all links on the left menu to communicate and clarify purpose of each (“As a first-time user I don’t see a difference between some of these [links], and I would just mash the button to figure out what I want.”)
  • Add links to Hours and Planning a Visit in the left menu. Only 1 participant looked for Hours in the FAQ but commented, “I can’t believe I had to do all of that for hours.” The other 4 participants went to the Library home page or to Google to find the answer.
  • After changes are made and the system has been live for a few months we should run usability tests again, which will give us a chance to fine-tune language and confirm fixes. I also need to finish looking at the Request System from an accessibility perspective to confirm full access to all researchers.

Protocol for UVA (Netbadge) users

Task/Question

Rationale

1 There is a signed copy of the book, Tough Guys Don’t Dance, in Special Collections. How would you look at it?

How would you describe the process this far in your own words?
If you were to click on the Request button, what do you think would happen next?
Is there anything unclear about this process so far?

  • Evaluate clarity of multiple request buttons
  • Evaluate effectiveness of dynamic shading
  • Listen for how much of the first and second request screens are read
  • Evaluate clarity of instructions modals:
  • do they understand where the book will be delivered;
  • do they acknowledge that they’ll choose a pick up date next;
  • do they understand they’re being taken into another system
  • Assess clarity of request process in participant’s own words
2 You need to read an article in the journal, Epoch, but it could be in 2001 or 2002 so you’ll need to see both. What do you do next?
  • Evaluate clarity of multiple request buttons
  • Evaluate visibility of “Please select ONE item to request” and tooltip, “Select only a single item to continue”.
  • Evaluate UVA new user registration process
3 Complete the request so you can see the journal later today.

How do you know you’ve got all the data you need
How do you know you’ve successfully submitted the request

  • Assess clarity of submission process from Aeon dashboard
  • Determine if * is understood to represent required fields
  • Evaluate clarity of date picker
4 How can you check to see today’s hours for Special Collections?
  • Assess visibility and clarity of FAQ
5 You’ve decided to wait until tomorrow to see the journal. Can you make that change?
  • Evaluate clarity of selecting request (TN#?) to edit request
  • Note if they instead use Outstanding Requests or other option on left menu
6 Is it possible to get a copy of the Epoch article instead?
  • Evaluate clarity of steps to have a copy made
  • Evaluate understanding of file types
7 Take a few moments to look at the dashboard. What are your impressions of this page?
What are some of the specific things you can do on this page?
  • Assess general impressions
  • Assess clarity of different request functions
8 Now we’re going to go into a new system for searching for unique Special Collections materials including unpublished manuscripts, university records, visual materials such as films and photographs, audio recordings, digital material, and more. This time I’m going to have you go through the request process as a non-UVA researcher, which has a different login procedure.

Please give me your first impressions of the page at https://archives.lib.virginia.edu/repositories/3/top_containers/371

  • Evaluate Archives at UVA record for clarity and completeness
  • Evaluate process for non-UVA researchers
9 You’d like to take notes on an item in the Poe papers collection. If you are a non-UVA researcher, what do you do next?
  • Evaluate visibility of request button and clarity of request button text
  • Evaluate visibility of First Time Users link
  • Evaluate First Time Users registration process
10 Go back to Virgo and begin the request process for Tough Guys Don’t Dance as a non-UVA researcher. Once you are in the Special Collections Request System, can you save your work without submitting?
  • Evaluate clarity of Save for later option.
  • Discover improvements to recommend.
  • Evaluate clarity of Awaiting User Review
  • Evaluate clarity of Unsubmitted vs. Outstanding Requests
11 Do you have the option to submit more than one request at once?
12 Is there anything confusing about this request process?
Is there anything missing from this page?
What would make this request process easier?
  • Last comments about request process and Aeon dashboard
13 On a scale of 1 to 5, where 1 is very easy and 5 is very hard, how would you rate your ease in completing these tasks?
  • Gauge overall experience with request process