- Purpose: Assess the usability of the refreshed Robertson Media Center pages Stakeholders: Students looking for and using media services on the web
- Survey dates: February 1-3, 2022
- Participants:
- 2nd year undergraduate, Nursing
- 2nd year undergraduate, Commerce
- Graduate student, Religious Studies
- 4th year undergraduate, Economics and Middle Eastern Studies
- 2nd year undergraduate, BACS
- Methodology: Participants were recruited from our existing research queue. A protocol was developed based on feedback of the previous design, and common tasks were identified. Testing was conducted synchronously on Zoom. URLs and a PDF of task slips were “pushed” to participants in Zoom chat, and participants shared their screen. Sessions were recorded and live captions were captured by Otter.ai to provide a reasonable transcript of each session. During the session participants answered questions and completed tasks to determine usability of the web site. Participants were compensated with a $25 Amazon e-card upon completion of the session. See full protocol.
- Objectives
- Determine what keywords are commonly used to search for the RMC website
- Determine how students navigate from the Library home page to the RMC pages
- Assess clarity and intuitiveness of Spaces, Equipment, and Learning categories
- Assess visibility of the new camera comparison and 3D printing studio pages
- Assess visibility of new subnav on RMC child pages
- Determine if common tasks can be successfully completed
Summary of Findings
- All five students had heard of Robertson Media Center and some had been to the space in Clemons but only one had actually used services for a class.
- Participants were asked to start on a web search page and to look for a studio in the Library that they could reserve to record a podcast. Four out of five students followed instructions and successfully found their way to an RMC page where they could reserve an audio studio. Search strings used were CLEMONS LIBRARY RESERVE; LIBRARY PODCAST UVA STUDIO; ROBERTSON MEDIA CENTER; and UVA LIBRARY RECORDING ROOM.
- Participants were then asked to navigate from the Library home page to the RMC page. Four out of five students followed instructions and eventually located the RMC page, but several commented that it was not prominently linked. Two participants saw RMC listed in Today’s Hours and clicked on “All Hours & Locations” while the other two clicked on “Explore Libraries & Labs” under the “Spaces and Equipment” heading. Shortly before testing began we changed the “All Hours” link to read “All Hours & Locations” which may have had a positive effect on findability.
- Terms used to describe the RMC main page were as follows:
- interactive, visually appealing, informational
- helpful, neat, cohesive
- organized, concise, general [leading to subpages with more detailed info]
- digital, accessible, innovative
- links to resources, calendar, general information
- The three categories on the RMC page are intuitive and clear. Two participants thought the Learning page was more geared toward instructors because of the headings “Consultations” and “Classroom support for instructors.” “Walk-up Equipment” was understood to mean no reservations are needed. One participant expected to find RMC classes linked to this page.
- Most participants struggled when asked to confirm if there was an RMC student advisory group, which didn’t neatly fall into the Spaces, Equipment, and Learning categories. The answer was in a block of text and was not a hyperlink, and only two out of five found it. Generally, the observed behavior is that they quickly scan for headings and prominent links and make quick decisions as to whether they’re on the right track, and back out if they think they’re headed down the wrong path.
- Participants had to bounce back and forth between RMC pages and calendar pages to complete tasks which led one to comment, “Going back [to the RMC page] isn’t as easy as I would like.” There are known limitations in using the calendar product, but we should work to integrate the reservation process into the RMC web pages for ease of use and a better user experience.
- None of the five participants found the “camera comparison” page to help them determine if a camera had a zoom lens or could do still photography. Some handled this task by doing Google searches and skimming descriptions on the reservations page, but only one participant found the answer and was confident that they’d successfully completed the task.
- When faced with a task to find and reserve a 3D printer, all participants headed for the Equipment or Learning pages. None found the 3D printing studio page. When asked specifically about finding a studio, participants used the Spaces page. All were eventually able to complete the task.
- There was minimal use of the subnav in favor of the breadcrumbs at the top of the subpages. There was common understanding that the Library logo would lead to the Library home page.
- On a scale of 1 to 5 where 5 is ‘very easy’, on average the participants ranked the ease with which they could find information and complete tasks as 4.
Recommendations
- Add a link to RMC classes on the Learning page. Rename heading to “Help and consultations” to be more inviting to undergraduates. Provide anchors for each section to make training section more prominent.
- Links to the camera comparison page should be added in the description for individual camera reservation pages and added on the Equipment page in the following two places:
Add prominent links on every RMC-related calendar page to the RMC equipment and space web pages.
- We can’t currently put reservable cameras on same page as walk-up cameras and related equipment, but we want one calendar page for all cameras and related equipment. Also want to be able to show 3D printers (on Equipment calendar) with 3D printer studios (on Spaces calendar). Equipment and Spaces calendars are currently completely separate.
- If calendars can’t be combined as requested above, navigation would be much easier with a subnav menu to link spaces and equipment calendars and related web pages (RMC equipment and space pages, Library home page, Library Services page).
- Trainings and instruction materials get lost in a semi-hidden tab “instructions” when viewing equipment calendar pages. Would like to hyperlink the work “instructions” in the description tab.
- Spaces calendars have a “week view” in addition to the default “day view.” If possible, add “week view” to all equipment availability calendars for consistent user experience.
- Fix reservation calendars so user can select multiple hours. Studios can be booked for a maximum of three hours but a user can’t click on a second and third hour block. We should at least mitigate the confusion it causes by prominently listing the maximum number of hours a studio can be booked and include instructions on how to do so in the space description above the reservation calendar.
- Longer term, consider if we should integrate the whole reservation process into the RMC equipment and spaces pages for a better use experience.
Full Protocol
Start: https://www.google.com/ (or anywhere not on Library website)
Target: https://www.library.virginia.edu/rmc and child pages
Goal: Warm-up
Q: Are you familiar with the Robertson Media Center? Have you used the RMC website? Have you physically been to the RMC in Clemons Library?
Rationale: Assess familiarity with RMC space and website, which may inform how well they complete tasks
Goal: Find RMC services by searching keywords or navigating from Library home page
Q: [Open your favorite web browser to any search page that is not a Library web page, such as google.com.] You are looking for a studio where you can record a podcast. Does the Library have one that you can reserve?
What makes you think you have or have not found what you are looking for?
Can you reserve a studio for next Friday? Book it for the maximum number of hours allowed.
Rationale: Assess ability to find RMC page, Spaces subpage, or audio studio reservation page. Note terms used to search. Note path navigated to RMC. Direct participant to the Spaces page if needed. Assess ability to reserve a studio for a future date.
Q: How would you find out what other services the RMC offers to you? [If appropriate: Can you do get there without using the browser back button?]
Rationale: Assess ability to navigate from calendar to Library home page to RMC. Note path if navigating or keywords used if searching.
Q: [If needed]: Go to the Library home page. How would you navigate to the Robertson Media Center page from here?
Goal: Three categories on RMC page should be intuitive
Start here: https://www.library.virginia.edu/rmc
Q: [before clicking] What do you think you’ll find under “Spaces”? [after clicking and reviewing page] How would you describe this category in your own words?
Rationale: Assess clarity and understanding of “Spaces” and two subcategories for equipment; Gain insight into user-centric terminology
Q: [before clicking] What do you think you’ll find under “Equipment”? [after clicking and reviewing page] How would you describe “Walk-up Equipment” in your own words? How would you describe “Reservable equipment” in your own words?
Rationale: Assess clarity and understanding of “Equipment”; Gain insight into user-centric terminology
Q: [before clicking] What do you think you’ll find under “Learning”? Can you name some possibilities for me? [after clicking and reviewing page] How would you describe this category in your own words?
Rationale: Assess clarity and understanding of “Learning”; Gain insight into user-centric terminology
Q: Describe the Robertson Media Center page in three words. What page elements make you think of those words?
Rationale: Impressions of design; visibility of intro text, subpages, lower sections
Goal: Complete common tasks
Go back to Task Slips PDF document and scroll down to Task 1.
TASK1: You’ve heard there is a student advisory group that helps to run the Robertson Media Center. Can you confirm if this is true?
Rationale: Assess efficacy of the Mission and FAQ pages.
TASK2: You’ve got a project to complete that requires a 3D printer, and you know you can use one in the Robertson Media Center. How would you find out what you need to know before you head over to Clemons Library?
Rationale: Find Spaces through subnav or from RMC main page, or navigate from Equipment to calendar pages and back to Spaces; Assess if 3D printing task leads to Equipment or Spaces, should FAQ be linked in either place, and make recommendations; Assess if the Equipment, Spaces, and/or FAQ pages answer common questions:
- Who can use a 3D printer
- How much does it cost to use a 3D printer
- Can I reserve it
- What training do I need
TASK3: You’re about to go to the RMC for your 3D printer appointment but wonder if there will be help nearby when you get there. What are your options going to be?
Rationale: Determine if help is readily available; Find links to “Learn more about RMC” and “read our FAQ”; correctly interpret information
TASK4: You need to use a video camera with a zoom lens to complete an assignment, but you’ll need to plan ahead if training is required. Outline your steps.
Rationale: Use subnav to get to “Learn about RMC equipment” or “Camera comparison information”; discern difference between “Camera equipment” and “High-end cameras” links; Assess visibility and effectiveness of sidebar nav; Assess clarity of instructions to get required training and reserving a video camera
TASK5: Now you’re thinking you might need a camera that can do both video and still photography. Does that mean you’ll need to do training and make reservations for more than one camera?
Rationale: Assess ability to navigate to the correct calendar page and tab for instructions; Assess visibility and effectiveness of sidebar nav; Assess clarity of instructions to get required training and reserving a multi-use camera
TASK6: Your friend told you she’d used a space in Clemons that is set up for using virtual reality headsets, but what you really want to do is create your own VR content. What is available to you?
Rationale: Find information pertinent to reserving and using VR studios; ask the following questions if not addressed:
- Which equipment can be checked out?
- What equipment/studios need to be reserved?
- How do I reserve a VR station?
- Can I log into my own Steam account on the VR machines
Goal: Gather final assessments
Q: What are your final impressions of the web pages you’ve worked with today? I’m interested in organization and clarity of information, as well as design.
Rationale: Final opinions on usability and possible improvements; determine if any opinion has altered since start
Q: On a scale of 1 to 5, where 1 is Very Hard and 5 is Very Easy, how would you rank the ease with which you can find information and complete tasks on these web pages?
Rationale: Provide a quantified measure on ability to complete tasks