User Experience Project ID: UX-2846, Analyze Tree-Test Results
Purpose: Assess usability and clarity of proposed website top level reorganization
Stakeholders: UVA Library staff and users
Testing dates: 11/13/17 – 11/14/17
Participants: 29 UVA undergraduates (paid)
Enrolled undergraduates were solicited via a banner on the Library web page to take a 15 minute test. An online tree-test tool by Optimal Workshop was employed to help determine the efficacy of a proposed information structure based on previous testing results. We created a simple hierarchy of 5 top categories for the Library website:
- About UVA Libraries
- Using the Library
- Help and requests
- Research and instruction
- Advanced technology
Each category had between three and nine subcategories. Undergraduate testers were asked to complete ten tasks, and were told it was possible to complete these tasks on the Library website. The tree-test results indicate how many testers found the correct path on their first try, how many found on their second or third try, and how many failed to find the correct path.
Summary of preliminary findings
- Undergraduates correctly identified categories about spaces (libraries, study areas, room reservations)
- Undergraduates correctly identified categories about searching for books and articles
- The category Advanced Technology was not well understood
- The task to find specialized software could only be successfully completed by selecting the “Advanced technology” category, but only 38% of testers selected the category immediately, and 41% of testers ultimately selected one of the other four categories
- Color printers and scanners (equipment) were not easily found under Using the Library
- 38% of testers found it immediately, an additional 28% found it eventually, and 34% did not find it at all
Deeper dive: task #8, “Find course materials on reserve for your class”
Only 12 testers (41%) navigated correctly to “Search course reserves” or “Search Virgo” in order to complete the task. Of the 17 that did not navigate correctly, 11 incorrectly looked for the answer under the “Help and requests” category (which may show a willingness to ask a librarian for help when they can’t find something on the website). Of those 17 failures, 10 selected “Research and instruction” (one of the correct categories) at some point, but still did not successfully complete the task. It took all 29 testers an average of 17.65 seconds to finish this task, which was considerably slower (by at least 8 seconds) than the time spent on the other tasks. Previous testing has indicated that some students are unfamiliar with the concept of course materials being held on reserve in a library. The high rate of failure coupled with the extended time it took to finish this task further suggests confusion with this concept.
The testing information gathered here continues to inform website organization projects. Some categories of information should be surfaced in order to be found by undergraduates, but other user groups (graduate students and faculty) also need to be tested.