UX-4102, User Testing for DH@UVa

Objectives

• Assess understanding and clarity of DH certificate requirements
• Assess understanding and clarity of DH certificate application process
• Assess clarity of what counts as an elective course and MoU
• Assess clarity and utility of menu headings, filters and nodes, About DH@UVa
• Assess understanding of Activate Your Profile
• Assess clarity of who/how to contact for help
• Explore initial likes/dislikes related to site organization and design

Stakeholders: DH@UVa site owners

Dev site: https://digitalhumanities.dev.uvaits.virginia.edu/

Testing date: November 26 – november 30, 2018

Participants: 5 UVA graduate students from English (2), Art, Anthropology, and Slavic departments

Methodology: usability testing with one participant, one facilitator, and one note taker in a quiet space. participants were solicited from DH@uva mailing list. they used a laptop and were asked to complete tasks on a newly-revised website to determine the usability of the site. the facilitator and note taker listened for comments and opinions, and noted behaviors.

Strong Findings

• None of the 5 participants had trouble finding and enumerating the steps to apply for the DH certificate. Most navigated by the DH Certificate link in the menu. One participant was briefly confused by the More about the graduate certificate in digital humanities section above the DH certificate requirements This participant also commented, “I think there is a mismatch between what I would expect from a digital certificate and the application process, which is so traditional. Why do I need words, why do I need a statement, when I’m applying to do something visual? What about art students?”
  • Recommendations: Remove More about the graduate certificate in digital humanities section to focus attention on enumerated steps. Alternative: work this section into About DH@UVa with visible links to requirements and application process. Reduce text blocks with diagrams or other images that can convey information in a non-textual way.
• All participants could find and convey the DH certificate requirements as well as how to get an elective course approved; and each demonstrated basic understanding of the MoU.
  • Recommendation: None
• 4/5 expressed the need to clarify the relationship between DH@UVa and IATH, Scholars Lab, and SHANTI in order to clarify the purpose of the DH@UVa website. One participant was still unclear at the end of the test: “I guess I’m also not 100% sure what DH@UVa is as compared to the Scholars Lab as compared to individual projects. Again, why am I going to this website?” From another participant: “I am really interested all digital stuff, but I am not sure what each center or office does…they all overlap, also Data Services. I don’t know who to contact. I have no idea how they’re different from DH.” This participant didn’t want to “have to read everything and figure out the differences.” A third participant: “What’s the difference between the three [pointing to the links in footer]? Are they sponsors? Needs to be made more clear.” This same participant wanted to see Makerspaces included. The one participant who did not express confusion about the centers or the purpose of the website found it to be “clear, organized, and inviting. Judging from what I see here I’d want to do this myself because it seems like a very organized program.”
  • Recommendations: Add new intro text, Welcome to DH@UVa or similar, above the fold on the home page stating clear purpose of the website and concise descriptions of how related centers intersect with DH. Retest to determine if there is an improvement or if further changes are needed.
• 4/5 found the DH@UVa Network area at least somewhat unclear and that it appeared “awkward,” “scattered,” “crowded,” and “feels like 50 things are competing for my attention.” Some expressed that there isn’t enough information to provide context, for instance in the People filter: “I’m wondering why these people are here. There should be something about who they are, not just their names and picture.” One participant wanted more filters to narrow down results further by subject and geographic location. 3/5 wanted the nodes pages to have a more obvious organization, calling the current order “random” and indicating that “alphabetical is what I expected to find.” Participants wanted a different structure on the nodes page (“Not tiles, but the full title, followed by the first few lines of text”) and one commented: “This is good, but it needs a description to explain what we are looking at.” 3/5 expressed a need for a robust and visible search function on this page.
  • Recommendations: Add context by including tooltips for each node button. Add description to each card. Reformat nodes pages in a grid with full titles and abstracts to facilitate scanning and evaluating content without having to click on every Learn More Consider duplicating the search box near the heading of the page for optimum visibility.
• All participants had no trouble figuring out where to go for help, but all had some familiarity with DH@UVa. Most knew Rennie and mentioned her by name. A related note: all 5 participants expected to find who administers DH@UVa on the About DH@UVa page under People (see #2 under Protocol below). 2 participants expressed that they expected to find the primary DH@UVa contact at the top of the People page, and the other 3 felt they had completed the task when they clicked on the People link and saw the headings for the Executive Board and Steering Committee. Only 1 participant eventually scrolled to the bottom of the page and located the DH@UVA Curriculum Development Team
  • Recommendations: Best practice would be to have a Contact link on the menu for optimum visibility. Move DH@UVA Curriculum Development Team section to the top of the People page.

Other Findings

• 3/5 participants expressed some uncertainty about what to expect when clicking on the Activate Your Profile Answers ranged from “to save things I’m interested in” to “receive information, subscribe to a newsletter, make your profile available to others to network with.” Best practice tells us that it would help provide context to have one sentence explaining why you’d want to activate your profile before clicking on the button.
  • Recommendation: Add a brief description below Activate Your Profile to outline the benefits of creating a DH profile.
• Resources is currently a bit of a catch-all: “To me it’s a miscellaneous list.” DH Organizations and Projects could easily be its own menu heading. 2/5 participants wanted to see organizations and projects from outside UVA. A listing of upcoming events are commonly found on a website’s top level page, so having a link to Events Calendar in the menu is unnecessary. One participant recommends: “Make sure this [Resources] page functions as a toolkit to people who end up doing the certificate. Sort of a gateway to skills you’d use in the certificate program.”
  • Recommendation: Remove Events Calendar from menu. Refocus Resources by pulling out DH Organizations and Projects and adding the heading to the menu. Consider creating a page for new DH scholars and put Jumpstart your DH project and Links for the new DH scholar there, and link to Apply for the DH Certificate.

Protocol

UX-2985, Improve Content on Graduate Student Services page

Objectives

• Evaluate category headings for clarity, accuracy, completeness
• Confirm understanding of breadcrumb navigation
• Assess if CTA (call to action) areas work as expected
• Explore likes/dislikes about descriptions, icons, boxes, design, wording
• Gain insights with version preferences

Stakeholders: UVA graduate students

Testing date: 9/26/18-9/28/18

Participants: 4 UVA graduate students (East Asian Studies; Anthropology; Classics; Public Policy)

Methodology: usability testing with one participant and one facilitator in a quiet space. participants are representative users. they use a laptop and are asked to complete tasks on a new website to determine the usability of the site. the facilitator listens for comments and opinions, and notes behaviors. Two versions of prototypes were shown to assess preferences.

FINDINGS

• Without a description the participants had some trouble correctly and fully defining what they would find under Publications: “Publications is the only one I’m unsure about. Does that mean I’m looking for publications?” Two participants expected “resources to help me publish my own works in various places” and “links for how to get published”. Even after reading the description and seeing the options under Publications, one commented: “I would expect to see more about how to research different publications. I’d expect it to be LESS dissertation-focused. So that strikes me as odd.” Another commented after reading the description: “That helps me understand that I’m the one doing the publishing.”
  • Recommendation: Change category to Publishing, which is more suggestive of the activity of creating scholarly content; and retest.
• Two participants struggled to correctly interpret Instruction until they read the description and saw the options there. The other two participants defined the category as offering “helping with teaching” and “networks of graduate teaching assistants,” and both mentioned that they expected to find a link to the Center for Teaching Excellence (not a Library service).
  • Recommendations: Add a link to UVA Center for Teaching Excellence under Move Scholars Lab and RMC links to Consultation. Additionally, reorder categories so that Consultation and Instruction are next to each other.
• All four participants preferred the prototype with descriptions (version 2):
  • “The second layout is more intuitive.”
  • “I feel like the emphasis on helping comes through more on [version 2]. You’ve got specifically ‘these resources can help’ under Publication, under Instruction you’ve got ‘the library can help’ That sort of language really sets up the library as a space you can come when you need help.”
  • “I like the graphics! And I like the discrete division. It’s visually appealing. And I like the descriptions underneath (the icons)… What is really helpful is it provides an explanation, and I don’t have to do the guesswork…I really do like these explanations. While the first version is graphically simplistic, and I do like simplicity, I also think that some of these [icons] are ambiguous, and the other version saves me time of knowing where to go and what to click on.”
  • Words used to describe version 2 include “clear,” “clean,” “descriptive,” “effective,” and “helpful.”
  • Recommendation: Use version 2 with descriptions.
• Two participants would use a link to the subject guides if one were provided on this page.
  • Recommendation: Add a link to subject guides under Consultation.
• Although being shown a mostly non-working prototype, participants seemed to correctly interpret the CTA links.
  • Recommendation:
• All four correctly interpreted how to use the breadcrumbs.
  • Recommendation:
• When asked to project what it might be like to use version 2 on tablet or mobile, two participants said they were unlikely to use the website on a device. Another participant commented, “I have a lot more tolerance for scrolling when it’s a smaller screen. I know that they can’t put [in] all the information.”
  • Recommendation: None at this time.

Version 1 prototype

Version 2 prototype

Protocol

UX-3822, Redesign staff profile page template

Purpose: Assess usability and clarity of redesigned staff directory with profile pages

Stakeholders: users and staff

Testing date: 8/31/18

Participants: 5 UVA undergraduates

Methodology: “Guerrilla” testing, in which UX staff, some in tyrannosaurus Rex costume, invite students entering Clemons Library to spend 10 minutes of their time to improve a library web page. Coffee and danishes offered to all passers-by.

FINDINGS

• Participants didn’t initially know what or who Library Subject Specialists were, but were able to grasp the concept (“for doing research, to see someone to talk to”) after spending a few moments looking at the page.
  • Structure of the Librarian Subject Specialist page is clear and effective. All could grasp that the organization was by school, then by department.

fig. 1, Librarian Subject Specialist view

• However, the concept of a “library department” was confusing to participants. When shown the Department tab participants said the page was “unexpected” and that it wouldn’t be useful to them
  • Recommendation: Consider tooltips or another method to give additional description and context for Librarian Subject Specialist and Department tabs
• The Profile pages were easily grasped and appreciated but not carefully read. Comments included:
  • “Ask Me About” should be higher up because it’s most important
  • Gender pronouns are “inclusive”
  • “What’ ORCID id?”
  • Page is “useful” and “helpful”
  • Recommendation: Put bulleted information at the top of the right column to improve ability to skim
  • Recommendation: Limit Bio and other text blocks to reduce visual clutter and increase white space
    

    fig. 2, profile page

• Participant suggestions to make the staff directory easier to use echo the recommendation above to give additional description and context with tooltips:
  • “Need a description for Department tab”
  • “Too many tabs, too many options”
  • “Need to understand when to use Subject Specialist or Department”

Protocol

UX-3833, Guerrilla test Robertson Media Center pages and equipment request process

Purpose: Assess usability and clarity of new RMc pages and equipment checkout pages

Stakeholders: users and staff

Testing date: 8/31/18

Participants: 5 UVA students

Methodology: “Guerrilla” testing, in which UX staff, some in tyrannosaurus Rex costume, invite students entering Clemons Library to spend 10 minutes of their time to improve a library web page. Coffee and danishes offered to all passers-by.

RMC site: https://www.library.virginia.edu/rmc

LibCal RMC equipment site:  https://cal.lib.virginia.edu/equipment?lid=241

Project phase one deliverables were to create a landing page and second-level pages. Based on the content, design, and architecture, users are to be able to understand the following:

• Who the space is for (students, faculty, community members)
• What services and equipment/spaces are available to whom and how to obtain them
• This is a place for making, learning, and teaching

The protocol was developed to elicit responses to assess if the goals were met, and to test the response to wording for equipment check out that does not require reserving in advance.

STRONG FINDINGS

• After perusing the content and images of the landing page, participants were asked what the space is for, and to describe the space using three adjectives. Their answers reflect a space for making and creating, equipment to use and check out, and help when needed.
  • Four out of five testers described the space as “high-tech” or “technological.” The specific terms “creative,” “useful,” “3-D,” “VR,” “studio,” “check-out,” and “advanced media” suggest that the participants see a space for using and exploring new technologies.
  • Participants used the terms “knowledgeable,” “helpful,” and “supportive,” and phrases such as “people to help,” “assists students,” “help you with stuff,” “classroom,” “tutorial,” “tech support,” and “equipment troubleshooting,” all of which convey help, learning, and teaching.
  • One participant offered that “faces and names [under heading, ‘Who We Are’] makes people more approachable.”
  • Other terms reflect favorably on the environment: “modern,” “variety,” “expansive,” “fresh,” “eclectic.”

• Four out of five participants correctly identified UVA students and faculty as the audience focus for the RMC. One participant thought RMC might be limited to “STEM or media studies majors.” Two participants also mentioned access by “the public” and “anybody outside UVA” but that they wouldn’t be primary users.
• No discernible problems with participants correctly interpreting the meaning of terms “No Reservations” or “First come, first served.” Similarly, the RMC equipment page heading, “Walk-up Equipment,” seems to provide enough context to be correctly understood. However, several participants expressed that they also assumed they’d be able to use equipment without training, or that there would be no lengthy check out procedure, or that there would be no need to sign out equipment at all. We recommend avoiding idioms and colloquialisms that may lead to confusion and misunderstanding, particularly for non-native English speakers. To ensure clarity we recommend using plain language phrasing, either “No reservations required” or “No reservations needed,” to be used consistently throughout the RMC and equipment request sites. Additionally, because “Walk-up Equipment” is easily misinterpreted, we recommend changing “Walk-up Equipment” to “No reservations equipment” or similar. Also please review the categories under that heading (for instance, video equipment and cameras must be reserved so instead belong under the heading, “Reservable equipment). Instead of “Browse the equipment collection” button use text more actionable like “Check out equipment” or “Use Equipment.”
• After having reviewed the RMC landing and second-level pages, participants largely correctly identified who the space is for and what goes on there. Participants were then shown the Equipment page and asked: “If you wanted to use a camera, what would you do next?” Three participants initially thought they would have to go to the RMC desk to check out a camera, and one participant wondered if he needed to use equipment in-house. Four out of five had to be directed to click on “Browse the Equipment Collection” to make a reservation online. Three saw the text, “Training & Reservation Required” and another thought they should search online for a tutorial. All participants saw the links for “Availability” and “More details” but two participants were still unclear what to do next. Most participants clicked on “More details” but didn’t read any of the text. They all saw the calendar and figured out the process of selecting a date by clicking on a green box and logging in via Netbadge. One participant expressed confusion about the difference between red and grey boxes on the calendar, and the use of the term “padding.”
  • Recommendations for all LibCal equipment items, based on above results: Hide “Availability” since reservations can be made under “More details.” Change “More details” to a button, “Use this equipment” that users will see as an actionable thing. Review content and messaging about requirements and training on item pages to reduce amount of text. Highlight the Instructions tab (which no test participant clicked on). Describe specifications of equipment separately from the process to train and check out. Use short, declarative bulleted lists rather than prose to improve clarity and make text more skimmable.

PROTOCOL