Research Results: Usability testing on new design for DH@UVa site

UX-4102, User Testing for DH@UVa

Objectives

  • Assess understanding and clarity of DH certificate requirements
  • Assess understanding and clarity of DH certificate application process
  • Assess clarity of what counts as an elective course and MoU
  • Assess clarity and utility of menu headings, filters and nodes, About DH@UVa
  • Assess understanding of Activate Your Profile
  • Assess clarity of who/how to contact for help
  • Explore initial likes/dislikes related to site organization and design

Stakeholders: DH@UVa site owners

Dev site: https://digitalhumanities.dev.uvaits.virginia.edu/

Testing date: November 26 – november 30, 2018

Participants: 5 UVA graduate students from English (2), Art, Anthropology, and Slavic departments

Methodology: usability testing with one participant, one facilitator, and one note taker in a quiet space. participants were solicited from DH@uva mailing list. they used a laptop and were asked to complete tasks on a newly-revised website to determine the usability of the site. the facilitator and note taker listened for comments and opinions, and noted behaviors.

Strong Findings

  • None of the 5 participants had trouble finding and enumerating the steps to apply for the DH certificate. Most navigated by the DH Certificate link in the menu. One participant was briefly confused by the More about the graduate certificate in digital humanities section above the DH certificate requirements This participant also commented, “I think there is a mismatch between what I would expect from a digital certificate and the application process, which is so traditional. Why do I need words, why do I need a statement, when I’m applying to do something visual? What about art students?”
    • Recommendations: Remove More about the graduate certificate in digital humanities section to focus attention on enumerated steps. Alternative: work this section into About DH@UVa with visible links to requirements and application process. Reduce text blocks with diagrams or other images that can convey information in a non-textual way.
  • All participants could find and convey the DH certificate requirements as well as how to get an elective course approved; and each demonstrated basic understanding of the MoU.
    • Recommendation: None
  • 4/5 expressed the need to clarify the relationship between DH@UVa and IATH, Scholars Lab, and SHANTI in order to clarify the purpose of the DH@UVa website. One participant was still unclear at the end of the test: “I guess I’m also not 100% sure what DH@UVa is as compared to the Scholars Lab as compared to individual projects. Again, why am I going to this website?” From another participant: “I am really interested all digital stuff, but I am not sure what each center or office does…they all overlap, also Data Services. I don’t know who to contact. I have no idea how they’re different from DH.” This participant didn’t want to “have to read everything and figure out the differences.” A third participant: “What’s the difference between the three [pointing to the links in footer]? Are they sponsors? Needs to be made more clear.” This same participant wanted to see Makerspaces included. The one participant who did not express confusion about the centers or the purpose of the website found it to be “clear, organized, and inviting. Judging from what I see here I’d want to do this myself because it seems like a very organized program.”
    • Recommendations: Add new intro text, Welcome to DH@UVa or similar, above the fold on the home page stating clear purpose of the website and concise descriptions of how related centers intersect with DH. Retest to determine if there is an improvement or if further changes are needed.
  • 4/5 found the DH@UVa Network area at least somewhat unclear and that it appeared “awkward,” “scattered,” “crowded,” and “feels like 50 things are competing for my attention.” Some expressed that there isn’t enough information to provide context, for instance in the People filter: “I’m wondering why these people are here. There should be something about who they are, not just their names and picture.” One participant wanted more filters to narrow down results further by subject and geographic location. 3/5 wanted the nodes pages to have a more obvious organization, calling the current order “random” and indicating that “alphabetical is what I expected to find.” Participants wanted a different structure on the nodes page (“Not tiles, but the full title, followed by the first few lines of text”) and one commented: “This is good, but it needs a description to explain what we are looking at.” 3/5 expressed a need for a robust and visible search function on this page.
    • Recommendations: Add context by including tooltips for each node button. Add description to each card. Reformat nodes pages in a grid with full titles and abstracts to facilitate scanning and evaluating content without having to click on every Learn More Consider duplicating the search box near the heading of the page for optimum visibility.
  • All participants had no trouble figuring out where to go for help, but all had some familiarity with DH@UVa. Most knew Rennie and mentioned her by name. A related note: all 5 participants expected to find who administers DH@UVa on the About DH@UVa page under People (see #2 under Protocol below). 2 participants expressed that they expected to find the primary DH@UVa contact at the top of the People page, and the other 3 felt they had completed the task when they clicked on the People link and saw the headings for the Executive Board and Steering Committee. Only 1 participant eventually scrolled to the bottom of the page and located the DH@UVA Curriculum Development Team
    • Recommendations: Best practice would be to have a Contact link on the menu for optimum visibility. Move DH@UVA Curriculum Development Team section to the top of the People page.

Other Findings

  • 3/5 participants expressed some uncertainty about what to expect when clicking on the Activate Your Profile Answers ranged from “to save things I’m interested in” to “receive information, subscribe to a newsletter, make your profile available to others to network with.” Best practice tells us that it would help provide context to have one sentence explaining why you’d want to activate your profile before clicking on the button.
    • Recommendation: Add a brief description below Activate Your Profile to outline the benefits of creating a DH profile.
  • Resources is currently a bit of a catch-all: “To me it’s a miscellaneous list.” DH Organizations and Projects could easily be its own menu heading. 2/5 participants wanted to see organizations and projects from outside UVA. A listing of upcoming events are commonly found on a website’s top level page, so having a link to Events Calendar in the menu is unnecessary. One participant recommends: “Make sure this [Resources] page functions as a toolkit to people who end up doing the certificate. Sort of a gateway to skills you’d use in the certificate program.”
    • Recommendation: Remove Events Calendar from menu. Refocus Resources by pulling out DH Organizations and Projects and adding the heading to the menu. Consider creating a page for new DH scholars and put Jumpstart your DH project and Links for the new DH scholar there, and link to Apply for the DH Certificate.

Protocol

1 [Start: https://digitalhumanities.dev.uvaits.virginia.edu/]
This is a site under development.
What is your initial impression of this web page?
  • Assess initial impressions
  • Assess clarity and utility of menu headings
2 (task) You are unfamiliar with Digital Humanities and want to know who administers it at UVA. Where do you start?
  • Assess clarity of  About DH@UVa
3 [Scroll down and point to 1.) DH@UVA Network and 2.) View all nodes button]
What happens here?
  • Assess understanding of filters and nodes
4 [Point to Activate your profile button]
What would you expect to happen when you click on this button?
When would you click on it?
  • Assess understanding of Activate your profile button
  • Assess understanding of a profile
  • Does Log in help to define Activate your profile
5 (task) Find out about the DH projects that others have done.
  • Assess location of Projects under Resources
6 Describe, in your own words, the requirements of the DH certificate.
  • Assess understanding and clarity of the DH certificate requirements
7 (task) You think that a particular course in your home department of History might count toward a DH certificate. How can you determine if it does?
  • Assess clarity of what counts as an elective course
  • Assess ability to navigate to course information
8 Describe, in your own words, the steps to receive permission for a non-DH course to count toward the DH certificate.
  • Assess clarity of the first steps taken to get a non-DH course to count as an elective
9 You’ve received verbal approval for an elective course from your instructor and the DH administrator. What’s your next step?
  • Assess ability to navigate to Memorandum of Understanding (MoU)
  • Assess clarity of the MoU requirement
  • Probe for understanding of MoU
10 Describe, in your own words, the steps to apply for the DH Certificate.
  • Assess clarity of the steps to take to sign up for the DH Certificate
  • Ensure that the numbered steps are included in the process before clicking on the application link
11 (task) You have a question about the application. How do you get help?
  • Assess clarity of who/how to contact if further information
    is needed
12 What are three adjectives you’d use to describe the process to apply for a DH certificate?
  • Gain insight into process to learn about and apply for DH certificate
13 What is missing on this website?
What is confusing on this website?
  • Assess understanding and utility of page
  • Gain insights into what is missing or confusing
14 How would you make this website easier to use?
  • Final comments

Leave a Reply