Research Results: Two Usability Tests Improve Virgo GIS

  • User Experience Project ID: UX-236, Geoblacklight User Research and Design
  • Purpose: Assess usability and clarity of the GIS Virgo (Geoblacklight) interface, redesign, test again
  • Stakeholders: U.Va. Library Geoblacklight project team and Virgo GIS users

First test comparing two interface designs

    • Test dates: 9/18-9/25/15
    • Test participants:
      • 2 GIS professionals
      • 4 students taking GIS classes
    • Methodology: In-person, individual testing on existing interface. Users were asked to complete a broad range of tasks relating to searching for,
      Wireframe redesign of GIS Virgo

      fig. 1: Wireframe redesign of GIS Virgo

      downloading, and emailing datasets. The user was left to decide to search with keywords in the search box or by zooming in on the map to narrow search results. Other tasks included identifying dataset attributes and layers, and using facets and search history. A new wireframe design was then shown (fig. 1) and feedback requested.

Summary of findings

  • Branding (UVA or Library) matters. “GeoBlacklight” meant nothing to testers and was not associated with Virgo or U.Va. Library. Testers want to understand scope/holdings of the database they’re searching.
  • Make it clear how to use the bounding box to search the map. 5/6 (all students) did not find the bounding box feature on their own.
  • Subject facet was used often but not successfully due to spotty metadata.
VirgoGISmodal

fig. 2: Virgo GIS with tutorial modal

Interim outcome: The interface was redesigned with a tutorial modal (fig. 2) and U.Va. branding. Facets were eliminated.

Second test, post-launch

  • Test dates: 2/23-2/25/16
  • Test participants:
    • 2 expert staff (Office of the Architect; Facilities Management)
    • 1 graduate student expert (Landscape Architecture)
    • 1 beginner graduate student (American Studies)
  • Methodology: In-person, individual testing after soft launch (Virgo GIS portal linked
    New Virgo GIS

    fig. 3: Virgo GIS

    into Virgo navigation header). The users were tested on navigation, search, download, and understanding of terminology. Tasks targeted map searching more so than previous test. See fig. 3.

Summary of findings

  • All 4 testers found the newest interface easy to use.
  • 3 out of 4 testers correctly used History, Saved Searches, and the tutorial modals.
  • The beginner tester was initially confused by some features but grasped concepts quickly.
  • Final observations on this interface:
    • “Easy to use. A lot of good data.”
    • “Straightforward. Impressed. Happy to see this. Will tell others to use.”
    • “Nice. Has everything I would want. Best GIS portal that I have used. Incredible. Impressed. Great job. Love it.”
    • “Nice, open, interesting, more tools and data than expected. Very cool, easy to use.”

Project status: Virgo GIS had a soft launch in January 2016 and a phase two is planned to implement more features. Based on the post-launch testing, the recommendation was made to create a Help page with definitions, explanation of features, and anomalies in data and presentation.

Project Files:  https://virginia.box.com/UX236-1 and https://virginia.box.com/UX236-2

Research Results: Dataset Deposit and Search in Libra

  • User Experience Project ID: UX-514 and UX-570, Complete user testing and analysis for Dataverse Deposit and Search functions in Libra, the online archive for University of Virginia scholarship
  • Purpose: Assess usability and clarity of the data deposit process and search function in Libra
  • Stakeholders: U.Va. Library Libra project team and Libra dataset users
  • Test dates: 10/29-11/4/15 and 11/17-11/20/15
  • Test participants:
    • 3 faculty (Education, Environmental Science, Pathology)
    • 5 graduate students (Education, Public Health, Biology)
    • 1 undergraduate (Systems Engineering)
  • Methodology: In-person testing in two phases. The first users were tested on uploading, describing, and publishing datasets to Dataverse, a dataset deposit and discovery interface developed at Harvard. Testers were shown two different Dataverse instances and were asked to complete a series of tasks on each. Testers were then asked which Dataverse interface was easier to understand and use. The second users were asked to perform searches and interpret findings on the preferred Dataverse instance.

Summary of findings

  • The deposit procedure was intuitive or learnable.
  • All 4 deposit testers wanted or expected to see the Add button on the initial page.
  • Deposit testers easily navigated around the site. All 4 deposit testers used breadcrumbs and upper left Dataverse icon to navigate back to start. Login link was expected or easily found on upper right.
  • Deposit testers easily used Add and Edit buttons, and were familiar with these terms as actionable.
  • All deposit testers stumbled a bit typing in dates in the required format: YYYY-MM-DD.
  • 3 out of 4 deposit testers were not confident or familiar with the term “metadata.”
  • Deposit testers liked icons, photos, and images, and felt they communicated valuable information.
  • All 5 search testers were familiar and comfortable with using facets and search box.
  • 4 out of 5 search testers were able to correctly define what a dataverse is. Definitions included “warehouse,” “schools or areas,” “a collection from an institution,” “a project,” and “specific to a department.” 4 out of 5 search testers understood the hierarchy: dataverses contain datasets which contain files.
  • Search testers found popup definitions useful.

Project status

Libra dataset deposit and search will have a soft launch with early adopters in March 2016.

Project Files: https://virginia.box.com/UX514 and https://virginia.box.com/UX570

Research Results: Catalog Advanced Search

Project details:

  • User Experience Project ID: UX-27 Virgo UI Design for Full-text and Exact Search
  • Purpose: Assess usability of proposed changes to Catalog Advanced Search in Virgo. These changes will allow the user to select “exact” searching and/or allow the user to select full-text searching for keywords
  • Stakeholders: All Virgo users
  • Test date: 5/22/15
  • Test participants: 1 graduate student and 2 undergraduates
  • Methodology: In-person with paper mock-up of proposed changes

Brief summary of findings:

  • Two of the three users correctly matched search results to the search strategies, indicating an understanding of how “Don’t stem” and “Full-text” work.
  • Users did not readily understand the meaning of “stem” and “full-text”, but found the Help text clear.
    • (Some minor changes were made based on user recommendations: Discussing both stemmed and non-stemmed searches, including a fuller explanation of what the “full-text” option does, and using the term “group” instead of “phrase.”)
  • One user specifically mentioned liking the checkboxes.
  • The graduate student was well-versed in Virgo Advanced Search and easily understood how the new features work.
    • Undergraduates demonstrated more confusion with terminology but used good logic to anticipate how the features work.

Project Files: https://virginia.box.com/s/2447w6c977dkqn16gyfclbap5qxi6d1m

Research Results: Available to Order in Virgo

  • User Experience Project ID: UX-544, Complete user testing and analysis for Available to Order in Virgo
  • Purpose: Assess usability and clarity of a new process on Virgo. Patrons find book records in Virgo and can select them to be purchased for the permanent Library collection. Electronic books are immediately accessible while physical books are ordered and delivered via the LEO or Ivy Stacks process to the requesting patron.
  • Stakeholders: U.Va. Library “Available to Order” project team
  • Test date: 11/6/15
  • Test participants:
    • 5 undergraduate students (2 second-years, 3 third-years)
    • 1 graduate student (first year in Batten School)
    • Methodology: “Guerrilla” testing, in which we approached students in the main hall of Alderman Library and asked for 5 minutes of their time in exchange for a candy bar. Testers were asked to complete six tasks on a server with test data to ascertain if the process was clear and if there were any areas of confusion.
availabletoorderclip

Fig. 1: Sample Virgo record seen by testers

  • Brief summary of findings:
    • The link to request an item, which sat below the Availability box, was not readily seen by 5 out of 6 testers. When asked to request an item, testers first found and clicked on the link about the new service (AKA the “About” page). The recommendation was therefore made to improve the visibility and clarity of the request link. The link was refashioned into a button to make it more actionable.
    • Testers were able to ascertain from the “About” page what an Available to Order item was, how much it would cost them, and how long it would take for the book to arrive.
    • When asked where they would get help, all testers indicated familiarity with the “Ask A Librarian” service or that they would be comfortable asking for help at a library service desk.
    • Testers had no trouble understanding that clicking on the “Continue” button would initiate the order request. Testers commented that they’d also like confirmation that the book wouldn’t be charged to them and to see more information about what happens next in the process. One tester commented that he’d like to have a shopping cart, like Amazon, where he could then review his order before submitting it.
  • Project status: Changes were made to clarify the flow of the request process and the “Available to Order” service was rolled out in December 2015.

Project Files: https://virginia.box.com/UX544

UX Team Priorities for Spring 2016

A high-level overview of UX Team priorities for the Spring 2016 semester is now available.

Highlights include:

  • Completing 2015 User Survey follow-ups and finalizing questions for 2016 User Survey.
  • Experimenting with ways to connect physical and virtual spaces.
  • Define and execute a mobile strategy; Launch v1 of a mobile app by end of the semester.
  • Implement technical efficiences and automation in web development process. Also establish a pattern library.
  • Create and present a web strategy.
  • Evaluate and improve signage.
  • Support Libra 2.0 work.
  • Conduct Virgo research in support of next version.
  • Complete a space use audit.

Questions? Please send them to me (Jill Heinze, Director, User Experience)!

 

Google is breaking out of the box with ‘Now on Tap’

The BBC reports on a Google innovation for mobile devices that analyzes the context of text to conduct searches with the push of a button, rather than typing in a search box:

The feature works with any app. And if someone wants to know something specific, they can trigger a contextual voice search by saying: “OK Google”.
One possibility would be asking: “Who’s the lead singer?” when a song’s name is displayed in Spotify.
“It’s search designed for the mobile world,” says Mr Singhal.
“You don’t have to switch windows to type information into one window and then go to another.”

The new functionality is being released as part of the latest version of the Android operating system.

The article outlines the business strategy behind the decision, the competitive landscape, and privacy concerns.

Will contextual search set the bar for the library search experience?

Outsmarting Robots is also Good UX

The article, “Will Smart Robots Take Your Job?,” from Ping! Zine was referenced in a U.Va. Today Daily Report. I was immediately impressed by the fact that the proposed qualities that will “future proof” jobs are also the very qualities that are essential for doing good UX work. Those key traits include:

  • Overcome cognitive blindness. “You can overcome your cognitive blindness by strengthening your critical thinking. Start asking yourself, Why do I believe this? What do I truly know? What don’t I know? What do I need to know?
  • Get good at not knowing. “Rather than pride yourself on what you think you know, become an adaptive learner—someone who knows what you don’t know and how to learn it by asking the right questions…”
  • Humility is a “silver bullet.” “Don’t be so consumed with being right—be consumed with constantly stress testing what you believe against new data. Treat everything you think you know as conditional, subject to modification by better data.”
  • Become an egoless collaborator. “The powerful work connections that will be needed to build successful organizations will result from relationships that are built by authentically relating to another person, recognizing their uniqueness, and doing so in a respectful way that builds trust.”
  • Sharpen your hands-on skills. “Artificial intelligence will in many ways make our lives better,” says Hess. “But it will also challenge all of us to take our skills to a higher level in order to compete and stay relevant.”

I can’t stress enough how important these skills are in UX work, and how difficult they are to attain! It takes a lot of restraint and practice to keep an open mind, and I for one purposefully try to get better at it with every user research project.

I’m interested to know if you recommend readings or techniques that help you do these things in practice.

I found Steve Portugal’s book, Interviewing Users: How to Uncover Compelling Insights, very useful in structuring questions in an open-ended way to prevent researcher bias from influencing results.

Update: Looks like you can find an instructional video featuring Portugal on interview techniques.

Research Results: Staff Website Information Architecture

Project details: 
  • User Experience Project ID: UX-16 Reorganize Staff Website
  • Purpose: Reevaluate the information architecture (IA) of the Staff Website in response to expressed user needs.
  • Stakeholders: U.Va. Library staff
  • Test dates: Launched 12/11/14
  • Test participants: 31 Library staff members
  • Methodology: Remote, online open card sort
  • Brief summary of findings:
    • There was little agreement about what content fit under categories labeled by department (ex: “Human Resources”). This reinforces best practice which advises against an IA based on organizational structure.
    • Major content areas/themes users identified include:
      • Forms
      • Resources for Managers / Resources for Employees (to be combined into a label such as “Employee Services”)
      • Computing
      • Library Buildings & Spaces
      • In an Emergency
      • Organizational Design
      • Announcements
  • Next steps:
    • UX work is complete. Library Administration will interpret and apply user research findings in revising the site, communicating with stakeholders as appropriate.

Project File (Summary of Findings):

 

Research Results: Interview with Expert Manuscript Researcher

Project details:
  • User Experience Project ID: UX-119 Manuscripts Discovery
  • Purpose: Have exploratory conversation with a visiting expert researcher to understand the research process and mental models related to manuscript research.
  • Interview date: 3/3/15
  • Interview participants: 1 expert researcher (J.D., Ph.D.), U.Va. Library User Experience Librarian
  • Methodology: Extemporaneous conversation

Brief Summary of Findings

  • Opening up collections to user-contributed annotations should be seriously evaluated in light of the expressed needs shared in this interview, as well as in previous U.Va. Library user research on manuscript research. This would address many of the most pressing user and librarian challenges, and add tremendous value to the user experience. Indeed, this annotating is already happening within physical collections via note-sharing among researchers.
  • When digitizing collections, wherever possible, good date information and high-resolution, quality-controlled scans should be prioritized. Document readers should recognize the scale at which users need to read and print materials.
  • In terms of creating quality user experiences, it appears that those systems that save researchers time are among the most impactful. One of the most positive, memorable experiences this user related was of being able to find everything he needed in about 15 minutes thanks to a card catalog organized by date – an experience described as “awesome.”
  • Librarians have significant opportunity to add value to the manuscript research experience by facilitating researcher-contributed information and collaboration, and elucidating the context and connections among collections in a more broad and systematic way. (Currently, this is accomplished primarily through one-to-one interactions, which requires users to find the right person at the right time.)

Interview Notes and Summary of Findings

Research Results: Clemons Media Classrooms Reservation Form

Project details:

 

  • User Experience Project ID: UX-3 Room Reservation Systems
  • Purpose: Assess usability of proposed changes to Clemons Media Reservation Form
  • Stakeholders: Clemons staff who moderate bookings; Faculty and grad students who book rooms for viewing media
  • Test dates: 2/4/15-2/10/15
  • Test participants: 4 graduate students who teach
  • Methodology: In-person and remote usability test
  • Brief summary of findings:
    • 3 out of 4 testers wanted to visit the room before teaching in it to be sure it would work for their classes. 2 testers mentioned that a photo or layout of the room might suffice.
    • 3 testers found and used the Check Virgo button. 1 tester called the Virgo popup window “fantastic”.
    • All 4 testers expected an auto-generated email confirming their submission, and that there would be another follow up to confirm the reservation.
    • None of the testers tried to submit both class reservations on the same form. Only 1 tester saw the + option to add another date/time; None seemed to read the instructions below the Reservation Dates/Times heading.

 

Project Files: https://virginia.box.com/s/avazp9xorzf2spurubd5zs8jbf63907m

Contents: