Research Results: Perform and assess a guerrilla test on new home page

Project Details

  • User Experience Project ID: UX-1372, Perform and assess a guerrilla test
  • Purpose: Assess usability of the new home page design
  • Stakeholders: UX team
  • Test date: 8/25/16
  • Test participants:
    • 5 undergraduates
    • 1 faculty
  • Methodology: “Guerrilla” testing, in which we approached students entering Clemons Library and asked for 10 minutes of their time in exchange for a candy bar or banana. Testers were asked for first impressions and to rank the three most important links or areas on the newly-designed page. Testers were also asked to complete seven tasks on both laptop and mobile versions, and then were asked to rank the ease with which they completed those tasks.


  • Initial impressions from four testers were that the page is “easy to navigate” and “organized.” One tester who had already used the page was “frustrated by it, can’t find anything.” Other comments: like the photo banner, like the prominence of the hours and Ask A Librarian on the banner.
  • Most important links were varied: Research had three votes; two votes each for Hours, Ask A Librarian, Services, and Find Study Spaces; and one vote each for Virgo, Events, Reserve a Room, and Printers and Photocopiers.
  • When asked to find the hours of a digital media lab, four testers correctly used the Hours dropdown menu in the sticky header or the “View All Hours” link. Two navigated to the “Digital Labs” link under Services so eventually found the correct information, but with some difficulty.
  • Most testers were familiar with Ask A Librarian as a way to get help finding an article. AAL was found in the sticky header, the banner, and the footer.
  • When asked to book a room for a study group, three selected the “Reserve a Room” link under Spaces. Two navigated to Services, then to “Reserve and Rent a Room”. One tester said he always finds the link by Googling “Book It”.
  • Finding an event that occurs next month in Brown library was difficult for four testers, but three eventually found it. One gave up. Two tried to use the search box by typing in Brown, which apparently does not search location, so they were not successful using that method. Most testers easily found and used the >> on the calendar to bring up the next month; and half used the library filter to find events in Brown.
  • When asked to find expert help in Sociology, three testers went to Ask A Librarian. One found the Consultations heading on the Services page, and one thought maybe he’d use the staff directory, but commented, “I wouldn’t know why I would be on this [library] site for a sociology expert.” None of the testers found the “Subject Liaisons” link under Services.
  • Other impressions of new design: two commented that they liked the blue; and the design was called “nice,” “easy to navigate,” “self-explanatory,” and “modern.” One tester’s impression was “not favorable” because he finds it similar to the new UVA home page design that “benefits outsiders.”
  • On the mobile design, 4/6 testers found the hamburger menu and used it to find About.
  • On the mobile design, 5/6 testers were successful in scrolling to find a recent announcement under News.
  • On a scale of 1-5, with 5 being very easy, 5/6 testers rated their ease in completing these tasks as 4 or 5.


Mobile App Ideation Session

On Friday, February 12, the UX Team conducted a Mobile App Ideation session.

Definition of ideation

:  the capacity for or the act of forming or entertaining ideas

Merriam-Webster. Merriam-Webster. Web. 17 Feb. 2016.

The purpose of the session was to give Library Staff the opportunity to take part in the Team’s Mobile App Project. The Mobile App Project is part of a larger project already underway to develop a solid and flexible web services layer. Participants learned about ideation and the project development process. By the end of the session, the participants had created prototypes that addressed user identified problems. During the Spring semester, these prototypes will be the basis for user testing and further development.

The Challenge was to create a “minimum viable product” for the Library’s mobile app that focuses on the needs of both graduate and undergraduate students.

  • Define Stage:
    • Participants divided into teams and examined feedback gathered from student focus group interviews. Needs/wants/observations were separated into categories. The goal was to form Insights based on these categories.
    • Participants then formed “How might we” questions, based on these Insights. Four questions that best aligned with student feedback were chosen.
    • In the next exercise, each participant composed a tweet, announcing the new Library App.
  • Diverge Stage:
    • In this stage, participants explored various ideas and drew a storyboard that illustrated their most viable idea.
  • Decide and Prototype Stages:
    • In the final exercise, each team selected the best individual idea and created a storyboard for presentation to the group. The storyboards will later be used for testing and development.


The ideation session resulted in the selection of four themes. The Library app should provide

  • A customizable, fully functional version of Virgo and ILL
  • The ability to reserve group study rooms, and display seat availability throughout each location
  • Location based information for navigation
  • Real-time information about the Library

In the last exercise, participants explored these themes and created five mock-ups for a proposed app.

Storyboard Summaries

  1. For a user wanting real-time information about study space availability: the user could select a specific library and a specific room. The app would indicate either percentage full, or provide a heat map to indicate seat availability.
  2. (Assumes User has opted-in to multiple notifications)
    User is in a bookstore when the Library app reminds the user to check the Library catalog before buying a book. The user scans the ISBN, finds the book, and chooses to be reminded to get the book when she is in the Library. User visits the Library, receives notification(s).  The app provides a stacks map and offers help locating the book. Another notification asks the user if she would like to consult databases based on the call number of her selection, or talk to a subject specialist. The user receives one final notification, asking for feedback on the experience.
  3. User has three objectives: print a paper, locate a specific study group, and check-out a movie. The app would show the locations of services (such as printers) and study spaces (quiet and collaborative), as well as provide the ability to search the Library catalog. In each case, the app would provide a floor plan, indicating the user’s location and path to the desired service/room/item.
  4. User types search into Search Box → Retrieves map of the Grounds → User chooses Alderman Library → App displays catalog record showing copies available → App displays floor plan of Alderman Library, indicating You are here → App displays layout of the stacks, indicating the location of the book for which the user searched.
  5. Premise: People want all kinds of information, not limited to databases. The app would connect people who have questions with people who have answers. There would be three channels from which to choose: Ask a Librarian, AI, and Hey (shout out). The last would be for questions such as “How many people are in line at the Coffee Café?” Users might get points for answering such questions.

Videos of storyboards:


Research Results: What Users Might Want on Digital Signs

The UX Team and internal stakeholders are working together to find the best way to leverage new, integrated, interactive digital signage to enhance the user experience.

To that end, UX hosted a staff brainstorming session in April to generate a large number of ideas for content we could present to users via digital signs and the Alderman kiosk. Staff then selected their most promising ideas to present to users for their feedback.

In May, we worked with staff to deploy whiteboards in Alderman, Clemons, and Brown where students could vote and comment on staff ideas, or add their own.

The results, while not scientific, are a good “gut check” for what might resonate with users, and are as follows (including number of “checks” in favor of idea):

  1. What study space is not full: 80
  2. Videos (exercise/dance breaks): 39
  3. Display quiet and collaborative spaces: 35
  4. Information about Safe Ride, Safe Walk: 35
  5. Local and library-wide events, day of event
    promotion: 27
  6. Interactive map of Grounds (w/shuttle/bus
    schedules, walking directions): 24
  7. Instant polls and surveys: 17
  8. Library staff Who’s Who: 16

Bar graph of user votes for digital signage ideas.

Bar graph of user votes for digital signage ideas, by library

Update: In another view of this data, the results change slightly when one subtracts the votes against and idea from the votes for an idea to arrive at the net vote:

  1. What study space is not full: +76
  2. Display quiet and collaborative spaces: +35
  3. Information about Safe Ride, Safe Walk: +35
  4. Videos (exercise/dance breaks): +34
  5. Local and library-wide events, day of event promotion: +27
  6. Interactive map of Grounds (w/shuttle/bus schedules, walking directions): +21
  7. Instant polls and surveys: +12
  8. Library staff Who’s Who: +10

You can find more detailed results in Confluence, and you can review the output of the Library staff brainstorm there as well.

The purpose of this exercise was to get a rough idea of content interests to create some helpful and interesting signs for users, but was not intended to be prescriptive. We’ll pilot ideas and test results as we assess how users interact with these signs.

LibGuides v2 Transition Resources

The Library will be transitioning to LibGuides v2 this summer (hooray!). This post summarizes the important points that guide owners will need to make a smooth transition.


  • The migration will take place 6/17, but v2 guides will NOT be live at that time.
  • Guide owners have until 8/1 to make changes in preparation for Fall semester.


A cross-Area group is available to help with transitional needs. Contacts and the kinds of questions they can address are as follows:

  • Renee Reighart (UX): Usability, writing for the web, content best practices
  • Jenn Huck (I&S): Technical transition details
  • Jill Heinze (UX): Usability, supporting user research
  • Tim Morton (CAD): Revised A-Z list; Resource metadata
  • Chris Ruotolo and Michele Claibourn (AE): Managerial support


The migration group held an information session on 5/31:

A member of the migration group is reaching out to those content owners who wanted but were unable to attend.

After the 6/17 migration, training sessions will be offered that focus on refining and optimizing guides in the v2 environment. Sessions will be announced to staff once they are scheduled.


The UX has created and made available numerous resources to facilitate transition:

Get Help!

We know that this transition is a lot of work for content owners, and we want you to know you’re not alone! If you have questions or concerns and aren’t sure who to talk to, feel free to contact Jill Heinze to find the assistance you need.

Thank you, guide owners!

Research Results: Brown Library Space Study

Project Details

  • User Experience Project ID: UX-720, Brown Space Study
  • Purpose: Assess Brown space usage in preparation for redesign of Brown 140 due to Total Advising project space reductions.
  • Stakeholders: UVA Library Total Advising Project Space Team, Staff housed in Brown Library
  • Test dates: 4/7-4/8/2016
  • Test participants: 111 (106 undergrad, 4 grad, 1 staff)
  • Methodology: Two separate approaches were taken to gather data. The first was a direct solicitation and handout method, where, for one hour, students entering Brown library were asked if they would participate in the survey and asked to return the survey to a box on the way out. 100 students were selected and agreed to participate, with at least 63 returning the survey. After Brown staff relayed to the UX members conducting the survey that many students had enquired about participating in the survey, a change in method ensued and a second 100 surveys were left on the return table in Brown for users to fill out at will. At the same time the initial 63 returns were recovered by UX staff. An additional 48 surveys were recovered by staff the next morning, 24 hours after the initial survey start. All surveys were then entered by hand into a QuestionPro instrument designed process the survey data.

Summary of Findings

  • Brown’s users are almost exclusively undergraduates and are, in general, happy with their study options, both for groups and individuals.
  • Noise, crowding, lighting, and available outlets are the users’ chief concerns.
  • Most users utilize the Main Floor.
  • Users of the stacks floors were more content with the environment.
  • Users view the Ground and Intermediate Floor as quiet floors.

Project File

Research Results: Two Usability Tests Improve Virgo GIS

  • User Experience Project ID: UX-236, Geoblacklight User Research and Design
  • Purpose: Assess usability and clarity of the GIS Virgo (Geoblacklight) interface, redesign, test again
  • Stakeholders: U.Va. Library Geoblacklight project team and Virgo GIS users

First test comparing two interface designs

    • Test dates: 9/18-9/25/15
    • Test participants:
      • 2 GIS professionals
      • 4 students taking GIS classes
    • Methodology: In-person, individual testing on existing interface. Users were asked to complete a broad range of tasks relating to searching for,
      Wireframe redesign of GIS Virgo

      fig. 1: Wireframe redesign of GIS Virgo

      downloading, and emailing datasets. The user was left to decide to search with keywords in the search box or by zooming in on the map to narrow search results. Other tasks included identifying dataset attributes and layers, and using facets and search history. A new wireframe design was then shown (fig. 1) and feedback requested.

Summary of findings

  • Branding (UVA or Library) matters. “GeoBlacklight” meant nothing to testers and was not associated with Virgo or U.Va. Library. Testers want to understand scope/holdings of the database they’re searching.
  • Make it clear how to use the bounding box to search the map. 5/6 (all students) did not find the bounding box feature on their own.
  • Subject facet was used often but not successfully due to spotty metadata.

fig. 2: Virgo GIS with tutorial modal

Interim outcome: The interface was redesigned with a tutorial modal (fig. 2) and U.Va. branding. Facets were eliminated.

Second test, post-launch

  • Test dates: 2/23-2/25/16
  • Test participants:
    • 2 expert staff (Office of the Architect; Facilities Management)
    • 1 graduate student expert (Landscape Architecture)
    • 1 beginner graduate student (American Studies)
  • Methodology: In-person, individual testing after soft launch (Virgo GIS portal linked
    New Virgo GIS

    fig. 3: Virgo GIS

    into Virgo navigation header). The users were tested on navigation, search, download, and understanding of terminology. Tasks targeted map searching more so than previous test. See fig. 3.

Summary of findings

  • All 4 testers found the newest interface easy to use.
  • 3 out of 4 testers correctly used History, Saved Searches, and the tutorial modals.
  • The beginner tester was initially confused by some features but grasped concepts quickly.
  • Final observations on this interface:
    • “Easy to use. A lot of good data.”
    • “Straightforward. Impressed. Happy to see this. Will tell others to use.”
    • “Nice. Has everything I would want. Best GIS portal that I have used. Incredible. Impressed. Great job. Love it.”
    • “Nice, open, interesting, more tools and data than expected. Very cool, easy to use.”

Project status: Virgo GIS had a soft launch in January 2016 and a phase two is planned to implement more features. Based on the post-launch testing, the recommendation was made to create a Help page with definitions, explanation of features, and anomalies in data and presentation.

Project Files: and

Research Results: Dataset Deposit and Search in Libra

  • User Experience Project ID: UX-514 and UX-570, Complete user testing and analysis for Dataverse Deposit and Search functions in Libra, the online archive for University of Virginia scholarship
  • Purpose: Assess usability and clarity of the data deposit process and search function in Libra
  • Stakeholders: U.Va. Library Libra project team and Libra dataset users
  • Test dates: 10/29-11/4/15 and 11/17-11/20/15
  • Test participants:
    • 3 faculty (Education, Environmental Science, Pathology)
    • 5 graduate students (Education, Public Health, Biology)
    • 1 undergraduate (Systems Engineering)
  • Methodology: In-person testing in two phases. The first users were tested on uploading, describing, and publishing datasets to Dataverse, a dataset deposit and discovery interface developed at Harvard. Testers were shown two different Dataverse instances and were asked to complete a series of tasks on each. Testers were then asked which Dataverse interface was easier to understand and use. The second users were asked to perform searches and interpret findings on the preferred Dataverse instance.

Summary of findings

  • The deposit procedure was intuitive or learnable.
  • All 4 deposit testers wanted or expected to see the Add button on the initial page.
  • Deposit testers easily navigated around the site. All 4 deposit testers used breadcrumbs and upper left Dataverse icon to navigate back to start. Login link was expected or easily found on upper right.
  • Deposit testers easily used Add and Edit buttons, and were familiar with these terms as actionable.
  • All deposit testers stumbled a bit typing in dates in the required format: YYYY-MM-DD.
  • 3 out of 4 deposit testers were not confident or familiar with the term “metadata.”
  • Deposit testers liked icons, photos, and images, and felt they communicated valuable information.
  • All 5 search testers were familiar and comfortable with using facets and search box.
  • 4 out of 5 search testers were able to correctly define what a dataverse is. Definitions included “warehouse,” “schools or areas,” “a collection from an institution,” “a project,” and “specific to a department.” 4 out of 5 search testers understood the hierarchy: dataverses contain datasets which contain files.
  • Search testers found popup definitions useful.

Project status

Libra dataset deposit and search will have a soft launch with early adopters in March 2016.

Project Files: and

Research Results: Catalog Advanced Search

Project details:

  • User Experience Project ID: UX-27 Virgo UI Design for Full-text and Exact Search
  • Purpose: Assess usability of proposed changes to Catalog Advanced Search in Virgo. These changes will allow the user to select “exact” searching and/or allow the user to select full-text searching for keywords
  • Stakeholders: All Virgo users
  • Test date: 5/22/15
  • Test participants: 1 graduate student and 2 undergraduates
  • Methodology: In-person with paper mock-up of proposed changes

Brief summary of findings:

  • Two of the three users correctly matched search results to the search strategies, indicating an understanding of how “Don’t stem” and “Full-text” work.
  • Users did not readily understand the meaning of “stem” and “full-text”, but found the Help text clear.
    • (Some minor changes were made based on user recommendations: Discussing both stemmed and non-stemmed searches, including a fuller explanation of what the “full-text” option does, and using the term “group” instead of “phrase.”)
  • One user specifically mentioned liking the checkboxes.
  • The graduate student was well-versed in Virgo Advanced Search and easily understood how the new features work.
    • Undergraduates demonstrated more confusion with terminology but used good logic to anticipate how the features work.

Project Files:

Research Results: Available to Order in Virgo

  • User Experience Project ID: UX-544, Complete user testing and analysis for Available to Order in Virgo
  • Purpose: Assess usability and clarity of a new process on Virgo. Patrons find book records in Virgo and can select them to be purchased for the permanent Library collection. Electronic books are immediately accessible while physical books are ordered and delivered via the LEO or Ivy Stacks process to the requesting patron.
  • Stakeholders: U.Va. Library “Available to Order” project team
  • Test date: 11/6/15
  • Test participants:
    • 5 undergraduate students (2 second-years, 3 third-years)
    • 1 graduate student (first year in Batten School)
    • Methodology: “Guerrilla” testing, in which we approached students in the main hall of Alderman Library and asked for 5 minutes of their time in exchange for a candy bar. Testers were asked to complete six tasks on a server with test data to ascertain if the process was clear and if there were any areas of confusion.

Fig. 1: Sample Virgo record seen by testers

  • Brief summary of findings:
    • The link to request an item, which sat below the Availability box, was not readily seen by 5 out of 6 testers. When asked to request an item, testers first found and clicked on the link about the new service (AKA the “About” page). The recommendation was therefore made to improve the visibility and clarity of the request link. The link was refashioned into a button to make it more actionable.
    • Testers were able to ascertain from the “About” page what an Available to Order item was, how much it would cost them, and how long it would take for the book to arrive.
    • When asked where they would get help, all testers indicated familiarity with the “Ask A Librarian” service or that they would be comfortable asking for help at a library service desk.
    • Testers had no trouble understanding that clicking on the “Continue” button would initiate the order request. Testers commented that they’d also like confirmation that the book wouldn’t be charged to them and to see more information about what happens next in the process. One tester commented that he’d like to have a shopping cart, like Amazon, where he could then review his order before submitting it.
  • Project status: Changes were made to clarify the flow of the request process and the “Available to Order” service was rolled out in December 2015.

Project Files:

UX Team Priorities for Spring 2016

A high-level overview of UX Team priorities for the Spring 2016 semester is now available.

Highlights include:

  • Completing 2015 User Survey follow-ups and finalizing questions for 2016 User Survey.
  • Experimenting with ways to connect physical and virtual spaces.
  • Define and execute a mobile strategy; Launch v1 of a mobile app by end of the semester.
  • Implement technical efficiences and automation in web development process. Also establish a pattern library.
  • Create and present a web strategy.
  • Evaluate and improve signage.
  • Support Libra 2.0 work.
  • Conduct Virgo research in support of next version.
  • Complete a space use audit.

Questions? Please send them to me (Jill Heinze, Director, User Experience)!