Research Results: Staff Website Information Architecture

Project details: 
  • User Experience Project ID: UX-16 Reorganize Staff Website
  • Purpose: Reevaluate the information architecture (IA) of the Staff Website in response to expressed user needs.
  • Stakeholders: U.Va. Library staff
  • Test dates: Launched 12/11/14
  • Test participants: 31 Library staff members
  • Methodology: Remote, online open card sort
  • Brief summary of findings:
    • There was little agreement about what content fit under categories labeled by department (ex: “Human Resources”). This reinforces best practice which advises against an IA based on organizational structure.
    • Major content areas/themes users identified include:
      • Forms
      • Resources for Managers / Resources for Employees (to be combined into a label such as “Employee Services”)
      • Computing
      • Library Buildings & Spaces
      • In an Emergency
      • Organizational Design
      • Announcements
  • Next steps:
    • UX work is complete. Library Administration will interpret and apply user research findings in revising the site, communicating with stakeholders as appropriate.

Project File (Summary of Findings):


Research Results: Interview with Expert Manuscript Researcher

Project details:
  • User Experience Project ID: UX-119 Manuscripts Discovery
  • Purpose: Have exploratory conversation with a visiting expert researcher to understand the research process and mental models related to manuscript research.
  • Interview date: 3/3/15
  • Interview participants: 1 expert researcher (J.D., Ph.D.), U.Va. Library User Experience Librarian
  • Methodology: Extemporaneous conversation

Brief Summary of Findings

  • Opening up collections to user-contributed annotations should be seriously evaluated in light of the expressed needs shared in this interview, as well as in previous U.Va. Library user research on manuscript research. This would address many of the most pressing user and librarian challenges, and add tremendous value to the user experience. Indeed, this annotating is already happening within physical collections via note-sharing among researchers.
  • When digitizing collections, wherever possible, good date information and high-resolution, quality-controlled scans should be prioritized. Document readers should recognize the scale at which users need to read and print materials.
  • In terms of creating quality user experiences, it appears that those systems that save researchers time are among the most impactful. One of the most positive, memorable experiences this user related was of being able to find everything he needed in about 15 minutes thanks to a card catalog organized by date – an experience described as “awesome.”
  • Librarians have significant opportunity to add value to the manuscript research experience by facilitating researcher-contributed information and collaboration, and elucidating the context and connections among collections in a more broad and systematic way. (Currently, this is accomplished primarily through one-to-one interactions, which requires users to find the right person at the right time.)

Interview Notes and Summary of Findings

Research Results: Clemons Media Classrooms Reservation Form

Project details:


  • User Experience Project ID: UX-3 Room Reservation Systems
  • Purpose: Assess usability of proposed changes to Clemons Media Reservation Form
  • Stakeholders: Clemons staff who moderate bookings; Faculty and grad students who book rooms for viewing media
  • Test dates: 2/4/15-2/10/15
  • Test participants: 4 graduate students who teach
  • Methodology: In-person and remote usability test
  • Brief summary of findings:
    • 3 out of 4 testers wanted to visit the room before teaching in it to be sure it would work for their classes. 2 testers mentioned that a photo or layout of the room might suffice.
    • 3 testers found and used the Check Virgo button. 1 tester called the Virgo popup window “fantastic”.
    • All 4 testers expected an auto-generated email confirming their submission, and that there would be another follow up to confirm the reservation.
    • None of the testers tried to submit both class reservations on the same form. Only 1 tester saw the + option to add another date/time; None seemed to read the instructions below the Reservation Dates/Times heading.


Project Files:



The UX Spring Research Calendar is Full!

If you’re unfamiliar with user experience research, you may not know about all of the planning and milestones essential to the process. Typically, any study entails:

  • Interviewing stakeholders and conducting needs assessments to clearly understand the end user perspective
  • Developing a protocol, or test, that we plan to have users conduct on our systems or services
  • Recruiting and scheduling users and note takers
  • Reserving equipment and space
  • Testing the protocol to make any necessary adjustments
  • Conducting tests
  • Collecting and analyzing results
  • Presenting findings
  • Making recommendations for changes
  • Ensuring changes are implemented per user requirements

This semester, we have already completed some tests and are analyzing results for the Staff Website information architecture and Libra ETD documentation. Another test regarding the Clemons media classroom reservation form will wrap up today, 2/10/15.

We’re actively planning 4 additional significant research projects:

  1. LibGuides v2
  2. Manuscripts Discovery (within Virgo)
  3. Libra Open Access needs assessment
  4. Research for various possible Virgo interface changes

To help you visualize how this work will be carried out this semester, I created a Research Calendar that highlights some significant milestones for these projects. (It’s important to note that I’m still working out how to accommodate some milestones, so I’ll be updating the document periodically.)

If you’d like to learn more, please contact me directly and/or plan to attend the February User Experience Community meeting on 2/11/15. You can find the agenda on the Staff Website.

In addition to these projects, we’ll be discussing suggestions the group received and how we might address them.

Hope to see you there!

What’s Happening in UX

A lot has been going on in UX here at the Library, and I thought you may like a brief update:

Recent Work:

    • User Survey: I’ve been working with Strategic Assessment Services (SAS) and Library Experience (LE) leadership to develop a revised draft of user questions for the 2015 survey. I think we’re close to a final draft of questions that relate to LE. Staff will be invited to review the entire draft when complete. We have approval to deploy the survey on March 16th and keep it in the field until the 27th.
    • Library Experience Area: I’ve been assisting LE leadership with planning for how to gather input to inform the area’s internal structure.
    • Staff Website: During a coffee session, it was advised that information about the organizational design be made more clear on the Staff Website. So, I created some pages to make the most important details more easily accessible (pending approval). In addition, I completed a user study in which staff reorganized content according to their preferences. I will analyze and use the results to improve the Site’s overall architecture.
    • Wearable Technology Event: On Tuesday 1/20, Doug and I hosted another session where staff could try out some wearable devices and explore their applicability to solving user needs. We plan to determine some ways staff can continue experimenting with newer technologies. PowerPoint slides from a User Experience Community meeting on the topic are available.
    • Libra Documentation: I completed a user study in which users responded to the organization of documentation about depositing electronic theses and dissertations (ETDs). In response to implied user needs, we created an ETD submission checklist that will be available in HTML and PDF form. I’m in the process of analyzing results and will rewrite and reorganize ETD documentation content accordingly.
    • Libra 2.0 User Requirements: I will work with Ellen to elicit user requirements for Open Access works. We’re considering early March for testing but the dates will depend on other projects in the queue.
    • Room Reservations and LibCal v2: I helped complete a revised form for reserving Clemons Media Classrooms. The UX Community approved the changes with minor edits. I’ll be doing user testing as soon as possible to identify any unforeseen issues. I also met with Jack who will scope out what’s needed to migrate to LibCal v2. The group assigned to improve our room reservation process will respond to Jack’s findings and make preparations to migrate. (We’re tentatively planning to do so over the summer.)
    • LibGuides v2: Tony, Starrie and I met with a LibGuides group to gather requirements for moving to v2 of LibGuides. Tomorrow (1/27), Tony and I will meet to review usage statistics, user research, and other data to inform wireframes for a new design. We’ll test design concepts with users once they’re complete.
    • LibAnalytics – Research Partners: I’m meeting with representatives from the Research Partners initiative on 2/2 to determine how their LibAnalytics dataset should be modified to meet their data collection needs.
    • Manuscripts Discovery: The second phase of testing for this project will involve having users identify and navigate manuscript materials within “full” Virgo. We’re targeting user testing for the first two weeks of March.
    • Virgo UI Work: Solr updates were made so that the default search would no longer include the full text of items in our collection. However, UI work needs to be completed to allow users to override this default when needed. I’ll be meeting with Tony soon to discuss how to move forward.

Upcoming Work

  • Most of the projects listed above are currently undergoing work.
  • I’ve also received requests for smaller-scale usability tests, including:
    • A request from the Circulation Community to test how Hold statuses appear in Virgo
    • A request from Special Collections to test the Oral Histories Collection interface
  • Priorities for this week (1/26) include:
    • Analyzing results from the Libra documentation and Staff Website user tests
    • Kicking off LibGuides v2 wireframing and user testing planning
    • Finalizing User Survey questions

If you’d like more details about any of these projects, priorities, or upcoming work, please contact me directly, and/or plan to come to the next User Experience Community meeting.

Our next UX Community meeting will take place on Wednesday, February 4th from 3-4:30pm in Clemons 201. We’ll discuss these items and also suggestions the Community has received.

Thank you!

Research Results: Special Collections Research Conversation

Project details:


  • User Experience Project ID: UX-18 Manuscripts Discovery Undergraduate Focus Group [Associated with UX-119: Manuscripts Discovery]
  • Purpose: Explore undergraduates’ reflections on using special collections as part of a course project.
  • Stakeholders: Primarily staff who work with special collections.
  • Test date: 11/25/14
  • Test participants: Approximately 16 students enrolled in USEM 1570
  • Methodology: Conducted a loosely-structured group interview with the class.
  • Project status: The interview was completed. The summary document highlights key themes and questions for further consideration.
  • Brief summary of findings:
    • The perceived uniqueness of special collections is a double-edged sword.
    • Students cite preparedness as a desired improvement.
    • Students value primary sources for their authenticity and have great appreciation for the physicality of materials.
    • Students believe primary sources bolster the credibility of their coursework.

Research Highlights & Summary

Research Results: Manuscripts Discovery (Round 1)

Project details:


  • User Experience Project ID: UX-119 Manuscripts Discovery [Round 1]
  • Purpose: Explore usability concerns related to test instance of hierarchical manuscript discovery interface. (See document, “Manuscripts Discovery – Summary of Issues“)
  • Stakeholders: Special collections repositories in Law, Health Sciences and Albert and Shirley Small Special Collections Library
  • Test dates: Preliminary testing: October 7-October 10; Final testing: October 31-November 5
  • Test participants: Preliminary: 3 users (1 undergraduate, 2 graduate students); Final test: 5 users (3 graduate students, 1 faculty, 1 staff)
  • Methodology: Conducted individual usability tests with each user (See Final Protocol)
  • Project status: Presentation with recommendations given to stakeholders 11/21/14.
  • Brief summary of findings:
    • Users were successful in navigating within collections and using the Collection Navigation feature.
    • Users understood the hierarchical arrangement of materials expressed in the interface.
    • Users could not successfully search across collections and identify all collections relevant to a topic.

Project files


Preliminary Thoughts on LibGuides Usability

Thanks to Steven Villereal for sharing this assessment of LibGuides from the University of Chicago Library.

As we revisit our own guides in light of implementing v2, I was struck by a number of findings that align with other usability research we’ve done here on unrelated topics.

Here are some things that stood out to me, and I invite you to share your own perspectives:

  1. Users are highly task-oriented. You may have learned this from countless usability studies, but users come to our web pages to get things done. As uncovered in this report, some users were so task-focused that they even wanted guides to relate back to specific courses. We should keep in mind the concrete tasks that draw users to our guides as we curate them.
  2. Users love tutorials. I’m not sure how universal this finding is, or to what extent users would actually use the videos, but this report echoes what I’ve heard in interviews with undergraduates – that they would like short video tutorials on basic library and resource use. It’s a reminder that we need to consider ways to communicate information aside from text.
  3. Users hone in on search boxes. In a number of tests I’ve done, the search box always figures prominently in user feedback. They love them, but they often have the wrong idea about what they do. In the case of our guides, the default search is set to search Virgo. I’m betting many of our users would find that surprising.
  4. Users hate long lists but want descriptive text. I’d summarize this finding as, users want just enough but not too much. This study found, “…they [users] wanted some kind of descriptive text for each link. They wanted to understand what each resource was useful for, but they did not want to click on links to the databases themselves to experiment and discover that on their own.” This is completely in-line with usability principles. Clicks are like currency. Users are reluctant to spend them unless they have a pretty good idea about what they’ll be getting.
  5. Users want human connections! This University of Chicago Library study noted, “Our group also found substantial evidence that users want to see librarians as real people. Pictures of Library buildings featured on LibGuides were consistently viewed negatively by participants while pictures of librarians themselves, contact information, and “Chat with a Librarian” were consistently viewed positively. The personal connection appeared important to the patrons we surveyed.” This sentiment aligns with personal interviews and usability studies I’ve performed, and I’m often surprised by how often it pops up. Users want individual names and faces to help orient them to the library. Regardless of technology’s ubiquity, the human element is still a vital ingredient in our virtual presence.

What findings caught your attention?

I look forward to doing some research on our own guides in the near future, and I’ll be sure to loop you in to that process.


Search is No Substitute for Good Site Architecture

“Well, if they can’t find it, users can just do a search, right?”

I’ve heard this comment enough in meetings that it’s worth reviewing why a site search should not be the solution to bad information architecture.

The Nielsen/Norman Group recently published a fantastic article noting 5 reasons why we shouldn’t rely on searching as a primary means for users to find content. Here are some take-aways from the piece:

  1. Searching requires that users have a good mental model of the domain area in which their searching, including knowledge about what attributes are most relevant in their search. This expectation isn’t realistic. In the case of academic library websites, we tend to organize information by buildings and locations, disciplines and subject specialties, and services, for examples. Users who don’t approach our site with the same mindset would be at a loss when conducting a site search.
  2. Searching demands users rely on their memory. Users have to remember specific keywords and how to structure the best search. That’s a lot of mental work, and, therefore, not user-friendly.
  3. Search takes more time and effort (it’s costly!). Think about how many times you’ve had to retype keywords on  your mobile phone because you touched the wrong key or auto correct intervened, and you can understand how searching demands more work from users than browsing.
  4. Site search usually works poorly.
  5. Users are bad at searching. Despite librarians’ best intentions and aspirations, poor search habits are ubiquitous. As stated in the article, “We keep seeing this over and over again: people have no understanding of what makes a good search query on a website. Their search mental model is corrupted by the big search engines and they expect search to work in the same way on every site.”

Yes, search is useful, particularly in cases where users have a good sense of what they’re looking for, but it is by no means an “out” for developing and maintaining a well-organized site that helps orient users to what we offer.

Research Results: Undergraduate User Interviews (1 of 7)

Project details:


  • User Experience Project ID: UX-111C Interview Analysis & Presentation (Undergraduate Interview S1)
  • Purpose: To understand ways in which users approach and execute research projects with a slight focus on Virgo article searching. Also, to test library staff assumptions about definitions of an “engaging” library.
  • Stakeholders: All Library staff
  • Interview dates: November – December 2013
  • Test participants: 12 users (7 undergraduates, 5 faculty)
  • Methodology: Conducted 1-to-1 semi-structured interviews. Recorded voice and also screens as users demonstrated Virgo searching.
  • Project status: Transcription and video editing of this undergraduate interview complete. Six more remain, as well as final analysis. Engagement portions of faculty interviews transcribed (not yet in Box).
  • Highlights: User…
    • Is a relative novice with academic research. (Has written 2 papers while at U.Va. that required few sources.)
    • Is very comfortable asking questions and seems to have started all of her research projects by consulting with faculty and/or Library staff.
    • Changes keywords as a recovery strategy when she can’t find what she’s looking for in Virgo (no mention of facets).
    • Highly values Library printing services and some computers. Checks out DVD’s but does not generally use libraries as reading/study spaces.
    • Believes major challenge to using Virgo is identifying synonyms in keyword searches.
    • Defines engagement as moving from passive to active: Generally, thinks of the library as a place that is there when you need it for research.
    • Thinks Library could do a better job of conveying what it offers outside of research materials for papers.
    • Considers library-sponsored events as a way to convey that the Library is an active part of the University community.
    • Believes the “one-stop-shop” idea proposed by Library staff is unrealistic.
    • Doesn’t use libraries as study spaces, but nevertheless would value designated quiet areas.
    • Liked that staff would not give up on her research project even when she was willing to change topics.

Project files