February22,2016

RachaelWinterling&LibraryAdministration
J.MurreyAtkinsLibrary
UniversityofNorthCarolinaatCharlotte
9201UniversityCityBlvd
Charlotte,NC282230001
Re:UsabilityTestsofAtkinsLibraryOCLCInterface
WewouldfirstliketoexpressourgratitudetoRachaelWinterlingandtheJ.Murrey
AtkinsLibraryteamforvisitingourclass.Theinformationprovidedtoushasgivenussome
valuableinformationregardingthelibrarysonlineinterface,enablingustodesignourusability
teststoyieldthebestresultspossible.ThisassignmentwasgiventousbyAssociateEnglish
ProfessorDr.GregoryWickliffinourENGL4181,WritingUserDocumentsclass.Themain
goalofthisassignmentwastoperformusabilitytestsontheAtkins/WorldCatLocaluser
interface,sothatwemayanalyzetheresults(successesanddifficulties)andpresentourfindings
tothelibraryteam.
Ouranalysisproducedseveralkeyfindings.Forthemostpart,ourparticipantswereable
tousetheinterfacefairlyeffectivelyandsuccessfully.Whentheydidencounteranyproblems,
theinterfaceprovidedthemwiththetoolsnecessarytocompletetheirtask.However,webelieve
thatourtestingandanalysishasgivenussomevaluableinsightintosomeofthedeficienciesof
theinterface.Duetothis,wefeelthatwecanprovidethelibraryteamwithafew
recommendationsthatwillfurtherimprovetheinterface.Ourfindingsinclude:

Usersthatwerenewtotheinterfaceencounteredsomedifficultieswiththeirfirsttask.
Asthesefirsttimeuserscompleteeachtask,theamountofproblemsquicklybecameless
frequent.
Thebasic,ordefault,searchgenerallyfailedtoproducethecorrectresults.
Afterfirstattemptingtousethedefaultsearchfunction,themajorityofusersmovedtothe
advancedsearchfunctioninordertosuccessfullycompletetheirsearch.
Asthesefindingsindicate,therearesomeopportunitiesforimprovementthatwouldhelp
streamlinetheuserexperienceandlessentheamountoffrustrationtheyexperiencewhen
utilizingthelibraryinterface.
Sincerely,
CodyDennard,StephanieLee,&NicholasAlcalde
Authors

cc:
RachaelWinterling,UsabilitySpecialistfortheJ.MurreyAtkinsLibrary
GregoryWickliff,Assoc.ProfessorofEnglish

UNC Charlotte J. Murrey Atkins Library

Website Beta Version: Usability Testing Report

ENGL 4181
February 23, 2016

By: Cody Dennard

Stephanie Lee
Nicholas Alcalde

Submitted to:
-

Rachael Winterling, Usability Specialist for the J. Murrey Atkins Library

Gregory Wickliff, Assoc. Professor of English

Executive Summary
Thepurposeofthisusabilityreportwas
to test a users ability to navigate and
successfully use the WorldCat Local
searchprogramandreportourfindings
back to the Atkins Library team. We
learned a lot from this report and
hopefully our findings will help to
improve the usability of the program
itself.
We asked multiple students with
different levels of experience with
WorldCat to complete 3 tasks for our
usability test. These tasks included
findingoutwhichschoolshaveacertain
bookwhengiventitleandauthorname,
findingthefirsteditionyearofacertain
book when given the title, and finding
howmanycopies/differenteditionsofa
certainbookwereattheUNCCharlotte
J.MurreyAtkinslibrarywhengiventitle
andauthor.Aftereachtestwasover,we
askedthestudentsaseriesofquestions
togetthemtodiscusstheproblemsthey
werehavingduringtheactualtest.We
also monitored their actions during the
testsowecouldseetheexactpartsthat
theywerestrugglingwith.
Someofthetestsubjectsstruggledmore
thanotherswithcertaintasks,butthey
were all able to ultimately complete
them. Some of the main issues we
discoveredwerethatwhenyousearched
abookbyitsexactnameitwouldnotbe
the first result and more

Another big issue we found was that

many of our users found themselves
havingtoresorttotheadvancedsearch
options because the default search bar
would rarely find what they were
lookingfor.
Aftersomediscussionwitheachofthe
participants, they all seemed to agree
thatwhile theywereultimatelyableto
find what they needed, they think it
shouldhavebeenmucheasiertodoso.
Therewerealotofunnecessarystepsin
the process and at times it got
frustrating.Justabouteveryoneregularly
usesGoogleanditservesasthestandard
for which all other search engines are
compared. When a student goes from
GoogletousingWorldCattheywillfind
amajordropoffineaseofuseaswellas
searchresultsuccess.
It is clear that WorldCat is still in its
early stages and has much to improve
on. The biggest suggestion our group
wouldmakeaftercompletingthisreport
wouldbetogetridofthedefaultsearch
bar altogether until the search result
algorithm has been significantly
improved. We believe replacing it and
havingtheadvancedsearchoptionsfrom
thebeginningwouldsavestudentsalot
of time and effort since many of our
users found themselves resorting to
advanced search after unsuccessful use
ofthedefaultsearchbar.

searching/filteringhadtobedonetofind
it.

Table of Contents
ExecutiveSummary.i
IntroductionandProcedure...1
KeyFindings2
1.

Notuserfriendlyforfirsttimeusersor

2.

Advanced search was more useful than

rookies..............2

usingthebasic...2
searchafteraddingfilterstoyourresults.

ConclusionsandRecommendations..3
Appendices...4
1.

A:UsabilityTests

.4
2.

B: Follow Up Questions

...5
References.6

List of Tables and Figures

GraphA:PriorExperiencewithInterface2

TableA:TaskTimesandAverages2

..

ImageA:GoogleSearch...3
ImageB:AdvancedBetaSearch.3

UNC Charlotte J. Murrey Atkins Library

Website Beta Version: Usability Testing Report
Introduction
The University of North Carolina at
Charlotte J. Murrey Atkins Library
introduced the new interface and online
worldwidecatalogWorldCatLocalinthefall
semester of the 2013 school year. Since
then,manyproblems withthesystemwere
foundandreported.
The system has changed somewhat since
then and is always under observation for
futureproblemstobeaddressedsothatthe
systemcanevolvetobetterservethestudents
and faculty of the University of North
CarolinaatCharlotte.
At the beginning of this spring semester,
Rachael Winterling came to speak to our
classaboutthesystemandhowshewould
likeourhelpinimprovingit.Sheshowedus
thedifferentaspectsofthesystemincluding
theBetasearchoptionwhichwouldbethe
focusofourproject.
Dr.Wickliffassignedusgroups;oursmade
upofundergraduatestudentsStephanieLee,
CodyDennard,AndNicholas Alcalde. He
thengaveustheinstructionsforourproject.
Three undergraduate students and one
graduate student participated in our studies
andallofthesessionswentverywell.Each
userwasabletocompleteeachtaskthatwas
giventothem.
Ourfindingsspecifiedinthefollowingreport
showdetailsofeachuniqueuserexperience
includingwhentheystruggledandsucceeded
withtheinterface.Theevidenceweprovide
willhelptheJ.MurreyAtkinsLibraryand

WorldCat to improve the system and its

usabilitysoitisabetterfitforthefaculty
and students of the University of North
CarolinaatCharlotte.

Procedure
Eachmemberofourgroupreachedoutto
friendsandcolleaguestofindparticipants
forourstudy.
Theusabilitytestswecreatedallowedusto
observe students performing specific
searchesusingthebetalibraryinterfaceon
a computer that the library provides for
studyrooms.Wecreatedspecifictasksthat
we tested before hand to ensure that the
usercouldhaveachancetocompletethem.
Weplannedforeachtestsessiontotakeno
morethanthirtyminutestoallowustime
tocommunicateanddiscussthetaskswith
theusersbefore,during,andafterthetests.
Thesessionswererecordedoncameraand
we recorded the screen and audio, which
wasourmainuseofmaterialtogatherdata.
Afterthetaskswerecompleted,weasked
eachuserfollowupquestionstogagehow
their experience went. We reviewed the
recordings of each session to collect and
evaluate data to then write this report,
whichwaswrittenanddesignedinGoogle
Documents.

Key Findings
1.
Not user friendly for
firsttimeorrookieusers.
Acommonanswertoourfollowupquestion,
Did you feel uncomfortable with
completinganyofthetasks?,wasthatthey
struggled at first, but with each task, the
system became easier to use because they
weremorefamiliarwithit.Mostoftheusers
hadbarelyusedthesystembeforethetest.

wasnt straight forward to them and they

struggledtomanipulatethesystemtofind
exactlywhattheywerelookingfor.
Oneuserstated, Thefirsttaskmademe
feel dumb I feel like searching for the
title exactly should have brought up the
book.

2.
Advanced search was
moreusefulthanusingthebasic
search after adding filters to
yourresults.
Our results indicated that users ended up
using the advanced search quite a bit to
complete their tasks. The search results
producedbyusingthedefaultsearchrarely
yielded the correct results. While the
advanced search function itself is quite
useful, we feel that our participants were
requiredtoutilizeitfartoooften.
To be specific, our participants werent
satisfiedwiththebasicsearchresultsand,
therefore,switchedtotheadvancedsearch
50%ofthetime.Inourteamsopinion,this
isfartoooften.

Graph A: Prior Experience With Interface

It also took the users quite some time to

complete tasks or even to realize that they
hadcompletedatask.Theinterfacejust

Itisourrecommendationthatthebasic,or
default, search function needs to be
improveduponsothat itproduces better,
more accurate, results. This would save
usersvaluabletimeandeffort,makingtheir
experience with the interface much more
enjoyable.

Task 1

Task 2

Task 3

Min. Time

1:00

0:50

0:44

Max. Time

3:19

2:20

4:53

Avg. Time

1:59.75

1:27.5

2:04.25

Table A: Task Times and Averages

Conclusions and
Recommendations
In conclusion, the beta Version of the
WorldCatLocaldatabasefortheJ.Murrey
AtkinsLibraryisaworkinprogress.While
it can definitely be helpful in finding
resourcesinthelibrarydatabase,ithasalot
ofthingsthatneedtoimprovedtohelpwith
theoverallusabilityofthesearchengine.If
itisabletomaketheaccommodationsthat
our group recommended in addition to
fixingafewotherminorproblems,itwould
makeitmucheasierandlessfrustratingfor
studentstouse.

Image A: Google Search

We believe that any search tool online

should be held to the standard of using
Google. It simply always knows what the
userislookingforandisabletodisplayits

resultsbasedonrelevancetotheuser,and
this is what WorldCat should strive to
accomplish.Whenausertypesintheexact
nameofabookorarticleitshouldbethe
firstresultthatcomesup,period.Itshould
not be necessary to apply more filters or
evenresorttotheadvancedsearchfeature
when you already have all the necessary
informationtofindsomething.
As we stated above, our biggest
recommendation for improving WorldCat
wouldbetoremovethedefaultsearchbar
and replace it with the advanced search
optionuntilthedefaultsearchalgorithmcan
besignificantlyimproved.Sincehalfofour
users needed to resort to the advanced
search option in order to find what they
werelookingfor,wethinktheyshouldhave
access to it from the beginning of their
search. It was originally designed as
somethingthattheusermayuseiftheyrun
into any troubles, not something that is
requiredhalfthetimejusttofindasingle
item.

Image B: Advanced Beta Search

Appendix A: Usability Tests

User Task
1

Description

Time Success Summary/Issues

ComicBookSearch Whichschoolshavea 02:20 Yes

copyofStarWars:
TheLastCommand
byDarkHorseComics

SearchesstarWars:thelastcommand,
searchesstarwars,searchesstarwarscomic
book,formatbook,searchesstarwarsthe
lastcommandcomicbook,sortbyrelevance
only,1stresult

PublishDateSearch Yearofthefirst
editionofTheGreat
Gatsby

01:50 Yes

KeywordFirstedition&titlethegreat
gatsby,sortbyrelevancyonly,goesback,
titlethegreatgatsby&keywordfirst
published,year1919to1930,1standonly
result

CopiesAvailable
Search

00:44 Yes

Searcheslordoftherings,1stresult,library
unccharlottelibraries,4thresult,format
book,3rdresult,sortbyrelevanceonly,3rd
result

ComicBookSearch Whichschoolshavea 03:19 Yes

copyofStarWars:
TheLastCommand
byDarkHorseComics

Searchesstarwars:thelastcommand,format
book,goestoadvancedsearch,titlestar
wars:thelastcommand&formatbook,
browserdifficulties,searchesdarkhorse
comics,searchesstarwarsthelastcommand
bydarkhorsecomics,6thresult

PublishDateSearch Yearofthefirst
editionofTheGreat
Gatsby

00:50 Yes

KeywordGreatGatsby&formatbook&
year19191930,1stresult

CopiesAvailable

01:50 Yes

AdvancedSearch,locationunccharlotte

TheLordofthe
RingsbyJ.R.R.
Tolkein.Howmany
copiesanddifferent
editionsandformats
areavailableat
UNCC.

TheLordofthe

Search

RingsbyJ.R.R.
Tolkein.Howmany
copiesanddifferent
editionsandformats
areavailableat
UNCC.

libraries,keywordthelordoftherings,
authortolkein,browserissues,1stresult

ComicBookSearch Whichschoolshavea 01:00 Yes

copyofStarWars:
TheLastCommand
byDarkHorseComics

Searchesstarwarsthelastcommand,adds
darkhorsecomics,6thresult

PublishDateSearch Yearofthefirst
editionofTheGreat
Gatsby

00:50 Yes

Titlegreatgatsby,years19201930,1st
result

CopiesAvailable
Search

00:50 Yes

Searchesthelordoftherings,1stresult

TheLordofthe
RingsbyJ.R.R.
Tolkein.Howmany
copiesanddifferent
editionsandformats
areavailableat
UNCC.

ComicBookSearch Whichschoolshavea 01:20 Yes

copyofStarWars:
TheLastCommand
byDarkHorseComics

SearchesStarWars,refinessearchbybooks,
addstheLastCommand,6thresult.

PublishDateSearch Yearofthefirst
editionofTheGreat
Gatsby

02:20 Yes

Keywordfirstedition&titlegreatgatsby,
backtoadvancedsearch,1920to1929forthe
years,1stresultbutscrollspastitwetellhim
itwasthe1925.

CopiesAvailable
Search

04:53 Yes

Advancedsearch,locationUNCcharlotte
libraries&keywordlordoftherings,4th
resultbutscrollspastandgoestothenext
page,contentfulltext,authordoesntsee
tolkein,goesbacktoadvancedsearch,tries
again,4thresult,formatbook,resultis3rd,
authorJ.R.R.Tolkein,resultis1st

TheLordofthe
RingsbyJ.R.R.
Tolkein.Howmany
copiesanddifferent
editionsandformats
areavailableat
UNCC.

Appendix B: Follow Up Questions

User1

Thefirsttaskwasthehardestforthem.Theyhadtosearchbyrelevancetofindit.Otherthingswere
comingupforStarWarseventhoughheputintheexacttitle.
Thethirdtaskwastheeasiestforthem.Thewantedresultshowedupfirst.
Theyhadtogothroughsometrialanderrorduringthefirsttwotasksthenfeltcomfortableforthethird
task.
Theyfinishedallofthetasks.
Theysuggestthatwhensearchingbyanexactname,theresultsshouldmatchthat.

User2

Thefirsttaskwasthehardestforthem.TheresultwouldntshowunlesstheyaddedthePublisherDark
HorseComicstothesearch.

Thesecondtaskwastheeasiestforthemsincetheycouldfiltertheirresultsbydate.
Thefirsttaskmadethemfeeldumbandthatheshouldnthavebeenthathardtofind.
Theyfinishedallofthetasks
Theysuggestthatthesearchfeaturebeimprovedbyaddingmoreoptionsintheadvancedsearch.
User3 Thesecondtaskwashardestforthemsincethedatabasedidnotspecifythattheresultwasthefirst
published,wehadtotellthemthatwhattheyfoundwascorrectbecausewehadGoogledthecorrectfirst
publishyear.
Thefirsttaskwastheeasiestforthem.Theyjusthadtoaddthepublisher,andtheresultcamerightup
forthem.
Theyneverfeltuncomfortableduringthetasksandtheyneverseemedtostrugglewiththem.
Theyfinishedallofthetasks.
Theydonthaveanysuggestions,thatthesystemseemsstraightforwardenough.
User4

Thethirdtaskwasthehardestforthem.Theysaidthekeywordsearchshouldbemorespecific.
Thefirsttaskwastheeasiestforthem.TheysearchedbyDarkHorseComicsandwereabletofindit
rightaway.Theysaidthesecondtaskwasalsoeasysinceyoucouldsearchbydate.
Theystruggledthemostwiththethirdtaskanditmadethemuncomfortable.
Theyfinishedallofthetasks.
Theirsuggestionistomaketheadvancedsearchthedefaultinsteadofthebasicsearch.

References
UsabilityTestParticipants:

SophomoreMechanicalEngineeringTechnologyStudent,UNCCharlotte,2/5/2016
SeniorMastersofScienceinInformationTechnologyStudent,UNCCharlotte,2/5/2016
SeniorFinancewithAccountingConcentrationStudent,UNCCharlotte,2/5/2016
SeniorSoftwareandInformationSystemsStudent,UNCCharlotte,2/10/2016

SupportingDocument:
DanielJackson&RobertFarnham,DiscoverWorldCatwithFacultyScientists:Ausability
analysisofthediscoveryinterfacefortheJ.MurreyAtkinsLibrary,March12,2014