Professional Documents
Culture Documents
510
Information Seeking through Students' Eyes 511
vices and systems. The specific method tween 1976 and 1996), expect to find all
we used was to ask students to record the information they need, for work and
4
their academic information-seeking ac- play, within easy electronic reach. For
tivities over seven days, including their them, learning takes place everywhere,
use of the Internet, consulting books, thanks to the portability of laptops, cell
discussing strategies with classmates, or phones and PDAs. Today's students move
meeting with faculty. Photographs and associatively through information rather
notes taken by the students were used to than linearly, having/ as one researcher
5
jog their memories as they recounted their put it, "hyperlinked minds."
activities during follow-up interviews. Traditional information literacy educa-
These accounts gave us a new under- tion has the potential to increase the use
standing of where, why, when, and how of library resources, but it has not yet
the students sought information. These changed the ways in which students ap-
insights generated ideas for improving proach research as they advance through
library services by accommodating the their college careers. Van Scoyoc and
ways in which students already conduct Cason found in 2006 that college juniors
their research. and seniors were no less likely to turn to
the Internet first for research than first-
Background and second-year students. At the same
The information landscape has changed time, despite evidence that they lack
dramatically over the past decade. basic information literacy skills, students
Google, with its single search box, speedy consistently rate themselves as effective
6
results display, and relevancy ranking has users of information.
changed user expectations for all search To understand how to apply the find-
interfaces. In addition, social networking ings in this study more broadly, it is im-
-features such as tagging support more portant to comprehend how M1T students
collaborative interaction among Internet vary in information-use practices from
users. Search tools and information are students at other universities. MIT enrolls
tightly integrated in today's electronic en- about 4,000 undergraduates and 6,000
vironment. For example, a user of Google graduate students each year. Of the 4,000
Scholar may access the citation, abstract, undergraduates, nearly 2,000 are in the
and even full text of a document all from school of engineering. Despite common
what appears to be one interface, masking stereotypes, MIvT students are more well
7
the fact that the index, the publication, rounded today than ever before. Today's
and the linking software come from three techies, the traditional mainstay of MIT's
separate providers. Finally, the sheer enrollment, are equally attractive to top
volume of information available online liberal arts schools and are actively re-
has grown enormously. These changes in cruited by them. Yet, recent comparative
the information environment mean that surveys show that MIT students are less
students are searching for information likely than their counterparts at other uni-
in different ways than they might have a versities to ask for help from librarians,
few years ago. are less likely to have found an interesting
Research has shown that students book through browsing at the library, and
expect to find most of their information are less likely to use the library for group
online.' They tend to turn to the Internet study and discussion. They are similar to
first, especially Google, when embark- their peers at other institutions in having
ing upon research.2 Today's students are used special collections in their work, and
busier than ever, and they highly value the extent to which they use computers in
the time-saving features of electronic requesting materials from libraries.'
information retrieval.3 Today's students, Given the rapidly changing informa-
known as Millennials (anyone born be- tion landscape and a desire to fundamen-
512 College & Research Libraries November 2008
tally examine how library systems need In this case, we believed the cultural
to evolve, MIT librarians wanted to find probe would help spark new insights
out more about what their own students into the information needs of our users by
were doing. MIT librarians designed a allowing us to see more of the breadth of
study inspired by the methodology used their information-seeking behavior. The
by Foster and Gibbons at the University 2005 MIiT Libraries' quantitative survey
of Rochester. In the Rochester study, stu- was sent to nearly 14,000 individuals-
dents were given cameras to document and almost half responded, including
their study habits. The photos the stu- over 50 percent of the student body and
dents took were later used in the course of 30 percent of the faculty. The answers to
interviews to help the students remember the more than 50 questions, including two
what they had done during a period of open-ended comment boxes, provided a
time. The open-ended, qualitative ap- rich amount of data concerning the com-
proach taken by the Rochester group munities' opinions of the quality and
provided data relevant to particular value of the MIT Libraries' resources and
questions of the researchers in addition services. However, the survey could not
to suggesting entirely new questions and provide details about how and whether
avenues for future research. One of the these resources and services were being
librarians described the process as a way used. We anticipated that seeing this
to "discover the unexpected."9 This sort of complete context would help highlight
ethnographic research is useful for chal- the service gaps that the MIT Libraries
lenging assumptions held by an organiza- needed to fill.
tion and informing plans for large-scale or Students took pictures, captured screen
long-term changes in systems.0 shots, and recorded notes of what steps
they followed every time they looked for
How the MIT Photo Diary Study was information related to their academic life
conducted over the course of a specific week. They
AfethodologV were then asked to bring the photos,
The Photo Diary Study used a methodol- screen shots, and notes to an interview
ogy from the field of anthropology known and to use those images and words to
as a "cultural probe." The premise behind jog their memories about the details
a cultural probe is that better results can of what they did. The undergraduates
be achieved by studying people in their participated during the spring semester.
own environment. Since it is not possible The graduate student portion of the study
to observe participants at all times, a occurred between semesters, which likely
device such as a camera or a notebook is caused an underreporting of course-
used by participants to help them record related goals and tasks. Each student
what they do and is later used to prompt received a $50 Amazon gift certificate in
memory of the events captured. This exchange for their time. Because we were
method is currently used in the field of in- interviewing students for this project, we
formation architecture to get a more com- secured permission and training from
plete picture of users and their culture." MIT's Committee on the Use of Humans
As a qualitative approach, a cultural as Experimental Subjects (COUHES).
probe tends to generate creative thinking For this study, we had hoped to re-
and insight related to a user group's be- cruit a manageable pool of 20 graduate
havior rather than statistically significant students and 20 undergraduate students,
data results. Nevertheless, results can be with participation in proportion to the
coded and analyzed to suggest trends enrollment in the five schools at MIT.
and to move beyond impressions and Despite the best efforts of the study
anecdotes, bringing a larger pattern of team, only 16 graduate students and
behavior into sharper focus. 16 undergraduates were found to par-
Information Seeking through Students' Eyes 513
TABLE 1 Analysis
Volunteers The interviewing teams'
School # of Graduate # of Ui ndergraduate notes were first analyzed
Students Students to identify and summa-
Art and Architecture I 0 rize major trends and is-
sues. Next, interviewers
Engineering 7 7 created cards for each
Science 3 3 student information-re-
Humanities, Arts 3 2 lated task documented
or described by the stu-
and Social Sciences dents, and each card was
Sloan School of 2 1 annotated to identify
Management the broader goal of that
Undecided NA 3 task and all methods
Total 16 16 used in completing the
task. The interviewers
then did a card sort to
ticipate; and, while participation was not group cards describing identical goals.
proportional to enrollment, we did have From these groupings, goal categories
student participants from each of MIT's were created. For example, the goal
five schools. We recruited from a pool category "course-related preparation"
of volunteers who had indicated their included individual goals of studying
willingness to give feedback for future for an exam, completing a homework
projects in the 2005 Library Survey. The assignment, and doing course readings.
focus of the study was solely on students, The process was repeated for tasks and
as they form the largest group of users of methods to create relevant categories
the MIT Libraries, and also because the for each set. Lists of information-seek-
time-intensive nature of the study made ing goals, tasks, and methods were
faculty participation unlikely. created from the cards. The cards were
coded with demographic data for each
Interviews student participant, including their
The interviews consisted of a list of ques- school, department, years at MIT, and
tions, including a few warm-up and dos- age. The coded data from the cards
ing questions, with most of the time spent were collated, standardized, and entered
on having each student tell the story of into a database from which quantita-
his or her week, showing photos, screen tive information-seeking patterns were
shots, and any notes prepared for the extracted. More than 500 methods were
student's diary. The interviewing teams employed for 275 tasks and goals in the
for this project consisted of eight librar- study sample.
ians, working in four teams of two people
each. At every interview, one librarian What We Learned from the MIT Photo
asked questions while the other took Diary Study
notes. Both librarians reviewed the notes One of the goals of the Photo Diary Study
after the interview was completed. These was to determine the range of information
notes represented a best effort by each activities of students at MIT. We broke our
of the interviewing teams, but no effort analysis into six major categories: goals,
was made to create an actual transcript tasks, methods, success, efficiencies, and
of the interview. This sometimes limited trusted resources. While we give a sum-
the ability of the researchers to check mary here, the complete report of our
back with original data when questions findings can be found at http://hdl.handle.
came up later. net/1721.1/33456.
514 College & Research Libraries November 2008
TABLE 2
Questions
Warm-up Questions What department are you in?
How long have you been at MIT?
Would you mind telling us your age?
# of times a month you use the physical
MIT Libraries?
# of times a month you use the electronic
resources of the MIT Libraries?
# of times a month you search for
information not using the MIT Libraries?
Interview Questions What were you looking for?
(used to flesh out details as needed) Where were you doing your research?
When were you doing your research?
What research strategies did you use?
What sources did you use?
How did you learn about these sources?
What devices did you use?
What worked?
What problems did you have?
How often do you do this task?
Closing Questions Are their any other tasks that you typically
do that you didn't do this week?
Do you ever do programming or scripting?
Have you ever used RSS, social
bookmarking tools, or Firefox extensions?
Goals ofStudent Information-seeking bine their research work with class work,
Activities the graduate students generally take
For every task performed by the stu- fewer classes per semester than under-
dents, the study team identified the goal graduates and typically are involved in
of the task. A summary of the students' research endeavors from the beginning of
goals, broken down by graduate and their degree program. With this study, we
undergraduate, provides a picture of the obtained a great deal of data about their
general differences between the goals research work but not how they approach
pursued by these two groups. course work.
As we expected, due to the fact that Despite the difference in timing of the
the graduate student portion of the study study for the two populations, there was
was conducted during a semester break, variation between the number of goal
graduate students were more focused on categories pursued by graduate students
their research, while undergraduate work (11 categories) and undergraduates (8
was defined by classroom assignments. categories). Undergraduates, unlike their
While it would have been advantageous graduate counterparts, tended to seek
to have a snapshot of graduate student information in support of social activities
work during a semester when they com- when not doing work for a class. Graduate
Information Seeking through Students' Eyes 515
TABLE 4
Task Categories From Student Interviews
Task Category # of Tasks # of Tasks Totals
Graduate Undergraduate
Students Students
Search for a known item 21 25 46
Search for information on a topic 20 21 41
Find facts or do a quick lookup 7 28 35
Search for a partially-known item 13 20 33
Take notes and organize information 13 3 16
Complete a class assignment 0 16 16
Check web sites/RSS/scripts/email for current 9 6 15
awareness
Study for class 0 15 15
Learn about a software program 9 4 13
Do course readings 0 10 10
516 College & Research Libraries November 2008
TABLE 5
Methods Used to Accomplish Tasks
Method Category Total # of Times Used
Occurrences First on a Task
Search Google 78 50
Go directly to a known URL not otherwise specified 60 38
Use the library catalog to browse or search 30 17
Search licensed citation databases 26 8
Use course web sites 23 14
Review notes/handouts taken in class/lab 17 9
Search e-resources finding tool 16 11
Read textbook 13 3
Consult other students 13 7
Consult with guest lecturer/faculty 12 8
Search Google Scholar 12 5
Search licensed full-text databases 10 5
Physically browse a collection 9 6
Refer to books in personal library 9 6
Retrieve a print resource from a library 9 4
Use Wikipedia 9 4
Information Seeking through Students' Eyes 517
TABLE 6
Maenft Pro uentlv UITed Methods for the Ton Four Tasks
Task Method No. of
Occurrences
Find facts or do a quick lookup Search Google 25
Search for a partially-known item Search Google 18
Search for information on a topic Search a citation database 17
Search for a known item Search the library catalog 16
Searching Google was the most fre- a trusted resource as a person, source, or
quently used method for finding facts tool that the student had used previously
and searching for a partially known or that had been recommended to them by
item. Searching the NMIT Libraries' catalog someone they trusted. For the first metric,
was the most frequently used method there were several occasions in the study
for known-item searching. For finding in which the student felt that s/he had been
information on an unknown topic, search- successful at a task, but the interviewers did
ing citation databases narrowly edged not agree with that assessment. Arating of
out Google as the most frequently used "efficient" did not take into account wheth-
method, though searching Google was the er the interviewer felt that the student had
method most frequently used first. Upon found an excellent source of information.
further analysis, the data showed that In a few cases the students were successful
when Google was used as a first method, and efficient, but the piece of information
it was necessary 58 percent of the time they found was less than optimal. Such an
to use at least one additional method to outcome is one limitation of this measure-
satisfy the requirements of the task. ment. The trusted resource metric was
developed once the interviewers realized
Success, Efficiency, and Trusted Sources how often students relied on particular
For each task, the study team recorded resources or persons to the exclusion of
three additional sets of metrics: 1) how others, even when they did not necessarily
successful the student judged him/herself have confidence that the resource would
to be at completing the task; 2) how efficient yield a successful outcome. In one example
the interviewer(s) felt the student had been of the use of a trusted resource, the student
at accomplishing the task; and 3) if the stu- used PubMed as a starting point for find-
dent used a "trusted resource." We defined ing articles not related to medical topics,
TABLE 7
Assessment of Success, Efficiency and Use of Trusted Sources in
Information-seeking Tasks
Graduate Undergraduate
Students Students
Yes Yes
Did the student judge him or herself successful at 86% 93%
completing his/her task?
In the opinion of the interviewer, was the student 77% 85%
efficient at accomplishing his/her task?
Did the student use a trusted person or tool in 80% 69%
accomplishing his/her task?
518 College& Research Libraries November 2008
TABLE 8
Top Four Task Categories And Success/Efficiency/Trusted Source Metrics
Sorted By Student Status
Graduate students Undergraduate students
Task Category Successful Efficient Used Successful Efficient Used
Trusted Trusted
Source Source
Search for a known item 86% 86% 76% 84% 84% 64%
Search for information 80% 40% 90% 82% 64% 57%
on a topic
Search for a partially- 62% 69% 92% 95% 75% 80%
known item
Find facts or do a quick 100% 100% 86% 96% 86% 86%
lookup
simply because she had previously had had recommended them. Trust in online
success with it when performing research resources is built from frequent use as
in a different subject area. We gleaned a documented in a recent OCLC study,"-
key fact from each of the three metrics we Word-of-mouth influences on service se-
recorded: students considered themselves lection and trust is also a well-documented
to be overwhelmingly successful; inter- phenomenon in the social sciences.3
viewers found the students to be relatively For tasks like topical searching or
efficient; and students often chose to use a searching for partially known items,
trusted resource. graduate students were, by their own
Table 7 summarizes the overall results admission, less successful and, by our
for these metrics. analysis, less efficient. However, the
The success, efficiency, and trusted graduate students were consistently in-
resource metrics were applied to the top volved in more complex research where
four task categories as demonstrated in discovering much-needed data may be
table 8. difficult or nearly impossible. For under-
Overall, the graduate students were graduate students searching for infor-
more likely than the undergraduate stu- mation on a topic, there were a notably
dents to use a trusted resource in accom- smaller percentage of tasks for which they
plishing their information-related tasks, relied on trusted resources, and yet they
though both relied heavily on trusted remained quite efficient in their pursuits.
resources. It is possible that the difference It is possible that this result is due to the
arose because of differences in the nature less complex nature of their inquiries as
of the work done by the two groups. compared to the graduate students.
Many graduate students work in a lab or With respect to products and services
research group and thus had colleagues provided by the MNT Libraries, students
working on the same or similar research mentioned a variety of resources. The
areas with whom they could consult. Un- students frequently recalled how and
dergraduates, however, are new to their when they learned about specific tools.
field of study and were often experiment- Many students learned about the re-
ing with new resources for information. sources via personal recommendations
Anecdotally, many students verbalized from faculty or other students. A small
that they had automatically gone to certain number discovered resources by a num-
resources because they had used them ber of methods, such as browsing the MIT
before or because someone they trusted Libraries' database of electronic resources,
Information Seeking through Students' Eyes 519
2. Take advantage of browser extensions within them. We would like our platforms
and toolbars that enable integrationof library to enable easy sharing of data with other
links on sites often used by students. For applications by third parties who see new
example, Firefox plug-ins enable direct uses for it. For instance, if all MIT Librar-
links from a title on Amazon to a title in ies' metadata for owned content contained
our catalog. Extensions like these make it within existing systems were available in
possible to incorporate and integrate our a metadata aggregator tool that had Web
services and links without needing the services enabled, we could allow others to
cooperation of the outside Web site. These create applications using that data.
are becoming more popular as users in the
outside world are finding useful ways to Implicationsfor Other MIT Libraries
link libraries, bookstores, and other Web Services
sites and services. For example, the MNT While the photo diary study provided a
Libraries now offer the LibX toolbar and significant amount of information to sup-
a Greasemonkey script that automates port decision-making for the evolution
lookup in the MIT Libraries catalog. of online tools, it inevitably also offered
3. Support the development of tools like direct support for other types of actions
browser extensions and toolbars within the and services that would enhance the in-
MIT community. In a technology-oriented formation-seeking experiences of the MIT
student community like MIT, we need to students. As in the case of the Rochester
invite our students to modify and improve study, these insights were an unexpected
library tools. In the study, we learned that benefit of the qualitative method we had
a small but significant number of students used, and the study team agreed that it
were interested in manipulating library was important to share these findings
data themselves, and many knew a fel- within the library system. Most notably:
low student who they believed would 1. Raise awareness: The Photo Diary
be interested in doing this. It would take Study echoed the results from the 2005
only a few students contributing their MIT Libraries Survey in that it showed
own ideas to begin to create novel and that students are often unaware of the vast
productive tools. The MIT Libraries have array of relevant and helpful information
begun to welcome such contributions by and tools available to them from the MIT
creating an "MIT Libraries Betas" page Libraries.
(http://libraries.mit.edu/betas/) where we 2. Reduce barriers to services: Because
are placing tools that aren't quite ready ease and speed of access are highly val-
for full-production mode or those that ued by MIT students, we recommended a
have been contributed by others. The first review of services to identify and address
student-contributed beta was a course- the barriers students experience in their
picking application written by computer use of libraries services.
science students and based on MIT Li- 3. Focus on custoinerservice:MIT students
braries' "Simile" research into integrating prefer to discover things on their own and
heterogeneous metadata. The betas site not to ask for help, except from trusted
has been highlighted periodically on the peers. It is critical that every interaction
MIT Libraries' homepage. with the MIT Libraries be positive in order
4. Migrate traditionallibraryfinding tools, to overcome this barrier as well as to allow
such as the catalog, to platforms thatenable Web the MIT Libraries to become part of the
services. In their current configuration, the students' networks of trusted resources.
library management system and the elec- 4. Close the skills gap: The undergradu-
tronic resources management (ERM) sys- ates interviewed in this study did not
tem hosted by the MIT Libraries are largely demonstrate the level of information-
inflexible because it is difficult to create seeking skills needed to carry them suc-
applications that access the data contained cessfully into graduate school. Continued
522 College & Research Libraries November 2008
efforts should be directed to assisting cur- and the other to investigate acquiring or
rent graduate students with getting up to building a metadata aggregator tool, such
speed, as well as preparing undergradu- as WorldCat Local. The MIT Libraries'
ates for future academic careers. Betas page was implemented in the sum-
5. Assist with personal iqformation man- mer of 2006.
agement: The number of tasks and the The MIT Libraries are one institution
amount of time the graduate students among many that are working to rapidly
devoted to information management develop systems-based solutions that will
were significant. Continued promotion radically improve information discovery
and instruction on bibliographic manage- for their users. From North Carolina
ment tools like Endnote, RefWorks, and State's innovative catalog interface using
Zotero will help students increase their Endeca software, to University of Roches-
productivity and provide an opportunity ter's eXtensible catalog effort, and many
for the MIT Libraries to partner with the others, big changes are underway. The
students in additional ways. MIT Libraries hope both to contribute to
this progress and to take advantage of
Conclusion innovations begun by others. We believe
Since the completion of the Photo Diary strongly that by staying focused on the
Study, the MIT Libraries have moved for- needs and information-seeking practices
ward on several of the recommendations of our students, we -willbe able to create
outlined here. Project SimpLR (http://lib- a sustained level of improvement in our
staff.mit.edu/simpl) has charged two task systems that will contribute to their ulti-
forces: one to implement a metasearch tool mate success in their academic lives.
Notes
1. Anna M. Van Scoyoc and Caroline Cason, "The ElectronicAcademic Library: Undergradu-
ate Research Behavior in a Library without Books," portal:Librariesand theiAcadeiy 6 (2006): 47-59;
Lesley M. Moyo, "Electronic Libraries and the Emergence of New Service Paradigms," Electronic
Libraq! 22 (2004): 220-30.
2. Van Scoyoc and Cason, "The Electronic Academic Library"; Christen Thompson, "In-
formation Illiterate or Lazy: How College Students Use the Web for Research," portal: Libraries
and the Academy 3 (2003): 259-68; Joan K. Lippincott, "Next Generation Students and Libraries,"
EDUCAUSE Review (2005), available online at http:/flibrary.case.edulksl/admin/slc/workingdocu-
ments/net__generation_students.pdf [Accessed 30 October 2007].
3. Studying Students: The UndergraduateResearch Projectat the UniversityofRochester, ed. Nancy
Fried Foster and Susan Gibbons (Chicago: Association of College and Research Libraries, 2007);
Lippincott, "Next Generation Students and Libraries."
4. Van Scoyoc and Cason, "The Electronic Academic Library"; Moyo, "Electronic Libraries
and the Emergence of New Service Paradigms."
5. Foster and Gibbons, Studying Students.
6. Van Scoyoc and Cason, "The Electronic Academic Library"; Foster and Gibbons, Stud4ing
Students; Thompson, "Information Illiterate or Lazy."
7. Marilee Jones, "New Kids on the Block: Observations on the Newest Generation of MIT
Students," MIT FacultyNeusletter 14 (2001): 12-15.
8. Institutional Research, Office of the Provost, MIUE CSEQ Report. C. Robert Pace and George
D. Kuh, College Student Experiences QuestionnaireResearch Programn (Indiana University Center for
Postsecondary Research and Planning, 5th ed., 2001).
9. Foster and Gibbons, Studying Students.
10. Catherine Courage and Kathy Baxter, UnderstandingYour Users:A PracticalGuide to User
Requirements Methods, Tools, and Techniques (San Francisco: Morgan Kaufman, 2004).
11. Hugh Beyer and Karen Holtzblatt, Contextual Design (San Diego: Academic Press, 1998);
Courage and Baxter, UnderstandingYour Users.
12. OCLC, Sharing,Privacyand Trust in Our Networked World: A Report to the OCLC Membership
(2007). Available online at wwwv.odc.org/reports/pdfs/sharing.pdf. [Accessed 24 October 2007].
13. Chip Walker, "Word of Mouth," American Demographics17 (1995): 38-45.
14. Chris Anderson, "The Long Tail," Wired 12 (2005): 170-77.
COPYRIGHT INFORMATION