Running head: MULTIMEDIA AWARD

Multimedia Award
Kay L. Venteicher
University of Maryland University College
July 12, 2015

MULTIMEDIA AWARD

2
Multimedia Award

Creating a multimedia learning object for use in the online environment requires a twostep process. First, implementation of learning and cognition theories within a multimedia
software, and secondly, utilizing an assessment instrument that will validate the quality for the
intended user and provide improvement feedback to the designer (Krauss & Ally, 2005, p. 1).
The use of the interactive multimedia resources with text and graphics improves the learning
process with the learner reconciling the cognitive dissonance (Krauss & Ally, 2005, pp. 2-3).
This paper will document the comparison and evaluation of the multimedia web sites to
determine which will contribute a greater merit, worth or value (Baumgartner & Payr, 1997, p.
2) to the learning process.
The Learning Object Review Instrument (LORI) by Leacock and Nesbit (2007) offers a
framework to evaluate and compare two learning objects (p. 1). The learning objects are two
multimedia websites, which have been selected for consideration for the multimedia award.
Review of the tutorials will provide the evaluation results detailing the pedagogical design and
usability for each of the two finalist products. Additionally, the strengths and weaknesses of the
pedagogical design and usability will reviewed to analyze strengths and weaknesses in design for
each learning object. The panel will evaluate the tutorial websites, Leadership Perception
Around the World and Team Dynamics and Building.
Measurable Learning Objectives
The LORI is an evaluation tool used to produce comments and ratings of the multimedia
evaluands. The judging panel members will produce expert appraisals for the two multimedia
products. The panel members will evaluate the nine areas in LORI: a) content quality, b)
learning goal alignment, c) feedback and adaption, d) motivation, e) presentation design, f)

MULTIMEDIA AWARD

interaction usability, g) accessibility, h) reusability, and i) standards compliance (Leacock &

Nesbit, 2007, pp. 44-45). Each area evaluated will receive comments recorded and a rating (i.e.
low to high, 1 5, 1 star 5 stars, and NA for not applicable if exercising opting out). The panel
scores will be averaged presenting an overall average for each area. The learning object with the
highest score is the winner (Nesbit, Belfer, & Leacock, 2004, p. 1).
Media Finalists
The multimedia evaluands demonstrate an instructivist pedagogical philosophy with the
Leadership Perception around the World employing exercises and comparison tools for graphic
clusters that lean towards a constructivist pedagogical philosophy (Reeves & Hedberg, 2003, pp.
190-191). The learning theory behind each is consistent with behavioral psychology (Reeves &
Hedberg, 2003, p. 192).
Description of Leadership Perception around the World
Leadership Perception around the World is the first finalist, an interactive multimedia
tutorial. It is a tutorial by Tara Wemsing (2011) of IE Business School. It provides a tutorial
that instructs the six (6) dimensions of leadership based on the 2004 publication titled Culture,
Leadership, and Organizations: The GLOBE Study of 62 Societies (Wemsing, 2011). It
contains three exercises, which provide the learner with specific dimensions of the different
countries as well as a comparative graph for countries/cluster on leadership dimensions. The
audience is college level general education. Pedagogical strengths include stated learning
objectives and accurate data and content broken into smaller, defined modules of information supported
by completion quizzes to evaluate learning. A pedagogical weakness is group activity is limited.

Usability strengths include a simple, easy to navigate narrated power point presentation. A
usability weakness is the tutorial is limited to internet accessibility.

MULTIMEDIA AWARD

Description of Team Dynamics and Building

Team Dynamics and Building is the second finalist, an interactive multimedia tutorial. It
is a tutorial by Roger Dunham (2011) of PSU Berks Center for Learning & Teaching, Penn State
University. It provides a tutorial that provides the foundation information of what is a team,
benefits associated with a team, effective team traits, and team member skills. The audience it
serves is college level general education (i.e., lower and upper division) and high school.
Pedagogical strengths include stated learning objectives and content broken into small, defined
modules of information with completion post-quizzes to evaluate learning. A pedagogical
weakness is group activity is limited. Usability strengths include a simple, easy to navigate
narrated power point presentation. A usability weakness is the tutorial is limited to internet
accessibility.
LORI Evaluation of Multimedia
The learning objects will be evaluated by the utilizing the LORI. The first panel member
evaluation is provided in Table 1, Leadership Perception Around the World and Table 2, Team
Dynamics and Building.
Table 1
Leadership Perception around the World.
LORI Item
Comments
Content Quality
Global geographical regions
/maps represented
Appropriate level of detail,
definitions & analysis
Unbiased and objective data
with comparison capability
Differences among
cultural/ethnic group
represented
Learning Goal Alignment
Learning goals declared in
both content and metadata

Score
5

MULTIMEDIA AWARD

Feedback & Adaptation

Motivation

Presentation Design

Interaction Usability

Targeted learner as
undergraduate and graduate/
leadership/ethics/marketing
Individual homework or selfpaced assignment
Sufficient to enable learner to
achieve goal
Exercises/quizzes and
feedback
Interactive comparison tools
with model demonstration
Individual learner can create
tailored response based upon
differential input
Learner profile not
maintained to influence
learning object behavior
Learners can select topic
comparison, post-test factual
information
Material highly motivating
Multimedia include maps,
cluster, and statistical
graphics
Reference material/data for
learner and teachers
Pictures /color aesthetically
/visually acceptable to
enhance efficient learning
Maps / cluster graphics are
colorful and clearly labelled
White background and
multiple colors for fonts,
legible
Free of visual clutter
Clear / concise definitions
within material; free of error
Site and layout very
simple/easy to navigate
within learning object
between exercises and
comparison tools
User interface is consistent
/predictable
Intuitive instructions within

MULTIMEDIA AWARD

each workshop
Accessibility
4
No portable device access
W3C Accessible / HTML
and Text
Internet Access,
Internet/Web browser
/computer access Window
7/IE 9/Firefox 4
Headphones/Speakers needed
for audio
Graphics require color
perception to understand
Reusability
5
Resource with videos, maps
and definitions, stand-alone
resource
Effective for broad range of
learners by providing
adjunctive content for use in
other context
Standards Compliance
5
Copyrighted by Creative
Commons (Attribution/
Noncommercial/ No
Derivatives 3.0 Spain
W3C accessible / HTML and
Text
Note: Adapted from Learning Object Review Instrument (LORI) - User Manual, by J. Nesbit,
K. Belfer, and T Leacock, 2004.
Table 2
Team Dynamics and Building.
LORI Item
Content Quality

Learning Goal Alignment

Comments
Key points presented/
appropriate level of detail,
definitions & analysis
Unbiased and objective
information
Logical evidence/argument
support claims
Differences among
cultural/ethnic group
represented
Learning goals declared in
content
Targeted learner as high

Score
5

MULTIMEDIA AWARD

Feedback & Adaptation

Motivation

Presentation Design

Interaction Usability

school, undergraduate and

graduate/ leadership/ ethics/
marketing
Individual and/or group stepby-step homework or selfpaced assignment
Sufficient to enable learner to
achieve goal
Interactive exercise/quizzes
and feedback
Individual learner cannot
create tailored response
based upon differential input
Learner profile not
maintained to influence
learning object behavior
Learners can select
topic/module comparison,
post-test factual information
Material motivating/limited
animation of PPT or
interactive quizzes
PPT presentation with audio
for modules
PPT pictures /color
aesthetically /visually
acceptable to enhance
efficient learning
Meaningful headings signal
content of text passages
White background and
multiple colors for fonts,
legible
Free of visual clutter
Clear / concise definitions
within material; free of error
Audio narration can be
turned on/off
Site and layout very
simple/easy to navigate
within learning object
between modules
User interface is consistent
/predictable
Intuitive instructions within

MULTIMEDIA AWARD

each module
Accessibility
4
No portable device access
W3C Accessible / HTML
and Text
Internet Access, Internet/
web browser /computer
access Window 7/IE
9/Firefox 4 required
Flash player 6.0 required
Headphones/Speakers
required for audio
Graphics require learner
color perception to
understand
Reusability
4
PPT with audio narration as a
resource with concepts and
definitions, stand-alone
resource
Effective for broad range of
learners by providing
adjunctive content for use in
other context
Standards Compliance
5
Copyrighted by Creative
Commons (Attribution/
Noncommercial/ No
Derivatives 3.0 United States
W3C accessible / HTML and
Text
Note: Adapted from Learning Object Review Instrument (LORI) - User Manual, by J. Nesbit,
K. Belfer, and T Leacock, 2004.
Summary of Decision
Leadership Perception around the World wins the multimedia award. The two evaluands,
which were selected as finalists for the competition were Leadership Perception around the
World garnering 42 points, with Team Dynamics and Building receiving a close 40 points.
Evaluation of the finalists was conducted with LORI, with Leadership Perception around the
World receiving higher scores in motivation and reusability. Leadership Perception around the
World provides additional reference material/data and interactive clusters/statistical graphics. It
provides reference material/data for learner and teachers. The information is unbiased with

MULTIMEDIA AWARD

objective data with a comparison capability that can create a tailored response based upon
differential input. Modules provide interactive exercises/quizzes accompanied by feedback. It
includes colored pictures and maps with colored fonts and a white background. The website is
intuitive for learner navigation. The learning object is accessible through internet access and is
copyrighted by Creative Commons as attribution/noncommercial/no derivatives.
Conclusion
Leadership Perception around the World was evaluated using the LORI as a better
multimedia product. It provided a clear evaluation scale and with additional descriptive
comments for the nine criteria items in the LORI. While both learning objects would provide for
an efficient learning multimedia product, determining the strengths and weaknesses allows for
selection of the product that will contribute the greater merit, worth or value (Baumgartner &
Payr, 1997, p. 2) to the learning process. Leadership Perception around the World provides a
greater learning opportunity over the Team Dynamics and Building multimedia product.

MULTIMEDIA AWARD

10
References

Baumgartner, P., & Payr, S. (1997). Methods and practice of software evaluation: The case of
the European Academic Software Award (EASA). Paper presented at the ED- MEDIA
97, Charlottesville.
Dunham, R. (2011, June 14). Team dynamics and building [Web tutorial]. Retrieved from
http://www.merlot.org/merlot/viewMaterial.htm?id=531495
Krauss, F., & Ally, M. (2005). A study of the design and evaluation of a learning object and
implications for content development. Interdisciplinary Journal of Knowledge and
Learning Objects, 1(1), 1-22.
Leacock, T. L., & Nesbit, J. C. (2007). A framework for evaluating the quality of multimedia
learning resources. Educational Technology & Society, 10 (2). 44-59. Retrieved from
http://www.ifets.info/journals/10_2/5.pdf
Nesbit, J.C., Belfer, K., & Leacock, T. (2004). Learning object review instrument (LORI) user
manual. Retrieved from
http://www.transplantedgoose.net/gradstudies/educ892/LORI1.5.pdf
Reeves, T.C., & Hedberg, J.G. (2003). Effectiveness evaluation. In T.C. Reeves & J.G. Hedberg
(Eds.), Interactive learning systems evaluation (pp. 173-203). New Jersey: Englewood
Cliffs Educational Technology Publications.
Wemsing, T. (2011, May 5). Leadership perception around the world. [Web tutorial]. Retrieved
from http://www.merlot.org/merlot/viewCompositeReview.htm?id=523576