Running heauei: 0SABILITY 0F AN 0NLINE TRAININu SITE 1

Iuea Papei
0sability of an Asynchionous, 0nline Tiaining Site foi Bistance Euucatois
















}essica E. Leauanae
0niveisity of Bawai'i at Nanoa
Sept 27, 2u1S


Running heauei: 0SABILITY 0F AN 0NLINE TRAININu SITE 2
0sabilitya tech buzzwoiuan umbiella teim that tiauitionally iefeis to uesign featuies that
enable something to be "easy-to-use" oi "usei-fiienuly" (Zachaiias P & Poylymenakou, 2uu9). Examining
the usability of an object, piouuct oi seivice is not something new to us...think coffee cups without hanules,
As Seen on Tv's Rock Pet, Smell-o-vision anu the poke featuie on Facebook. Even without having
conuucteu usability testing, I woulu aigue that none of these piouucts have valiu usable meiit. Eveiyuay,
we unconsciously anu consciously assess the puipose of a piouuct oi seivice anu whethei oi not it appeals
to oui human expeiience.
As stuuents, immeiseu in the use of technology to auvance euucation, we aie constantly evaluating
the "usability" of tech tools foi euucation. Peihaps the conveisation below illustiates this point:
!"#$ &'(")*+ ,- ).- /""0*-1 2" 3--2 2"450(2 "6 +" 7") ,842 2" ("9 "4 ':79-;<
'83$ &=-2>. ).- /""0*-1? ")6 +"@. 86- .(86-+ "4 2(- +65A-B<
!"#$ &C6- 7") ).540 '@6--4@8.2 2" (".2 7")6 2)2"658*. "6 D")E)#-;<
'83$ &'@6--4@8.2? D")E)#- 5. #*"@:-+ "4 ")6 .@(""* 4-2,"6:B
!"#$ &F84 7") 9".2 2(- /""0*- @*8.. .52- 8++6-.. 2" 2(- G"6)3? =8)*538 +"-.4>2 9*87 45@- ,52( 37
5H("4-B<
'83$ ')6-? 7") .(")*+ 6-8**7 0-2 8 E8#*-2B IA-672(540 ,"6:. "4 8 E8#*-2B

Common eveiyuay conveisations like the one above, uemonstiate one's ability to quickly assess, evaluate
anu ueciue on a tool of choice uepenuent on vaiiables such as ability to meet the uefineu puipose, ease of
use, functionality of featuies anu the usei's level of expeiience with the tool in question.
The puipose of my usability evaluation is to evaluate leainei expeiience foi Bistance Euucatois of
an online Bigh School who complete theii }ob 0iientation tiaining thiough an online tiaining site that is
asynchionous, self-paceu anu inteiactive. In this papei, the teim *-864-6 -J9-65-4@- is uefineu as metiics
that measuie leainei satisfaction anu functional usability.
The iising cost of ueliveiing tiaining to employees has uiiven many coipoiate manageis to migiate
tiauitional tiaining methous ovei to eLeaining. In auuition to cost-savings, eLeaining pioviues benefits
such as flexibility, convenience anu self-paceu instiuction (Stiothei, }uuith B., 2uu2). Bowevei, how uo
uecision makeis anu key stake holueis ueciue what system, methou, ueliveiy oi platfoim of eLeaining
piouuces effective iesults in teims of cost to implement anu impiovements in leaining.
In auuiessing auult leaineis, ieseaicheis Zachaiias anu Polylymenakou believe, asynchionous
eLeaining shoulu be uesigneu anu uevelopeu in ways that aie intiinsically motivating (Zachaiias P &
Poylymenakou, 2uu9). Zahaiias anu Poylymenakou fuithei exploie the use of usability testing to measuie
specific intiinsic motivatois in theii stuuy titleu, K-A-*"9540 8 L.8#5*527 IA8*)825"4 M-2("+ G"6 -N=-864540
C99*5@825"4.$ !-7"4+ O)4@25"48* L.8#5*527B Inteiactive content that met online peuagogy foi couise uesign,
content sequencing anu couise layout anu oiganizational weie most useful when using leainei satisfaction
Running heauei: 0SABILITY 0F AN 0NLINE TRAININu SITE S
metiics to conuuct usability testing (Sun Pei-Chen, Tsai Ray }., Fingei ulen, Chen Yueh-Yang, & Yeh
Bowming, 2uu8).
Buiing the summei of 2u1S, I was taskeu with ueveloping a tiaining solution that woulu allow
teacheis at Eugenuity
i
, the ability to complete job oiientation tiaining in a timefiame of thiee uays, piioi to
them ieceiving stuuents
ii
. At that time, oui cuiient tiaining solution incluueu a single synchionous tiaining
session, hosteu in Blackboaiu Collaboiate oi Auobe Connect. This session typically lasteu thiee-houis. Aftei
the session, new hiies weie sent a link to a libiaiy of piintable woiu uoc. iesouices that iangeu fiom
maiketing uocuments to piocess flow chaits to hunuieu page Banubooks of Instiuction. They weie then
assigneu stuuents without any fuithei tiaining oi mentoiing. Because of the complexity of theii job uuties
anu the low quality tiaining methou anu ueliveiy, teacheis often expiesseu feeling oveiwhelmeu anu
unueitiaineu befoie they hau even been assigneu stuuents.
uiven the tight ueauline I was given to uelivei a functioning piototype, I ueciueu to uevelop a
tiaining poital to host all of oui cuiient tiaining infoimation. This poital woulu be accessible 24 houis a
uay, 7 uays a week. The tiaining woulu be ueliveieu as self-paceu, asynchionous anu allow teacheis to
woik at theii own pace. They weie fiee to auvance in the couise once they felt they hau masteieu concepts.
They woulu also be able to ietuin to the tiaining mateiial at any time as a iesouice. Content woulu be
upuateu on a iegulai basis anu shoulu seive as the piimaiy tiaining iesouice foi teacheis iegaiuing job
iesponsibilities anu how to complete them.
The geneial appioach was to migiate all legacy papei tiaining mateiial into asynchionous, inteiactive
tiaining mouules, being caieful not to meiely ie-cieate uigital copies of what was alieauy pioviueu in the
piintable foimats. The eLeaining tiaining mateiial was cieateu using Auobe Captivate. The units incluue
mouules with embeuueu viueo tutoiials anu assessments, moueiateu uiscussion foiums with tiainei
feeuback, piintable ueliveiables foi uownloau anu evaluation suiveys anu checkpoints (Nollei, Leslie,
1998). I began by categoiizing infoimation following Bick anu Caiey's Besign Nouel as much as possible,
howevei because of time constiaints, the uesign evolveu into iapiu piototyping.
Ny mouel foi implementation was gieatly influenceu by the TCC confeience sessions I attenueu
Spiing of 2u1S. These sessions weie focuseu on ietaining quality teacheis anu Bistance Euucatois by
pioviuing flexible, asynchionous tiaining (uibbons, Beathei S., 2uu1). The mouel I auopteu foi Eugenuity
teacheis tiaining was uesigneu aftei Paik 0niveisity's mouel foi suppoiting theii team of 6uu+ Bistance
Euucatois
iii
. Actual site uevelopment began on }uly 1
st
anu the site launcheu piematuiely on August 8
th
,
2u1S, (business uecision) thiee weeks befoie the oiiginal ueauline. The neeus assessment anu uesign
uevelopment took place ovei a six-week peiiou with a team of one Instiuctional Besignei (myself) anu one
auuio technician who pioviueu the voice-oveis. Thus, the website is in extieme neeu of clean anu
Running heauei: 0SABILITY 0F AN 0NLINE TRAININu SITE 4
compiehensive usability testing in oiuei to measuie leainei expeiience to impiove oveiall functionality
anu usability.
The taiget population foi this stuuy will be cuiient Eugenuity Leau viitual Instiuctois. These incluue
teacheis fiom all content aieas. I will solicit paiticipation fiom 1S Eugenuity Leau viitual Instiuctois anu
aim at completing usability evaluations on at least 6 instiuctois total.
Paiticipants will complete the following scheuule of foui online evaluations anu one usability stuuy as they
complete the }ob 0iientation Tiaining
iv
:
!"#$%#&'() +,-. /.$'".0, 1.&2(3 4%0-(5. +'6')7
Pie-Tiaining
Evaluation
Embeuueu uoogle
Foim
Location: lanuing
page, site navigation
lockeu until
completion
To gathei uemogiaphic uata incluuing: age,
genuei, expeiience teaching online, tiauitional
teaching expeiience, subjectcontent aiea,
computei expeiience, attituuinal questions
towaius eLeaining
Befoie
beginning
asynchionous
tiaining
0nit 1: Evaluation Embeuueu uoogle
Foim, completion
iequiieu
0sing Kiikpatiick's Foui- Level Evaluation Nouel,
gathei uata that measuies leainei satisfaction
segmenteu out into leainei satisfaction metiics
anu functionality metiics.
Post 0nit 1
0nit 2: 0sability
Stuuy
Blackboaiu Live
Session, viueo feeu,
scieen shaiing,
iecoiuing anu guiueu
navigation anu toui
uathei live paiticipant feeuback anu leainei
expeiience uata as useis navigate the 0nit anu
complete the assignments. Builu tasks following
Kiug's 0sability Testing anu using Likeit scales
foi consistency.
Pie-0nit 2
Final tiaining
Evaluation
Embeuueu uoogle
Foim, completion
iequiieu
0sing Kiikpatiick's Foui- Level Evaluation Nouel,
gathei uata that measuies leainei satisfaction
segmenteu out into leainei satisfaction metiics
anu functionality metiics.
Post Final
0nit
To summaiize, the challenge of cieating a quality online, asynchionous, leaining enviionment can
be scaleu when uesigneis incoipoiate compiehensive usability testing that not evaluates the functional
elements of the uesign but also the cognitive elements that incoipoiate online peuagogy to cultivate
positive leaining expeiiences. When uesigneis can cieate leaining enviionments that meet instiuctional
goals, as measuieu by actual useisuecision makeis anu leaineis succeeu. The key is conuucting usability
testing that is uesigneu specifically to evaluate eLeaining anu not just tiauitional foimats. Bata gatheieu
fiom this type of thoiough, compiehensive usability testing pioviues coipoiate manageis uata to make
infoimeu uecisions that can fuel solutions to impiove an oiganization anu auvance leaining thiough quality
leainei expeiience.

i
Eugenuity is an online euucation anu blenueu leaining solution that pioviues piouucts anu seivices to ovei S,uuu school uistiicts
nationwiue. With a laige anu uiveise stuuent bouy in giaues 6-12, Eugenuity faces the challenge of pioviuing highly qualifieu
teacheis to eniolleu stuuents in any given content aiea, at any given time, in any given location of the 0niteu States baseu on client
neeus anu in compliance with State anu National iequiiements.
Running heauei: 0SABILITY 0F AN 0NLINE TRAININu SITE S

ii
Cuiiently, the Instiuctional Seivices team is maue up of 27u pait-time anu full-time teacheis who aie ceitifieu in thiee oi moie
States uepenuing on the geogiaphic locations of theii stuuent eniollments. Stuuent to teachei iatios can iange fiom 6u-SSu
stuuents pei teachei.
iii
Examining the Role of the Acauemic Biiectoi in Neuiating 0nline Instiuctoi Nissteps, Cathy Tayloi }B, Paik 0niveisity
2. Reciuiting, Tiaining, Nentoiing, 0bseiving, Evaluating anu Naintaining 0nline Aujunct Faculty, Beniy Roehiich, Ph.B., Paik
0niveisity
S. Pioviuing Piofessional Bevelopment 247: Restiuctuiing how we uelivei tiaining, Benise Swett, Foothill College

iii
0nit 2: Navigating oui Systems
Nouule 2: Exploiing the Stuuent Infoimation System
Nouule S: Exploiing oui Leaining Nanagement System
Nouule 4: 0sing Live Chat Suppoit in Live Peison
Nouule S: Bosting Youi Fiist Synchionous Blackboaiu Session

8.9.0.):.5;
Anaiaki, Fiiouz. (2uu4). Beveloping an Effective anu Efficient eLeaining Platfoim. P42-64825"48* Q")648* "G
2(- F"39)2-6? 2(- P42-64-2 84+ M8480-3-42, RS(2), S7-6S.
Cook, Baviu. (2uuS). The Reseaich We Still Aie Not Boing: An agenua foi the stuuy of Computei-Baseu
Leaining. C@8+-35@ M-+5@54-, TU(6), S41-S48.
uibbons, Beathei S. (2uu1). Anuiological anu Peuagogical Tiaining Biffeiences foi 0nline Instiuctois.
V4*54- Q")648* "G K5.284@- =-864540 C+3545.26825"4, W(S). Retiieveu fiom
http:www.westga.euu~uistanceojulafall4Sgibbons_wentwoith4S.html
uolu, Sanfoiu Bi. (2uu1). A Constiuctivist Appioach to 0nline Tiaining foi 0nline Teacheis. Q")648* "G
C.74@(6"4"). =-864540 X-2,"6:., Y(1), 2S.
uunasekaian, A., NcNeil, Ronalu, & Shaul, Bennis. (2uu2). E-leaining: ieseaich anu applications. P4+).2658*
84+ F"33-6@58* E6854540, ZW(2), 44-SS. uoi:1u.11u8uu1978Su21u417S28
}ung, Insung. (2uu1). Issues anu Challenges of Pioviuing 0nline In-seivice Teachei Tiaining: Koiea's
Expeiience. P42-64825"48* [-A5-, "G [-.-86@( 54 V9-4 84+ K5.284@- =-864540, S(1), 1-18.
Nollei, Leslie. (1998). Besigning communities of leaineis foi asynchionous uistance euucation. I+)@825"48*
E-@(4"*"07 [-.-86@( \ K-A-*"93-42, W](4), 11S-122.
Stiothei, }uuith B. (2uu2). An Assessment of the Effectiveness of e-leaining in Coipoiate Tiaining
Piogiams. E(- P42-64825"48* [-A5-, "G [-.-86@( 54 V9-4 84+ K5.284@- =-864540, Z(1). Retiieveu fiom
http:www.iiioul.oiginuex.phpiiioulaiticleview8S16u
Sun Pei-Chen, Tsai Ray }., Fingei ulen, Chen Yueh-Yang, & Yeh Bowming. (2uu8). What uiives a successful e-
Leaining. An empiiical investigation of the ciitical factois influencing leainei satisfaction.
F"39)2-6. \ I+)@825"4, (Su), 118S-12u2.
Zachaiias P, & Poylymenakou. (2uu9). Beveloping a 0sability Evaluation Nethous foi e-Leaining
Applications: Beyonu Functional 0sability. SY(1), 7S-98.

Running heauei: 0SABILITY 0F AN 0NLINE TRAININu SITE 6