Professional Documents
Culture Documents
Vladimir Prikazsky
Arnold Bosman
p.kostkova@ucl.ac.uk
Vladimir.prikazsky@ecdc.europa.eu
Arnold.Bosman@ecdc.europa.eu
ABSTRACT
Highly specialized professional communities of practice (CoP)
inevitably need to operate across geographically dispersed area members frequently need to interact and share professional
content. Crowdsourcing using wiki platforms provides a novel
way for a professional community to share ideas and collaborate
on content creation, curation, maintenance and sharing. This is
the aim of the Field Epidemiological Manual wiki (FEMwiki)
project enabling online collaborative content sharing and
interaction for field epidemiologists around a growing training
wiki resource.
However, while user contributions are the driving force for
content creation, any medical information resource needs to keep
editorial control and quality assurance. This requirement is
typically in conflict with community-driven Web 2.0 content
creation. However, to maximize the opportunities for the
network of epidemiologists actively editing the wiki content
while keeping quality and editorial control, a novel structure was
developed to encourage crowdsourcing a support for dual
versioning for each wiki page enabling maintenance of expertreviewed pages in parallel with user-updated versions, and a
clear navigation between the related versions.
Secondly, the training wiki content needs to be organized in a
semantically-enhanced taxonomical navigation structure enabling
domain experts to find information on a growing site easily. This
also provides an ideal opportunity for crowdsourcing. We
developed a user-editable collaborative interface crowdsourcing
the taxonomy live maintenance to the community of field
epidemiologists by embedding the taxonomy in a training wiki
platform and generating the semantic navigation hierarchy on the
fly. Launched in 2010, FEMwiki is a real world service
supporting field epidemiologists in Europe and worldwide. The
crowdsourcing success was evaluated by assessing the number
and type of changes made by the professional network of
epidemiologists over several months and demonstrated that
crowdsourcing encourages user to edit existing and create new
content and also leads to expansion of the domain taxonomy.
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or
distributed for profit or commercial advantage, and that copies bear this notice and
the full citation on the first page. Copyrights for third-party components of this
work must be honored. For all other uses, contact the owner/author(s). Copyright
is held by the author/owner(s).
DH15, May 1820, 2015, Florence, Italy.
ACM 978-1-4503-3492-1/15/05.
http://dx.doi.org/10.1145/2750511.2750523
General Terms
crowdsourcing, semantic web, evaluation, field
epidemiology, taxonomy
Keywords
FEMwiki, social and Semantic Web, user engagement,
evaluation
INTRODUCTION
2
BACKGROUND AND RELATED
WORK
Crowdsourcing owns its growing popularity to the simple fact
that a large number of users can make a small effort on a shared
task enabling a large scale collaborative work performed easily
[1]. Crowdsourcing has many forms and could be implemented
over a number of platforms. Typically, collaborative Web 2.0
technologies enable users to create and modify content in a
shared repository instead of merely being passive consumers. In
addition to sharing the work, the risk of bottlenecks is reduced.
The most well known example of crowdsourcing must be
Wikipedia1 with over 4.7 million articles, being increased every
day with over 800 new articles as of March 2015. However, user
contributions remain sparse. Wikipedia has also been studied as a
cultural phenomena reaching trusted level of information through
crowdsourcing [2].
Large wikis such as Wikipedia can be difficult to navigate, as
they are large at repositories and there is no native support for
organising the content. However, for domain-specific wikis, this
problem could be overcome by organizing the pages according to
semantic taxonomy representing the domain entities (also called
nodes) forming the basis for content navigation using the parentchild relationship (entity becomes a wiki page). Semantic
ontologies and taxonomies are used in a wide variety of
disciplines. Perhaps their most notable successes are in the life
sciences and medicine (for example, the Open Biological and
Biomedical Ontologies (OBO) [3], MeSH2, ICD-113 and
SNOMED-CT4). In such domains, the ontologies are usually
highly formal but require a considerable amount of expertise,
time and effort to build. Stevens and Jupp [4-5] argue that many
other medical ontologies are rather taxonomies as they do not
follow first order logic relationships between entities but better
describe the complex medical domain.
(the ontology
http://wiki.dbpedia.org/Ontology)
6 http://semantic-mediawiki.org/
7 http://www.neli.org.uk
is
accessible
at
T
Therefore, desig
gning a collabo
orative Web 2.0 wiki utilizing
ccrowdsourcing while
w
keeping editorial contrrol over quality
y
remains an issue. Further, engag
ging users in sem
mantic navigation
taaxonomy mainteenance for domaain wikis remain
ns attractive but a
ssuitable user-friendly interface is essential for
f unsupervised
ccollaborative inp
put form domain experts.
Inn this paper wee propose a solu
ution to these tw
wo problems: th
he
F
FEMwiki framew
work and cond
duct the initial evaluation. Useer
eengagement hass been a grow
wing discipline with developed
m
models for asseessment [15] bu
ut methods forr encouragemen
nt
remain an open problem.
p
3
THE FEMWIKI
F
CROWDSO
OURCING
F
FRAMEWO
ORK
F
FEMwiki consissts of a wiki-baased repository, user forums fo
or
ddiscussion of wik
ki pages, and usser personal profiles where userrs
ccan give more information aboutt themselves.
T
The schematic organisation off the wiki part of FEMwiki is
i
sshown in Figuree 2. Wiki pages - representing a term from th
he
ddomain of field epidemiology - may contain tex
xts and graphicss,
aand are organised
d into a hierarch
hical structure.
3.1
FEMwikii Dual Versiioning Editoorial
Model
Figure 2. A sch
hematic diagram
m of the FEMw
wiki framework
content org
ganisation.
Inn addition to naavigating by folllowing links to other
o
wiki pagess,
thhe main organisational featuree is a navigatiion hierarchy of
o
ssemantically con
nnected parent-child wiki pages derived from
m
ssemantic taxonom
my of the epidem
miology domain
n. Although therre
iss no strict meeaning given to
o the parent-ch
hild relationship
p
bbetween wiki pages
p
in the actually
a
platforrm, semantically
y
cconnected pareent-child pagess create a naaturally formed
nnavigation taxon
nomy representin
ng the semanticcs of the domain
kknowledge. Each
h node in the tax
xonomy is a wikii page, which can
hhave text and graphics, as well
w
as child paages. Additionaal
ffeatures, such as tagging, may asssign more terms to a single wik
ki
ppage flexibly. Paages can also co
ontain cross refference (untyped
d)
3.2
FEMwikii Semantic N
Navigation
Model
One off the main challlenges of impleementation of seemantic
technollogies is the cosst of developmeent and maintennance of
domainn ontologies annd taxonomies. This is of paarticular
importaance in life crittical domains w
where the need tto keep
the ont
ntology up to daate is paramounnt. User-friendliiness of
ontologgy editors is another challennge. In the FE
EMwiki
framew
work, we utilizeed the wiki useer interface userrs have
been uusing for collaboorating editing of the medical content
for enttirely different ppurpose: the wikki page also servves as a
user-fri
riendly taxonom
my editor, thus, offering a seeamless
experieence to users.
Thereffore, in order to elicit more editts from users thee entire
field eppidemiology taxxonomy is displayed on the navvigation
page (rrather than just ppages with existting content). A colour
codingg is used to ddraw user attennding to emptyy pages
("stubss") and to distinnguish between various types ccontent,
see Figgure 6. The taxonnomy editor suppports colour-codding for
the duaal versioning of pages: (A) YEL
LLOW: link to thhe latest
versionn of the page (B
B) GREEN: link to the expert-reeviewed
(and appproved) page, cclicking on the ttext approved vversion"
will leaad to the review
wed version. Fuurther, (C) QUE
ESTION
MARK
K: pages that doo not have an exxpert-reviewed version
(indicaated by the quesstion mark iconn), the link will lead to
the lateest version, and finally, (D) GR
REEN ONLY: inndicates
pages where the latesst version is alsso the expert-reeviewed
versionn, the link leadss to this commonn version. Any edits to
the paage will cause a new latest vversion to be ccreated.
Finallyy, (E) RED illustrates (and visuaally draws attenntion to)
to pagges tagged as stubs" where content has noot been
developped yet.
F
Figure 5. Thee page historry at FEMwiiki framework
k,
illlustrating expeert-reviewed an
nd unapproved pages.
p
By sim
mply looking at the colour-codded taxonomy bbrowser,
user caan see which pagges have expert--reviewed versioons, and
can eitther choose to ssee that version,, or a later unappproved
version if one eexists (Figure 6)). The user can aalso see
stub" pages marked in reed - this featture is
specifically deesigned to highllight parts of thhe wiki
content that neeed to be filledd in, and to enccourage
users to start thhis process.
4
THE FEMWIKI
F
EVOLUTIO
E
ON AND
E
EVALUATIION RESUL
LTS
A
As outlined in th
he Introduction section, the Fieeld Epidemiology
y
M
Manual Wiki (FEMwiki) [16], funded
f
by the ECDC
E
(Europeaan
C
Centre for Disease Prevention and Control), is used by field
eepidemiologists to maintain a reepository of kno
owledge used fo
or
trraining purposes. It was develo
oped and hosted
d by City ehealth
R
Research Centree until January
y 2012, and was
w subsequently
y
m
migrated to ECDC. FEMwiki was developed
d using Telligen
nt
C
Community softtware8 (which provides
p
typicall wiki functionss,
ssuch as editing, and conflict reesolution). The FEMwiki servees
pprimarily field epidemiologists
e
o
in Europe - thiis community of
ppractice was inveestigated by Fow
wler et al [17].
44.1
The evolution
e
off the FEMwiki content
T
The basis of the FEMwiki was the a training manual
m
developed
bby a training prrogramme run by ECDC, EPIIET, which waas
oorganised into 17 chapters. Each
h chapter was originally
o
written
bby a trainer/lectu
urer in the EPIE
ET programme - the manual waas
inntended to be sttudied and also taught
t
like a tex
xtbook, from starrt
too finish. Durin
ng the process of converting the manual to
F
FEMwiki, an editorial
e
board was appointed to oversee th
he
pprocess of review
wing each chaptter and convertiing it into a wik
ki
ppage(s) [16]. Wh
here possible, th
hese were the original
o
lecturerss,
ootherwise, new senior
s
experts were
w appointed.
T
The first version
n of FEMwiki retained
r
the chap
pter sequence of
o
thhe EPIET manu
ual, with a hom
me page for eaach chapter. Th
he
ssemantic nature of the FEMwiiki platform (i.ee. the taxonomy
y
bbrowser) was no
ot utilised at thiis initial stage (see
(
Figure 7-1)).
U
Utilising the FEMwiki
F
frameework potentiaal for semantiic
taaxonomic repressentation and navigation provideed an opportunity
too develop a tax
xonomy of publiic health and fieeld epidemiology
y
nnot covered by existing
e
medical taxonomies. In order to organisse
thhe content, a taxonomy
t
was developed in consultation
c
witth
8
http://telligent.ccom
4.2
4.3
(3) FEMwiki
(May 2012)
283
278
90
75
n/a
12
Number of deleted
nodes
Number of renamed
nodes
Inheritance richness
n/a
17
n/a
3.77
3.65
4.4
Figure 8. FE
EMwiki monthly
y numbers of pa
age edits and
forum
m posts
DISCU
USSION AN
ND FUTUR
RE WORK
C
Crowdsourcing has
h numerous forms
fo
and modeels. In this study
y,
w
we demonstrated novel fraamework, FEM
Mwiki, enabling
ccollaborative deevelopment of a domain taxo
onomy and wik
ki
ccontent through a single user-friiendly interface and successfully
eevaluated the FEMwiki portal ev
volution with reaal-world users.
H
However, there are some intteresting lesson
ns learned. Th
he
trransition from th
he original EPIE
ET training manu
ual to the curren
nt
F
FEMwiki knowledge repository and the subseq
quent online web
22.0 evolution sho
ows a clear patteern of moving aw
way from a lineaar
ccourse structure towards a colllection of articlles based on th
he
sstructure of thee domain. Also
o the number of
o levels in th
he
hhierarchy was reduced, which seems to indicate a movemen
nt
toowards simplify
ying the organisation of the co
ontent (which is
i
pperhaps not so important forr students follo
owing a coursse
sstructure). It is important to realise
r
that the process is stilll
oongoing, as pagees may be addeed, stub pages may
m be modified
d,
aand possibly co
omplex rearrang
gements will taake place to th
he
taaxonomy.
F
Future work, jo
ointly with EC
CDC, includes investigation of
o
sspecific incentiv
ves increasing users interest in contribution
n,
eeither to the taxo
onomy or to the wiki itself and further
f
evaluation
oof content and taaxonomy growth
h over the years. This is currently
y
aan ongoing project aiming to identify
i
and und
derstand the key
y
ccharacteristics off online commu
unity developmen
nt and long-term
m
ssustainability.
CONC
CLUSIONS
C
Crowdsourcing using
u
wikis platfforms provides a unique way fo
or
a geographically
y dispersed profeessional communities of practicce
too collaborate on
o content creattion, curation, maintenance
m
an
nd
ssharing. User-friendly navigation
n structure allow
wing a quick and
eeasy access to growing
g
collecttion of resourcees is required to
ssupport the conteent growth.
L
Launched in 2010, FEMwiki iss a real world social
s
combinin
ng
ccollaborative wik
ki service supporting field epiidemiologists in
E
Europe and worrldwide. With over 100 000 page
p
views, oveer
11000 users signeed up, ongoing user-driven sem
mantic taxonomy
y
uupdates and maiintenance by cro
owdsourcing to the professionaal
ACKNOW
WLEDGMENTS
We accknowledge EC
CDC for fundinng this project,, Julius
Weinbeerg and Mike Catchpole forr their input innto the
FEMw
wiki taxonomy. M
Many thanks also go to formerr CeRC
researcchers Martin Szzomszor and Sim
mon Hammond for the
implem
mentation of thee current versionn of the FEMw
wiki and
David Fowler for hiss input into preevious versions of this
paper aand the research study.
REFEREN
NCES
[1] J H
Howe. The Rise of Crowdsourcinng. Wired, June 2006
[2] S. Lindgren. Crow
wdsourcing Know
wledge. Culture
Unnbound, Volumee 6, 2014, pp. 6009627, 2014
[3] B S
Smith, M Ashburrner, C Rosse, J Bard, W Bug, W
Ceuusters, LJ Goldbberg, K Eilbeck, A Ireland, CJ
Muungall, N Leontiis, P Rocca-Serraa, A Ruttenberg,, SAs sunta Sansone, R
RH Scheuermannn, N Shah, PL
Whhetzel, and S Lew
wis. The OBO F
Foundry: coordinnated
evoolution of ontoloogies to support biomedical dataa
inteegration. Naturee biotechnology, 25(11):1251, 5,,
Noovember 2007.
[4] S Juupp, R Stevens, S Bechhofer, P Kostkova, and Y
Yeesilada. Semanticc navigation: Onntology or Know
wledge
Orgganisation Systeem? In NETTAB
B2007 A Semanttic
Weeb for Bioinform
matics Goals Toools Systems
Appplications, 20077.
[5] S Juupp, R Stevens, S Bechhofer, Y Yesilada, and P
Koostkova. Knowleedge Representattion for Web
Naavigation. SWAT
T4LS, 2008.
[6] C B
Bizer, J Lehmannn, G Kobilarov, SSoren Auer, C
Be cker, R Cyganiaak, and S Hellmaann. DBpedia - A
cryystallization poinnt for the Web of Data. Web Sem
mantics
Sciience Services aand Agents on thhe World Wide W
Web,
7(33):154{165, 2009.
[7] N S
S. Friedland, PG
G. Allen, G Matthhews, M Witbrock, D
Bax
axter, J Curtis, B Shepard, P Miraglia, J Angele, S
Staaab, E Moench, H Oppermann, D Wenke, D Israael, V
Chhaudhri, B Porterr, K Barker, J Faan, SY Chaw, P Y
Yeh, D
Teccuci, and P Clarrk. Project halo: Towards a digitaal