10/08/2015

CommunicatetoVaccinate:theCOMMVAC
systematicreview
JessicaKaufman
ResearchOfficerandPhDCandidate
GuestLecture AppliedHealthResearch
10August2015

latrobe.edu.au/chcp

CRICOSProvider00115M

Outline
Communicationforchildhoodvaccination
Identifyingatopicforasystematicreview
Definingthereviewsquestion(PICO)
Searchingandscreeningstudies
Assessingstudyquality
Evidencesynthesis
Findingsandimplications

10/08/2015

Asnapshotofchildhoodvaccination
GLOBAL: In2013,21.8millioninfantsworldwidedidnotreceivethefull
seriesofbasicimmunisations(WHO,UNICEF2014)
LOCAL: In2014,Victoriahad7outbreaksofmeasleswith75notifiedcases
(RUSSO2015)

Strategiestoincreaseuptake:

Supplysideinterventions:vaccineaccessand
availability,newtechnologies,healthpersonnel
Demandside/consumerlevelinterventions:
communication(e.g.remindersto
parents,education,decisionaids)

Examplesofcommunicationstrategies(interventions)for
childhoodvaccination
Informingoreducatingpeopleaboutrisksandbenefitsof
vaccinesandwhereandwhentogetthem
Remindingorrecallingtoappointments
Teaching peoplehowtocommunicatemoreeffectively
Supporting parentsandhelpingfacilitatedecisionmakingabout
vaccination
Involvingcommunitymembersinplanningandevaluating
vaccinationprograms

10/08/2015

Whydoweneedevidencebased vaccinationcommunication?
NEEDTOKNOWWHATWORKS:
manytrialsbutfeweffortstosynthesise availableevidenceglobally
Effective communicationenables:

Rapidrebuttalofmisinformation
Communitydialoguetoachievewidespreadsupport
Educationofparents,communitiesandvaccinators
Consistentandeffectiveadvocacyofvaccinebenefits

Ineffective communicationmay:

Consumetimeandresources
Underminetrustinhealthsystem
Havenooradverseeffectson
healthoutcomes

Vaccinepreventablediseaseoutbreaks
5

CommunicatetoVaccinate(COMMVAC)projects
COMMVAC1(2010 2012)andCOMMVAC2(20132017)
Aim:Tobuildtheevidencebaseandimprovecommunicationabout
childhoodvaccinationswithparentsandcommunitiesinLowandMiddle
IncomeCountries(LMICs)
InternationalresearchcollaborationwithpartnersfromAustralia,Norway,
France,Switzerland,SouthAfrica,Nigeria,Mozambique,andChile

10/08/2015

Evidencemapping
Runtestsearchfortrials
(CENTRALdatabase)and
otherliterature(MEDLINE)
Countarticlesevaluating
eachintervention
Maparticlestothe
communicationthey
evaluated

Orange=Trials

Blue=Nottrials

Systematicreviewan
interventionwherethereare
trials

Identifyingatopicforasystematicreview
Deliberativeforumswithkeystakeholderstoprioritise review
topics

Representativesofgovernmentalandnongovernmentalagencies,
consumergroupsandexperts
Facetoface(NewYork,Ottawa)andonline
Identifiedhighprioritycommunication

10/08/2015

COMMVACsystematicreviews
Goldstandardforevidenceof
effectiveness

Facetofaceinterventionsfor
informingoreducatingparents
aboutearlychildhoodvaccination
Communitydirectedinterventions
forinformingand/oreducating
aboutearlychildhoodvaccination
Kaufman, J., Synnot, A., Hill, S., Willis, N., Horey, D., Lin, V., et al. (2013).
Face to face interventions for informing or educating parents about early
childhood vaccination. Cochrane Database Syst Rev, 5, CD010038.
Saeterdal, I., Glenton, C., Austvoll-Dahlgren, A., Munabi-Babigumira, S., &
Lewin, S. (2014). Community-directed interventions for informing and/or
educating about early childhood vaccination. Cochrane Database Syst
Rev, 11, CD010232.

Definingthereviewsquestion

Population

Intervention

Comparison

Outcomes

10

10/08/2015

Population
Included

NotIncluded

Healthworker
motivation
program

(involvecommunicationandconsumersorcommunities)

Reminders
forparents

Vaccine
programme
funding
Trainingfor
health
professionalto
improvetheir
communication
skills

Trainingforhealth
professionalsin
vaccinedelivery

Chartprompt
remindersfor
health
professionalsthat
appearduring
consultations

Incentivesfor
consumersorhealth
professionals

More
convenient
organisation
ofcare

Vaccination
support
hotline

Parentheld
symptom
diarytotrack
adverse
events
Community
vaccination
events

Parentingskills
trainingfor
newparents

Community
coalitionstoplan
andimplement
vaccineprogram

Interpreters

Educational
materials
forparents

Decisionaids

11

Definingthereviewsquestion

Population

Intervention

Comparison

Outcomes

Parents/
caregivers

12

10/08/2015

Intervention
Complexnonclinicalintervention
Cleardefinitionattheprotocolstage

Informing v.educating:literaturereviewindicatesthereisno
agreementaboutthedifferencebetweeninformingandeducating

Relevantinterventionsarethosewhichmakeconsumersawareofthe
practicalorlogisticalfactorsassociatedwithvaccinationorseektoenable
themtounderstandthemeaningandrelevanceofvaccinationfor
themselves,theirfamilyorcommunity.(KAUFMAN2013)

13

Definingthereviewsquestion

Population

Intervention

Comparison

Outcomes

Parents/
caregivers

Anyfacetoface
communicationto
informoreducate

14

10/08/2015

Comparisons
Whatistheinterventioncomparedtowhenitisevaluated?

Facetofacecommunicationdirectedtoindividualparentsv.usualcare

Facetofacecommunicationdirectedtogroupsofparentsv.usualcare

Facetofacecommunicationdirectedtoindividualparentsv.facetoface
communicationdirectedtogroupsofparents
FacetofacecommunicationtypeAv.facetofacecommunicationtypeB

15

Definingthereviewsquestion

Population

Intervention

Comparison

Outcomes

F2Fdirectedto
individualparentsv.
usualcare
F2Fdirectedto

Parents/
caregivers

Anyfacetoface groupsofparentsv.
communicationto usualcare
informoreducate F2Fdirectedto
individualparentsv.
groupsofparents
F2F typeAv.F2F
typeB

16

10/08/2015

Outcomes
Whatmightbeaffectedbythisintervention?(e.g.behaviours,healthstatus,
thoughtsorbeliefs)
Whichoutcomesmightbeimportanttodifferentpeople(e.g.parentsv.
doctors?)

Systematicreviewshavetorespondtotrialsbutcanalsoleadfutureresearch
withcarefullyselectedoutcomes

Reviewliteraturefirsttounderstandtheoutcomesdifferenttrialsareusing

Processoutcomes(e.g.knowledgeandintentiontovaccinate)areasimportant
asendpointoutcomes(e.g.vaccinationstatus)

Needtoconsidertimepoints too whenwastheoutcomemeasured?

17

Definingthereviewsquestion
P

Population

Intervention

Comparison

Outcomes
*Vaccinationstatusofchild

F2Fdirectedtoindividual *Parentsknowledgeor
understandingofvaccination
parentsv.usualcare

Parents/
caregivers

Anyfaceto
face
communication
toinformor
educate

F2Fdirectedtogroupsof
Parentsintentiontovaccinate
parentsv.usualcare
child
F2Fdirectedtoindividual Parentsexperienceofthe
parentsv.groupsof
intervention
parents
Costofimplementingthe
F2F typeAv.F2F typeB
intervention
Adverseeffectsofintervention

Facetofaceinterventionsforinformingoreducatingparentsaboutearlychildhoodvaccination

18

10/08/2015

Searchingandscreeningstudies
Professionallydevelopedsearchstrategyacrossmany
databases
Twoauthorsscreenresultsseparatelyandcompare
Inclusioncriteria

Studydesign:RCTsorclusterRCTs

Childrenandvaccines:Childrenunder5or6(schoolage),any
WHOrecommendedroutineearlychildhoodvaccine(HPV
excluded)

Interventions:notcombinedwithanyother
intervention(e.g.reminders,improvedaccess)

19

Includedstudies
Screened9119abstractsandtitles
Included7studies

6RCTsand1clusterRCT

Settings

HIC:Australia(2studies),Canada,UnitedStates
LMIC:Pakistan(2studies),Nepal
Mixofhomeandclinic

Participants

2978participantsacross7studies
Groupsorindividualparents,expectantparentsorotherguardians
3studiestargetedhighriskmothers(drugusers,adolescents,lowsocio
economicstatus)

Interventions

Mixofsingleandmultisessioninterventions

20

10

10/08/2015

Outcomesmeasured
Primaryoutcomes

Vaccinationstatusofchild(measuredbymost studies)
Parentsknowledgeorunderstandingofvaccination(measuredbysome
studies)

Secondaryoutcomes

Parentsintentiontovaccinatechild(measuredbyno studies)
Parentsexperienceoftheintervention(measuredbyno studies)
Costofimplementingtheintervention(measuredbyone study)
Adverseeffectsofintervention(measuredbyno studies)

21

Assessingthestudyquality
Quality:themethodologicalstrengthofarelevantstudy,orhowableitis,throughitsdesign
anditsconduct,topreventsystematicerrors,orbias (RYAN2013)
Biasedstudiesaremorelikelytoproducemisleadingresultsthanthosethatarerigorously
designedandconducted (RYAN2013)
RISKOFBIAS:
Type ofbias

Relevantdomainsinthe
CollaborationsRiskofbias
tool

Description

Selectionbias

Systematic differencesbetween
baselinecharacteristicsofthe
groupsthatarecompared

Performance
bias

Systematicdifferencesbetween
groupsinthecarethatisprovided,
orinexposuretofactorsotherthan
theinterventionsofinterest.

Detectionbias

Systematicdifferences between
groupsinhowoutcomesare
determined

Attritionbias

Systematic differencesbetween
groupsinwithdrawalsfromastudy

Incompleteoutcomedata

Reportingbias

Systematic differencesbetween
reportedandunreportedfindings

Selectiveoutcomereporting

Sequencegeneration
Allocationconcealment

Blindingofparticipantsand
personnel
Otherpotentialthreats to
validity

Blindingofoutcome
assessment
Otherpotential threatsto
validity

(HIGGINS2011)
22

11

10/08/2015

Synthesising
theevidence

23

Synthesising theevidence
Metaanalysisiscombiningandanalysingdatafrommorethanonestudyat
once(BASTIAN2014)
Heterogeneity isvariationordiversity

Intheparticipants,interventionsandoutcomes(clinicalheterogeneity)

Instudydesignandriskofbias(methodologicalheterogeneity)
Intheinterventioneffectsbeingevaluated(i.e.theobservedeffectsaremore
differentfromeachotherthanonewouldexpectduetochancealone)
(statisticalheterogeneity)(HIGGINS2011)

Ifcalculationofstatisticalheterogeneityishigh,yourassumptionsareunsafe
andyoushouldnotpoolyourresults.

24

12

10/08/2015

Narrativesynthesis
Narrativesynthesis:describingresultsofincludedstudiesinawrittensummary

Groupstudiesbyintervention,comparison,orothermeaningfulcategory
Providethesameelementsofinformationabouteachstudyinthesameorderto
avoidbeingselectiveoremphasisingcertainfindings

Example:
Comparison1:Facetofaceinterventionsdirectedtoindividualparentsversus
control

Primaryoutcome:Immunisationstatus
o

Immunisationstatusmeasured3monthsafterasinglesessionintervention
Populationwasdiverse
Overalleffectwasuncertain
Evidenceoflowquality
Descriptionofeachstudyseffect

(KAUFMAN2013)

25

Forestplots
Forestplot:avisualrepresentationoftheresultsofeachstudy
Horizontallinesareconfidenceintervals,ormarginoferrorforeachresult
BelowisaNARRATIVESYNTHESISforestplot.Ametaanalysiswouldinclude
adiamondatthebottomshowingtheoverallsummary

26

13

10/08/2015

Forestplots
Forestplot:avisualrepresentationoftheresultsofeachstudy
Horizontallinesareconfidenceintervals,ormarginoferrorforeachresult
BelowisaNARRATIVESYNTHESISforestplot.Ametaanalysiswouldinclude
adiamondatthebottomshowingtheoverallsummary

27

Summaryof
findingstable
Organisedbythe
outcomes
Includesassessment
ofquality,
descriptionofeffect
andhowcertainwe
areofthateffect
Includedatthestart
ofallCochrane
reviews

28

14

10/08/2015

Summaryoffindings narrative
Facetofacestrategiesdonotconsistentlyimproveeitherimmunisation
ratesorparentknowledgeandunderstandingofvaccination,butthe
evidencewaslowtoverylowqualityfortheseoutcomes
Onlyonestudymeasuredcostandnostudiesmeasuredotheroutcomes
Theresultsofthisreviewarelimitedbythesmallnumberofincluded
studies,smallnumberofoutcomesmeasuredandproblemswithstudy
methods

29

Implicationsforresearchandpractice

Effectsizemaydependonbaselinevaccinationrates
Impactmaydifferforpopulationswithlowliteracyorlimitedaccessto
printedoronlinematerials
Futurestudiesshouldmeasureprocessoutcomesincludingknowledgeor
intentiontovaccinate

30

15

10/08/2015

Nextsteps
COMMVAC2project createevidencebased
globalguidanceonimplementingvaccination
communicationinterventionsforprogram
managersandpolicymakersinLICs
PhD Developacoreoutcomesetso
futureresearchcanhelpinformbetter
communicationdesignanddelivery,
andfuturetrialscanbemetaanalysed

31

References
Bastian,H.(2014)5keythingstoknowaboutmetaanalysis.ScientificAmericanBlogNetwork.
http://blogs.scientificamerican.com/absolutelymaybe/5keythingstoknowaboutmeta
analysis/
HigginsJPT,GreenS(editors).CochraneHandbookforSystematicReviewsofInterventions
Version5.1.0[updatedMarch2011].TheCochraneCollaboration,2011.Availablefrom
www.cochranehandbook.org
KaufmanJ,SynnotA,RyanR,HillS,HoreyD,WillisN,LinV,RobinsonP.Facetoface
interventionsforinformingoreducatingparentsaboutearlychildhoodvaccination.Cochrane
DatabaseofSystematicReviews2013,Issue5.Art.No.:CD010038.
Russo,K.(2015)ImmunisationUpdateApril2015.NetworkingHealthVictoria.
www.lmmml.org.au/LiteratureRetrieve.aspx?ID=191955 (accessed6/8/15).
RyanR,HillS,PrictorM,McKenzieJ;CochraneConsumersandCommunicationReviewGroup.
StudyQualityGuide.May2013http://cccrg.cochrane.org/authorresources (accessed30/7/15).
Saeterdal I,LewinS,AustvollDahlgrenA,GlentonC,MunabiBabigumiraS.Interventionsaimed
atcommunitiestoinformand/oreducateaboutearlychildhoodvaccination.Cochrane
DatabaseofSystematicReviews2014,Issue11.Art.No.:CD010232.
WHO,UNICEF.(2014)Progresstowardsglobalimmunizationgoals 2013.
http://www.who.int/immunization/monitoring_surveillance/slidesglobalimmunization.pdf
32

16

10/08/2015

Thankyou

www.commvac.com
j.kaufman@latrobe.edu.au

latrobe.edu.au/chcp

CRICOSProvider00115M

17