9/23/2015

Howtoreadandunderstandascientificpaper:aguidefornonscientistsViolentmetaphors

Violentmetaphors
Thoughtsfromtheintersectionofscience,
pseudoscience,andconflict.

Howtoreadandunderstandascientificpaper:aguide
fornonscientists
August25,2013January31,2015
JenniferRaff
Update(8/30/14):Ivewrittenashorterversionofthisguideforteacherstohandouttotheirclasses.If
youdlikeaPDF,shootmeanemail:jenniferraff(at)utexas(dot)edu.
Lastweekspost(Thetruthaboutvaccinations:YourphysicianknowsmorethantheUniversityof
Google(http://violentmetaphors.com/2013/08/14/thetruthaboutvaccinationsyourphysician
knowsmorethantheuniversityofgoogle/))sparkedaverylivelydiscussion,withcomments
fromseveralpeopletryingtopersuademe(andtheotherreaders)thattheirpaperdisproved
everythingthatIdbeensaying.WhileIencourageyoutogoreadthecommentsandcontribute
yourown,hereIwanttofocusonthemuchlargerissuethatthisdebateraised:whatconstitutes
scientificauthority?
Itsnotjustafunacademicproblem.Gettingthesciencewronghasveryrealconsequences.For
example,whenacommunitydoesntvaccinatechildrenbecausetheyreafraidoftoxinsand
thinkthatprayer(ordiet,exercise,andcleanliving)isenoughtopreventinfection,outbreaks
happen
(http://blogs.dallasobserver.com/unfairpark/2013/08/theres_a_measles_outbreak_at_v.php).
Beskeptical.Butwhenyougetproof,acceptproof.MichaelSpecter
Whatconstitutesenoughproof?Obviouslyeveryonehasadifferentanswertothatquestion.But
toformatrulyeducatedopiniononascientificsubject,youneedtobecomefamiliarwithcurrent
researchinthatfield.Andtodothat,youhavetoreadtheprimaryresearchliterature(often
justcalledtheliterature).Youmighthavetriedtoreadscientificpapersbeforeandbeen
frustratedbythedense,stiltedwritingandtheunfamiliarjargon.Irememberfeelingthisway!

Readingandunderstandingresearchpapersisaskillwhicheverysingledoctorandscientisthas 1/8

http://violentmetaphors.com/2013/08/25/howtoreadandunderstandascientificpaper2/

9/23/2015

Howtoreadandunderstandascientificpaper:aguidefornonscientistsViolentmetaphors

Readingandunderstandingresearchpapersisaskillwhicheverysingledoctorandscientisthas
hadtolearnduringgraduateschool.Youcanlearnittoo,butlikeanyskillittakespatienceand
practice.
Iwanttohelppeoplebecomemorescientificallyliterate,soIwrotethisguideforhowalayperson
canapproachreadingandunderstandingascientificresearchpaper.Itsappropriateforsomeone
whohasnobackgroundwhatsoeverinscienceormedicine,andbasedontheassumptionthathe
orsheisdoingthisforthepurposeofgettingabasicunderstandingofapaperanddeciding
whetherornotitsareputablestudy.
ThetypeofscientificpaperImdiscussinghereisreferredtoasaprimaryresearcharticle.Itsa
peerreviewedreportofnewresearchonaspecificquestion(orquestions).Anotherusefultypeof
publicationisareviewarticle.Reviewarticlesarealsopeerreviewed,anddontpresentnew
information,butsummarizemultipleprimaryresearcharticles,togiveasenseoftheconsensus,
debates,andunansweredquestionswithinafield.(Imnotgoingtosaymuchmoreaboutthem
here,butbecautiousaboutwhichreviewarticlesyouread.Rememberthattheyareonlya
snapshotoftheresearchatthetimetheyarepublished.Areviewarticleon,say,genomewide
associationstudiesfrom2001isnotgoingtobeveryinformativein2013.Somuchresearchhas
beendoneintheinterveningyearsthatthefieldhaschangedconsiderably).
Beforeyoubegin:somegeneraladvice
Readingascientificpaperisacompletelydifferentprocessthanreadinganarticleaboutsciencein
ablogornewspaper.Notonlydoyoureadthesectionsinadifferentorderthantheyre
presented,butyoualsohavetotakenotes,readitmultipletimes,andprobablygolookupother
papersforsomeofthedetails.Readingasinglepapermaytakeyouaverylongtimeatfirst.Be
patientwithyourself.Theprocesswillgomuchfasterasyougainexperience.
Mostprimaryresearchpaperswillbedividedintothefollowingsections:Abstract,Introduction,
Methods,Results,andConclusions/Interpretations/Discussion.Theorderwilldependonwhich
journalitspublishedin.Somejournalshaveadditionalfiles(calledSupplementaryOnline
Information)whichcontainimportantdetailsoftheresearch,butarepublishedonlineinsteadof
inthearticleitself(makesureyoudontskipthesefiles).
Beforeyoubeginreading,takenoteoftheauthorsandtheirinstitutionalaffiliations.Some
institutions(e.g.UniversityofTexas)arewellrespected;others(e.g.theDiscoveryInstitute
(http://www.discovery.org/))mayappeartobelegitimateresearchinstitutionsbutareactually
agendadriven.Tip:googleDiscoveryInstitutetoseewhyyoudontwanttouseitasascientific
authorityonevolutionarytheory.
Alsotakenoteofthejournalinwhichitspublished.Reputable(biomedical)journalswillbe
indexedbyPubmed(http://www.ncbi.nlm.nih.gov/pubmed).[EDIT:Severalpeoplehave
remindedmethatnonbiomedicaljournalswontbeonPubmed,andtheyreabsolutely
correct!(thanksforcatchingthat,Iapologizeforbeingsloppyhere).CheckoutWebofScience
(http://thomsonreuters.com/webofscience/)foramorecompleteindexofsciencejournals.
Andpleasefeelfreetoshareotherresourcesinthecomments!]Bewareofquestionablejournals
(http://scholarlyoa.com/individualjournals/).
Asyouread,writedowneverysinglewordthatyoudontunderstand.Youregoingtohaveto

http://violentmetaphors.com/2013/08/25/howtoreadandunderstandascientificpaper2/

2/8

9/23/2015

Howtoreadandunderstandascientificpaper:aguidefornonscientistsViolentmetaphors

Asyouread,writedowneverysinglewordthatyoudontunderstand.Youregoingtohaveto
lookthemallup(yes,everyone.Iknowitsatotalpain.Butyouwontunderstandthepaperif
youdontunderstandthevocabulary.Scientificwordshaveextremelyprecisemeanings).
Stepbystepinstructionsforreadingaprimaryresearcharticle
1.Beginbyreadingtheintroduction,nottheabstract.
Theabstractisthatdensefirstparagraphattheverybeginningofapaper.Infact,thatsoftentheonlypart
ofapaperthatmanynonscientistsreadwhentheyretryingtobuildascientificargument.(Thisisa
terriblepracticedontdoit.).WhenImchoosingpaperstoread,Idecidewhatsrelevanttomyinterests
basedonacombinationofthetitleandabstract.ButwhenIvegotacollectionofpapersassembledfordeep
reading,Ialwaysreadtheabstractlast.Idothisbecauseabstractscontainasuccinctsummaryoftheentire
paper,andImconcernedaboutinadvertentlybecomingbiasedbytheauthorsinterpretationoftheresults.
2.IdentifytheBIGQUESTION.
NotWhatisthispaperabout,butWhatproblemisthisentirefieldtryingtosolve?
Thishelpsyoufocusonwhythisresearchisbeingdone.Lookcloselyforevidenceofagendamotivated
research.
3.Summarizethebackgroundinfivesentencesorless.
Herearesomequestionstoguideyou:
WhatworkhasbeendonebeforeinthisfieldtoanswertheBIGQUESTION?Whatarethe
limitationsofthatwork?What,accordingtotheauthors,needstobedonenext?
Thefivesentencespartisalittlearbitrary,butitforcesyoutobeconciseandreallythinkaboutthecontext
ofthisresearch.Youneedtobeabletoexplainwhythisresearchhasbeendoneinordertounderstandit.
4.IdentifytheSPECIFICQUESTION(S)
Whatexactlyaretheauthorstryingtoanswerwiththeirresearch?Theremaybemultiple
questions,orjustone.Writethemdown.Ifitsthekindofresearchthattestsoneormorenull
hypotheses,identifyit/them.
Notsurewhatanullhypothesisis?Goreadthis
(http://psychology.ucdavis.edu/faculty_sites/sommerb/sommerdemo/stat_inf/null.htm),thengobacktomy
lastpostandreadoneofthepapersthatIlinkedto(likethisone
(http://pediatrics.aappublications.org/content/125/6/1134.full.pdf+html))andtrytoidentifythenull
hypothesesinit.Keepinmindthatnoteverypaperwilltestanullhypothesis.
5.Identifytheapproach
WhataretheauthorsgoingtodotoanswertheSPECIFICQUESTION(S)?
6.Nowreadthemethodssection.Drawadiagramforeachexperiment,showingexactlywhat 3/8

http://violentmetaphors.com/2013/08/25/howtoreadandunderstandascientificpaper2/

9/23/2015

Howtoreadandunderstandascientificpaper:aguidefornonscientistsViolentmetaphors

6.Nowreadthemethodssection.Drawadiagramforeachexperiment,showingexactlywhat
theauthorsdid.
Imeanliterallydrawit.Includeasmuchdetailasyouneedtofullyunderstandthework.Asan
example,hereiswhatIdrewtosortoutthemethodsforapaperIreadtoday(Battagliaetal.2013:
ThefirstpeoplingofSouthAmerica:NewevidencefromYchromosomehaplogroupQ
(http://www.plosone.org/article/info:doi/10.1371/journal.pone.0071390)).Thisismuchlessdetail
thanyoudprobablyneed,becauseitsapaperinmyspecialtyandIusethesemethodsallthe
time.Butifyouwerereadingthis,anddidnthappentoknowwhatprocessdatawithreduced
medianmethodusingNetworkmeans,youdneedtolookthatup.

http://violentmetaphors.com/2013/08/25/howtoreadandunderstandascientificpaper2/

4/8

9/23/2015

Howtoreadandunderstandascientificpaper:aguidefornonscientistsViolentmetaphors

Youdontneedtounderstandthemethodsinenoughdetailtoreplicatetheexperimentthatssomething
reviewershavetodobutyourenotreadytomoveontotheresultsuntilyoucanexplainthebasicsofthe
methodstosomeoneelse.
7.Readtheresultssection.Writeoneormoreparagraphstosummarizetheresultsforeach
experiment,eachfigure,andeachtable.Dontyettrytodecidewhattheresultsmean,justwrite
downwhattheyare.
Youllfindthat,particularlyingoodpapers,themajorityoftheresultsaresummarizedinthe
figuresandtables.Paycarefulattentiontothem!YoumayalsoneedtogototheSupplementary
OnlineInformationfiletofindsomeoftheresults.
Itisatthispointwheredifficultiescanariseifstatisticaltestsareemployedinthepaperandyoudonthave
enoughofabackgroundtounderstandthem.Icantteachyoustatsinthispost,buthere
(http://explorable.com/statisticstutorial),here(http://owl.english.purdue.edu/owl/resource/672/05/),and
here(http://bobhall.tamu.edu/FiniteMath/Module8/Introduction.html)aresomebasicresourcestohelpyou.
ISTRONGLYadviseyoutobecomefamiliarwiththem.
THINGSTOPAYATTENTIONTOINTHERESULTSSECTION:
Anytimethewordssignificantornonsignificantareused.Thesehaveprecisestatistical
meanings.Readmoreaboutthishere(http://www.dcscience.net/?p=6518).
Iftherearegraphs,dotheyhaveerrorbars(http://en.wikipedia.org/wiki/Error_bar)onthem?For
certaintypesofstudies,alackofconfidenceintervalsisamajorredflag.
Thesamplesize.Hasthestudybeenconductedon10,or10,000people?(Forsomeresearch
purposes,asamplesizeof10issufficient,butformoststudieslargerisbetter).
8.DotheresultsanswertheSPECIFICQUESTION(S)?Whatdoyouthinktheymean?
Dontmoveonuntilyouhavethoughtaboutthis.Itsokaytochangeyourmindinlightoftheauthors
interpretationinfactyouprobablywillifyourestillabeginneratthiskindofanalysisbutitsareally
goodhabittostartformingyourowninterpretationsbeforeyoureadthoseofothers.
9.Readtheconclusion/discussion/Interpretationsection.
Whatdotheauthorsthinktheresultsmean?Doyouagreewiththem?Canyoucomeupwithany
alternativewayofinterpretingthem?Dotheauthorsidentifyanyweaknessesintheirownstudy?
Doyouseeanythattheauthorsmissed?(Dontassumetheyreinfallible!)Whatdotheypropose
todoasanextstep?Doyouagreewiththat?
10.Now,gobacktothebeginningandreadtheabstract.
Doesitmatchwhattheauthorssaidinthepaper?Doesitfitwithyourinterpretationofthe
paper?
11.FINALSTEP:(Dontneglectdoingthis)Whatdootherresearcherssayaboutthispaper?
Whoarethe(acknowledgedorselfproclaimed)expertsinthisparticularfield?Dotheyhave

http://violentmetaphors.com/2013/08/25/howtoreadandunderstandascientificpaper2/

6/8

9/23/2015

Howtoreadandunderstandascientificpaper:aguidefornonscientistsViolentmetaphors

Whoarethe(acknowledgedorselfproclaimed)expertsinthisparticularfield?Dotheyhave
criticismsofthestudythatyouhaventthoughtof,ordotheygenerallysupportit?
HeresaplacewhereIdorecommendyouusegoogle!Butdoitlast,soyouarebetterpreparedtothink
criticallyaboutwhatotherpeoplesay.
(12.Thisstepmaybeoptionalforyou,dependingonwhyyourereadingaparticularpaper.But
forme,itscritical!IgothroughtheLiteraturecitedsectiontoseewhatotherpaperstheauthors
cited.Thisallowsmetobetteridentifytheimportantpapersinaparticularfield,seeiftheauthors
citedmyownpapers(KIDDING!.mostly),andfindsourcesofusefulideasortechniques.)
Nowbraceformoreconflictnextweekweregoingtousethismethodtogothroughapaperon
acontroversialsubject!Whichonewouldyouliketodo?ShallwecritiqueoneofthepapersI
postedlastweek?
UPDATE:Ifyouwouldliketoseeanexample,youcanfindonehere
(http://violentmetaphors.com/2013/09/08/anexampleofhowtoreadavaccinesafetystudy/)

IgratefullyacknowledgeProfessorsJosBonnerandBillSaxtonforteachingmehowtocritically
readandanalyzescientificpapersusingthismethod.Imhonoredtohavethechancetopass
alongwhattheytaughtme.
Doyouhaveanythingtoaddtothisguide?Acompletelydifferentapproachthatyouthinkisbetter?
Additionalquestions?Linkstootherresources?Pleaseshareinthecomments!
Science

journalclub,science

397thoughtsonHowtoreadandunderstandascientificpaper:
aguidefornonscientists
1. Pingback:AGuidetoLookingSmartontheInternet:HowtoFindandEvaluateOnline
Information
2. Pingback:DearParents,YouAreBeingLiedTo|ISmartNetwork
3. LINDARICE
May18,2015/7:58pm
Thiswasagreatarticle.Iwantedtoletyouknow,Ireferenceditonmylastblogpost:
http://midwifelindarice.com/indefenseoftheotherside/.Itriedtotacklesomegeneraladvice
onfilteringhealthinformationinthemedia,Idloveyouropinion.
Reply
4. Pingback:BufaleuntantoalchiloCarigenitori,vistannomentendoBufaleuntantoalchilo
5. Pingback:SeasononFlourishingScience|
6. MRTNLM
http://violentmetaphors.com/2013/08/25/howtoreadandunderstandascientificpaper2/
June30,2015/3:58am

7/8