DraftofNovember1,2008:commentswelcome

Whatiscomputation?

Inthe19thcentury,oursocietyfundamentallychangedthewayitproducedmaterialgoods. Whereasproductionhadpreviouslydependedonhumanandanimallabor,thesteamengine allowedmuchofthatlabortobetransferredtomachines.Thismadethemanufactureofmany goodsandservicesmoreefficient,andthereforecheaper.Itmadeotherkindsofproducts availablethatwouldotherwisehavebeenimpossibletomanufacture.Atthesametime,it causedwidespreadeconomicdislocation,puttingmanyskilledtradesmenoutofworkand forcingmuchofthepopulationtoleavetheirruralcommunitiesforcities,radicallychangingthe veryenvironmenttheylivedin.Consumerculture,television,corporations,andtheautomobile, wouldnothavebeenpossible,oratleastasinfluential,withoutthisautomationofproduction, betterknownastheindustrialrevolution. Wenowhearwereinthemidstofaninformationrevolution.Liketheindustrialrevolution,it involvesautomationthetransferofhumanlabortomachines.However,thisrevolution involvesnotphysicallabor,butintellectuallabor.Whilewecannotknowtheultimate consequencesofthesechanges,informationtechnologyhasplayedanincreasinglycentralrole inoursocietyoverthelasttwodecades.Inthe1950s,ifyouandyourfriendswantedtogotoa movie,youwouldnthavestartedbywalkingtothelibrary.Buttodayyouwouldntthinktwice aboutcheckingtheinternetforreviewsandshowtimes.Similarly,averagepeoplewouldnt havepublishedtheirdiariesinthe1950s,buttodaypeopleblogabouteverythingfrom internationalpoliticstotheirpetsfavoritetoys. Yetforalltheimportanceofthesetechnologies,peoplestilltendtothinkofcomputationas beingaboutnumbers,andcomputerscienceasbeingabouttheesotericworshipofacronyms andpunctuation. Computationisnttiedtonumbers,acronyms,punctuation,orsyntax.Butoneofthethingsthat makesitsointerestingisthat,inallhonesty,itsnotentirelyclearwhatcomputationreallyis. Weallgenerallyagreethatwhensomeonebalancestheircheckbook,theyredoing computation.Butmanypeoplearguethatthebrainisfundamentallyacomputer.Ifthatstrue, doesitmeanthat(all)thoughtiscomputation?Orthatallcomputationisthought?Weknow thatthevirtualenvironmentsincomputergamesarereallyjustcomputationalsimulations. Butitsalsobeenarguedthattherealuniverseiseffectivelyacomputer,thatquantumphysics CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome isreallyaboutinformation,andthatmatterandenergyarereallyjustabstractionsbuiltfrom information.Butiftheuniverseisjustcomputation,thenwhatisntcomputation?Andwhat doesitevenmeantosaythatsomethingisorisntcomputation? Computationisanideainflux;ourcultureisintheprocessofrenegotiatingwhatitmeansby theterm.Onehundredyearsago,nearlyeveryonethoughtofcomputationasbeingamental operationinvolvingnumbers.Andbeingamentaloperation,itcouldonlybedonebypeople. Today,theoverwhelmingmajorityofwhatweconsidertobecomputationisdonebymachines. Butatthesametime,westillsomehowassociateitwiththought.InWesternculture,wetend totakeourcapacityforthoughtasthecentraldistinctionbetweenourselvesandotheranimals. Moreover,weviewourspecificthoughtsandfeelingsasbeingoneofthemajorconstituentsof ourpersonalidentity.Sothoughtisconstitutivebothofourcollectivehumanityandofour individualidentities. Changingourideasaboutcomputationchangesourideasaboutthoughtandsoourideasabout ourselves.Itreverberatesthroughourculture,producingexcitement,anxiety,andcountlessB moviesaboutvirtualrealityandcyberthingies. Andyetforallourcollectivefascinationwithcomputation,whatwemeanbythetermisstill somewhatmysterious.

Questionsandanswers
Ivesaidthatcomputationisntfundamentallyaboutnumbers.Nevertheless,letsstartby thinkingaboutarithmetic,sincewecanallagreethatitsanexampleofcomputation.Suppose youasksomeone: Whatssevenplusthree? Iftheyreplyten,thentheyjustperformedacomputation.Sowecanprovisionallythinkof computationasakindofquestionanswering.Inthecaseofaddition,thequestionalways involvesapairofnumbersandtheanswerisalwaysanothernumber.Foranyparticular additionquestion,theresacorrespondingnumber,determinedbythepairofnumbers,whichis thedesiredanswer. Nowsupposeyouaskthemwhatsonemillion,sevenhundredeightytwothousand,six hundred,andseventyeightplusthreemillion,twohundredninetytwothousand,seven hundredandfour?Unlesstheyreespeciallygoodatmentalarithmetic,theywontbeableto keepallthenumbersintheirheadandsotheyllhavetotakeoutpencilandpaper,askyouto repeatthequestion,writeitdown,performtheadditiononpaper,andreadofftheanswer. Butnowarithmeticisnolongerjustamentaloperation.Itsalsoaphysicaloperation:it involvesmanipulationandchangeofphysicalobjects(thepencilandpaper).Wereusedto thinkingofthearithmeticwedoasbeingamentalthingthattakesplaceinourheads.But CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome inthiscase,itsspreadoutbetweenapersonshead,hands,pencil,andpaper.Anotherchange fromthesevenplusthreeexampleisthatwhilewepresentedthenumberstothepersonasa setofsoundsinspokenEnglish,theythenrepresentedthemassymbolsonpaper,presumably Arabicnumerals. 1 Whatsinterestinghereisthatnoneofthesedifferencesmatter.Solongasthepersoncomes upwiththerightanswerinwhateverrepresentationalsystemorsystemstheyhappenedtobe using,weconsiderthemtohavesuccessfullydonethearithmetic.Putanotherway,itdoesnt matterwhichpathwetakethroughthediagrambelow:

writing

add2

writing

write

read

speech

add1

speech

Wellcallthistheprincipleofbehavioralequivalence:ifapersonorsystemreliablyproduces therightanswer,theycanbeconsideredtohavesolvedtheproblemregardlessofwhat procedureorrepresentation(s)theyused.Behavioralequivalenceisabsolutelycentraltothe modernnotionofcomputation:ifwereplaceonecomputationalsystemwithanotherthathas thesamebehavior,thecomputationwillstillwork.

Thefunctionalmodel
Letstrytobemorepreciseaboutwhatwemeanbysayingcomputationisakindofquestion answering.Firstofall,therearedifferentkindsofcomputationproblems.Soitsfairtoaskan addingmachinewhat3+7is,butnotwhatthecapitalofIndiais.Theabilitytodoatasklike additionisroughlytheabilitytoansweracertainprescribedsetofquestionswiththecorrect answer.Sowecanthinkofacomputationalproblemasasetofpossiblequestions,eachof whichhasadesiredcorrectanswer.
1

Infact,thenumbersthemselvesneverevenmadeanappearance,sincetheyrecompletelyintangible;weonlyhaveaccessto themthroughrepresentations.

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome Inthecaseofadditionquestions,theonlyrelevantpartsofthequestionarethenumberstobe added.Foranaddingmachine,thewhatisthesumofpartofanadditionquestionis irrelevantbecausethatstheonlykindofquestionitcananswer.Sowelldispensewiththe irrelevantpartsandjustsaythequestionconsistsoftherelevantbits,inthiscasethepairof numberstobeadded.Similarly,welldistilltheanswerdowntojustanumber. Sowhilewestilldontknowwhatcomputationisintheabstract,wecanatleastsaysomething aboutwhatwemeanbyacomputationalproblem.Itsagroupofrelatedquestions,eachof whichhasacorrespondingdesiredanswer.Wellcalltheinformationinthespecificquestion theinputvalue(s)andthedesiredanswertheoutputvalue.Foranyinput,thereisa correspondingdesiredoutput.Thenotionsofinputandoutputarethemostbasicconceptsof computation:computationisforthemomenttheprocessofderivingthedesiredoutput fromagiveninput(s). Ifyouvetakentherightmathcourses,thenyoumayrealizethatwhatwerecallinga computationalproblemisthesameaswhatmathematicianscallafunction.Afunctionisjust aspecificationofoutputvaluesforanygiveninputvalue(s).Sineandcosinearefunctions;they specifyoutputvalues(numbers)foreverypossibleinputnumber.Additionisalsoafunction, butoftwoinputsratherthanone.Andmoreesotericoperationsinmathematics,likethe derivativecanalsobethoughtofasfunctions,althoughtheyrefunctionswhoseinputsare themselvesotherfunctions.Butfunctionsdonthavetobeaboutnumbers.Forexample,an MP3playercomputesafunctionwhoseinputisacompressedsongandwhoseoutputisan audiowaveform. Aprocedure(akaanalgorithm,program,routine,orsubroutine)isaspecificmethodfor determininganoutputvaluefromasetofinputvalues.Assumingtheprocedurealways producesthesameoutputforagivensetofinputs(e.g.itdoesntinvolverollingdiceorother nondeterministicoperations),thenitactslikeafunctioninthatithasastrictcorrespondence betweeninputsandoutputs.Thedifferencebetweenproceduresandfunctionsisthat functionsonlyspecifywhattheiroutputsare,whereasaprocedurespecifieshowtocompute them.Therearetypicallymanydifferentproceduresforcomputingagivenfunction,butaswe shallsee,therearesomefunctionswecantcompute.WecanprovisionallythinkofComputer Scienceasthestudyofproceduralknowledge:howtodescribeprocedures,constructthem, andcomparecompetingproceduresforcomputingthesamefunction. Wevebeenassumingherethattheonlyimportantpartofaproceduresbehaviorisitsoutput. Wellcallthisthefunctionalmodelofcomputation:proceduresarewaysofcomputing functionsandsotwoproceduresarebehaviorallyequivalentifandonlyiftheycomputethe samefunction(althoughtheymaydifferintheresourcestheyrequiresuchaslengthoftimeor amountofscratchpaper).Thisisalimitedviewofcomputation,butitcoversasurprising amountofground.Rememberthattheinputsandoutputsofourfunctionsdonthavetobe numbers.TheycanperfectlywellbethingslikeMP3filesorimagesfromawebcam.

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome

Procedures
Letsconsideronelastarithmeticexample.Supposeyoureatarestaurantandwanttoleavea 20%tip.Asacomputationalproblem,thisinvolvestakinganumberasinputandcomputing 20%ofit.Ifwecalltheinputcforcheck,thenwerecomputingthefunction: tip(c)=0.2c Mostpeoplearentespeciallygoodatcomputingpercentagesintheirheadsbecauseitinvolves multiplyingmultidigitnumbers;ifIaskedyouwhat73%wasof151.05,youdprobablyhaveto takeoutpencilandpaperoruseacalculator.But20%hastheusefulpropertythatwecanuse thefollowingprocedure: 1. Doublethenumber 2. Erasethelastdigit (assumingthebillandtiparewholenumbers) Thisismucheasierbecausemostpeoplecandoublenumbersofafewdigitsintheirheads, whereascomputinghighermultiplesisharderforthem.Ontheotherhand,itsassumeswe representthenumberasastringofdecimaldigitssothatremovingadigitdividesthenumber byten.IfthewaitpersonwritesthecheckinRomannumerals,youreoutofluck(quick:whats XX%ofMMCMXLIV?).

Representation
Inpractice,inputsandoutputsarealwaysencodedinsomespecificrepresentation.Often,we canignorewhichrepresentationisusedandjustthinkabouttheprocedureasmanipulatingthe informationintheinputratherthanasmanipulatingthecomponentsofaspecific representation.Consequently,computationisoftenreferredtoasinformationprocessingand thecomputingindustryastheinformationtechnologyindustry.However,asthisexample shows,thechoiceofrepresentationcanaffectourchoiceofalgorithm.Consequently, ComputerScienceisalsoveryconcernedwithstudyingthepropertiesofdifferentkindsof representationsordatastructures. Partofthereasonrepresentationmattersisthatitaffectswhatsimplerproceduresyoucantake forgranted.Proceduresarealwayswrittenintermsofothersimplerprocedures.Inthe algorithmabove,weassumedthereaderalreadyknewhowtodoubleanumberandhowto eraseadigit,sincethosearerelativelyeasytodoindecimalnotation.However,ifwewere teachingittoachildwhohadntlearnedmultiplication,wemightinsteadsay: 1. Addthenumbertoitself 2. Erasethelastdigit Thisistechnicallyadifferentprocedure,sinceitreplacesmultiplicationwithaddition,butsince itcomputesthesameoutput,itsbehaviorallyequivalenttotheoriginalone.

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome Nowsupposethechildhasntlearnedtodomultidigit addition,andyetwestillwantthemtocomputeourtip forus,perhapsastortureofayoungersibling.Thenwe couldsay: 1. Startwiththelastdigit 2. Addittoitself 3. Iftheresultismorethanten,thenwritethe1 abovethenextdigit 4. Ifthereareanydigitsleft a. Movetothenextdigit b. Gotostep2,rememberingtoaddin the1aboveit,ifyoucarrieda1over 5. Otherwise,erasethelastdigit Thisprocedureexplainscomputinga20%tipona multidigitnumberintermsofsingledigitaddition.But supposethepoorchildcantadd,butcanonlycount, yetisinexplicablyeagertopleaseyou,theirsadistic oldersibling.Thenwecouldtellthepoorchildto: 1. Startwiththelastdigit 2. Addittoitselfbydoingthefollowing a. Writea0underneathit b. Ifthenumberunderneathisthesame asthedigitthatwestartedwith,then gotostepe c. Counteachnumberupbyone d. Gobacktostepb e. Iftherewasacarrywrittenabovethe originaldigit,thencountitupone moretime 6. Iftheresultismorethanten,thenwritethe1 abovethenextdigit 7. Ifthereareanydigitsleft a. Movetothenextdigit b. Gotostep2,rememberingtoaddin the1aboveit,ifyoucarrieda1over 8. Otherwise,erasethelastdigit Butofcourse,todothat,youwouldhavetobereally sadistic.

Levelsofabstraction
Wedescribedmultiplicationin termsofaddition,whichbroke downintoaddingdigits,andthen counting.Inpractice,wewould makethisexplicitbydescribing proceduresintermsofsub procedures: Todoubleanumber:add ittoitself Toaddtwonumbers:add thelastdigits,thenadd thenextdigits,withthe carry,etc. Toaddtwodigits: repeatedlyadd1toone whilesubtractingone fromtheotheruntilthe seconddigitiszero.

Ifweweretodiagramallthe proceduresinvolved,itmightlook likethis:

Mostprogramsbreakupinto layerslikethis.Proceduresare builtontopofsimpler procedures,whichusuallyarebuilt ontopofstillsimplerprocedures, untilwegetdowntothelevelof theprimitiveoperationsthatthe computershardwarecanexecute directly.However,youcannever describeaprocedure completely;youcanonly describeitintermsofother proceduresthereaderalready understands.

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome

Theimperativemodel
Thefunctionalmodelisaprettygoodmodelofwhatwemeanbycomputation.Itcoversallof mathandalotofprogramming.Butitassumesthatcomputationsalwaysreadaninput,think forawhile,writeanoutput,andthenstop.Theonlyaspectofaprogramsbehaviorthe functionalmodelisconcernedwithistherelationbetweenitsinputanditsoutput;itconsiders proceduresthatgeneratethesameoutputforthesameinputstobebehaviorallyequivalent. Ontheotherhand,thestepsofourtipcomputingproceduresinvolvescribblingonscratch paper.Infact,theybasicallyworkbygraduallyaccumulatingscribblesuntiltheanswerisonthe page.Andthoseindividualstepsofwriting,erasing,andrewritingthescratchpaperarent necessarilybestthoughtofascomputationsinthissenseofhavingspecificinputsandoutputs. Asithappens,computersworkinternallyinmuchthesameway;theirfundamentaloperations involvereading,writing,anderasinglocationsonwhatamountstoelectronicscratchpaper. Butwhataboutthisprocedure: 1. 2. 3. 4. 5. 6. 7. DrawanXatthetopofthepage DrawanXunderthecurrentX ErasetheXabovethenewX IfthenewXisntatthebottomofthepage,gotostep2 DrawanXontopofthecurrentX ErasetheXbelowthenewX IfthenewXisatthetopofthepage,gotostep2,otherwisetostep5

Thisiseffectivelyasimpleanimationprocedure.ItmakesanXmoveupanddownonthe screen,orinthiscase,thepaper.Itneverproducesanoutputinthesenseofthefunctional modelofcomputation.Infact,itneverevenstopsrunning.Sowehaveaprocedurethat: Canbefollowedmechanically Doessomethinguseful,and Doesitusingexactlythesametechniquesthatnormalproceduresuse

Yetitisntconsideredacomputationunderthefunctionalmodelbecauseitdoesntcomputea functionperse. Clearlythefunctionalmodelsnarrowlimitationofaprogramsbehaviortoitsoutputisa significantlimitation. 2Manyprogramsdontreturnresultsintheusualsense.Thesoftwareon yourcellphoneneverstopsrunning,orsoonehopes.Similarly,whenyouuseabackup programtomakecopiesofyourfiles,theprogramsmodificationofyourbackupmediumisnt

2

Thisissomewhatunfairtothefunctionalview.Itseasytomakeotherfunctionalmodelsofcomputationthatcanembracethis kindofcomputation.Ourpurposehereisnottochooseonemodeloveranother,buttodrawattentiontotwodifferentwaysof thinkingaboutcomputation.

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome anincidentalsideeffectoftheprogram,asitwaswhencomputingatip;itsthewholereason fortheprogram. Programstepsthatinvolvemodifyingthecomputersmemoryortakingsomeactioninthe worldarecalledcommandsorimperatives 3.Inpractice,thebasicinstructionsfollowedbyyour computerareimperatives.Programsaresequencesofimperativesthatthecomputerexecutes inorder.Asinourtipcalculatingexample,imperativescanbestrungtogethertocomputea function.Butaswiththebackupprogramexample,theycanalsojustbestrungtogetherto manipulatethecomputersmemoryinamannerwefinduseful. Thisgivesusanothermodelofwhatcomputationis,whichwellcalltheimperativemodel: proceduresaresequencesofcommands(imperatives)thatmanipulaterepresentations.The advantageofthismodelisthatitslessrestrictivethanthefunctionalmodel:itcoversanykind ofmanipulationofrepresentations,regardlessofwhethertherearedesignatedinputsand outputsinthefunctionalmodelssense.Asweshallseelater,however,itsoftenmoredifficult tothinkaboutprogramsatthislevel.Onereasonisitsoftenmoredifficulttospecifywhatthe programissupposedtodo.Inthefunctionalmodel,theresasimplespecificationofa computationalproblemthefunctiontobecomputedandacomparativelyeasywayto decidediftheprogramiscorrect:justcheckwhethertheprogramcomputesthecorrect function.Inthelessrestrictiveimperativemodel,itcanbedifficulteventospecifywhata programissupposedtodo,muchlessdeterminewhetheritdoesit. Nowthatwehaveacoupleofdifferentwaystothinkaboutcomputation,letsthinkaboutwhat wemeanbycomputersandwhatmakesthemunusualasakindofmechanism.

Interpretation
Whatwevelearnedsofaristhatcomputationinvolvesthemanipulationofrepresentationsby followingsomespecifiedprocedure,butthatthesamecomputationcanbeachievedbymany differentprocedures,representations,andmechanisms.Aslongastheproceduresare behaviorallyequivalent,theyreinsomesenseinterchangeable.Thedifferencebetweenwhat wevecalledthefunctionalandimperativemodelsaboveislargelyamatterofwhataspectsof theproceduresbehaviorareconsideredrelevanttothetaskathand. Whatmaybelessobviousisthatwhatwevecalledproceduresabovearethemselves representations.Theyrejusttext.Andassuch,theycanbemanipulatedlikeanyothertext: copied,modified,erased,etc...Whatmakesthemdifferentfromotherrepresentationsisthat theycanalsobefollowed(executed).Theprocessofreadingandfollowingastored representationofaprocedureiscalledinterpretation.

FromtheLatinimperaretocommand.

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome Programmabledevicesaresimplydevicesthatcanstoreandinterpretrepresentationsof procedures.Acalculatoronlyoperatesononekindofrepresentationnumbers.Buta programmablecalculatorprovidesawayofrecordingkeypressesandplayingthembacklater, asiftheywerebeingpressedbytheuser.Thosekeypressesformastoredrepresentationofa procedure,andtheplaybackprocessisasimpleformofinterpretation. Programmabilityisimportantbecauseitmeansthatasingledevicecanbemadetodomany differentfunctions,notbyphysicallyreconfiguringit,butsimplybychangingastored representationinsidethedevice.Computerswouldbenowherenearthesocialoreconomic forcetheyarenowifyouhadtobuyseparatecomputersforwordprocessing,email,andediting photos.Noneofthoseindividualfunctionswouldhavebeenworththecostofanearlypersonal computer.Butthefactyoucanbuyonepieceofhardwareandhaveitperformawiderangeof tasksmakescomputerseconomicallyviable.

Everythingistext
Moderndigitalcomputersaredefinedbytwospecialattributes.Thefirstistheir programmability.Thesecondisthefactthatalthoughtheycanmanipulatemanydifferentkinds ofrepresentationspictures,sounds,programs,textallthosedifferentkindsofdata representationsareultimatelyencodedwithinasingleuniformmetarepresentation. 4Inpoint offact,themetarepresentationusedbycomputersisperfectlyfamiliartoyou.Computers supportabasicalphabetofsymbolsandthenstringthosesymbolstogethertoencode information.Inotherwords,computersusetext. Themaindifferencebetweencomputertextandourtextisthatcomputersuseasimpler alphabetwithjusttwosymbols,ratherthanthe26ofouralphabet.Becauseithastwosymbols, itsknownasbinary(aftertheLatinbimeaningtwo).Overthelast50years,computer scientistshavefiguredoutcleverwaysofencodingotherrepresentationsassequencesofbinary symbols,includinghumantext,programs,pictures,sound,andmoreabstractkindsofdatasuch asnetworksofrelationships. Fortunately,youwontneedtolearntothinkinbinaryforthiscourse,althoughyoucanlookat theappendixifyourecurious.Mostprogrammersrarelydealwithbitstringrepresentations directly,ifever.Whatmattersisthatallrepresentationscanbeencodedinsomeuniform representationalmedium.Thechoiceofbinarytextisultimatelyamatterofengineering convenience:itseasiertomakeareliableelectronicmemorycircuitifitonlyhastoreliably distinguishbetweentwosymbols.Earlycomputersexperimentedwithmorecomplicated alphabets,butitturnedouttobecheaperandeasiertorepresentcomplicatedalphabetsusing
4

Tosaythatsomethingisametathingyistosaythatitsakindofhigherlevelthingythatcansomehowbeusedtodescribe, construct,orreasonaboutallthedifferentpossiblelowerlevelthingies.Soametarepresentationisarepresentationthatcanbe usedtoconstructorrepresentotherrepresentations.

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome groupsofbinarysymbolsthantobuildmemorycircuits thatcoulddistinguishmanydifferentsymbols.

AutomatedTesting
Manysoftwaredevelopment companiesuseautomated systemsthatrunatnightwhen programmershavefinished working.Theymayperform lengthytasksthatprogrammers wouldntbewillingtowaitfor duringthedayortesttheprogram forproblems.Forexample,some gamecompaniescheckfor unintendedconsequencesto changesintheirprogramsby runningthelatestversionofthe gameatnight,puttingitthrough asetoftestinputs,andcapturing screenimagesfromtheplay sequence.Theythencompare thosescreencapturestothe screencapturesfromtheprevious day,andemailanydiscrepancies totheprogrammers.Each morning,theresultsofanew batchofautomatedtestsare waitinginthedevelopersemail.

Metaprogramming
Somoderndigitalcomputersarecharacterizedby: Aunifiedmetarepresentation(binarytext strings)inwhichallotherrepresentationscan beencoded Asetofbuiltincommandsformanipulating binarytext Theabilitytostoresequencesofcommands (i.e.procedures)inmemoryasbinarytext Theabilitytointerpret(execute)previously storedprocedures

Thefactthatproceduresarestoredinthesamebit stringformatasotherdatahasprofound consequences;itmeansthatthesamecommandsused toexamineandmanipulatedatacanbeusedto examineandmanipulatetheproceduresthemselves. Andthatmeanswecanwriteproceduresthatexamine, manipulate,andevencreate,otherprocedures.This practiceiscalledmetaprogramming:theuseof softwaretoautomatethedevelopmentofothersoftware.

Thebestexampleofmetaprogrammingistheuseofhigherlevelprogramminglanguages.As wediscussed,computersultimatelydescribealldataintermsofverysimpleoperationson binarytext.Shockingasitmayseem,humansarenotgoodatthinkinginbinary.Moreoverthe operationsthecomputerunderstandsaresohighlysimplifiedthatitcaneasilytakehundredsof thousandsofmachinelanguagecommandstodescribeasophisticatedprogram.Writinga programwithhundredsofthousandsofcommandsisgenerallybeyondtheabilityofahumanto doreliably;itsimplyinvolveskeepingtrackoftoomanydetails.Ontheotherhand,keeping trackofdetailisoneofthethingscomputersdoverywell.Soratherthanforcinghumansto programinmachinelanguage,wedesignprogramminglanguagesthatareeasierforpeopleto understand,thenwriteprograms,calledcompilers,totranslatethemintomachinelanguagefor us. Therearemanyotherkindsofprogramdevelopmentprograms,someofwhichyouwillusein this class. Optimizers are programs that rewrite other programs to eliminate certain simple kinds of inefficiencies, making them run faster. Debuggers are programs that monitor and control the execution of other programs, allowing a programmer to observe a programs CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome behavior as it runs, and to collect forensic evidence when the program fails. Static checkers scanprogramsbeforeexecutionforcertainkindsofcommonerrors.Revisioncontrolsystems keep track of different versions of a program, who made what changes, and when, so that problematicchangescanbeeasilyundone.Testingsystemsrunprogramsagainsttestsuites largecollectionsoftestinputstosearchforinputsthatcausethemtomisbehave. The information technology industry wouldnt be possible without metaprogramming. Humansarebyfarthescarcestandmostexpensiveresourceinthecomputerindustry.Justas the industrial revolution required the ability to use machines to manufacture other machines, largescalesoftwaredevelopmentrequirestheabilitytoautomatemundanedevelopmenttasks, freeingprogrammerstofocusonthetasksthatrequirereasoningandcreativity.Consequently, the overwhelming historical trend in software development has been to automate more and more of the development task, even at the expense of writing programs that require more powerfulcomputers.

Simulation
Although all modern computers share the same broad traits, theyre not identical. Simple microcontrollers, such as the ones that control small appliances, would likely have commands for adding numbers in binary, but multiplication might have to be programmed in terms of addition.ThedigitalsignalprocessorsincellphonesandDVDplayers,ontheotherhand,have fairlyelaboratecommandsetsforarithmeticbecausethatswhattheyspendmostoftheirtime doing. Butevenwhen computershavesubstantiallysimilarbuiltin commands,theyprobably wont encode those commands in binary in exactly the same manner. Thats the reason PCs cantrunprogramswrittenforMacintoshes(atleastolderMacs),andviceversa. However,sinceallcomputersstoredataasbitstrings,theresnoreasonwecantstorea procedurewrittenforonecomputerinthememoryofanother.Giventhat,weoughttobeable towriteametaprogramforonecomputerthatreadsandexecutesprogramswrittenfor another.Themetaprogramwouldmakethenewcomputerbehaviorallyequivalenttotheold one:itcouldinterpret/executetheprogramsoftheoriginalcomputerperfectly,althoughwitha significantspeedpenalty.Programsthatreadotherprogramsanddowhattheysayarecalled interpreters;interpretersthataredesignedtomakeonecomputerbehavelikeanotherare calledsimulatorsoremulators.SimulatorsareoftenusedtorunPCprogramsonMacintoshes orvideogameswrittenforoldgameconsolesonnewercomputers. Ofcourse,wesaidthatitseemedlikeweoughttobeabletowriteaprocedureforone computertosimulateanothercomputer.Butcouldwereallydoitinpractice?Thereareafew obviousissues,suchasthatacomputerwith1megabyteofmemoryprobablywontbeableto simulateonewith100gigabytes.Butifweassumethesimulatingmachineisgivenenough memory,canonecomputeralwayssimulateanother?Itsalwayspossiblethatonecomputer mighthavesomekindofspecialcommandthatsimplycannotbesimulatedbythecommandsof another,andthereforethatitmightbeabletocomputefunctionsthattheothercannot.Is CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome thereasetofcommandsthataresufficientforsimulatinganykindofcomputernomatterwhat itscommandsetis? Wehavetheanswer,andthatanswerisprobably.

Universality
ThefirstmachinesthatcouldsimulateothercomputersweredesignedbyBritishmathematician AlanTuring,andsoarenowreferredtoasTuringmachines. 5MostTuringmachineswerent evenprogrammable,butTuringshowedthattherewasaparticularkindofTuringmachine, whichhecalledtheUniversalMachine,whichcouldtakerepresentationsofotherTuring machinesandsimulatethem.ThatmeanttheUniversalMachinewasnotonlyprogrammable,it couldcomputeanythingthatanyTuringmachinecouldcompute. ThedetailsofTuringmachinesarentimportanttothisdiscussion.Whatmattersisthatthey cansimulateallthecomputersweremanagedtothinkup,andconsequentlycancompute anythingthosecomputerscancompute.Thismeansanycomputerthatsabletosimulate TuringsUniversalMachineisalsoabletosimulateanycomputer,andthereforeableto computeanythingweknowhowtocompute.ThispropertyofbeingabletosimulateTuring machines,andthereforebeingabletosimulateanyknowncomputer,iscalledTuring completeness,Turingequivalence,orjustuniversality. TheamazingthingisthatnearlyalldigitalcomputersareTuringcomplete.Infact,some stunninglysimplemechanismsareTuringcomplete. 6Oneofthepracticalconsequencesofthis isthathardwaredesignerscanlargelyignoretheissueofwhethertheircommandsetsare universal(theyare)andfocusonfindingcommandsetsthatcanbemadetorunveryquicklyin hardware. 7 Ofcoursethisdoesntprecludethepossibilitythatsomeonemightsomedaycomeupwitha radicallydifferentdesignforacomputerthatcouldntbesimulatedbyaTuringmachine.But thusfar,everycomputationalsystemthatsbeendevisedhaseitherbeenTuringcomplete(can simulateandbesimulatedbyanyotherTuringcompletesystem)orisweakerthanaTuring
5

AlanTuring,Oncomputablenumbers,withanapplicationtotheEntscheidungsproblem,ProceedingsoftheLondonMathematical Society,Series2,42(1936),pp230265.NotethatTuringmachinesweremoreaconceptualdesignthanapracticalengineering design.AlthoughTuringdidworkonthedesignofactualworkinghardware,hisworkonTuringmachinescamebeforethe constructionofrealcomputerswaspractical.

MinskyandBlumprovedinthe1960sthatatwocountermachinewasTuringcomplete.Atwocountermachineisamachine whoseonlydatarepresentationisapairofnumbers(nonnegativeintegers,infact),andwhoseonlycommandsaretoaddor subtract1fromoneofthenumbersandtocheckwhetheroneofthemwaszero.

7

Infact,currentmodelsofthePentiumprocessor(bothIntelandAMD)effectivelydontrunthePentiumcommandsetbutonly simulateit.TheyeachhavetheirowninternalcommandsetandeffectivelycompilePentiumprogramsintoprogramsintheinternal commandsetastheyrunthem.

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome machine(i.e.cantsimulateaTuringmachinebutcanstillbesimulatedbyaTuringmachine).So itisgenerallyassumedthatanyfunctionthatcanbecomputedcanbecomputedbyaTuring machine.ThisisknownasChurchsThesis,orsometimesastheChurchTuringHypothesis, after20thCenturymetamathematicianAlonzoChurch.

Thelimitsofcomputation
Youmayhavenoticedacertaincarefulnessofwordinginourdefinitionofuniversality:wedidnt actuallysaythatuniversalmachinescouldcomputeanything,onlythattheycouldcompute anythingthatothermachinescouldcompute.Itturnsoutthatnotonlyaresomeproblems provablyuncomputable,butinasenseitstheverypowerofuniversalmachinesthatmakes thoseproblemsuncomputable. Anyonewhosusedacomputerknowstherearetimeswhenaprogramwillhang:youtellitto dosomethinganditjustsitsthere,presumablydoingsomething,butneveractuallygettingback toyouwithananswer.Thisgenerallyhappensbecauseofsomeerrorbytheprogrammer.For example,ifwetakeourtipcomputingprocedure: 1. 2. 3. 4. Startwiththelastdigit Addittoitself Iftheresultismorethanten,thenwritethe1abovethenextdigit Ifthereareanydigitsleft a. Movetothenextdigit b. Gotostep2,rememberingtoaddinthe1aboveit,ifyoucarrieda1over 5. Otherwise,erasethelastdigit

Andwemiscopyitsothatitreads(changewritteninboldface): 1. 2. 3. 4. Startwiththelastdigit Addittoitself Iftheresultismorethanten,thenwritethe1abovethenextdigit Ifthereareanydigitsleft c. Movetothenextdigit d. Gotostep1,rememberingtoaddinthe1aboveit,ifyoucarrieda1over 5. Otherwise,erasethelastdigit

Thenacomputerfollowingthisprocedurewillrunforeverbecauseitwillkeepstartingoverwith thefirstdigitandnevergettingpaststep4.Thiskindofbugiscalledaninfiniteloopthe programrepeatedlyrunsthesamepieceofcodewithouteverhalting. Itwouldbeextremelyusefulifyouroperatingsystemcouldautomaticallydetectwhethera programishungandinformyouimmediatelywhenithappenedratherthanmakingyouwait. Thisiscalledthehaltingproblem:givenaprogramandapotentialinputforit,determine whethertheprogramwouldeverhalt(finish)ifitwererunonthatinput.Theproblemisthat CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome someprogramsjustnaturallytakealongtime,sothefactthataprogramhasrunforalongtime doesntnecessarilymeanthatitshung.Itsalwayspossiblethatitsabouttofinishandgiveyou ananswer. 8 Atfirst,itseemsthatweoughttobeabletouseasimulatortosolvethehaltingproblem.After all,asimulatortellsustheexactoutputaprogramgenerates.Thatsactuallymoreinformation thanweneed:wejustneedtoknowwhetherthereisanoutput,notwhatitsvalueis. Unfortunately,thisdoesntwork.Theproblemisthatsimulatorsjustblindlyfollowthe instructionsoftheprogramstheysimulate;iftheprogramitssimulatingrunsforever,sowill thesimulator. Letsthinkforamomentaboutwhatitwouldmeanforthehaltingproblemtobecomputable. Ifthehaltingproblemwerecomputable,wecouldwriteametaprocedurecallithalts? thattooktwoinputs,aprocedureandaninputtoruniton,thatwouldinfalliblyoutputyesor nodependingonwhetherthespecifiedprocedurewouldhaltifrunonthespecifiedinput. Thatturnsouttobethedifficultywiththehaltingproblem,becauseitleadstoaparadox.Forif wecouldsolvethehaltingproblemifwecouldwritethehalts?procedurewecouldwrite anotherprocedure,callitsmartass,thatlookedsomethinglikethis: 1. Followthehalts?procedure,givingitasinputsthisprocedure(smartass),andwhatever inputwasgiventosmartass 2. Ifhalts?sayssmartasswouldhalt,thenrunforever 3. Ifitsayssmartasswouldrunforever,thenhalt. Inotherwords,wecouldwriteaprocedurethatusedhalts?toaskquestionsaboutitself,and thendotheoppositeofwhathalts?saiditwoulddo.Ifsmartasshalts: Thenhalts?willoutputyes Andsosmartasswouldrunforever Oops,Iguesshaltsshouldreallyanswerno Andsmartasswillhaltafterall Andsohalts?willoutputyesafterall Etc.etc.

Thismeansthatthehaltingproblemmustbeuncomputablebecauseassumingwascomputable meantassumingwecouldwritehalts?,whichmeantwecouldwritesmartass,whichleadtoa contradiction.Whenanassumptionleadstoacontradiction,theassumptionmustbewrong.

Youmayhavehadasimilarexperiencewhilewebsurfing.Youneverknowwhetherapagelookupistakingalongtimebecause theserverisdownorjustbecauseitsoverloaded,soyouholdlittledebateswithyourselfoverwhethertokeepwaitingortostopit andtrysomethingelse.

8

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome Thisproof,orrather,amuchlongerandmorerigorousversionofit,wasduetoTuring. 9Its importantbecauseitshowsthat: ThereareproblemsthatareuncomputablebyTuringmachines,andsobyany mechanicalsystemweveeverbeenabletodevise Worseyet,theyincludeproblemswemightactuallycareabout,likecheckingwhethera programisbroken Ultimately,theexistenceofuncomputableproblemsisduetotheverypowerof generalpurposecomputationand,specifically,tothecapacityformetacomputation.

However,theuncomputabilityofthehaltingproblemisoftenmisunderstood,soletsbevery clearonwhatitdoesanddoesntentail.Itdoesntmeanwecanneverdetermineifaprogram halts;infact,wecansolvethehaltingproblemformanyoftheprogramsthatwecareaboutin reallife.Whatwecantdoiswriteaninfallibleprogramfordeterminingwhetheranarbitrary pieceofprogramtextwillhaltforanarbitraryinputtext;anyprogramwewriteforthehalting problemmustnecessarilygetitwrongsomeofthetime,evenifitgenerallygetsitrightinthe caseswereallycareabout. Thehaltingproblemhasalsobeenusedtoarguethatthebrainmustnotbeacomputerbecause peoplecantellwhetherprogramswillhaltandcomputerscant.Regardlessofwhetherornot thebrainisbestthoughtofasacomputer,peopleareactuallyjustlikecomputersonthehalting problem:wecansolveitalotofthetime,butwealsogetitwrongsometimes.

Imitation,equivalence,andintelligence
Theresanoldpartygamecalledtheimitationgameinwhichtwopeople,classicallyaman andawoman,hideindifferentroomswhiletheothergueststrytoguesswhichiswhichby submittingwrittenquestionsandreceivingwrittenanswers.Theideaistoseewhetherthe hiderscanfooltherestoftheguests.Intheterminologyweveusedabove,thepeopleinthe roomsaretryingtoactbehaviorallyequivalenttooneanother.Obviously,thisdoesntmean thatiftheimpersonatorswintheyveactuallychangedidentitiesorsexes. 10 Butimagineweplaythegamenotwithtwopeople,butwithonepersonandonecomputer. Whatifthecomputercouldfoolpeopleintothinkingitwashuman?Inanimportant1950 paper,ComputingMachineryandIntelligence,AlanTuring(ofTuringmachinefame)arguedthat ifacomputercouldfoolhumansintothinkingitwashuman,thenitwouldhavetobe consideredtobeintelligenteventhoughitwasntactuallyhuman.Inotherwords,intelligence

Ibid. Thatis,sexinthebiologicalsense.Onemightarguethattheyvetemporarilychangedtheir(sociallydefined)genderidentities.

10

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome isultimatelyabehavioralphenomenon. 11Thatis,anentityisintelligentbecauseitbehaves intelligently,notbecauseitscomposedofsomespecialsubstancesuchaslivingtissueora soul. 12Moreover,intelligenceisacomputationalphenomenon,amenabletocomputational analysis. Togetherwithcybernetics,andearlierworkonthetheoryofcomputation,Turingspaperset thestageforthedevelopmentofartificialintelligenceinthe1950sand60s.Thesethen providedascientificmodelforthestudyofmentalrepresentationsandtheprocessesthat operateonthem,whichwerethenimportedintofieldssuchaspsychologyandlinguistics, leadingtothedevelopmentofthefieldofcognitivescienceinthe1970s.

Computationalneuroscience
Westartedwiththeideaofcomputationbeingaprocessthathumansperformintheirbrains, andthenlookedathowspecificinstancesofthatprocesscouldbemodeledatacoarselevelby mechanicalprocesses.Thatleduseventuallytoboththelimitsofmechanicalprocedures,and theclaimthatitwasnonethelesstheoreticallypossibleforamachinetobeintelligent. Conversely,itsbecomecommoninourculturetotalkaboutthebrainasbeingakindof computer.Atthispoint,youmayobjectIdontfeellikeacomputer.Soletstalkaboutwhat itmightmeantosaythatthebrainisakindofcomputer.Certainly,thebraincando computation.Butthatdoesntmeanthatifyoucutopensomeonesskullyouregoingtosee somethingthatsrecognizablylikeaMacoraPC.Butasweveseen,computationisallabout behavioralequivalence.SotherealquestioniscouldabrainandaPC,inprinciple,simulateone another?Onesideisobvious.ApersoncansimulateaPC,bywritingdownthecontentsofthe computersmemoryon(awholelotof)scratch paperandthenmanuallyexecutingthe instructionsinitsprogram,oneatatime.Itwould beincrediblyboring,butitwouldbeconceptually possible.Sotherealquestioniswhethera sufficientlybigPCcouldsimulatethehumanbrain. Thebrainisanetworkofroughly100billioncells calledneurons,togetherwithothertissuesthat helptheneuronsfunction.Toafirst approximation,eachneuronhasasetofinputs, calleddendrites,thatreceivesignalsfrom Pyramidalneuronfrommousecerebralcortex neighboringneuronsandanoutput,theaxon,that (fromWikipedia)
11

Althoughwidelyaccepted,thecomputationalviewisnotuniversallyaccepted.Seeforexample,Searle,John.R.(1980)Minds, brains,andprograms.BehavioralandBrainSciences3(3):417457.
12

Notethatthisisntanargumentastowhetherpeoplehavesoulsornot,simplythatitspossibletobeintelligentwithoutone.Its notatheologicalclaim.

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome producesasignalbasedontheinputsthecellis receivingandthecellsowninternalstate.Each neuronsaxonisconnectedtodendritesofmanyother neurons.Whenoneneuronstimulatesanother neuron,itpredictablyincreasesordecreasestherate orlikelihoodofthesecondneuronstimulatingother neurons.Ifitspredictable,thenitshouldbepossible towriteacomputerprogramthatpredictsand simulatesit,andindeedtherearemanycomputational modelsofdifferentkindsofneurons.Butifeach individualneuroncanbesimulatedcomputationally, thenitshouldbepossibleinprincipletosimulatethe wholebrainbysimulatingtheindividualneuronsand connectingthesimulationstogether. Whiletheresadifferencebetweenshouldbe possibleandisactuallypossible,thisisthegeneral ideabehindtheclaimthatwecanunderstandthebrain computationally.Assumingitsright,ithassome importantconsequences.Foronething,ifbrainscan besimulatedbycomputers,thencomputerscouldbe programmedtosolveanyproblembrainscansolve, simplybysimulatingthebrain.Butsinceweknowthat computerscantsolvecertainproblems(the uncomputableproblems),thatmustmeanthatbrains cantsolvetheuncomputableproblemseither.Inother words,therearefundamentallimitstothepossibilityof humanknowledge. Althoughwerealongwayfrombeingabletofully simulateabrain,computationalneuroscience,inwhich scientiststrytounderstandneuralsystemsas computationalprocesses,isanimportantandgrowing areaofbiologicalresearch.Bymodelingneuralsystems ascomputationalsystems,wecanbetterunderstand theirfunction.Andinsomeexperimentaltreatments, suchascochlearimplants,wecanactuallyreplace damagedcomponentswithcomputingsystemsthat are,somuchaspossible,behaviorallyequivalent.

WhoamI?
Simulationalsoraisesquestions aboutpersonalidentity.Theresa longtraditioninWesternculture ofidentifyingtheselfwithones thoughts.Butifourbrains,and thusourthoughts,canbe simulated,towhatextentdoes thatmeanweourselvescanbe simulated?Istheparticular substanceofourconstruction biologicaltissuesomehow crucialtoouridentity,orcouldwe, inprinciple,mapoutallthe neurons,synapses,andconnection strengthsinourbrains,andmake simulationsofourselves?There arepeoplewhoarguewecould liveforeverbydownloading ourselvesintosilicon. Werealongwayfrombeingable tosimulateafullbrain,muchless mapalivingone.Forthe foreseeablefuturetheseissues willbeconfinedtosciencefiction andphilosophypapers. Butinmanywaysthequestionof whetherwewilleveractuallybe abletodownloadourselvesinto siliconislessimportantthanhow ideasofcomputingandsimulation arealreadychangingsocietys attitudesaboutselfandidentity. Asartificialagentsbecomemore lifelike,willwestarttoviewthem asrealpeople?Willwesomeday haveaMarioLiberationFront arguingforlegalrightsforvideo gamecharacters?

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome

ComputationalX(forallX)
Thegeneralapproachofunderstandingthebehaviorof systemsbyconstructingcomputersimulationsofthem isntlimitedtobiology.Anysystemwhosebehavioris predictablecanbesimulated,oratleastapproximated, insoftware.Incomputationalscience,theoriesabout thenaturalworldcanbetestedmorepreciselyby embodyingthemincomputationalsimulationsand comparingtheirbehaviortotheobservedbehaviorin thephysicalworld.Forexample,incomputational chemistry,thebehaviorofatomscanbesimulated underconditionsthatcouldnotbeadequately controlledinthelaboratory.Incomputationalbiology, simulationisusedtounderstandthefoldingofproteins insideofcellstobetterunderstandtherelationship betweentheiraminoacidsequencesandtheir functionalstructure.Thesecanthenbeusedin applicationssuchascomputationaldrugdesigntotry tocustomdesignmoleculeswithdesired pharmacologicalbehavior.Inengineering, computationalsimulationallowsprototypedesignsto betestedincaseswheretheconstructionofphysical prototypeswouldbeprohibitivelyexpensiveor dangerous.Infinancialengineering,computermodels ofthebehaviorofmarketsareusedtodesignfinancial instrumentsthatgiveinvestorsbettercontrolovertheir risklevels.

Computational everything
Quantumphysicshastwounusual features:first,physicalproperties arequantized,thatis,limitedtoa setofdiscretevaluesastheyare incomputers,ratherthanatrue continuum,and,second,that physicalsystemsbehave differentlydependingonwhere andhowtheyreobserved, thereforeunderstandingthe behavioroftheuniverserequires explicitlyreasoningaboutwhat informationisavailableaboutitat whattimes. Theinterpretationofquantum physicshasalongand controversialhistory.However, oneminorityinterpretation, pancomputationalism,arguesthe universeisbestthoughtofasa computeritself.Butthenwhat wouldthisevenmean?Notice thatthedefinitionwevebeen usingofcomputationthatit involvesthemanipulationof representationsdoesntwork anymore.Iftheuniverseisa computer,whatisitrepresenting? Itself?Yourconsciousness?If computationwithout representationispossible,thenwe havetoaskalloveragainthe questionofwhatcomputationis.

Okay,sowhatiscomputation?

Thepointoftheforegoingwasnottoarguethatthe universeisacomputerorthatweshouldalldownload ourselvesintosilicon,butratherthatcomputationisan ideainflux.Ourcultureisintheprocessof renegotiatingwhatitthinkstheconceptscomputationandcomputerreallymean.Computation liesonaconceptualfaultlinewheresmallchangescanhavemajorconsequencestohowwe viewtheworld.Thatmakesitaveryinterestingfieldtofollowrightnow. ItsaninterestingexercisetotypedefinecomputationintoGoogleandlookatthedefinitions youget.Theyincludedefinitionslikedetermining something by mathematical or logical
methods, which doesnt apply well to the idea of DNA being a computer program, or finding a solution to a problem from given inputs by means of an algorithm, which is essentially the

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome
functional model that, as we saw, doesnt apply well to things like computer games, or for that matter brains (although it may apply well to individual parts of brains). Another possible answer is that computation is information processing, although that then begs the question of what we mean by information.

Apragmaticdefinitionwouldbethatcomputationiswhat(modern,digital)computersdo,soa systemiscomputationalifideasfromcomputersareusefulforunderstandingit.Thisis probablytheclosestdefinitiontohowrealpeopleusetheterminpractice.Asweveseen, therearealotofdifferentkindsofsystemsthatarecomputationalunderthisview. Butthatisntaverysatisfyingdefinition.Anotherviewwouldbethatcomputationiswrapped upwiththestudyofbehavioralequivalence.Underthisview,computationistheprocessof producingsomedesiredbehaviorwithoutprejudiceastowhetheritisimplementedthrough silicon,neurons,orclockwork.Thekeytheoreticalissuesincomputationarethen: Howdowespecifythedesiredbehavior? Thefunctionalmodelgivesusaconceptuallysimpleapproach,butitisnteasily applicabletoallproblems Howdoweanalyzenaturalsystemsweathersystems,stockmarkets,brainsusing theconceptualtoolsofcomputation? Whatkindsofbehaviorarepossibletoproducemechanically? Whatisthecostofproducingagivenkindofbehavior?Aretheresomekindsthatare prohibitivelyexpensive?(Answer:youbet!) Howdowepiecetogetherexistingcapabilities(behaviors)toproducenewkindsof behavior?

Inpractice,mostcomputationalsystemsthatweengineerwillbebuiltusinggeneralpurpose, programmablecomputers.Forthesesystems,wealsohaveanadditionalsetofconcerns: Howcanweautomateasmuchoftheprogrammingtaskaspossible? Whatarecommonbuildingblocksofprogramsthatwecanreusewithouthavingto rebuildthemeachtime? Howdowedesigncomputerhardwaretorunasfastaspossible? Howdowedesigncomputerlanguagestobeflexibleandeasytounderstand?

Finally,thereareanumberofpsychologicalandsocialissuesweneedtothinkabout: Whatkindsofcomputationalsystemsareeasyforpeopletouse? Howcanweusehumanpsychologytodesignsystemsthatpeoplefindrewarding?For example,whatkindsofgamesorotherinteractiveentertainmentsystemsarenaturally engaging? Howcanwedesignlargesystemssothattheyreeasierforhumanstounderstand? Howcanagroupofpeopledivideupaprogrammingtaskinsuchawaythattheycan worktogethereffectively?

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome Whataretherisksofinformationtechnology?Dowereallywanttomakeairplanesthat arecontrolledbycomputers,sothatifprogramcrashes,theplanedoestoo?Do databasesandsurveillancetechnologygivegovernmentsandcorporationstoomuch power? Howcanweuseinformationtechnologytoimprovepeopleslives? o Canweuseautomationtohelptheelderlytoliveindependently,ordisabled peopletomoveabouttheirenvironmentsbetter? o Howwouldinnercityschoolsbedifferentifwehadonelaptopperchild? o Canwebuildautomatedtutoringsystemstoalloweveryonetohaveoneon onehelpwhentheyneedit? o Whatsortsofautomationwouldbeusefulandsustainableforpeoplein developingcountries?

Wewonthavetimetogointoallofthesequestionsinclass.Buttheseareactiveresearch topicsincomputerscience. Computationisabroad,richfield.Ithashaddeepinfluencesonourlivesandculture.Ifyour PC,cellphone,andwebaccesssuddenlydisappeared,youwouldprobablyhavetoradically reconfigureyourlife,eventhoughthesehaveonlybeenwidelyavailableforthelast15years. Itsunlikelythatthechangestosocietyaregoingtostopanytimesoon.Bylearningabout computation,youcanbeinapositiontohelpmakesurethosechangesareforthebetter.

Appendix:binaryrepresentationofdata
Althoughcomputersessentiallystoretheirinformationastext,theyuseamuchsimpler alphabetthanhumanlanguages.Fortunately,thechoiceofalphabetreallydoesntmatter.The Japanesehiraganacharacterset,forexample,hasaseparatecharacterforeachpossiblesyllable inthelanguage.TheRomanalphabet,however,usesletterstorepresentindividualsounds withinasyllable.Fortunately,wecanperfectlywellwritetheJapanesetext: (English:Ilikeit) intheRomanalphabetbyrepresentingeachhiraganacharacterwithasetofRomancharacters, inthiscase,theRomancharactersthatwouldbepronouncedthesameasthehiragana character: su ki de su yo ToformtheRomanization: sukidesuyo Inthiscase,wehappenedtoencodethehiraganacharactersetintheRomancharactersetin suchawaythattheEnglishpronunciationoftheRomanizationwouldbeagoodapproximation CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome ofthecorrectJapanesepronunciation.Butwedontstrictlyspeakinghavetodothat.Wecould justcallthefirst26hiraganacharactersaa,ab,ac,etc.,andthencallthenext26ba, bb,bc,etc.Orwecouldnumberthen: 01 02 03 04 05 Toformtheencoding: 0102030405 Sincethealphabetdoesntreallymatter,electricalengineersarefreetodesigncomputer hardwareusingwhateveralphabetisconvenientforelectroniccircuits,thenencodewhatever alphabetwereallywanttouseasshortstringsinthecomputersnativealphabet.Thatalphabet turnsouttohaveonlytwosymbols. 13Thistwoletteralphabetiscalledbinary(fromLatinbi, two).Whenwesaythatinformationisencodedinbinary(ordigitally),wejustmeanthat itsrepresentedasaseriesofsymbolsinthebinaryalphabet. Youmaywellhavebeentoldthatcomputersstoreeverythingasnumbers,butthatisntstrictly true;binaryisultimatelyjusttextwithatwoletteralphabet.However,ifwechoosetointerpret thoselettersasdigitsofanumber,thenwecanrepresentnumbersinbasetwo.Basetwoislike baseten,exceptweveonlygottwodigits0and1andsowehavetocarrywhenwegopast 1ratherthanwhenwegopast9.Usingbase2,wecanwriteanysidenumberasaseriesof binarysymbolsorbits(shortforbinarydigit): 0is0000 1is0001 2is0010 3is0011 4is0100 5is0101 6is0110 7is0111 8is1000

Forthosewhoareelectricallyinclined,heresthegeneralintuitionbehindthis.Ultimately,informationinacomputeris representedasvoltagesorcurrentsonawire.Manycomputers,forexample,usevoltagesbetween0and5volts.However,its easyforexternalinfluences(electricalinterference,powersupplyvariation)tomakesmallchangestothevoltageonawire.Ifwe trytorepresent26differentsymbolsusingdifferentvoltagesbetween0and5V,thenthedifferencebetweenthevoltagesfor differentsymbolsisaround0.2V,whichmakesiteasyforexternalinfluencestochangethewirefromonesymboltoanother.Ifwe onlyhavetwovalidvoltages0Vand5Vthenevenlargeamountsofinterferencewontchangethevoltageenoughtoconfusethe circuitry.Inbiology,thebrainusesthissametacticwhentransmittinginformationoverlargedistances.Ratherthanusinga continuouslyvaryingvoltage,neuronsuseonlytwovoltagesandchangehowoftenthevoltageishigh.

13

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome etc.

Ofcourse,wecanalsorepresentEnglishtextbyusinggroupsofbinarysymbolstoreferto letters.Themostcommonrepresentationfortextatthemoment 14usesgroupsof8bits, playfullycalledbytes,foreachletter: Ais01000001 Bis01000010 Cis01000011 Dis01000100 etc.

ThewordBADwouldthenberepresentedbygroupingthebitstringrepresentationsofthe individuallettersintoonelargebitstring: B A D 01000010 01000001 01000100 However,theold8bitcodecanonlyrepresenttheRomanalphabet,theArabicnumerals,anda fewextrasymbols.Recently,thecomputingindustryhasbeenmovingtoa16bitcode,called Unicode,whichcanrepresentnotonlytheRomanalphabet,butalsothecharactersetsofall majorlanguagesoftheworld. 15Forexample,theUnicodeencodingfor would be:

0011000001011001 0011000001001101 0011000001100111 0011000001011001 0011000010001000

Digitalmedia
Hopefullybythispointweveconvincedyouthatyoucanrepresentanytextinbinary,nomatter whattheactualalphabetofthetextis,andthatwecanrepresentnumbersastext,and thereforeinbinary.Whataboutimages?Imagescanberepresentedasseriesofpixels,eachof whichhasaspecificbrightness,andrecordingthebrightnessesofeachpixelasanumber. 16 Sinceeachnumbercanberepresentedinbinarytext,thewholethingcanberepresentedin binarytext.Andwevealreadyseenthatprocedurescanberepresentedastext,sinceallthe
14

ThisistheAmericanStandardCodeforInformationInterchangeorASCIIcode,themoremodernversionofwhichisthe InternationalStandardsOrganizationsstandardnumber646orISOcode. AcompletelistingofallthescriptsrepresentedinUnicodeisavailableontheweb.CheckoutthepagesonCuneiformandRunic.

15

Thisistechnicallyonlyawayofapproximatinganimage,sincewethinkofimagesasbeingcontinuousandaspotentiallyhaving finerdetailsthancanbecapturedbythepixels.However,wecanapproximatetheimagetoanydesiredlevelofaccuracybyusing morepixels.Similarly,othercontinuousquantities,suchasthebrightnessofanindividualpixel,canonlybeapproximated,butcan beapproximatedtoanarbitraryaccuracybyusingmorebitsjustasnumberscanbewrittenmoreorlessexactlyindecimalnotation byusingmoredigitsafterthedecimalpoint(moreplacesofaccuracy).

16

CopyrightIanHorswill2007,2008ian@northwestern.edu

DraftofNovember1,2008:commentswelcome procedureswevediscussedsofarhavebeenpresentedtoyouasEnglishtext.Although computersuseamorepreciseandefficientencodingfortheirproceduresthanEnglish,the fundamentalideaisthesame.Otherkindsofrepresentationscanbeencodedinbitstrings usingvariousclevertechniquesthatcomputerscientistshavedevelopedoverthelast50+years.

CopyrightIanHorswill2007,2008ian@northwestern.edu