You are on page 1of 6

Columbia University in the City of New York | New York, N.Y.

10027
DEPARTMENT ! "TAT#"T#C" $%& Mathemati's (uil)in*
+,,- (roa)way
Phone. /+%+0 &1234$1+54
!a6. /+%+0 $$43+212
W 6501: STOCHASTIC PROCESSES
Fall Semester 2003
Professor Ioannis Karatzas


COURSE SYLLABUS


Review of Conditional Distributions and Expectations.
The simple Random Walk and its properties; gamblers ruin problem. Markov Chains in
discrete time; notions of commun-icating classes, stopping times, recurrence, transience,
harmonic functions, invariant mea-sures; ergodic properties.
Markov Chains in continuous time: birth-and-death processes, Poisson process, pure-
birth processes, queueing processes. Branching processes and their properties.
Martingales in discrete time: invariance under transforms, Doob decomposition,
optional sampling theorem, applications, upcrossings, convergence. Brownian motion:
construction, basic properties, sample paths, refection principle, hitting times,
quadratic variation.
Elementary introduction to stochastic integration and its connection with martingale
transforms, Itos rule, applications, connections with the heat equation. Martingales in
continuous time. Elementary introduction to stochastic diferential equations and
difusions.


Required Text:
G. LAWLER: "Introduction to Stochastic Processes". Chapman & Hall (1995).

Recommended Texts:
J.R. NORRIS: "Markov Chains". Cambridge University Press (1997).
B. OKSENDAL: "Stochastic Diferential Equations". 3
rd
Edition, Springer-Verlag (1992).
R.B. ASH: "Basic Probability Theory". J. Wiley & Sons (1972).

A DETAILED SYLLABUS FOLLOWS
. Lecture #1: Tuesday, 2 September
Overview of the topics in the course: Simple Random Walk. Random Walk. Processes with
Independent Increments. MARKO propert!. Martin"ale propert!. #ROW$ian motion.

. Lecture #2: Thursday, 4 September
%lementar! approach to the simple random walk: &am'ler(s ruin pro'lem) a'sorption
pro'a'ilities) e*pected duration of "ames. $otions of Recurrence and +ransience.

. Lecture #3: Tuesday, 9 September
,om'inatorial approach to the simple random walk. -istri'ution of the first.return.time to
the ori"in.
Assignment # 1: Read R. Ashs Chapter on the Combinatorial Approach to the Simple Random
Walk. Problems 1 8, from Ash, pp. 190-191. Due on Thu. 18 September.

. Lecture #4: Thursday, 11 September
+he MARKO propert!. MARKO ,hains. Initial distri'utions) transition pro'a'ilities /for one
and several da!s0. $otions of communication) e1uivalence classes) irredu.ci'ilit!.

. Lecture #5: Tuesday, 16 September
2irst hittin" times and their elementar! properties. $otion of stoppin" time. +he Stron"
MARKO propert!.

. Lecture #6: Thursday, 18 September
%1uivalent formulations of the MARKO propert!. ,onditional Independence. Statement and
proof of the Stron" MARKO propert!. Applications: "eneratin" functions and distri'utions for
first.passa"e.times) and for first.return.times) 'ased on the Stron" MARKO propert!.
Assignment # 2: Not to be handed in.
1. Read sections 1.1 1.7 in Norris.
2. Norris, Problems (Pbs.) # 1.1.2 1.1.4, 1.2.1, 1.5.1, 1.5.3.
3. Lawler, Problems (Pbs.) # 1.2, 1.5(a,b), 2.4.

. Lecture #7: Tuesday, 23 September
Recurrence and +ransience for MARKO ,hains: definitions and 'asic properties. Application
to the simple random walk in two and three dimensions.

. Lecture #8: Thursday, 25 September
Invariant measures and invariant distri'utions for a MARKO ,hain. %*amples. -ou'l!.
stochastic matrices. Positive. and null.recurrence. %*istence of invariant distri'utions.
%r"odic theorem for MARKO ,hains.
Assignment # 3: Lawler, Problems # 1.9 - 1.11, 1.13, 2.2 2.5. Due on Tue. 7 October.

. Lecture #9: Tuesday, 30 September
Periodicit! for a MARKO ,hain. +he 3conver"ence to e1uili'rium3 theorem for irredu.ci'le)
aperiodic chains) via the couplin" method. +ime.reversal) the detailed.'alance e1uations.

. Lecture #10: Thursday, 2 October
-efinition and properties of conditional e*pectations. ,onditional e*pectation as least.
s1uares predictor. -efinition and 'asic properties of Martin"ales. +he -OO# decomposition.

. Lecture #11: Tuesday, 7 October
+he optional samplin" theorem) application to simple random walk and to the "am'ler(s ruin
pro'lem. Martin"ales associated with MARKO ,hains) harmonic functions4 connections with
recurrence and transience.
Assignment # 4: Lawler, Problems # 5.3 5.7, 5.9, 5.10. Due on Tue, 14 October.

. Lecture #12: Thursday, 9 October
#asic facts a'out martin"ale transforms. 5pcrossin"s ine1ualit!) the conver"ence theorem for
martin"ales.

. Lecture #13: Tuesday, 14 October
Martin"ale transforms4 invariance of martin"ales under stoppin". Optional samplin" theorem:
for 'ounded stoppin" times) for 'ounded martin"ales) other sufficient conditions. Application
of optional samplin" to the simple random walk.

. Lecture #14: Thursday, 16 October.
-efinition and properties of the #ROW$ian motion. Martin"ale and MARKO properties.
-istri'ution of the first.hittin".time to a certain level4 null recurrence. ,omputation of the
pro'a'ilit! of 6ero.crossin" durin" a "iven time.interval.
. Lecture #15: Tuesday, 21 October. MID-TERM EXAMINATION.

. Lecture #16: Thursday, 23 October
&am'ler(s ruin pro'lem for #ROW$ian motion4 martin"ale methods. Stron" MARKO propert!.
7uadratic variation of the #ROW$ian path4 variations of all other orders.
Assignment # 5: Read Chapter 8 in Lawler. Problems # 8.4 8.10. Not to be handed in.

. Lecture #17: Tuesday, 28 October
Stochastic inte"ral for simple processes4 elementar! properties) connections with martin"ale
transforms.

. Lecture #18: Thursday, 30 October
Stochastic inte"ral for "eneral adapted processes. Properties. I+O(s chan"e.of.varia'le rule.
%lementar! computations for stochastic inte"rals.
Assignment # 6: Due on Tue, 11 November.
1. Read Chapter 9 in Lawler, do Problems # 9.1, 9.2.
2. Oksendal, Problems # 3.2, 3.4, 3.5 (p.29); # 4.2, 4.3, 4.6, 4.7 (pp. 38-40); 5.1, 5.2, 5.10(I)
(pp. 54-57).
. Lecture #19: Tuesday, 4 November. UNIVERSITY HOLIDAY

. Lecture #20: Thursday, 6 November
ariants of the I+O formula. Applications. Ar"ument for the proof. +he heat e1uation4
fundamental solution) solution of a "eneral ,A5,89 pro'lem. Interpretation in terms of
#ROW$ian motion.

. Lecture #21: Tuesday, 11 November
ariants of the heat e1uation /coolin") heat.transfer0) interpretation of solutions in terms of
#ROW$ian motion. 2%9$MA$.KA, formula) the arc.sine law. -IRI,8:%+ pro'lem4 computation
of the e*pected e*it.time for multi.dimensional #ROW$ian motion in a 'all.

. Lecture #22: Thursday, 13 November
-efinition and properties of the POISSO$ process. &enerali6ations: pure.'irth process) 'irth.
and.death process. %*amples: M/M/1) M/M/k) and M/M/infinity 1ueues. %*ponential
distri'ution of the interarrival times. Some 'asic properties of the e*ponential distri'ution.
Assignment # 7: Read Chapter 3 in Lawler. Problems 3.1 3.5, 3.7 3.12. Not to be handed in.

. Lecture #23: Tuesday, 18 November
MARKO ,hains in continuous.time. Independence and e*ponential distri'ution of ;ump.
times) transition mechanism. +ransition pro'a'ilities) ,hapman.Kolmo"orov e1uations)
invariant measures.

. Lecture #24: Thursday, 20 November
Optimal Stoppin" for discrete.parameter Markov ,hains) and for #rownian motion /notes from
-!nkin < 9ushkevich0.
Assignment # 8: Read Chapter 4 in Lawler. Problems 4.1, 4.2, 4.6, 5.14. Due Tue. 2 December.
. Lecture #25: Tuesday, 25 November
-iscrete.time Markov ,hain em'edded in a ,ontinuous.time Markov ,hain) discussion of
recurrence and transience. #irth.and.death processes4 criteria for positive and for null
recurrence. %*amples: invariant measures for the M/M/1) M/M/k) and M/M/infinity 1ueues.
. Lecture #26: Thursday, 27 November. THANKSGIVING HOLIDAY

. Lecture #27: Tuesday, 2 December
Stochastic -ifferential %1uations) the 'asic e*istence and uni1ueness theorem of I+O. %*plicit
solutions for linear S-%(s. %*amples. $otion and computation of infinitesimal "enerator.

. Lecture #28: Thursday, 4 December
&IRSA$O(s +heorem: Motivation and proof.
. Lecture #29: Tuesday, 9 December
&IRSA$O(s +heorem: application to #rownian motion with drift) computation of the
distri'ution of first.passa"e.times.

. Lecture #30: Thursday, 11 December. FINAL EXAMINATION

You might also like