z.

zheng 1
Probability and Statistics
Prof. Zheng Zheng

z. zheng
2
Course Outline
Named as a National Quality Curriculum Taught
in English for International Students by the
Ministry of China
18 class lectures: Sept. 25, 2014-Jan. 22, 2015
Time: Thursday 9:00-11:45 am
Many sessions will be video-recorded this
semester
1 midterm (~1st week of Dec., TBD) and 1 final (last
week of Jan. 2015, TBD)
z. zheng
3
Lecturers
Experienced Teaching Team (3 Full:
Prof. Zheng Zheng zhengzheng@buaa.edu.cn (Probability)
Prof. Huaping Xu xuhuaping@buaa.edu.cn (Stochastic)
Prof.Yichuan Yang ycyang@buaa.edu.cn (Statistics)
Lecturer.Xin Zhao x.zhao@buaa.edu.cn (Probability)
Teaching Assistants/ Office Hour:
- Liya Shu: email: 403196123@qq.com 15652915285
- Guangnan Chen: email: 304055353@qq.com 13126732863
- Address: New Main building F703, Monday 19:00-20:00
z. zheng
Important Course Information (1/3)
Course website:
http://www.ee.buaa.edu.cn/oldeeweb/html/eng
lish/common/Excellent%20course.html

Lecture notes
Usually available a few days before each lecture.
Access them from the Lectures notes link of the
course website (to be updated).
4
z. zheng
5
Important Course Information (2/3)
Textbook: A. Papoulis, Probability, Random
Variables, and Stochastic Processes, McGraw-Hill
Press, 3rd Ed, 1991.

Credits: 3 credits (48 hrs)/ 2 credits option (32 hrs,
finishes by the mid-term exam) .

Grade Breakdown: homework 30% (50%); mid-
term 30% (50%) ; final 40%.
z. zheng
6
Ground Rules
- No iPad, laptop and mobile phones during the
exams (therefore, dont record the slides and
your notes only with your iPad)!
- Arrive on time, especially for sessions to be
recorded.
- Hand in your homework at the beginning of
each lecture.
Important Course Information (3/3)
z. zheng
7
Course Aims


Understand random outcomes and random
information.
Understand probability theory.
Learn random variables.
Know-how of random processes.
Learn how to analyze statistical
information.
7
z. zheng
8
Course: Table of Contents
1. Introduction
Concept of probability and
probability spaces
Elementary probability theory
Conditional probability and Bayes'
theorem
2. Repeated Trials
Combined experiments
Bernoulli trials
Poisson theorem
3. Random Variables and Distribution
Discrete and continuous random
variables
Functions of random variables
Joint and marginal distributions
Independence
4. Expectation
Mean, variance and covariance
Conditional distribution and
conditional expectation
Least squares estimation for
Gaussian random vectors

5. Limit theorems
Laws of large numbers
Central limit theorem

6. Statistics
Parameter estimation
Hypothesis testing

z. zheng
9
Why Study Probability and Statistics?
Decisions have impact on your life and in the world
What make decisions hard
Uncertainty (Randomness)
z. zheng
10
Why Study Probability and Statistics?
What is Uncertainty or Randomness ?
Operator will receive a call in next one hour
z. zheng
11
Why Study Probability and Statistics?
What is Uncertainty or Randomness ?
Variations in value of 10O resistor
Electromagnetic Interference
No. of users in network at any instant
Uncertainty is everywhere.
z. zheng
12
Why Study Probability and Statistics?
Can you give some example?
z. zheng
13
Why Study Probability and Statistics?
Can we measure uncertainty?
z. zheng
14
Why Study Probability and Statistics?
Probability and statistics are the language of
uncertainty.
We are able to explicitly include uncertainty
into decision making using probability.
Statistics provide data about uncertain
relationships.
z. zheng
15
Flaws of averages: average depth is 3 feet
Can Ignore Uncertainty in Decision-Making?
Average Depth
3 ft.
z. zheng
16
Engineering Decisions Involving Uncertainty
16
Civil Engineering
Design input
- Statistics of vehicles
- Max. no. of vehicles at an instant
- Weight of vehicles
Selection of Materials
- Analysis of Materials
Behavior under stress
Calculate chances of failure using
load statistics
z. zheng
Mobile Network Design
Uncertain No. of users
Random connection time
Use probability theory to calculate
capacity and cell size
Call dropping probability to ascertain
network quality
Random channels
- Indoor
- Outdoor
- Mobile users
Telecommunications
Engineering Decisions Involving Uncertainty
17
z. zheng
18
Signal/Image Processing
Engineering Decisions Involving Uncertainty
Noise Removal
Noisy Image
Clear Image
Noise is random.
Noise removal requires estimation of noise statistics.
Filter is designed based on probability theory.
z. zheng
19
Wind parameters
- Wind speed are time dependent
- wind direction
Requires statistical approaches for
location selection and design
Average power generation
Requirement analysis
Material selection
Wind Turbines
Engineering Decisions Involving Uncertainty
z. zheng
20
Industrial Engineering
Quality of product
Average length of life of light bulb?
Cant test all bulbs !!!
Use sampling technique and use
statistical method to determine expected
life.
Engineering Decisions Involving Uncertainty
z. zheng
21
Business Management
. and many more
Analyzing customer demands to
determine manufacturing, marketing
and prices etc.
Engineering Decisions Involving Uncertainty
z. zheng 22
What is probability?
One thing that makes decision hard is uncertainty.
Probability measures uncertainty formally, quantitatively.
Probability is the mathematical language of uncertainty.

What is probability of getting 5
at first attempt ?
22
z. zheng 23
What is Statistics?
Statistics are numbers that summarize
the results of a study/system.
Steps in statistics
- Collect data
- Organize or arrange the data
- Analyze the data
- Infer general conclusions
Date Jan Feb Mar Aprl
1 2 4 1 3
2 0 1 0 1
3 3 1 3 1
4 0 0 0 0
5 1 3 1 3
6 0 0 0 0
7 0 0 0 0
8 2 2 2 2
9 0 4 0 4
10 3 1 3 1
11 0 1 0 1
12 1 0 1 0
13 0 3 0 3
14 0 0 0 0
15 2 0 2 0
16 0 3 0 3
17 3 0 3 0
18 0 0 0 0
19 1 2 1 2
20 0 4 0 4
21 0 1 0 1
22 2 1 2 1
23 0 3 0 3
24 3 0 3 0
25 0 0 0 0
26 1 2 1 2
27 0 4 0 4
28 0 1 0 1
29 0 1 0 1
30 0 4 0 4
23
z. zheng 24
Link between Probability and Statistics
Model Data
Probability
Statistics
System
Prediction: looking forward
looking backward
Refine
Model
24
z. zheng 25
What is probability? (1/3)
Classical definition:
P(A)=N
A
/N
N: # of all possible outcome
N
A
: # of outcomes favor A
A priori
Equal Probability assumption
Example:
Balls in a box.

z. zheng 26
What is probability? (2/3)
Relative frequency definition:

N: # of experiments
N
A
: # times of As occurrence
A posteriori
Example:
Toss a coin, many, many times
n
n
A P
A
n
lim ) (

=
z. zheng
27
What is probability? (3/3)
Axiomatic definition:
Experiment under repeatable conditions
Elementary events , i
O consists of , i

A, B, C, are subsets of O, denoted as
e.g. implies

{ }
1 2
, , , , =
k

e A
. e
O c A
z. zheng 28
Set Theory--Definition
Set: a collection of elements

Subset: consists of elements that are also
element of the set A
Belong or not belong to a set:
Empty (null) set:
{ }
1 2 n
, ,. . . , = A
{ } 0/

i
A e ,
i
A e ,
A B c
z. zheng 29
Operations on Subsets
Union
Intersection
Complement
Transitivity Property: If and ,
A B
B A
A B A
B A
A
A
{ }
{ }
| or
| and
A B A B
A B A B
, , ,
, , ,
= e e
= e e
{ }
| A A , , = e
c A
B Ac
c B
z. zheng 30
More definitions
Mutually exclusive (M.E) or Disjoint
B A
| = B A
1
A
2
A
n
A
i
A
j
A
A partition of O: a collection of
mutually exclusive subsets of O
1
and
i j i
i
A A A
=
= C = O
z. zheng 31
De-Morgans Laws
A B A B =
A B
B A
A B
B A
A B
B A
A B A B =
A B
B A
A B
A B
A B
A B
z. zheng 32
Field
A collection of subsets of a nonempty set
forms a field F if



It is easy to show that etc.,
also belong to F. Thus
, , B A B A
{ }. , , , , , , , , 0 B A B A B A B A B A F =
z. zheng 33
Axioms of Probability
For any event A, the probability of A
assigned as P(A) satisfies:
i) P(A)0 (Probability is a nonnegative
number)
ii) P()=1 (Probability of the whole set is
unity)
iii) If AB=,then P(AB)=P(A)+P(B)
z. zheng 34
Some Basic Rules


for not M.E. A and B
). ( 1 ) or P( 1 ) P( ) ( ) P( A P A A A P A A = = + =
( )
0. P C =
? ) ( = B A P
A B A
B A
, B A A B A =
( ) ( ) ( ) ( ) P A B P A AB P A P AB = = +
( ) ( ) ( ) P B P BA P BA = +
( ) ( ) ( ) ( ) P A B P A P B P AB = +
Proof:
z. zheng 35
Conditional Probability
P(A|B) = Probability of the event A given
that B has occurred.
We define provided

This definition satisfies:


, if
,
) (
) (
) | (
B P
AB P
B A P =
. 0 ) ( = B P
0 ) | ( > B A P
1 ) | ( = O B P
) | ( ) | ( ) | ( B C P B A P B C A P + = . 0 = C A
z. zheng 36
Properties of Conditional Probability
If ,
If ,


if and
A B c
( | ) 1. P A B =
B Ac
( | ) ( ). P A B P A >

=
=
n
i
i i
A P A B P B P
1
) ( ) | ( ) (
| =
j i
A A O =
=

n
i
i
A
1
Total Probability Theorem
z. zheng 37
Bayes Theorem
P(A) represents the a priori probability of the event
A. Suppose B has occurred, and assume that A
and B are not independent. Bayes rule takes into
account the new information (B has occurred)
and gives out the a posteriori probability of A
given B.
( | ) ( ) ( | ) ( ) P A B P B P B A P A =
) (
) (
) | (
) | ( A P
B P
A B P
B A P =