CHAPTER 3

TWO-VARIABLE REGRESSION MODEL:

THE PROBLEM OF ESTIMATION

3.1 (1) Yi = /31+ /32% + ui . Therefore,

E(Yi IXi ) = /31+ /32% ui)
= flu + fi2X + E ), since the /3's are constants and X
is nonstochastic.
= /31+132X , since E(ui I Xi ) is zero by assumption.

(2) Given cov(uiui) = 0 for V for all i,j (i*j), then

cov(YiYi) = E Ulf; - E(Yi)][Yi - E(Y.i)] }
= E(ueui), from the results in (1)
= E(ui)E(uf), because the error terms are not
correlated by assumption,
= 0, since each ui has zero mean by assumption.

(3) Given var(ui\Xi) = cr2, var (Yi\Xi) = E[Yi - E(Yi)]2= E(ui2) =

var(ui\Xi) = a2, by assumption.

3.2 Yi Xi yi xi xiyi xi2

4 1 -3 -3 9 9
5 4 -2 0 0 0
7 5 0 1 0 1
12 6 5 2 10 4

sum 28 16 0 0 19 14

Note: Y = 7 and :f = 4

A A

Exy, 19
Therefore, /32 = —14 — 1.357; /31= Y-162 X = 1.572
EXt2
3.3 The PRF is: Yi = /31+ fl2Xi + ui
Situation I: flu= 0, /32 =1, and E(ud = 0, which gives E(YilX ) = Xi
Situation 2:131=1, fl2 = 0 , and E(ui)= (Xi - 1), which gives

E(Yi ) = Xi
which is the same as Situation 1. Therefore, without the assumption
E(u i) = 0, one cannot estimate the parameters, because, as just shown,
one obtains the same conditional distribution of Y although the
assumed parameter values in the two situations are quit different.

11
3.4 Imposing the first restriction, we obtain:
A A A

E w= E (Y - X31 - (2 Xi) = 0
1
Simplifying this yields the first normal equation.
Imposing the second restriction, we obtain:
A A A

E.,x; = E [(Yi — 131 - 132 Xi)Xi ] = 0

Simplifying this yields the second normal equation.
The first restriction corresponds to the assumption that E(ui\X;) = 0.
The second restriction corresponds to the assumption that the
population error term is uncorrelated with the explanatory variable
Xi, i.e., cov(uiXi) = 0.
3.5 From the Cauchy-Schwarz inequality it follows that:
E(XY)2 <1
E(X2)E(Y2) —
I (xiyi)2
Now r2— _______________ 1, by analogy with the Cauchy-Schwarz
Exi2 E.142
inequality. This also holds true of p2 , the squared population
correlation coefficient.
3.6 Note that:
E xiyi E xiyi
13 yx =__________ d )6 .t y = ___
E Xi an
E yi2

Multiplying the two, we obtain the expression for r2, the squared
sample correlation coefficient.
A A

3.7 Even though )6 yx . )6 xy =1, it may still matter (for causality and
theory) if Y is regressed on X or X on Y, since it is just the product
A A

of the two that equals 1. This does not say that flyx = 13 xy .
i

., — = X =n +1
3.8 The means of the two-variables are: 2 and the

correlation between the two rankings is:

r—
xyiI (1)
Exi E yi 2 2

where small letters as usual denote deviation from the mean values.
Since the rankings are permutations of the first n natural numbers,

12
 2
E xi 2 =Ex2 ( x) _
_ n(n +1)(2n +1) n(n +1)2 n(n2 —1)
n 6 4 12
and similarly,
y,2 = n(n —1) _
2

12 , Then
2
d2 =E(x,- Yi) = E(yy,2
2n(n +1)(2n +1)
2
2E
6 d
Therefore, E X Y = n(n+_ 1)(2n
6
+1)
2
(2)

EXEY
Since Ex y , = XE , using (2),
n we obtain

E d2
(3)
2
d2
n(n + 1)(2n +1) n(n +1)2 n(n2 _1)
3 2 4 12
Now substituting the preceding equations in (1), you will get the answer.

A A A A-

3.9 (a) fli = Y — fi2Xi and al = Y — /32 x [Note: xi = (Xi - X )]

= Y, since V n
xi=„

A
x2 E 2
E A Xl2 a
var( flu ) — ________________ a and var( al) — _______ a
2 2

nExi2 nExi2 n

Therefore, neither the estimates nor the variances of the two

estimators are the same.

Exiy , E xiyi
A

(b) /32 = ______ and en = ____, since xi = (Xi - yV )

13
Exi2 Ex,2
 It is easy to verify that var( /32) = var( a2) —
x,2

That is, the estimates and variances of the two slope estimators are
the same.

(c) Model II may be easier to use with large X numbers, although

with high speed computers this is no longer a problem.

3.10 Since E x, = Eyi = 0 , that is, the sum of the deviations from mean

value is always zero, x = y = 0 are also zero. Therefore,

A A-

= y - )32 x = 0. The point here is that if both Y and X are

expressed as deviations from their mean values, the regression line

will pass through the origin.

A E(x,-x)( yi-y) E x,y,

fl2 = _____________________ , since means of the two
E Xr 2

E(x, - X) 2

variables are zero. This is equation (3.1.6).

3.11 Let Zi = aXi + b and Wi = cYi + d. In deviation form, these become:

zi = ax; and wi = cyi. By definition,

Eziwi acExiyi
r2 _____________________________ — r1 in Eq.(3.5.13)
zit E wi aC 2
xi2 E yi 2

3.12 (a) True. Let a and c equal -.1 and b and d equal 0 in Question 3.11.

14
 (b) False. Again using Question 3.11, it will be negative.

(c) True. Since rxy = ryx > 0, Sx and Sy (the standard deviations of X
and Y, respectively) are both positive, and ryx = fly Sx,— and rxy =
Sy
Sy
flxy— , then f3xy and flyx must be positive.
Sx

3.13 Let Z = X1 + X2 and W = X2 and X3. In deviation form, we can write

these as z = x1 + x2 and w = x2 + x3. By definition the correlation
between Z and W is:
Ez,w, E(xi+ x2)(x2+ x3)
rte,
I s l E z i 2 E w i t
2
E(xi+x2) E (x2 + x3)2
Ex22
5 because the X's are

\ I(E xi2 + x22)( x2 2 + x32)

uncorrelated. Note: We have omitted the observation subscript for
convenience.
2
a
________________ — —1, where a2 is the common variance.
V(20.2 + 20.2) 2
The coefficient is not zero because, even though the X's are
individually uncorrelated, the pairwise combinations are not.
As just shown, Ezw. a-2 , meaning that the covariance between z

and w is some constant other than zero.

3.14 The residuals and fitted values of Y will not change. Let
Yi = + fl2X w and Yi = al + a2Zi +ui , where Z = 2X
Using the deviation form, we know that
A ExY
) 62 — ___ , omitting the observation subscript.
x2

2Exiyi

4E xi2

15
 fli= Y — 1 62 X; al= Y —a2 Z= (Note: Z = )
That is the intercept term remains unaffected. As a result, the fitted
Y values and the residuals remain the same even if Xi is multiplied
by 2. The analysis is analogous if a constant is added to Xi.

3.15 By definition,
2

(Eyo)i)2 [E(Pi-Fu'i)(.vi) E.A yi2

Eyi 2 xEs „2 ) ( E y ,2 ) ( E s , i 2 ) yi 2

E02x02 j322Ex,2
since E.piii =o. Eyi2 E yi2 — r2, using (3.5.6).

3.16 (a) False. The covariance can assume any value; its value depends
on the units of measurement. The correlation coefficient, on the
other hand, is unitless, that is, it is a pure number.

(b) False. See Fig.3.11h. Remember that correlation coefficient

is a measure of linear relationship between two variables. Hence,
as Fig.3.11h shows, there is a perfect relationship between Y and
X, but that relationship is nonlinear.

(c) True. In deviation form, we have

yi = +
Therefore, it is obvious that if we regress y, on the slope
coefficient will be one and the intercept zero. But a formal proof can
proceed as follows:
If we regress yi on 5,i , we obtain the slope coefficient, say, as: a
yiyi ftEx, A32
______ =
a = ___________ , y 722 = 1 ' because
-2 /3 2 xi
E 2 p yi = and E xi = /3 E x2 i for the two-variable
/3^Xt
model. The
intercept in this regression is zero.

3.17 Write the sample regression as: Y, = ;131 + it,. By LS principle, we

want to minimize: E elf 2 = E (Y, — )6'1)2 . Differentiate this equation

16
 with the only unknown parameter and set the resulting expression to
zero, to obtain:
d(1^1
') 2I(Y, —A)(-1)= 0
d
which on simplification gives h., = Y ,that is, the sample mean. And
2
a
we know that the variance of the sample mean is where n is the
-11- ,

n
sample size, and .2 is the variance of Y. The RSS is
RSS 31,2.
E0,1_17)2 = E yi2 and 6.2 = ________________. It is worth adding the
(n —1) (n —1)
X variable to the model if it reduces Q2 significantly, which it will if
X has any influence on Y. In short, in regression models we hope
that the explanatory variable(s) will better predict Y than simply its
mean value. As a matter of fact, this can be looked at formally.
Recall that for the two-variable model we obtain from (3.5.2),
RSS = TSS - ESS
= EYF
=
Therefore, if A
is different from zero, RSS of the model that
contains at least one regressor, will be smaller than the model with no
regressor. Of course, if there are more regressors in the model and
their slope coefficients are different from zero, the RSS will be much
smaller than the no-regressor model.

Problems

3.18 Taking the difference between the two ranks, we obtain:

d -2 1 -1 3 0 -1 -1 -2 1 2
d2 4 1 1 9 0 1 1 4 1 4 ;E d 2 = 26

Therefore, Spearman's rank correlation coefficient is

6Ed 2
6(26)
rs =1 _______ =1 ________ — 0.842
n(n2 1)— 1 0(1 02 -1)
Thus there is a high degree of correlation between the student's
midterm and final ranks. The higher is the rank on the midterm, the
higher is the rank on the final.

3.19 (a) The slope value of -4.318 suggests that over the period 1980-
1994, for every unit increase in the relative price, on average, the
(GM'S) exchange rate declined by about 4.32 units. That is, the

17
 dollar depreciated because it was getting fewer German marks for
every dollar exchanged. Literally interpreted, the intercept value of
6.682 means that if the relative price ratio were zero, a dollar would
exchange for 6.682 German marks. Of course, this interpretation
is not economically meaningful.

(b) The negative value of the slope coefficient makes perfect

economic sense because if U.S. prices go up faster than German
prices, domestic consumers will switch to German goods, thus
increasing the demand for GM, which will lead to appreciation of
the German mark. This is the essence of the theory of purchasing
power parity (PPP), or the law of one price.

(c) In this case the slope coefficient is expected to be positive, for

the higher the German CPI relative to the U.S. CPI, the higher the
relative inflation rate in Germany which will lead to appreciation
of the U.S. dollar. Again, this is in the spirit of the PPP.

3.20 (a) The scattergrams are as follows:

120

100_

80_
8

0°

0
0

20.. °
8
..°°°°°
0
40 do do 100

PRODBUS

120_
100._

80..

03
U-
0
0
60_

40_

20_
0
40 do 100 lio
PRODNFB

18
 (b) As both the diagrams show, there is a positive relationship
between wages and productivity, which is not surprising in view
of the marginal productivity theory of labor economics.

(c) As the preceding figures show, the relationship between wages

and productivity, although positive, is not linear. Therefore, if we
try to fit a straight line regression model to the data we may not get
a good fit. In a later chapter we will see what types of models
are appropriate in this situation. But if we routinely fit the linear
model to the data, we obtain the following results.

Wagebus = -109.3833 + 2.0039 Prodbus

se = (9.7119) (0.1176) r2 = 0.8868

Wagenfb = -123.6000 + 2.1386 Prodnfb r2 = 0.8777

se = (11.0198) (0.1312)

where bus = business sector, nth = non-farm business sector

prod = productivity as measured by output per hour and wage =
compensation per hour.
As expected, the relationship between the two is positive.
Surprisingly, the r2 value is quite high.

3.21 Ey, Ex, Ex,y, Ex,2 y2

Original data: 1110 1700 205500 322000 132100
Revised data 1110 1680 204200 315400 133300
Therefore, the corrected coefficient of correlation is 0.9688

3.22 If you plot these variables against time, you will see that generally
they have moved upward; in the case of gold there is considerable
price volatility.

(b) If the hypothesis were true, we would expect /3 2

(c) Gold Price r = 186.183 + 1.842 CPI t se =

(125.403) (1.215) r2 = 0.150

NYSEt = -102.060 + 2.129 CPIt

se (23.767) (0.230) r2 = 0.868

It seems the stock market is a better hedge against inflation than

gold. As we will see in Ch.5, the slope coefficient in the gold price
equation is not statistically significant.

19
 3.23 (a) The plot is as follows, where NGDP and RGDP are nominal and
real GDP. 10000 8000

8000

6000

4000

2000
4000

2000
(b)
NGDPt = I _ NGDP RGDPI - 986.3317 +
201.9772 time

se = (1907.715) + 128.7820 r2 = 0.9277

RGDPt = 1907.715 + 128.7820

se = (45.1329) ( 1.9666) r2 = 0.9914

(c) The slope here gives the rate of change of GDP per time period.

(d) The difference between the two represents inflation over time.

(e) As the figure and regression results indicate, nominal GDP has
been growing faster than real GDP suggesting that inflation has been
rising over time.

3.24 This is straightforward.

3.25 (a) See figure in Exercise 2.16 (d)

(b) The regression results are:

fc = —198.126+1.436X,
se= ( 25.211) (0.057)

r2 = 0.966
where Y= female verbal score and X = male verbal score.

(c) As pointed out in the text, a statistical relationship, however

strong, does not establish causality, which must be established a
priori. In this case, there is no reason to suspect causal relationship
between the two variables.

20
3.26 The regression results are:

fc = —189.057 +1.285X,
se= ( 40.927)(0.082)
r2 =0.918
3.27 This is a class project.

21