Coding Theory and Cryptography, Homework 1

Sonam Kumar September 3, 2012

Solution 1
Three fair dice are thrown. X
1

,X
2

,X
3

are the values taken by the 1st, 2nd, 3rd dice. Since the dice are all fair, thus probability of getting i {1,2,3,4,5,6} is p
i

= P(X = i) and p
1

...p
6

= 1/6. If p
i

denotes the probability of the 1st dice, q

j

of the 2nd and r

k

of the 3rd, we have p

ijk

=p
i

q
j

r
k

, because tossing 3 dice is mutually independent. Then, H(X

1

,X
2

,X
3

)= p
ijk

.logp
ijk

= i
j k i

p
ijk

.logr
k

Here,

p
ijk

.logp
i

p
ijk

.logq
j

p
ijk

.logp
i

= ( p
ijk

)logp
ii j k

= i
j k i j

)logp
i

= ( p
ij

). Similarly, p
i

logp
i

= H(X
1

i j k j

). p
ijk

.logq
j

= p
j

.logq
j

= H(X
2


i j k k

). Therefore, H(X
1

p
ijk

.logr
k

= p
k

.logr
k

= H(X
3

,X
2

,X
3

)= p
i

.logp
i

p
j

.logq
j

p
k

.logr
ki j

k = H(X
1

) + H(X
2

) + H(X
3

). 2 1

, Now, X = X
1

+X
2

+X
3

. Let
i

,
j

are the values taken by the random variables X

1

,X
2

,X
3

. Then, p
c

= P(X = c) = = P(X 1 +X
2

+X
3

= c)
( i

P(X
1

=
i

,X
2

=
j

,X
3

=
k

) =
+

j + k =c) ( i

P(X
1

=
i

)P(X
2

=
j

)P(X
3

=
k

) =
+

j + k =c) (

p
i

q
j

r
k

Now, H(X) =
+ j + k =c)

p
c

.logp
c

=
c

( i

p
i

.q
j

.r
k

.log(p
i

p
j

) =
+ j + k =c) ijk

p
i

.q
j

.r
k

.log(p
i

p
j

p
k

)
i

p
i

.q
j

.r
k

.log(p

p
j

p
k

) =
i j k i j k i j k

p
i

.q
j

.r
k

.log(p
k

) = p
i

.q
j

.r
k

.log(p
i

) p
i

.q
j

.r
k

.log(p
j

) p
i

.logp
i

q
j

.logq
j

r
k

.logr
ki j

k = H(X
1

). 2

Solution 2
X, Y are discrete random variables. Without loss of generality, lets assume a
1

) + H(X
2

) + H(X
3

Xa
n

and b
1

Yb
m

. Then, we can legally say: H(X + Y |X) =

(a i

)) = P(X + Y = a
i

+b
j

|X = a
i

).log(P(X + Y = a
i

+b
j

|X = a
i

,b j )

(a i

)) But then, P(X = x, Y = y|X = x) = P(Y = y). Therefore, H(X + Y |X) = P(X = a
i

,Y = b
j

|X = a
i

).log(P(X = a
i

,Y = b
j

|X = a
i ,b j )

(a i

P(Y = b
j

|X = a
i

).log(P(Y = b
j

|X = a
i

))
,b j

) = H(Y |X) 2 2

Example: We need a function g on two variables X, Y taking finite number of values, such that H(g(X, Y )|X) = H(Y |X). Let X, Y be independent random variables taking values 0,...,(n 1) with probabilites n 1 . Let g(X, Y ) = XY . Then, H(g(X, Y )|Y ) = = H(XY |X)
i

H(XY |X = i).P(X = i),0 i n 1 = H(XY |X = 0)P(X = 0) + ... + H(XY |X = n 1)P(X = n 1) = H(XY } |X = 0) n1 {{ 1 n } = + ... + H(XY |X = n 1) 1 n1 n
i=0

H(XY |X = i) = n1 P(Y |X) = P(Y |X) 2

Solution 3
X( n is a random variable taking values 0,1,...,n with probabilites P(X = k) = k ) pkqnk, with 0 <p< 1 and q = 1 p. Then, H(X) = n
k=0

P(X = k).log(P(X = k)) = ( nk

) pkqnk.log( ( nk ) pkqnk) = n
k=0 n

( n
k=0

k ) pkqnk.log(pk) ( nk ) ) = nplogp n ( n
k=0

k ) pkqnk.log(qnk) ) n
k=0

( nk ) pkqnk.log(
pkq(n1)k

n1

( n( k 1 ) pkq(n1)k nqlogq
n1

n1k n ( n
k=0 k=0 k=0

k ) pkqnk.log( ( nk ) ) ) = nplogp.(p + q)n1 nqlog(q). (p + q)n1 ) = nplogp.1 nqlogq.1 n ( n ( k ) pkqnk.log( nk n (

n
k=0

k )
( k=0

) pkqnk.log( ) ) n{plogp + qlogq} pk,qnk, nk 0 0 k n. 2 3 ( nk

Solution 4
X is a random variable taking values v
1

,...,v
n

. We are given that E[X] = and X is a random variable of maximum entropy. Let e
i

Now, = ev
i

n
i=1

eSi . E[X] = ,S = e
i

,q
i

= n
i=1

v
i

= p
i

n
i=1

) E[X] E[ln(S)] = p
i

ln(e

n
i=1

ln(S)} = p
i

{ln(e
i

)p
i

n
i=1

eSi ) = p
i

ln( n
i=1

) p
i

ln(q
i

n
i=1

) = H(X) H(X) + E[ln(S)] We see p

i

ln(p
i

n p
i

ln(q
i

) n p
i

ln(p
i

), because X has the maximum entropy, and the equality i=1 holds iff q
i=1 i

i. Thus, p
i

ev i= =p
i

n ev
i

, and A = n ev i,a = 1. i=1 2

i=1

Solution 5
Given n n board. We go through the following algorithm: Suppose that the squares on the board are numbered from 1 through n2 in some suitable fashion, and only the player who has put the Queen knows the method of numbering. Now we ask questions, which can be answered in Yes or No. If the answer is Yes, remember 1 and remember 0 if the answer is No. Now, collect all these 0,1 bits, and see which square it is numbered by. This takes log
2

n time. The questions can be placed as nodes in a binary tree. The depth of the tree is the lower bound for the algorithm. It is log
2

n. log
2

n T(n) log
2

n 2 5