Professional Documents
Culture Documents
Protocols
1. Introduction
Carl A. Sunshine
Information Sciences Department, The Rand Corporation,
.1700 Main Street, Santa Monica, California 90406, USA
Yogen K. Dalai
Xerox Systems Development Department, 3408 Hillview
Avenue, Pal6.Alto, California 94304, USA
Transport protocols are designed to provide t'uJ.]y reliable
communication between processes which must communicate
over a less reliable medium such as a packet switching network (which may damage, lose, or duplicate packets, or
deliver them out of order). This is typically accomplished by
assigning a sequence number and checksum to each packet
transmitted, and retransmitting any packets not positively
acknowledged by the other side. The use of such mechanisms
requires the maintenance of state informatiol: describing the
progress of data exchange. The initialization and ~,~ahltenance
of *his state information constitutes a connection between
the two processes, provided by the transport protocol
programs on each side of the connection. Since a connection
requires significant resources, it is desirable to maintain a
connection only while processes are communicating. This
requires mechanisms for opening a connection when needed,
and for closing a connection after ensuring that all user data
have been properly exchanged. These connection management procedures form the main subject of this paper. Mechanisms for establishing connections, terminating connections,
recovering from crashes or failures of either side, and for
resynchronizing a connection are presented. Connection
management functions are intimately involved in protocol
reliability, and if not designed properly may result in deadlocks or old data being erroneously delivered in place of
current data. Som~ protocol modeling techniques useful in
analyzing cennecrion management are discussed, using
verification of connection establishment as an example. The
paper is based on experience with the Transmission Control
Protocol (TCP), and examples throughout the ~aper are
taken from TCP.
Carl Sunshine received a PhD in computer science from Stanford Utliversity in 1975 where he worked on
analysis, design, and implementation
of communication pratocols for computer networks. Since 1975 he has
been with the Rand Corporation,
where he is involved in research on
computer network protocols, network interconnectioa network planning, distributed systems, and operating systems. Dr. Sun~;hine is active in
IFIP TC6.1, the Internetwork Working Group.,
Yogen K. Dalai received his BTech
degree in Electrical Engineering from
the Indiap Institute of Technology,
Bombay, in 1972, and his MS and
PhD degrees in Electrical Engineering
from Stanford Univ,~rsity in ~973
and 1977 respectively. He is currently with the Xerox Systems Development Department in Palo Alto
working on the design, analysis and
implementation of computer communication protocols, and the interconnection of computer networks. His research interests also
include local networks, distributed system~ architecture,
packet switching~ broadcast protocols and operating systems.
Dr. Dalai is a member of the ACM and IEEE.
l
OPEN
% /
Opening
packet
exchange
tain a connection only while processes are communicating, and then to terminate the connection and free
the resources when the processes are done. This leads
to mechanisms for opening a connection when
needed, and for closing a connection after insuring
that all user data have been properly exchanged.
In opening and then closing a connection, it is
convenient to describe the protocol machines as going
through a number of states. Each machine starts in
the NoMctive state in which no connection exists.
On command from the user process, a machine may
actively initiate connection establishment (Opening
state), or passively wait for connection establishment
to be initiated from the remote end (Listening state).
Once the connection is established (Established
state), either user may terminate it, placing the
protocol machine in a Closing state. Once the connection has been closed, the protocol machine is in the
NotActive state again. Figure 1 illustrates the lifecycle of a connection as it passes from one: state to
another. The Opening and Closing states may in
reality consist of several substates depending on the
protocol used for accomplishing the desired effects.
These connection management procedures fo~m
the main subject of this paper. They include procedures for dealing ~ith crashes or failures of either side
of a connection as well as opening and dosing
connections under normal circuinstances. This paper
does not discuss the multiplexing function of
NotActive
.,
packet
exchange
LISTEN
',,.L~
Closing
Listening
....
packet
exchange
CLOSE
~I
45.5
Established
I. . . .
Fig. 1. Comlcctionlife-cycle.
456
transport protocols which is independent of connection management functions for individual connections. Connection management functions are intimately involved in protocol reliabiiity, and if not
designed properly may result in deadlocks or old data
being erroneously accepted in place of current data.
This paper is based on our experience with the
Transmission Control Program (TCP) [5,7,8]. Our
efforts in designing TCP have resulted in continuing
changes to the original lnternetwork Protocol
described by Cerf and Kahn [4]. We will use TCP
to designate the class of transport protocols based
on the Intemetwork Protocol. Lessons are drawn
from various stages of TCP development throughout the paper, so the reader is cautioned that some
examples do not reflect the current TCP. In particular, the formal analysis in Section 6 is based on
an early version of TCP. We will use TP and TCP to
stand for both the protocol and the program that
implements it.
TCP wzs designed with strong "worst case"
assumptions about the underlying transmission
medium. In particular it was assumed that packets
could be damaged, lost, duplicated, and delivered out
of order, with a widely varying delay. All of these
events are known to occur through various "natural"
causes in packet switching networks, ltowever, TCP
was not designed to solve the additional problems
imposed by "malicious intruders" [15] which require
procedures involving encryption. In considering
connection management mechanisms within a TP, it
will be helpful to use examples of the dialogue
between two TPs. All of our examples will be drawn
from TCP, but similar scenarios exist in other
transport protocols. These dialogues consist of the
exchange of packets between TCPs A and B and will
be illustrated as follows:
Each packet contains a sequence number, some
optional control information, an optional acknow.
ledgement, and some optional data, represented in
the following notation:
( Seq #)(Control )( Ack #)(data ).
Each line of the dialogue consists of a packet label in
pareatheses followed by the activity at A where
"<- -" signifies the packet being received at A, "- ->"
sign,.'fies ~he packet being transmitted at A, " ..... "
signifies that A is unaware of the packet at that time,
and ' T ' indicates no activity. Next appears a descrip.
tion of the packet in the notation described earlier.
Lastly, the activity at process B is described, where
2.Connection establishment
Since tiansport protocols use sequence numbers to
validate at, lying packets (check for duplicates and
correct order), one of the main function of connection establi'~hment is to properly initialize sequence
number parameters used by the protocol. Each TP
has a Next Sequence Number (NSN) that it will assign
to the ne'~t (new) packet transmitted, and an
Expected Sequence Number (ESN) which it expects
to see on the next arriving packet. NSN must be
initialized to some Initial Sequence Number (ISN)
when a connection is started, and ESN in the
opposite TP must be set equal to ISN. Sequence
numtiers may be assigned to various size units of
information from a full packet to a bit. In TCP, each
octet is assigned a sequence number, and each packet
carries the sequence number of the first octet
contained and a count of the number of octets in the
packet. This facilitates fragmentation of packets at an
intermediate point and their later reassembly at their
destination as described in [4].
Many transport protocols employ the simple
(1)
(2)
(3)
(3) 2
(4)
OPEN
LISTEN
Opening
Listening
- ->
I
<-- ->
- ->
<- CLOSE
457
- .>
accept
<-.....
- ->
<--
Not active
CLOSE
Not active
I
I
I
I
I
I
OPEN
LISTEN
Opening
Listening
(5)
- ->
(6)
(7)
(3) ~
(8)
<-- ->
.....
<-I
(Seq 0 )( Ark 3 )
(Seq 3)(data JK )
(Seq 3 )(data DE )
.',Seq 0 )(Ack 5 )
delayed
retransmission
terminate connection
- ->
accept
<-.....
new data delayed
- ->
ola duplicate arrives
~-old data accepted
458
A
OPEN
(I)
Opening
pick ISN = x
- ->
(2)
<--
LISTEN
Listening
I
( Seq x )( SYN )
(3)
(4)
(Ack x + 1)
t
<..
set ESN = x + 1
<.pick ISN = y
<Seq y )(SYN )
set ESN = v + 1
- ->
Established
[
<Seqx + 1 )(Ack y + 1)(data AB)
Established
Listening
Listening
(1)
.....
(2)
<-discard
set ESN = z + 1
(Ack z + 1 )
I
pick ISN = y
I
<--
(4)
set ESN = y + 1
. .>
(Seq y )( SYN )
t
(Ack y + 1 )
I
(5)
pick ISN : x
. .>
<-discard
Established
I
(Seq x )(SYN)
dis,:ard
I
(6)
oM connection request
(Seq z)(SYN )
(3)
459
(Seq y + l )(Ack z + l )
I
Fig. 4. Deadlock with simpleestablishment.
460
(1)
OPEN
Opening
LISTEN
I,istening
pick ISN = x
--~>
(Seq.0)(SYN)
!
I
remetnber x
pick ISN = y
t,
(Seq y)(SYN)(Ack x + 1 )
(2)
I
I
set ESN = y + 1
Established
I
(3)
-->
(Seq x + 1 )(Ack y + 1 )
--~,
set ESN = x + 1
Established
I
I
A
(1)
Not active
.....
B
Listening
(Seq z)(SYN)
I
I
(2)
<-invalid
(3)
.->
I
Not active
remember z
pick ISN --- y
(Seq y)(SYN)(AcK z ~ 1 )
I
( Seq z + 1 )(RST )( Ack y + l )
reject SYN
Listening
Fig. 6. Rejections of old SYN packet with 3 way handshake.
461
Forbidden Zone
seq # s
~~:~'i~i!
res y nchroniz
!i .i! .
.-- "~4!~::"%Ii
....----"
2S
~'::i'.:.~'.:~';~.
_ _LZ
~.i~I~:i ? ~ "":
~'~!."~
actual
~~:::H
I : ' , ! .r..,:
i l I...'F.,'..J.
='.':L, ." "-"
~.i."-ransmission rate i ~ t ~ ~
,.--I"
,,
!~;!!_:!
I
~!!
time t
,.
462
............
I
L
~....,...[......
m
sl~
Sa
seq # s
S1
sO
II
actual ~ransmission
time t
i!0
tb
t1
(1)
l.:stablislaed
Established
I
CLOSE
(2)
Closing
- ->
(3)
<-Not
data in transit
,1...
463
A requests termination
Closing
B complies
Not active
464
9). I f full delivery is important, the high level protocol must wait to close the connection until all data
has been successfully transmitted. More seriously, if
the initiating TP times out it can not be certain
whether the other side has received the FIN and the
reply was lost, or whether the other side never
received the FIN and is still open.
4.4. Acknowledged FIN exchange
E~t'.'..,tshed
(1)
Established
(Seq 29 )(Ack 78 )( data HIJKL )
(3)
(4)
A requests termination
Closing
(Seq 78 )(FIN)(Ack 35)
( Seq 35 )(Ack 79 )
Not active
data in transit
I
I
('l,~s: rt[,
(2)
,,.I.
B complies
Not active
Fig. 10. Connection termination using acknowledged FIN exchange.
Established, CLOSE
(1)
(2)
Established, CLOSE
(Seq 29 )(FIN )(Ack 78 )
(Seq 78 )( FIN )(Ack 29 )
,.==o
Closing
(2)
(1)
465
*..H
Closing
( Seq
(Seq 29
( Seq 30
( Seq
78 )(FIN )(Aek 29 )
)( FIN )(Ack 78 )
)(Aek 79 )
79 )(Ack 30)
Not active
<~--
Not active
Fig. 11. Simultaneous exchange of confirming FINs.
5. Failure recovery
The protocal mechanisms described so far adequately manage connections as long as tile protocol
machines on each side continue to function normally.
Unfortunately, hosts and their TPs occasionally crash
with loss of state intormation. When a host fails and
subsequently restarts, all knowledge of TP connections is lost. Connections which were established
become half open with file failed si~i~ thinking they
are not active, while the other side believes the
connection is still established. Half open connections
Established
Established
(Seq 29 )( Ack 78 )( data ABCDE)
(I)
-->
(2)
Closing
- ->
I
I
CLOSE
I
(3)
<--
(4)
- ->
(Seq 78 )(FIN)(Ack 35 )
(Seq 35 )(Ack 79)
...,,
Not active
(3)1
.....
I
I
466
B
I
crash
OPEN
Established
Opening
(1)
(2)
( Seq 6 )(SYN )
( Seq 300 )( Ack 100)
I
rejected
(3)
I
I
(1) 2
believable
Not active
( Seq 6 )( SYN )
..o.a
B
I
crash
Not active
(1)
Established
(2)
I
(Seq 100)(RST)(Ack 302)
believable,
Not active
Not active
Fig. 14. Half-open connection discovery and reset.
467
468
"
--
NotActive
SYN arrives
OPEN
SYN arrives ~
@,
set
set timeout
timeout
or Retry
I
~,
SYN-Received 1
K-~
~"~ SYN-Sent
I
ACK arrives
sOb'ished V
469
Event
Next
State
SYN
self
SYN
self
SYN-ACK
self
Data
self
ACK
ES
RST
NA
ACK, RST
self
OPEN
self
Retrans
Quit
Retry
self
self
self
470
~SSx)(NA)(SYNx,R~(x)} . . . . . . . . . . .
"# T #~
-i~
(SSx.~(SRy)(SYNx,SYN.ACKy(Z),RST(x),RST(v))
-II, T 1,,,.
,,,,I,
(SSx)(SRy)(SYNx,SYN. ACKy(z),RST(Y))
,.1'
(NA)(SRy)(SCY~ACK, ,(Z))
. . . . . . . .
4...
,I...
(SSx~(SRy)(SYNx,SYN- ACKy(z),RST(x))
-,T,-
) (NA)(NA)0
(SSx)(SSy}(SYNx,SYNy) (""-'--"
=
~ (S':x)(NA)(SYNx) (
(SSx)~SR,, ) (SYNx,SYN- ACKy(X))
(ES)(~;Ry)(SYN. ACKy(x),ACK(Y))
x-'>
(ES)(ES)O
y(,- (,-.y
x -,~ -,->x
Process~IProcess~/Current\
(NA) NOT ACTIVE
(SSx) SYN SENT WITH SEQ # x
.($Ry) SYN RECEIVED AND A
SYN SENT WITH SEQ # Y
(ES) ESTABLISHED
X ~ incoming(expected) seq. #
Y ~ outgoing seq. #
7. Conclusions
Connection management functions are intimately
involved in transport protocol reliability and require
careful design if errors are to be avoided under unusual
but possible circumstances. Special care must be
taken to avoid the possibility of confusing packets
from different connections between the same pair of
processes since old packets may persist in the network and be delivered during later conne,~tlons. This
requires a unique identitier for each new connection,
or careful selection of the initial sequence number
(ISN) to be used. A clock may be used to reliably
determine the ISN even after protocol failures, but
clock-based schemes require resynchronization of the
connection at certain points. The 3 way handshake
provides a means for reliably synchronizing the two
sides of the protocol to establish a connection while
minimizing the amount of state information that
must be maintained for inactive connections. This
involves the exchange of special synchronization
control packets (SYN) at the start of a connection.
Control packets (FIN) may also be used to
terminate a connection. In a graceful closing, the
protocol guarantees that all data sent before the close
command is issued will be delivered. In an immediate
closing, data in transit may or may not be delivered,
but both sides know that the connection has
terminated. A unilateral closing or abort does not
guarantee that resources on the other side of the
connection have been released. Halt'open connections
resulting from failure of on side of the protocol can
also be terminated by an exchange of control packets.
While infonnal analysis of scenarios is a very useful
tool for designing connection management mechanisms, it is not adequate for a fully reliable analysis.
More precise models defining the detailed operation
of the protocol machines on each side of the connection must be developed for this purpose. One such
model specifying the machines as f'mite automata
augmented by context information is presented and
471
Acknowledgements
The development of the Transmission Control Protocol
which provided the basis for this work was carried out while
the authors were at the Stanford University Digital Systems
Laboratory, supported in part by the Defense Advanced
Research Projects Agency (under contract number MDA90376C-0093) and by the National Science Foundation Graduate
Fellowship Program. Vint Cerf and Robert Kahn provided
many of the original ideas behind TCP and Vint Cerf has
been a constant participant in discussions on improvements.
The Cyclades research group under Louis Pouzin also contributed many early ideas through discussions i~. IFIP TC 6.1.
Ray Tomlinson suggested several important modifications
including the 3 way handshake. Bili Plummer, Dick Karp, Jim
Mathis, and Bob Metcalfe also contributed significantly to
the work. The continuing support of TCP development by
ARPA has made this wr.,rk l~ossible.
Appendix
The following relationships between the clock parameters
presented in section 5 are useful in analyzing resynchronization.
S = 32 bits
(1)
(2)
C = 2PD sec
(3)
472
numbers is
s=bt.
Hence the point of intersection gives t = R.
bR = B(R - C + L)
Example parameters
A suitable choice of the various parameters is
R =(C-L)I(1-b/B)
= 00 when b - B
= C - L when b = 0
(4)
(5)
(6)
D=
14 bits,
1 sec,
q =
1~8bits,
L~30secs.
References
[ 1] D. Belsnes, Single Message Communication, IEEE Trans.
on Communication, Vol 24, No 2, February 1976,
pp. 190-194.
[2] G.V. Bochman, Logical Verification and Implementation of Protocols, Proc. Fourth Data Communications
Syrup., Quebec, Canada, October 1975, pp. 7.15-7.20.
IEEE 75CH 1001-7DATA)
[3] G.V. Bochmann and J. Gecsei, A Unified Method tbr
the Specification and Verification of Protocols, Proc,
IFIP Congress, Toronto, Canada, August 1977, pp.
229 -234.
[4] V.G. Cerf, and Robert E. Kahn, A Protocol for P,lcl-.:et
Network Intercommunication, IEEE Trans. on Communication, COM-22, May 1974, pp. 673-648.
[5} V.G. Cerf, Y.K. Dalai, and C.A. Sunshine, Specificiation
of lnternet Transmission Control Program, INWG Note
72, revised December 1974.
[6] V.G. Cerf, TCP Resynchronization, Digital Systems Lab
Technical Note Number 79, Stanford University,
January 1976.
[7] V.G. Cerf, Specificiation of Internet Transmission Control Program, TCP (Version 2), March 1977, (available
from DARPA/IPTO).
[8] V.G. Cerf and J.B. Postel, Spccificiation of lnternet
Transmission Control Program, TCP (Version 3),
January 1978, (available from USC/ISI).
[91 Y.K. Dalai, More on Selecting Sequence Numbers, Proc.
A CM SIGCOMM/SIGOPS h~terprocess Communications
lorkshop, Santa Monica, March 1975, pp. 25-36.
(A CM Operating Systems Review, 9, 3, July 1975). Also
INWG Protocol Note 4, October 1974.
1101 Y.K. Dalai, Estabishing a Connection, INWG Protocol
Note 14, March 1975.
1111 A. Danthine and J. Bremer, Modeling and Verification
of End-to-End Transport Protocol, Proc. Syrup. on
Computer Network Protocols, Liege, Belgium, February
1978.
1121 J.G. Fletcher and R.W. Watson, Mechanisms for a
Reliable Timer-Based Protocol, Proc. Syrup. on Com.
puter Network Protocols, Liege, Belgium, February
1978.
[131 L.L. Gadick and R. Rom, Reliable llost-to-Host Protocols: Problems and Techniques, Proc. Fifth Data Communications Syrup., Snowbird, Utah, September 1977,
pp, 4.58-4.6,';. (IEEE 77CH1260-9C).
473
7O.
[24] R.S. Tomlinson, Selecting Sequence Numbers, Proc..
ACM SIGCOM/SIGOPS lnterprocess Communication~
Workshop, Santa Monica, Calitornia, March 1975, pp.
11-23. (A CM Operating Systems Review, Vol. 9, No. 3,
July 1975) Also INWG Protocol Note No. 2, August
1974.
D.C.
Walden, A System for Interprocess Communica[25]
tion in a Resource Sharing Computer Network, Comm.
ACM 15, 4, April 1972, pp. 221-230.