Avoiding Being Overwhelmed By

Connectivity Complexity
David Cuthbertson

Overview

Understand the Problem(s)

Standards can help or hinder
New technology
gy challenges
g
Organisational challenges
Labelling not the same as naming
Case Study(s)

Personal Background
g
Personal Experience

Industry Groups and Frameworks

Network
Troubleshooting
Cabling and Network
Installations
Managed Services
Voice/Data

Skills
Awareness
Communication

Infrastructure Management
Practices

Naming
N
i
Labelling
Change Process

Baselining
B
li i
Toolsets
Visualization

Change
g is Constant
Information requirements force technical change
Information and Data
Applications
Virtual
Hardware
Physical Infrastructure

Infrastructure responds to change

Change
g Affects Management
g
Governance & control force organizational change
Projects

Operations

Do it faster
Zoning /
Partitioning

Management

Consolidate
/ Optimize
Reduce
costs

Use of
Partners

Security

Change
Records
Capacity
Reporting

Regulators

No
Downtime!
Data Loss
Prevention

Physical Infrastructure Management

Practice and processes are impacted

How Do We Cope?
p
Adopt industry best practices where they exist
Ensure operational processes are not the weakest link
Ensure clarity on roles and practices

Make the complexity easier to manage

Clear naming, labelling and documentation

Reduce the number of data sets needed for governance and control

The Case Studyy

Two data centers (co-lo)
- approx 400
00 racks
k
- 36200 fiber/copper connections
- 5800 patches
- 600 servers
Im just overloaded and cant get others to understand

4 spreadsheets inventory + connectivity per DC

(>1.8 million cells) + >6000 other spreadsheets
Maintained by a data center manager no onsite staff

Simple Questions
1.

Is every device in the data centers or equipment rooms currently documented,

along with how it is connected in a system that supports both planning and
operations information needs? NO

2.

Does existing connectivity documentation for data centers and shared equipment
rooms use consistent naming conventions across all locations, devices and
connections? DONT KNOW

3
3.

Are updates
A
d t to
t inventory
i
t
and
d connectivity
ti it d
data
t d
duplicated
li t d iin spreadsheets,
dh t
diagrams and databases in a consistent way? PARTIALLY

4.

When planning choices are confirmed

When devices are added / removed
With changes in backbone cabling or patching

Can detailed work packages be created for internal or external engineers, without
visiting site beforehand,
beforehand or being there during changes? NO

The Impact
p On Operations
p
Increasing project costs and delays
Overloading
O l di off kkey engineers
i
/ iindividuals
di id l
Reverse engineering with workshops, site surveys, etc

Increased risks
The human factor (guess, assume, worry, forget, etc.)
Huge security holes

Strategic initiatives often limited in scope and benefit

Asset management doesn
doesntt really help DC management

Localised attempts to retain/manage knowledge of

configurations
g
spreadsheets,
p
, diagrams
g

Information Sets
Documents
Project
j
p
plans
Design and specifications
Commissioning documents / contracts
Manuals and procedures
Lists
Inventory actual and planned
Connectivity paths, end points
Capacity space, power, connectivity
Support warranty, maintenance
Diagrams
Backbone cabling - physical, logical
Topology Network, SAN, voice, power
Layouts - Floor plans, racks, cable tray
W kfl
Workflow
Changes,
Ch
iinterfaces
t f

Requesting
q
g a Connection -1
Device

Connection

Device

SW-BHAM-09

Port

CAT6A Copper

23

NIC 1

Port

Device
1

Port

Connection

Port

Device
2

SW-BHAM-09

23

To be filled in by DC team

NIC 1

UK_BIRM_BLADE-03

This was the initial spreadsheet used to capture connectivity requests

Requesting
q
g a Connection -2
Device

Connection

Device

SW-BHAM-09

Port

CAT6A Copper

23

NIC 1

Port

Device
1

Port

Connection

Port

Device
2

SW-BHAM-09

23

To be filled in by DC team

NIC 1

UK_BIRM_BLADE-03

Port Type
Speed
VLAN

Room
Rack
Position
Make
Model
Asset Number
IP Address
dd

Room
Rack
Position
Make
Model
Asset Number
IP Address
dd

Port Type
Speed
VLAN

Colour
Cable Label
Connection type
Length

The spreadsheet just

went from 4 to 30 columns wide!

Namingg & Labellingg

Ports and Connections

Equipment use the physical label?
Port name
- 1 or 01 or 001?
- 2/1 2\1 2/01 SL2/1 Port 2/1 Gig 2/1 Fe2/1 Slot 2/09
- Mgmt MGT Con Console ILO Net Mgmt
- NIC 1 Eth A Net 0 hba0 bge1 12F1 primary
Cable Labels
1) port
4) full path

2) local devices
5) cable unique id

3) end devices
6) path unique ID

And Which Cablingg Topology?

p gy
Top Of Rack

End Of Row

Could involve
1 to 5 patches

Multi-path
p
Options
p
RACK
N01

RACK
N02

Zone A

Zone B

01 06 07-12
01-06
07 12

01 06 07-12
01-06
07 12

Network
R k
Racks

RACK
N03

RACK
N04

Zone C

Zone D

01 06 07-12
01-06
07 12

01-06 07-12 13-18 19-24

01 06 07-12
01-06
07 12

01-06 07-12 13-18 19-24

RACK
A01

RACK
A02

Server Racks

Veryy Multi-path
p

One unit = 48 modules @ 6 LC ports per module = 288 ports

One rack @ 6 Units = 288 modules = 1728 ports

So Increase The Spreadsheet

p
Size
Device

Port

23

Room
Rack
Position
Make
Model
Asset Number
IP Address

Port Type
Speed
p
VLAN

Patch

Colour
Cable Label
Type
Length

PPA
Port

Patch
Panel
A

05

AA

Port Type
Speed
p

Contact
Purpose
Request date
Request Number
Project Code / Ref

Room
Rack
Position
Make
Model

Back
to
Back

Type
Length
g

Patch
Panel
B

PPB
Port

AC

05

Room
Rack
Position
Make
Model

Port Type
Speed
p

Patch

Colour
Cable Label
Type
Length

Work number
Path reference
Scheduled date
Installed status
Path length

The spreadsheet data has now

grown to up to 60 columns
wide

Port

Device

NIC 1

Port Type
Speed
p
VLAN

Room
Rack
Position
Make
Model
Asset Number
IP Address

But With Structured Wiringg

1. Cabinet
Patching

4. Services
Presentation

3. Service
Patching

2. Zone and
Backbone Cabling

Network
Patch
P l
Panel

Copper

Patch
P l
Panel

Patch
P l
Panel
Patch
Panel

Storage
KVM

Patch
Panel

Fiber

Patch
Panel

It makes sense to present services in patch panels

Expand The Spreadsheet(s) a Bit More

P

23

05

AA

B
To
B

AB

05

B
To
B

13

As common practice is to present Core

infrastructure services at cross
connects.
So you may be connecting to a service
panel not a device
panel,

25

B3
/04

B
to
F
3/
04

Switch Ports
Presented
In Patch Panel

01

01

01

01

A
12

12

-LP H

12

-LP H

12

-LP H

-LP H

01

01

01

01

A
12

L-1

12

A
H
P

-LP H

12

-LP H

L- H
P

01

01

01

01

12

12

12

43
43
43

-LP H
12

-LP H

12

-LP H

-LP H

12

-LP H

L- H
P

01

01

23

22
22

21
21

20
20

19
19

43

42
42

41
41

40
40

39
39

38
38

37
37

36
36

35
35

34
34

33
33

32
32

31
31

30
30

29
29

28
28

27
27

26
26

29
29

28
28

27
27

26
26

25

24

23

22

25
25

24
24

23
23

22
22

21
21

20
20

19
19

01

A
01

37

36
36

35
35

34
34

33
33

32
32

31
31

30
30

29
29

28
28

27
27

26
26

25
25

24
24

23
23

22
22

21
21

20
20

19
19

A
01 1L- PH
A
2

A
01 PL- H
12 A

A
01 LP- H
12 A

A
01 LP- H
12 A

A
01 LP- H
12 A

A
01 LP- H
12 A

01

01

12

12

A
H
P

01

01

12

12

-LP H

12

-LP H

12

-LP H

01

01

01

01

12

-1L

12

A
H
P

-LP H

12

L- H
P

L- H
P

01

01

01

01

12

-LP H

12

L- H
P

12

L- H
P

12

L- H
P

25

24
24

23
23

22
22

21
21

20
20

19
19

18
18

12

01

12

L- H
P

A
01 -LP H
12 A

01 E-1B T
2 A

A
01 1-L PH
A
2

01

01

12

L- H
P

12

-LP H

12

L- H
P

12

L- H
P

25

01

26

L- H
P

27

26

01 E-1B T
2 A

43

42
42

41
41

40
40

39
39

38
38

37
37

36
36

35
35

34
34

33
33

32
32

31
31

30
30

29
29

28
28

27
27

26
26

25
25

24
24

23
23

22
22

21
21

20
20

19
19

18
18

A
01 1-L PH
A
2

A
01 -LP H
12 A

01 E-1B T
2 A

A
01 -LP H
12 A

01 E-1B T
2 A
01 E-1B T
2 A
01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A

0 B
1 -1 ETA
2

43

A
01 LP- H
12 A

A
01 -LP H
12 A

A
01 -LP H
12 A

A
01 -LP H
12 A

42

41
41
41

40
40
40

39
39
39

38
38
38

37
37
37

36
36
36

35
35
35

34
34
34

33
33
33

32
32
32

31
31
31

30
30
30

29
29
29
29

28
28
28
28

27
27
27
27

26
26
26
26

25
25
25
25
25

24
24
24
24
24

23
23
23
23
23

22
22
22
22
22

21
21
21
21
21

20
20
20
20
20

19
19
19
19
19

18
18
18
18
18

01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A
01 E-1B T
2 A

01 E-1B T
2 A

0 B
1 -12 ETA

01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A

0 B
1 -12 ETA

01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A

0 B
1 -1 ETA
2

43

42

01 E-1B T
2 A

43
A
01 -LP H
12 A

A
01 LP- H
12 A

A
01 -LP H
12 A

A
01 -LP H
12 A

A
01 PL- H
12 A

A
01 -LP H
12 A

A
01 1-L PH
A
2

A
01 -LP H
12 A

12

A
H
P

L- H
P

12

-LP H

L-1

12

L- H
P

A
01

01

01

01

12

L- H
P

12

-LP H

12

L- H
P

12

L- H
P

Cabling Subsys
stem 1

01

01

01

01

A
01 PL- H
12 A

38

A
01 -LP H
12 A

39

A
01 -LP H
12 A

A
01 -LP H
12 A

-LP
12

A
-LP
12

A
01 PL- H
12 A

A
01 -LP H
12 A

01

01

A
01 LP- H
12 A

-12

A
LP
AH

A
01 L P
-12 H A

01

A
01 L P
-12 H A

A
01 L P
-12 H A

A
0 -1 LP H
1 2 A

A
0 -1 LP H
1 2 A

A
0 -1 LP H
1 2 A

A
0 -1 LP H
1 2 A

13

A
01 -LP H
12 A

12

L- H
P

12

L- H
P

12

-LP H

12

L- H
P

12

L- H
P

A
01 -LP H
12 A

01

01

01

01

A
H
2P

A
H
2P

01-

A L

01

L
A
H
2P

A
01-

01-

A
01 1-L PH
A
2

A
01 -LP H
12 A

A
01 -LP H
12 A

A
H
P

12

-LP H

12

L- H
P

12

L- H
P

12

L- H
P

A
01 -LP H
12 A

12

L- H
P

12

L- H
P

L-1

12

L- H
P

01

01

A
01 -LP H
12 A

01

01

01

01

01

01

10

A
01 PL- H
12 A

11

10

A
01 1-L PH
A
2

11

A
01 -LP H
12 A

A
H
P

14

13

A
01 -LP H
12 A

12

-LP H

12

L- H
P

L-1

12

L- H
P

42

A
01 -LP H
12 A

01

01

01

01

43

01 E-1B T
2 A

3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3
3

2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2
2

1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1

43
43
43
9

28

27

10

29

28

-1 T
E
B

11

29

01

12

11

30

13

12

11

31

30

14

13

12

11

32

31

-1 T
E
B

14

13

12

11

33

32

01

14

13

12

11

34

33

01

14

13

12

11

18

14

13

12

11

19

35

34

10 E-1B T
2 A

14

13

12

11

20

19

36

35

0 B
1 -12 ETA

14

13

12

11

21

20

37

36

14

13

12

11

21

38

37

14

13

12

11

22

39

38

14

13

12

11

23

22

40

39

14

13

12

11

24

23

41

40

-1 T
E
B

14

13

12

11

25

24

42

41

-1 T
E
B

4
14

13

12

25

43

42

01

4
14

13

26

5
14

27

26

6
15

28

27

43

-1 T
E
B

16

15

29

28

01

17

16

15

29

17

16

15

30

-1 T
E
B

17

16

15

31

30

01

10

17

16

15

32

31

10

17

16

15

33

32

10

17

16

15

34

33

TA
E

10

17

16

15

35

34

-1 T
E
B

10

17

16

15

36

35

-1 T
E
B

10

17

16

15

37

36

-1 T
E
B

10

17

16

15

38

37

10

17

16

15

39

38

-1

10

17

16

15

40

39

01

10

18

17

16

15

41

40

01

10

18

17

16

42

41

10

18

17

43

42

10

7
18
18

43

01

-LP H

12

-LP H

L-1

01

38

37

39

38

L- H
P

40

39

01

41

40

L- H
P

42

41

01

43

42

A
01 LP- H
12 A

43

11

43
A

12

30

13

31

30

01 E-1B T
2 A

01 E-1B T
2 A

14

32

31

01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A
0 B
1 1- ETA
2

15

33

32

-1 T
E
B

-1 T
E
B

1
16

-1 T
E
B

2
17

11

-1 T
E
B

3
18

01

01

01

01

4
19

01 E-1B T
2 A

5
20

01 E-1B T
2 A

6
21

01 E-1B T
2 A

7
22

0 B
1 -1 ETA
2

01 E-1B T
2 A

01 E-1B T
2 A

10

01 E-1B T
2 A

10

10

01 E-1B T
2 A

11

10

10

11

10

22

11

11

12

12

13

12

13

14

13

12

14

E
TA

15

14

13

15

15

14

16

-1 T
E
B

15

17

-1 T
E
B

16

18

-1 T
E
B

17

16

19

18

17

16

20

-1

18

17

21

01

18

23

22

01

19

24

23

01

20

19

25

24

21

20

19

26

25

22

21

20

27

26

22

21

28

27

23

22

29

28

24

23

29

25

24

23

30

26

25

24

23

31

30

26

25

24

32

31

26

25

32

27

26

33

-1 T
E
B

28

27

34

33

-1 T
E
B

29

28

27

35

34

29

28

27

36

35

-1 T
E
B

29

28

36

-1 T
E
B

30

29
42

37

01

31

30

40

01

32

31

30

37

01

33

32

31

30

40

01

34

33

32

31
34

33

01 E-1B T
2 A

01 E-1B T
2 A

35

34

33

32
35

34

01 E-1B T
2 A

11

36

35

0 B
1 -1 ETA
2

12

11

37

36

01 E-1B T
2 A

13

12

38

37

01 E-1B T
2 A

14

13

39

38

01 E-1B T
2 A

14

40

39

01 E-1B T
2 A

15

41

40

01 E-1B T
2 A

16

15

42

41

0 B
1 -1 ETA
2

17

16

43

42

01 E-1B T
2 A

18

17

43

01 E-1B T
2 A

18

41

A
01 LP- H
12 A

A
01 -LP H
12 A

A
01 -LP H
12 A

A
01 -LP H
12 A

A
01 -LP H
12 A

A
01 L-1 PH
A
2

A
01 -LP H
12 A

A
01 -LP H
12 A

10

A
01 -LP H
12 A

10

A
01 -LP H
12 A

11

A
01 -LP H
12 A

12

11

A
01 -LP H
12 A

13

12

A
01 -LP H
12 A

14

13

A
01 L-1 PH
A
2

14

A
01 -LP H
12 A

15

A
01 -LP H
12 A

16

15

A
01 -LP H
12 A

17

16

Cabling Subsystem 3

18

17

41

14

AP Distributor

19

18

Cabling Subsystem 4

A
H
P

20

19

Distributor D

12

-LP H

12

-LP H

-1L

12

-LP H

12

-LP H

12

-LP H

21

20

Cabling Subsystem 4

01

01

01

01

01

01

22

21

01 E-1B T
2 A

10

A
01 LP- H
12 A

24

23

10

A
01 PL- H
12 A

25

24

10

A
01 LP- H
12 A

25

-1 T
E
B

10

A
01 -LP H
12 A

26

01

01 E-1B T
2 A

36

35

34

33
A
01 LP- H
12 A

27

26

37

36

35

34
A
01 -LP H
12 A

28

27

A
01 -LP H
12 A

29

28

-1 T
E
B

A
01 -LP H
12 A

01 E-1B T
2 A

29

01

10 BE-1 T
2 A

01 E-1B T
2 A

38

37

36

35
A
01 1-L PH
A
2

30

TA
E

A
01 -LP H
12 A

31

30

-1 T
E
B

39

38

37

36
A
01 LP- H
12 A

-1 T
E
B

32

31

30

40

39

38

37
A
01 1-L PH
A
2

01 E-1B T
2 A

01

-1 T
E
B

33

32

31

-1

41

40

39

38
A
01 -LP H
12 A

01 E-1B T
2 A

01

34

33

32

41

40

39
A
01 -LP H
12 A

12

01

12

-LP H

01 E-1B T
2 A

-LP H

23

22

24

23

25

24

26

25

27

26

28

27

29

28

-1 T
E
B

29

-1 T
E
B

30

-1 T
E
B

31

30

TA
E

32

31

33

32

-1

33

01

34

01

35

34

01

36

35

38
B

36

39
1

12
01 B E
-12 T A

38
0

LP
-12 H A

A
0 -1 LP H
1 2 A

A
0 -1 P
L H
1 2 A

01

A
0 -1 LP H
1 2 A

15

01 B E
-12 T A

39
A

37

42

01

01 B E
-12 T A

40

43

37

01

A
0 -1 LP H
1 2 A

A
0 -1 LP H
1 2 A

A
0 -1 LP H
1 2 A

A
0 -1 LP H
1 2 A

16

B
01 - E T
12
A

42

Cabling Subsystem 2

B
0 -1 E T
1 2 A

43
I

B
0 -1 E T
1 2 A

40

B
0 -1 E T
1 2 A

B
0 -1 E T
1 2 A

41

B
0 -1 E T
1 2 A

41

A
01 PL- H
12 A

A
01 L-1 PH
A
2

A
01 PL- H
12 A

A
01 PL- H
12 A

A
01 PL- H
12 A

A
01 PL- H
12 A

A
01 PL- H
12 A

A
01 PL- H
12 A

01

H
A

PL12

A
01 L-1 PH
A
2

01

H
A

-LP
12

A
01 LP- H
12 A

01

01

01

01

H
A

12

H
A

12

12

-LP

12

-LP H

PL-

PL-

Cabling Subsyystem 1

12

12

-1 T
E
B

-LP H

-1 T
E
B

-LP H

-1 T
E
B

-1 T
E
B

01

01

01

01

01

2
-1 T
E
B

01

2
-1 T
E
B

B
01

35

34

33

A
01 -LP H
12 A

10

01

36

35

34

37

36

35

38

37

36

01

38

37

42

41

40
0 B
1 -1 ETA
2

42

41
A
01 -LP H
12 A

Cabling Subsystem 5

38

B
0 -1 E T
1 2 A

Cabling Subsystem 5

11

10

39

01 B E
-12 T A

B
0 -1 E T
1 2 A

12

11

12

13

12

12

42

B
0 -1 E T
1 2 A

-LP H

14

13

-LP H

15

14

-1 T
E
B

16

15

-1 T
E
B

17

16

TA
E

01

18

17

-1 T
E
B

01

19

18

20

19

-1

A
H
P

21

20

01

22

21

01

12

23

22

01

L-1

24

23

-LP H

25

24

26

25

40

01

27

26

01

43

B
0 -1 E T
1 2 A

28

27

29

28

12

30

29

-1 T
E
B

12

31

30

B
0 -1 E T
1 2 A

B
0 -1 E T
1 2 A

01

-LP H

32

31

-1 T
E
B

2
01

-LP H

33

32

-1 T
E
B

34

33

-1 T
E
B

35

34

-1 T
E
B

-1 T
E
B

01

36

35

-1 T
E
B

01

01

37

36

01

37

01

T
E

A
H
P

41

40

01

1-2

41

01

01 E-1B T
2 A

-1 T
E
B
-1 T
E
B

01

01

12

01 E-1B T
2 A

-1 T
E
B

2
01

L-1

01 E-1B T
2 A

-LP H

A
T

2
BE-1

-1 T
E
B

01

01

01

A
T

T A
E
B

-1 E
B
T

1-2

01
0

01

2
BE-1

-1 T
E
B

-1 T
E
B
2

01

39

01

01 E-1B T
2 A

Cabling Subsystem 5

01

-1
E
B

01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A

01 E-1B T
2 A

42
40

39

TA
E

43
01 E-1B T
2 A

1
41

40

-1 T
E
B

1
42

41

40

2
42

41
-1

01

01

01 E-1B T
2 A

4
A

5
A

6
2

01 E-1B T
2 A

7
-1 T
E
B

01 E-1B T
2 A

01

10

10

11

-1 T
E
B

12

11

TA
E

13

12

14

13

01

14

-1

15

16

15

17

16

42

17

18

19

18

20

19

21

20

-1 T
E
B

21

-1 T
E
B

22

01

23

22

01

24

23

25

24

25

26

27

26
38

T
E

28

27
39

28
42

1-2

29

39

40

-1
E
B

42

43
-1 T
E
B

01

30

29

01 E-1B T
2 A

31

30

0 B
1 -1 ETA
2

A
-1 T
E
B

-1 T
E
B

2
01

32

31

01 E-1B T
2 A

2
E
TA

-1 T
E
B
01

-1
B

01
1
0

33

32

01 E-1B T
2 A

34

33

A
-1 T
E
B
-1 T
E
B

A
01

01

-1 T
E
B
-1 T
E
B

2
01

01

35

34

36

35

37

36

BE-1

A
-1 T
E
B

-1 T
E
B

2
01

38

37

01

01

-1 T
E
B
E
TA
01

-1
B

01
1
0

38

0 B
1 -1 TAE
2

39

41

40

-1 T
E
B

41

38

39

43
2

38

BE-1

39

BE-1

42
01

-1 T
E
B

A
-1 T
E
B

A
-1 T
E
B
2
01

2
01

42
01

Distributor D

43
01

43
01

Cabling Subsystem 5

And Cabling Extends Beyond The DC!

D I RE C

16

15

12

21

20

19

18

17

16

15

14

13

12
I

43

42

21

20

19

18

17

16

15

14

13

12

11

Define Active Device Namingg

Device Name
1) Logical Name
2) Type of device / location
3) Asset Number

US-SW-DC02-03P
Cisco 6509 DC02-B03-U2
Asset HW0078732

DTE/DCE

ACTIVE

PWR MGMT

RESET

STATUS

SYSTEM

WS-X6K-SUP1A-2GE

SW-BHAM-CORE1.Card1
PCMCIA

PORT 2

PORT 1

100%

1%
SWITCH LOAD

CONSOLE

SUPERVISOR 1

NK
LI

EJECT

NK
LI

FAN
STATUS

SW-BHAM-CORE1.Card3
WS-X6748-GE-TX

11
2

13
14

12

23
24

25

35
26

37
38

36

47
48

SW-BHAM-CORE1.Card4

WS-X6748-GE-TX

10/100/1000
GE MOD

48 P ORT

STATUS
2

10

11

12

13

11
2

14

15

16

17

18

19

20

21

22

23

24

13
14

12

23
24

25

26

27

28

29

30

31

33

32

34

35

FABRIC ENABLED
RJ45

36

25

35
26

37

38

39

40

41

42

43

44

45

46

47

48

37
38

36

47
48

SW-BHAM-CORE1.Card5

48 P ORT

STATUS
1

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

10/100/1000
GE MOD

25

26

27

28

29

30

31

33

32

34

35

FABRIC ENABLED
RJ45

36

37

38

39

40

41

42

43

44

45

46

47

48

Device component Switch Card

US-SW-DC02-03P Card3
Cisco 6509 DC02-B03-U2.Slot03
A
Asset
HW00 8 3
HW0078737

SW-BHAM-CORE1.Card9

C6509-E-FAN

WS-C6509-E

INPUT 1
100-240V ~
16A
60/50 Hz

INPUT 1
100-240V ~
16A
60/50 Hz

INPUT 2
100-240V ~
16A
60/50 Hz
L
AL

RU
N

RU
N

INST

INPUT
OK
1
CISCO SYSTEMS, INC.

Device component Blade Server

INPUT 2
100-240V ~
16A
60/50 Hz

L
AL
INST

FAN
OK

INPUT
OK

OUTPUT
FAIL

SW-BHAM-CORE1

SWITCH MUST BE IN OFF O POSITION TO INSTALL/

REMOVE POWER SUPPLY. FASTENER MUST BE FULLY
ENGAGED PRIOR TO OPERATING POWER SUPPLY.

CISCO SYSTEMS, INC.

FAN
OK

OUTPUT
FAIL

SWITCH MUST BE IN OFF O POSITION TO INSTALL/

REMOVE POWER SUPPLY. FASTENER MUST BE FULLY
ENGAGED PRIOR TO OPERATING POWER SUPPLY.

UX-NY0445-PROD
HP BL685C BLNY05-DC02-B03-U2.Slot03
Asset HW0078143

Just Make the Spreadsheet

p
Bigger!
gg
P

23

05

AA

B
To
B

AB

05

13

B
To
B

25

3
/04

10

10

11

11

12

12

13

13

14

14

15

15

16

16

17

17

18

18

19

19

20

20

21

21

22

22

23

23

24

24

The spreadsheet(s) become un-manageable

a))
b)
c)
d)

Too manyy blank cells

Too big to easily comprehend and check
Updating is a nightmare to ensure consistency and accuracy
Lots of manual cutting/pasting to other spreadsheets

B
to
F
3/
04

And To Manage
g The Request
q
Equipment Positioning

Data Center
Change Request

New
Equipment
?

Equipment
Placement

N
New
Infrastructure
?

Assess
Space /Power
Connectivity

Y
Allocate
location

New
Infrastructure
Build

Design

Design
Connection
Path(s)

Create
Work Order

Install
Allocate
location

Allocate
PP ports

Create
Work Order

Allocate
Resources

Install
& Test

Patch
& Test Path

N Infrastructure
New
I f t t
Patch to
Equipment

Connectivity

Test

Installed
And Working

Define Infrastructure Namingg

1

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

24 Port Copper Patch Panel

12
11
10
09
08
07
06
05
04
03
02
01

A/
B

B/
C

C/
D

A/
B

B/
C

C/
D

A/
B

B/
C

C/
D

A/
B

B/
C

C/
D

A/
B

B/
C

C/
D

A/
B

B/
C

C/
D

A/
B

B/
C

C/
D

A/
B

B/
C

C/
D

A/
B

B/
C

C/
D

A/
B

B/
C

C/
D

A/
B

B/
C

C/
D

A/
B

B/
C

C/
D

288 Port Fibre Panel

12
11
10
09
08
07
06
05
04
03
02
01

Patch Panel
AB
B03-AB
B03-5
US-NY-DC05-H1-B03-5
PP B03-AB-U5
PP B03-AB-U5 to H07-AC-U2
PP B03-AB-U5 to H07-AC-U2/H06-AG-U9
PPC B03-AB-U5 to H07-AC-U2:Ports 01 to 24

P t Selection
Port
S l ti
6 Copper RJ45 Ports

6 Fiber LC Ports

AB-A Ports 1-6

H07-AC-B Ports 7-12
PCI AB-A Ports 1-6 to H07-AC-B Port 7-12
AC-03B Ports 1-6
PFI B03-AB-03B Ports 1-6 to H07-AC-12A Ports 1-6

Infrastructure Data Issues

1.

Names and labelling of devices, ports, patch panels are often

inconsistent. Keep them as simple as possible.

2.

Working practices will dictate when an update is carried out

3.

Use external standards if helpful TIA606B, TIA942

As equipment vendors use various names for ports, decide on your standard
Labels and administration identifiers may be different!

Request,
q
, design,
g , issue design
g docs,, make change,
g , complete
p
A spreadsheet will be locked by one user, so look at a database option

You need other views / perspective to manage connectivity

Capacity reporting, device views

Diagrams Rack layout, backbone architectures

In Practice
In an estate of about 400 racks
32, 000 copper / 22, 000 fiber backbone connections
9,000 paths
20 000 patch connections
20,000
Master spreadsheet(s) with connection paths
9,000 rows
Master spreadsheet(s) for backbone
54,000 rows
Approx. 4000 change spreadsheets identified
300,000 rows
Switch p
port allocations a tab p
per switch //card
16,000
,
rows
San switch port allocations a tab per SAN switch
Plus asset list(s), 800 rack layouts, backbone diagrams

Some Reasons To Change

g

Faster implementation time demanded

Identify and reclaim unused capacity
Improve switch resource utilisation
Service provider / partner labor costs
Individual overload / resignation
Failed audit / compliance
Fewer accidents / mistakes
Upgrades / refurbishment

Physical connectivity data highlights the current understanding of inventory

and dependency. An easy target for auditors!

In Hindsight
g
1. Maybe a spreadsheet wasnt the best way to start
- It gives structure, but
b isnt
best
b suited
d to data
d
center connectivity
- Loading a database is 3 spreadsheets - 1)racks 2)devices 3)connections

2. One day you will have to establish a baseline inventory and

maintain it before you can manage connectivity
-

Inaccuracies in data / naming are often present in other systems

Without resolving naming issues, audits/ data capture may be wasted

3. Managing connectivity has to cope with the real world

- Complex device build switches, servers, cabling distribution, pret
terminated
i t d cassettes,
tt MPO
MPO, bl
blown fib
fibre, di
diverse paths,
th etc.
t

The Bigger
gg Picture Think Forward

Physical locations
Hardware inventory
Physical connectivity
Logical connectivity
Rackk capacity
i
Port capacity
Asset lists
Project plans
Work orders
Alternate paths / SPOF
Monitoring toolsets

Reducingg Data Sets Tomorrow

Excel

Visio

Floor box list

Floor plan

Cabinet list

Equipment room floor plan

Patch panel list

Backbone cabling diagram

Inventory

Network diagram

Inventory

Rack diagram

Inventory

Server connectivity diagram

Example
p Link Visio to Data

Embed device / link data

in diagrams
Standard Visio Feature!

A More Strategic
g Approach
pp

Infrastructure
Database(s)
&
Reporting

In Summary - When?
1.

Every device in the data center and how it is connected is currently documented
in a system that supports both planning and operations information needs

2.

Updates to inventory and connectivity data are duplicated in spreadsheets,

diagrams and databases in a consistent way

When devices are added / removed

With changes in backbone cabling or patching

3
3.

Detailed
D
il d workk packages
k
ffor d
data center changes
h
can b
be created
d ffor iinternall or
external engineers, without visiting site beforehand or being there during
changes.

4.

Current connectivity documentation for data centers and shared equipment

rooms uses consistent naming conventions across all locations, devices and
connections

Additional Material
www.tiaonline.org

ANSI/TIA568C, ANSI/TIA606B, ANSI/TIA942

www.bicsi.org

ANSI/BICSI-002

www.squaremilesystems.com
Free SMS Visio utilities
Downloads and videos
Webinars/videos
Visio automation, documenting cabling, etc.
www.assetgen.com
Evaluation software
Webinars

Free DCIM/CMS evaluation version

Data center practices, Visio integration