EXECUTIVE GUIDE

Back to TOC

EXECUTIVE GUIDE

Next-generation data centers:

Opportunities abound

IT transformation begins at the data center, as enterprises

embrace technologies such as virtualization and cloud computing
to build adaptable, flexible and highly efficient infrastructures
capable of handling today’s business demands.

Sponsored by APC
www.apc.com
 EXECUTIVE GUIDE

Table of Contents
Profile: APC by Schneider Electric 3
Introduction 4
Next-generation data centers: The new realities................................................................................................................................. 4

Trends 5
Elastic IT resources transform data centers ...................................................................................................................................... 5
What IT pros like best about next-generation technology................................................................................................................. 7
Thinking outside the storage box............................................................................................................................................................ 9

Emerging technologies 12
One in 10 companies deploying internal clouds..................................................................................................................................12
Google, Microsoft spark interest in modular data centers, but benefits may be exaggerated.................................................14
The big data-center synch.....................................................................................................................................................................16

Best practices 18
Seven tips for succeeding with virtualization....................................................................................................................................18
Don’t let the thin-provisioning gotchas getcha..................................................................................................................................21
The challenge of managing mixed virtualized Linux, Windows networks..................................................................................... 23

Inside the Data Center 26

The Google-ization of Bechtel ............................................................................................................................................................ 26
Why San Diego city workers expect apps up and running in 30 minutes or less........................................................................ 30

Sponsored by APC
www.apc.com 2
 EXECUTIVE GUIDE
Back to TOC

Profile: APC by Schneider Electric

APC by Schneider Electric, a global leader in critical power and cooling services, provides

industry leading product, software and systems for home, office, data center and factory

floor applications. Backed by the strength, experience, and wide network of Schneider

Electric’s Critical Power & Cooling Services, APC delivers well planned, flawlessly installed

and maintained solutions throughout their lifecycle. Through its unparalleled commitment

to innovation, APC delivers pioneering, energy efficient solutions for critical technology and

industrial applications. In 2007, Schneider Electric acquired APC and combined it with MGE

UPS Systems to form Schneider Electric’s Critical Power & Cooling Services Business Unit,

which recorded 2007 revenue of $3.5 billion (€2.4 billion) and employed 12,000 people

worldwide. APC solutions include uninterruptible power supplies (UPS), precision cooling

units, racks, physical security and design and management software, including APC’s

InfraStruXure® architecture the industry’s most comprehensive integrated power, cooling,

and management solution. Schneider Electric, with 120,000 employees and operations in

102 countries, had 2007 annual sales of $25 billion (€17.3 billion). For more information on

APC, please visit www.apc.com. All trademarks are the property of their owners.

Sponsored by APC
www.apc.com 3

Introduction
Next-generation data centers:
The new realities
By Beth Schultz
S
everal years ago, Geir Ramleth, CIO of construction
giant Bechtel, asked himself this question: If you could
build your IT systems and operation from scratch today,
would you recreate what you have?
Ramleth certainly isn’t the only one who
has contemplated such a question. And the
answer always seems to be “no.” Yesterday’s
technology can’t handle the agility required
by today’s businesses.
The IT transformations often begin at the
data center. At Bechtel, for example, Ramleth
began its migration to an Internet model
of computing by scrapping its existing data
centers in favor of three new facilities (see
“The Google-ization of Bechtel,” page 26).
The next-generation data centers are all
about adaptability, flexibility and responsive-
ness, and virtualization is a building-block
technology.
The numbers show how tactical virtualiza-
tion has become. As discussed in “Seven tips
for succeeding with virtualization,” more than
4 million virtual machines will be installed
on x86 servers this year and the number of
virtualized desktops could grow from less
than 5 million in 2007 to 660 million by 2011,
according to Gartner. More telling, however, is
the shift from a tactical to strategic mindset.
The popularity of virtualizing x86 server and
desktop resources has many enterprise IT
managers reassessing ways to update already
virtualized network and storage resources, too.
“Enterprise IT managers are going to have
to start thinking virtual first and learn how
to make the case for virtualization across IT
disciplines,” advises James Staten, principal
analyst at Forrester Research.
In the article “What IT pros like best about
next-generation technology,” Michael Geis,
director of IS operations for Lifestyle Family
Fitness, says he has discovered how strategic
storage virtualization can be.The fitness chain
set out to resolve a performance bottleneck
originating in its storage network, and wound
up upgrading its data center infrastructure in
a way that not only took care of the problem
but also drastically changed how it made
storage decisions.
Heterogeneous storage virtualization
provides much-needed flexibility in the data
center.“Five years ago, the day you made the
decision to go with EMC or IBM or HP or
anyone, you might get a great discount on the
first purchase, but you were locked into that
platform for three to five years,” Geis says. But
now, every time the chain adds storage, its cur-
rent vendor, IBM, has to work in a competitive
situation, he says.
If virtualization is the starting point, an
Internet model of computing a la Bechtel’s
vision, increasingly is the end goal. Many
experts believe that corporate data centers
ultimately will be operated as private clouds,
much like the flexible computing networks
operated by Internet giants Amazon and
Google, but specifically built and managed
internally for an enterprise’s own users.
Besides Bechtel, corporations already
building their own private clouds include
such notable names as Deutsche Bank, Merrill
Lynch and Morgan Stanley, according to
The 451 Group.The research firm found in
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
a survey of 1,300 corporate software buyers
that about 11% of companies are deploying
internal clouds or planning to do so. As
discussed in “One in 10 companies deploying
internal clouds,” that may not seem like a
huge proportion, but it’s a sign that private
clouds are real.
As Vivek Kundra, CTO for the District of
Columbia government says, private clouds
are “definitely not hype.” Kundra, for one,
says he plans to blend IT services provided
from its own data center with external cloud
platforms such as Google Apps. And Gartner
predicts that by 2012 private clouds will
account for at least 14% of the infrastructure
at Fortune 1000 companies, which will benefit
from service-oriented, scalable and elastic IT
resources.
Of course, on their way to the cloud-based
data center of the future enterprises will dis-
cover, deploy – and possibly discard – myriad
other technologies.
For example, some might embrace the
concept of modularization, the idea of
packaging servers and storage in a container
with its own cooling system. Microsoft and
Google have piqued interest in this approach
with their containerized data center schemes.
Others might look to integration, focusing
on an emerging technology such as Fibre
Channel over Ethernet to bring together data
and storage on one data center fabric. Still
others might turn the notion of the traditional
storage-area network on its nose, using Web
services that link applications directly to the
storage they need.
In the new data center, possibilities
abound.
Schultz, formerly Network World’s New Data
Center editor, is now a freelance writer. She can
be reached at bschultz5824@gmail.com.
4

Next-generation data centers:
opportunities abound
Section 1
Trends
Elastic IT resources transform data centers
Several IT trends converge as data centers evolve
to become more adaptable, Gartner says
T
By Jon Brodkin, Network World, 12/04/2008
The enterprise data center of the
future will be a highly flexible and
adaptable organism, respond-
ing quickly to changing needs
because of technologies like
virtualization, a modular build-
ing approach, and an operating
system that treats distributed
resources as a single comput-
ing pool.
The move toward flexibility in all data
center processes comes after years of
building monolithic data centers that
react poorly to change.
“For years we spent a lot of money
building out these data centers, and the
second something changed it was: ‘How
are we going to be able to do that?’”
says Brad Blake, director of IT at Boston
Medical Center.“What we’ve built up is so
specifically built for a particular function,
if something changes we have no flex-
ibility.”
Rapidly changing business needs and
new technologies that require extensive
power and cooling are necessitating a
makeover of data centers, which represent
a significant chunk of an organization’s
capital costs, Blake notes.
For example,“when blade servers
came out that completely screwed up all
of our matrices as far as the power we
needed per square foot, and the cooling
we needed because these things sucked
up so much energy, used so much heat,”
he says.
Virtualization of servers, storage,
desktops and the network is the key
to flexibility in Blake’s mind, because
hardware has long been tied too rigidly to
specific applications and systems.
But the growing use of virtualization is
far from the only trend making data cen-
ters more flexible. Gartner Group expects
to see today’s blade servers replaced in
the next few years with a more flexible
type of server that treats memory, proces-
sors and I/O cards as shared resources
that can be arranged and rearranged as
often as necessary.
Instead of relying on vendors to decide
what proportion of memory, processing
and I/O connections are on each blade,
enterprises will be able to buy whatever
resources they need in any amount, a far
more efficient approach.
For example, an IT shop could com-
bine 32 processors and any number of
memory modules to create one large
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
server that appears to an operating system
as a single, fixed computing unit. This
approach also will increase utilization
rates by reducing the resources wasted
because blade servers aren’t configured
optimally for the applications they serve.
Data centers also will become
more flexible by building in a modular
approach that separates data centers into
self-contained pods or zones which each
have their own power feeds and cooling.
The concept is similar to shipping con-
tainer-based data centers, but data center
zones don’t have to be enclosed. By not
treating a data center as a homogenous
whole, it is easier to separate equipment
into high, medium and low heat densities,
and devote expensive cooling only to the
areas that really need it.
Additionally, this separation allows
zones to be upgraded or repaired without
causing other systems to go offline.
“Modularization is a good thing. It gives
you the ability to refresh continuously
and have higher uptime,” Gartner analyst
Carl Claunch says.
This approach can involve incremental
build-outs, building a few zones and
leaving room for more when needed. But
you’re not wasting power because each
zone has its own power feed and cooling
5
 EXECUTIVE GUIDE
Back to TOC

Section 1: Trends • • •
supply, and empty space is just that. This is some functionality stripped out of the the meta operating system, building in
in contrast to long-used design principles, general purpose operating system, a meta zones and pods, and more customizable
in which power is supplied to every operating system to manage the whole server architectures – are helping build
square foot of a data center even if it’s not data center will be necessary. toward a future when IT can quickly
yet needed. The meta operating system is still provide the right level of services to users
“Historical design principles for data evolving but is similar to VMware’s Virtual based on individual needs, and not worry
centers were simple – figure out what you
have now, estimate growth for 15 to 20
years, then build to suit,” Gartner states. “Newly built data centers often opened with huge areas
“Newly built data centers often opened of pristine white floor space, fully powered and backed
with huge areas of pristine white floor
space, fully powered and backed up by up by a UPS, water and air cooled, and mostly empty.
a UPS, water and air cooled, and mostly With the cost of mechanical and electrical equipment, as
empty. With the cost of mechanical and
electrical equipment, as well as the price
well as the price of power, this model no longer works.”
of power, this model no longer works.” Carl Claunch , analyst, Gartner
While the zone approach assumes
that each section is self-contained, that Datacenter Operating System. Gartner about running out of space or power. The
doesn’t mean the data center of the future describes the concept as “a virtualiza- goal, Blake says, is to create data center
will be fragmented. Gartner predicts that tion layer between applications and resources that can be easily manipulated
corporate data centers will be operated distributed computing resources … that and are ready for growth.
as private “clouds,” flexible computing utilizes distributed computing resources “It’s all geared toward providing that
networks which are modeled after public to perform scheduling, loading, initiating, flexibility because stuff changes,” he says.
providers such as Google and Amazon yet supervising applications and error “This is IT. Every 12 to 16 months there’s
are built and managed internally for an handling.” something new out there new and we
enterprise’s own users. All these new concepts and technolo- have to react.”
By 2012, Gartner predicts that private gies – cloud computing, virtualization,
clouds will account for at least 14% of the
infrastructure at Fortune 1000 companies,
which will benefit from service-oriented,
scalable and elastic IT resources.
Private clouds will need a meta
operating system to manage all of an
enterprise’s distributed resources as a
single computing pool, Gartner analyst
Thomas Bittman says, arguing that the
server operating system relied upon so
heavily today is undergoing a transition.
Virtualization became popular because
of the failures of x86 server operating
systems, which essentially limit each
server to one application and waste tons
of horsepower, he says. Now spinning up
new virtual machines is easy, and they
proliferate quickly.
“The concept of the operating system
used to be about managing a box,”
Bittman says.“Do I really need a million
copies of a general purpose operating
system?”
IT needs server operating systems with
smaller footprints, customized to specific
types of applications, Bittman argues. With

Sponsored by APC
www.apc.com 6

Section 1: Trends • • •
What IT pros like best about
next-generation technology
Flexibility, costs savings and eco-friendly operations make the list
By Ann Bednarz, Network World, 10/20/2008
D
Do hundreds of gallons of used
vegetable oil belong anywhere near
a data center, let alone inside? Phil
Nail thinks so.
Nail is CTO of AISO.Net, whose Romoland,
Calif., data center gets 100% of its electricity
from solar energy. Now he’s considering waste
vegetable oil as an alternative to using diesel
fuel in the Web hosting company’s setup for
storing solar-generated power.
“We’re never opposed to trying something
new,” says Nail, who has eliminated nearly 100
underutilized stand-alone servers in favor of
four IBM System x3650 servers, partitioned
into dozens of virtual machines using VMware
software.
Server virtualization fits right into AISO.Net’s
environmentally friendly credo.The company
increased its average server-utilization level by
50% while achieving a 60% reduction in power
and cooling costs through its consolidation
project.
For Nail, virtualization technology lives up to
the hype. Nevertheless, it isn’t always easy for IT
executives to find the right technology to help
a business stay nimble, cut costs or streamline
operations. Read on to learn how Nail and
three other IT professionals made big changes
in their data centers, and what they like best
about their new deployments.
No more vendor lock-in
Vendor lock-in is a common plight for IT
storage buyers. Lifestyle Family Fitness, however,
found a way out of the shackles.The 56-club
fitness chain in St. Petersburg, Fla., set out to
resolve a performance bottleneck it traced to
its storage network, and wound up upgrading
its data center infrastructure in a way that
not only took care of the problem but also
extended the life of its older storage arrays.
Lifestyle’s users were starting to notice that
certain core applications, such as membership
and employee records, were sluggish. IT staff
confirmed the problem by looking at such
metrics as the average disk-queue length
(which counts how many I/O operations are
waiting for the hard disk to become available),
recalls Michael Geis, director of IS operations
for the chain.“Anything over two is considered
a bottleneck, meaning your disks are too slow.
We were seeing them into the 60, 80 and 100
range during peak times,” he says.
After rolling out IBM’s SAN Volume
Controller (SVC), queue lengths settled back
below two, Geis says.The two-node clustered
SVC software fronts EMC Clariion CX300 and
IBM System Storage DS4700 disk arrays, along
with two IBM Brocade SAN switches. SVC lets
Lifestyle combine storage capacity from both
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
vendors’ disk systems into a
single pool that is manageable
as a whole for greater utiliza-
tion.An onboard cache helps
speed I/O performance; SVC
acknowledges transactions
once they’ve been committed
to its cache but before they’re
sent to the underlying storage
controllers.
A key benefit of virtualizing
storage is the ability to retain
older gear, rather than doing
the forklift replacements
typically required of storage
upgrades, Geis says.“We didn’t
have to throw away our old
legacy equipment. Even though
we’d had it for a few years, it
still had a lot of performance
value to us,” he says. Lifestyle
uses the new IBM storage for
its most performance-sensitive
applications, such as its core
databases and mail server, and
uses the EMC gear for second-
and third-tier storage.
Using storage gear from
more than one vendor adds management
overhead, however.“You have to have a relation-
ship with two manufacturers and have two
maintenance contracts.There’s also expertise
to think about. Our storage team internally has
to become masters of multiple platforms,” Geis
says.
The payoff is worth it, however:“Now that
I’ve got storage virtualization in place with
the SVC, my next storage purchase doesn’t
have to be IBM. I could go back and buy EMC
again, if I wanted to, because I have this device
in-between,” Geis says.
Heterogeneous storage virtualization gives
buyers a lot more purchasing flexibility -- and
negotiating power.“Every time we add new
storage, IBM has to work in a competitive
7

Section 1: Trends • • •
situation,” Geis says.“Five years ago, the day you
made the decision to go with EMC or IBM or
HP or anyone, you might get a great discount on
the first purchase, but you were locked into that
platform for three to five years,” he says.
What Geis likes best is the flexibility the
system affords and “knowing that I don’t have
to follow in the footsteps of everybody else’s
storage practices, that I can pick and choose the
path that we feel is best for our organization.”
Getting more out of resources
A key addition to Pronto.com’s Web
infrastructure made all the difference in its
e-commerce operations. Fifteen million people
tap the Pronto.com comparison-shopping site
each month to find the best deals on 70 million
products for sale on the Web. If the site isn’t
performing, revenue suffers.
Pronto.com wanted to upgrade its load
balancers in conjunction with a move from a
hosting provider’s facility in Colorado to a Vir-
ginia data center operated by parent company
IAC (which also owns Ask.com, Match.com
and Evite, among other Internet businesses).
The New York start-up invested in more than
load balancing, however, when a cold call
from Crescendo Networks led to a trial of the
vendor’s application-delivery controllers.
Load balancing is just one aspect of today’s
application-delivery controllers, which combine
such capabilities as TCP connection manage-
ment, SSL termination, data compression,
caching and network address translation.The
devices manage server requests and offload
process-intensive tasks from content servers to
optimize Web application performance.“Our
team knew load balancing really well, but we
didn’t know optimization.And we didn’t know
that optimization would be something that we’d
really want,” recalls Tony Casson, director of
operations at Pronto.com.
When Casson and his team tried out
Crescendo’s AppBeat DC appliances, however,
they were convinced. In particular, the devices’
ability to offload TCP/IP and SSL transactions
from Web servers won them over.
A major benefit is that Pronto.com can delay
new Web-server purchases even as its business
grows.“It really has extended the life of our
server platform,”Casson says.“The need for us to
purchase new front-line equipment has been
cut in half. Each Web server can handle approxi-
mately 1.5 times the volume it could before.”
Small investment, big payoff
IT projects don’t have to be grand to be
game-changing.A low-priced desktop add-on
is yielding huge dividends for the Miami-Dade
County Public Schools.
The school district installed power-manage-
ment software on 104,000 PCs at 370 locations.
By automatically shutting down desktops that
aren’t in use, the software has reduced the
district’s average PC-on time from 20.75 hours
per day to 10.3 hours. In turn, it’s shaved more
than $2 million from the district’s $80 million
annual power bill.
Best of all, because Miami-Dade already was
using software from IT management vendor
BigFix for asset and patch management, adding
the vendor’s power-management component
cost the district just $2 more per desktop.“There
was very little effort to implement the program
once we defined the operating parameters,”
says Tom Sims, Miami-Dade’s director of
network services.
Automated shutdown
Besides saving money, the power-manage-
ment project has kick-started a new wave of
green IT efforts – something that’s as important
to the school district’s managers as it is to
users.“We all want to save energy and keep
the environment clean and functional for our
kids, more so because we are a public school
system,” Sims says.
Miami-Dade is working on the program’s
second phase, the goal of which is to let IT
administrators customize shut-down times on a
school-by-school basis.That will result in more
power saved and more reductions in carbon
emissions.
The district also is eyeing the chance for
even bigger savings by turning off air condi-
tioning units in areas where desktop computers
are powered off. IP-controlled thermostats will
enable Miami-Dade to coordinate PC and air
conditioning downtime.“The potential cost
savings is even bigger than the desktop-power
cost savings,” Sims says.
For that to happen, the IT group has been
working more closely with facilities manage-
ment teams — a collaboration Sims expects to
grow. “There are IP-driven devices that will inter-
face with all kinds of facilities equipment.These
devices allow remote management and control
by a central office via the organization’s data
network. So, the possibilities seem limitless.”
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
Staying green
Going green is more than a buzzword for
AISO.Net, which from its inception in 1997
has espoused eco-friendly operations.
Other companies are buying carbon
offsets to ease their environmental con-
science, but energy credits aren’t part of the
equation for AISO.Net. The company gets its
data center electricity from 120 roof-mounted
solar panels. Solar tubes bring in natural sun-
light, eliminating the need for conventional
lighting during the day; and air conditioning
systems are water-cooled to conserve energy.
As it did with its building infrastructure,
AISO.Net overhauled its IT infrastructure
with green savings in mind. By consolidating
dozens of commodity servers to four IBM
servers running VMware’s Infrastructure 3
software, it upped utilization while lowering
electricity and cooling loads. CTO Nail sold
the leftover servers on eBay.“Let somebody
else have the power,” he says.
For Nail, green business is good business,
and that’s what he likes best about it.“Maybe
it costs a little bit more, but it definitely pays
for itself, and it’s doing the right thing for the
environment,” he says. Customers like environ-
mentally friendly technology too:“More and
more companies are looking at their vendors
to see what kind of environmental policies
they have,” he says.
Today AISO.Net is designing a rooftop
garden for its data center; it estimates
the green roof could reduce cooling and
heating requirements by more than 50%. It’s
also looking for an alternative way to store
solar-generated power. That’s where the waste
vegetable oil comes in. Nail wants to replace
the company’s battery bank, which stores
power from the solar panels, with a more
environmentally friendly alternative. The
idea is to retrofit a small generator to run not
on diesel fuel but on recycled vegetable oil
acquired from local restaurants and heated
in 55-gallon drums. The generator in turn
would feed power to air conditioning units,
Nail says.
The idea came from seeing others use
waste vegetable oil to run cars, Nail says.“We
figured, why can’t we take that technology
and put it into something that would run our
air conditioning?” he notes.“We’re kicking that
around, trying to design it and figure out how
we’re going to implement it.”
8

Section 1: Trends • • •
Thinking outside the storage box
Unbridled growth in data storage and the rise in Web 2.0
applications are forcing a storage rethink. Is this the end
of the storage-area network as we know it?
W
By Joanne Cummings, Network World, 01/26/2009
With storage growth tracking at
60% annually, according to IDC,
enterprises face a dire situation.
Throwing disks at the problem
simply doesn’t cut it anymore.
Andrew Madejczyk, vice president
of global technology operations at
pre-employment screening com-
pany Sterling Infosystems, in New
York, likens the situation to an
episode of “House,” the popular
medical drama.
“On ‘House,’ there are two ways to
approach a problem.You treat the symptoms,
or you find out what the root cause is and
actually end the problem,” Madejczyk says.
“With storage, up until now, the path of least
resistance was to treat the symptoms and buy
more disks” – a method that surely would ignite
the ire of the show’s caustic but brilliant Dr.
Gregory House.
Were the doctor prone to giving praise, he’d
give a call out to enterprise IT managers who
are rethinking this traditional approach to
storage. He’d love that technologists are willing
to go outside their comfort zones to find a
solution, and he’d thrive on the experimenta-
tion and contentiousness that surround the
diagnosis.
House probably would find an ally in
Casey Powell, CEO at storage vendor Xiotech.
“Everybody acknowledges the problem and
understands it, but nobody’s solving it.As
technologists, we have to step back, look at the
problem and design a different way,” Powell
says.
Optimizing the SAN
Today most organizations store some
combination of structured, database-type and
unstructured file-based data. In most cases, they
rely on storage-area network (SAN) technolo-
gies to ensure efficiencies and overall storage
utilization, keeping costs down as storage needs
increase.
In and of themselves, SANs aren’t enough,
however. Enterprises increasingly are turning to
technologies that promise to provide an even
bigger bang for the buck, including these:
• Data deduplication, which helps reduce
redundant copies of data so firms can shrink
not only storage requirements but also
backup times.
• Thin provisioning, which increases storage
utilization by making storage space that has
been overprovisioned for one application
available to others on an as-needed basis.
• Storage tiering, which uses data policies
and rules to move noncritical data to slower,
less expensive storage media and leave
expensive Tier 1 storage free to handle only
the most mission-critical applications.
• Storage resource management software,
which helps users track and manage storage
usage and capacity trends.
“In the classic SAN environment, these tools
don’t just provide a partial solution.They allow
you to make fundamental improvements,” says
Rob Soderbery, senior vice president of Syman-
tec’s Storage and Availability Management
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
Group. Clients that have pursued
these strategies even have been able
to freeze storage spending for a year
at a time, he says.“And when they get
back on the storage spending cycle,
they get back on at about half the
spending rate they were at before,”
he adds.
Although few IT executives
report such dramatic reductions in
storage spending, many are pursuing
such strategies.
At Sterling, for example, moving
from tape- to disk-based backups
via Sepaton’s S2100-ES2 virtual tape
library reduced the time it takes for
nightly backups from 12 to just a few
hours, Madejczyk says. Sepaton’s deduplication
technology provides an added measure. In
addition, he has virtualized more than 90% of
his server environment,“reducing our footprint
immensely,’” and implemented EMC thin
provisioning and storage virtualization.
Still, his company’s storage needs keep
growing, Madejczyk says.“In this economy,
Sterling is being very responsible and careful
about what we spend on,” he says.“We’re
concentrating on the data-management part of
the problem, and we’re seeing results. But it’s a
difficult problem to solve.”
Tom Amrhein, CIO at Forrester Construction
in Rockville, Md., has seen similar growth.The
company keeps all data associated with its
construction projects in a project management
database, so the vast majority of that stored data
is structured in nature. Regulatory and compli-
ance issues have led to increased storage
needs nonetheless.
“Most companies need to keep their tax
records for seven years, and that’s as long as
they need to keep anything,”Amrhein says.
“But tax records are our shorter-cycle data.
Depending on the jurisdiction, the time we
could be at fault for any construction defect is
up to 10 years – and we’re required to have the
9

Section 1: Trends • • •
same level of discovery response for a project
completed nine years ago as we would for a
project that’s been closed out two weeks.”
Forrester Construction has cut down a bit
on storage needs by keeping the most data-
intensive project pieces – building drawings,
for example – on paper. “Because the scanning
rate is so high and paper storage costs so low,
retaining those as physical paper is more cost-
effective,”Amrhein says.
The real key to keeping costs in check,
however, is storage-as-a-service, Amrhein says.
IT outsourcer Connectria hosts the company’s
main application servers, including Microsoft
Exchange, SQL Server and SharePoint;
project management, finance, CRM and Citrix
Systems. It handles all that storage, leaving
Forrester Construction with a predictable, flat
monthly fee.
“I pay for a set amount of gigabytes of
storage as part of the SLA [service-level agree-
ment], and then I pay $1 per gig monthly for
any excess,”Amrhein explains.“That includes
the backup, restore and all the services around
those. I’m paying $25K a month to Connectria,
plus paying for about 10GB over my SLA
volume.That overage is a wash.”
For the firm’s unstructured data, Forrester
Construction uses Iron Mountain’s Connected
Backup for PCs service, which automatically
backs up all PCs nightly via the Internet. If a
PC is not connected to the Internet at night,
the user receives a backup prompt on the next
connection.
“With 60% of the people out of the office,
this is perfect for us,”Amrhein says.“Plus, Iron
Mountain helps us reduce the data volume by
using deduplication,” he says.“Even for people
on a job site with a wireless card or low-speed
connection, it’s just a five- or 10-minute thing.”
Still, the unstructured side is where the
construction company sees its biggest storage
growth. E-mail and saved documents are the
biggest problem areas.
The rise in Web 2.0 data
Forrester Construction is not alone there. In
the enterprise, IDC reports, structured, transac-
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
tional data will grow at a 27.3% compounded
annual rate over the next three to five years.
The rise in unstructured, file-based data will
dwarf that growth rate, however. IDC expects the
amount of storage required for unstructured,
file-based data to increase at an unprec-
edented 69.4% clip. By 2010, enterprises for the
first time will find unstructured storage needs
outstripping traditional, structured storage
demands.
The rub here is that although SANs are
extremely efficient at handling structured,
transactional data, they are not well optimized
for unstructured data. “SANs are particularly
ill-suited to Web 2.0, scale-out, consumer-ori-
ented-type applications,” Symantec’s Soderbery
says.“No. 1, the applications’ architecture is
scale-out, so you have hundreds or thousands
of systems working on the same problem
instead of one big system, like you would have
with a database.And SANs aren’t designed that
way.And No. 2, these new applications – like
storing photos on Facebook or video or display
ads or consumer backup data – are tremen-
dously data intensive.”
Symantec hit the wall with this type of data
in supporting its backup-as-a-service offering,
which manages 26 petabytes of data, Soderbery
says.“That probably puts us in the top 10 or
20 storage consumers in the world.We could
never afford to implement a classic Tier 1 SAN
architecture,” he says.
Instead, Symantec went the commodity
path, using its own Veritas Storage Foundation
Scalable File Server software to tie it all together.
“The Scalable File Server allows you to add
file server after file server, and you get a single
namespace out of that cluster of file servers.
This in turn allows you to scale up your applica-
tion and the amount of data arbitrarily.And the
software runs on pure commodity infrastruc-
ture,” Soderbery explains. Plus, the storage
communicates over a typical IP network vs. a
more expensive Fibre Channel infrastructure.
Symantec’s approach is similar to that of the
big cloud players, such as Google and Amazon.
com.“We happen to build packaged software to
enable this, whereas some of the early adopters
built their own software and systems. But it all
works the same way,” Soderbery says.
The prudent approach to storage as it
continues to grow, Soderbery says, is to optimize
and use SANs only for those applications that
merit them – such as high-transaction, mission-
critical ERP applications. Look to emerging
commodity-storage approaches for more
10

Section 1: Trends • • •
scale-out applications, such as Web 2.0, e-mail
and interactive call-center programs.
Does that mean enterprises need to support
SANs and new cloud-like scale-out architec-
tures to make sure they’re managing storage as
efficiently as possible? Perhaps.
Eventually, however, the need to support
unstructured, scale-out data will trump the
need to support structured, SAN-oriented data,
IDC research shows.With that in mind, smart
organizations gradually will migrate most appli-
cations off SANs and onto new, less expensive,
commodity storage setups.
A new enterprise approach
One interesting strategy could provide an
evolutionary steppingstone in the interim:
using Web services. Championed primarily by
Xiotech, the idea is to use the Web-services APIs
and standards available from such organiza-
tions as the World Wide Web Consortium
(W3C) as the communications link between
applications and storage.
“The W3C has a nifty, simple model for how
you talk between applications and devices. It
includes whole sets of standards that relate to
how you provision resources in your infrastruc-
ture, back to the application,” says Jon Toigo,
CEO of analyst firm Toigo Partners International.
“All the major application providers are
Web-services enabled in that they ask the
infrastructure for services. But nobody on the
storage hardware side is talking back to them.”
Nobody, that is, except Xiotech.
Xiotech’s new Intelligent Storage Element
(ISE) is the first storage ware to talk back,
although other vendors quickly are readying
similar technology,Toigo says. ISE, based on
technology Xiotech acquired with Seagate
Technology, is a commodity building-block of
storage, supporting as many as 40 disk drives
plus processing power and cache that can be
added to any storage infrastructure as needed.
Xiotech claims ISE can support anything from
high-performance transactional processing
needs to scale-out Web 2.0 applications.
All storage vendors should work to Web-
services-enable their hardware and software so
they can communicate directly with applica-
tions, Xiotech’s Powell says.This would preclude
vendor lock-in and let enterprises build storage
environments using best-in-breed tools instead
of sticking with the all-in-one-array approach.
“They’d be able to add more storage, services or
management, without having to add everything
monolithically to a SAN,” Powell says.
Eventually Web services support will have
virtualized storage environments realizing even
greater efficiencies, to the point where applica-
tions themselves provision and de-provision
storage.“Today, when we provision storage, we
have to guess, and typically, we either over- or
underprovision,” Powell says.“And then, when
the user is no longer using it, we seldom go
back and reclaim the storage. But the applica-
tion knows exactly what it needs, and when it
needs it.Via Web services, it can request what
it needs on the fly, and as long as its request is
within the parameters and policies we set up
initially, it gets it.”
Web services already have proved an effi-
cient storage tack at ISE user Raytown Quality
Schools in Missouri, says Justin Watermann,
technology coordinator for the school system.
The system went with Xiotech shortly after it
moved to a new data center and created an
all-virtual server infrastructure.A big plus has
been Xiotech’s Virtual View software, which uses
Web services to communicate with VMware’s
VirtualCenter management console for its ESX
servers,Watermann says. He can manage his
virtualized server and storage infrastructure
from a single console.
“When you create a new data store,Virtual
View shows you what port and LUN [logical
unit number] is available to all of your ESX
hosts in that cluster,”Watermann says.“And
when you provision it, it uses Web services to
communicate with VirtualCenter, and says,‘OK,
this is the data store for these ESX hosts.’ And
you automatically have the data store there and
available to use.You don’t even have to refresh
or restore.”
That bit of automation saves on administra-
tion, but enabling the application to do the
provisioning and de-provisioning would be an
even greater boon,Watermann says.“It’s really
hard to get more staff, and you only have so
many hours in the day. If you don’t have to tie
your staff up with the repetitive tasks of carving
up space and assigning it, so much the better.”
If they build it . . .
Now Watermann is using a Web service
from Xiotech partner Eagle Software to improve
the school system’s backup times.
Eagle, a storage reseller, provides Raytown
with backup software from CommVault
Systems.“The Web-services tool lets us mirror
our data, pause that mirror, attach that to the
backup server, back it up, then disconnect it
from the backup server, unpause the mirror, and
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
re-sync that data so we don’t have to push it
across our Ethernet network,”Watermann says.
Like Eagle, companies are starting to
develop pieces to put Web-services-enabled
storage all together,Watermann adds. Such
small, point approaches are the norm today,
but experts say that in five or 10 years, every
application and every device will use some
kind of software glue, such as Web services, to
provide a range of storage and IT services in an
automated, efficient manner.
“It would have to be levels of software
that create services that include computing
resources, network and storage infrastructure,
and the retention and reliability metrics
associated with all of those components,”
Symantec’s Soderbery says.“It will be a combi-
nation of the point solutions we see now, like
VMware and SAN virtualization via Cisco and
Brocade, plus thin provisioning, replication
and deduplication.We’re going to require all
of those things to work in concert with a level
of software that ties them together cohesively
to provide those appropriate levels of service
to the application.”
Others, including Ken Steinhardt, CTO
for customer operations at EMC, are less
optimistic.“If someone could write something
magical that does things that we’d love to have,
wouldn’t that be great?” he asks rhetorically.
That would take a miracle, Steinhardt says.
“The tools to write software are out there, but
the catch is it’s just not that simple.You need to
be able to have a solution that works broadly
across a range of applications as well, and
typical, highly consolidated environments run
a mix of broad, diverse apps, not just a single
application. I don’t see it happening,” he says.
The Web services model is too much of a
stretch to Steinhardt:“From a storage perspec-
tive,Web services are a completely separate
issue.We’re talking about storing zeros and ones
on a storage device.That’s pretty agnostic to the
individual application and always has been,”
he says.
Not so, analyst Toigo asserts.Web services
provide a common framework, so by default,
they can support every application.“People
need to tell their vendors,‘Look, I’m not buying
your junk anymore if I can’t manage it using
this common [Web services] metaphor,’” he
says.“That puts you in the driver’s seat.”
Cummings is a freelance writer in North
Andover, Mass. She can be reached at joanne@
cummingslimited.com.
11
 EXECUTIVE GUIDE
Back to TOC

Next-generation data centers:

opportunities abound
Section 2
Emerging technologies
One in 10 companies deploying internal clouds

E
By Jon Brodkin, Network World, 12/15/2008

Enterprise IT shops are start- At the center of cloud computing is prise’s distributed computing resources.
ing to embrace the notion of a services-oriented interface between a It’s not clear exactly how fast this
provider and user, enabled by virtualiza- technology will advance. VMware plans to
building private clouds, model- tion, says Gartner analyst Thomas Bittman. release what might be considered a meta
ing their infrastructure after “When I move away from physical to operating system with its forthcoming
public service providers such virtual machines for every requirement, Virtual Datacenter Operating System.
as Amazon and Google. But I’m drawing a layer of abstraction,” Bitt- But cloud computing is less a new
while virtualization and other man says.“What virtualization is doing technology than it is a way of using
is you [the customers] don’t tell us what technology to achieve economies of
technologies exist to create server to get, you just tell us what service scale and offer self-service resources that
computing pools that can allo- you need.” are available on demand, The 451 Group
cate processing power, storage While virtualization technologies for says. Numerous enterprises are taking on
and applications on demand, servers, desktops and storage are readily this challenge of building more flexible,
available, Gartner says to get all the ben- service-oriented networks using existing
the technology to manage those
efits of cloud-computing enterprises will products and methodologies.
distributed resources as a whole need a new meta operating system that Thin clients and virtualization is the
is still in the early stages. controls and allocates all of an enter- key for Lenny Goodman, director of the
The corporations building their own
private clouds include such notable
names as Bechtel, Deutsche Bank, Morgan
Stanley, Merrill Lynch and BT, according
to The 451 Group. The research firm found
in a survey of 1,300 corporate software
buyers that about 11% of companies are
deploying internal clouds or planning
to do so. That may not seem like a huge
proportion, but it’s a sign that private
clouds are moving beyond the hype cycle
and into reality.
“It’s definitely not hype,” says Vivek
Kundra, CTO for the District of Columbia
government, which plans to blend IT
services provided from its own data center
with external cloud platforms like Google
Apps.“Any technology leader who thinks
it’s hype is coming at it from the same
place where technology leaders said the
Internet is hype.”

Sponsored by APC
www.apc.com 12
 EXECUTIVE GUIDE
Back to TOC

Section 2: Emerging technologies • • •

desktop management group at Baptist While Kundra and Goodman have cloud. To build it on your own is quite an
Memorial Health Care in Memphis, Tenn. begun thinking of themselves as internal ambitious project. Where I see more enter-
Baptist uses 1,200 Wyse Technology thin cloud providers, many other IT shops prises going is down the path of renting
clients, largely at patients’ bedsides, and view cloud computing solely as it relates clouds that have already been built out by
delivers applications to them using Citrix to acquiring software-as-a-service and some service provider.”
XenApp application virtualization tools. on-demand computing resources from There is room for both internal and
Baptist also is rolling out virtual, customiz- external providers such as Salesforce. external cloud computing within the
able desktops to those thin clients using “Cloud computing is definitely the same enterprise, though. In Gartner’s view,
Citrix XenDesktop. hot buzzword,” says Thomas Catalini, a corporations that build their own private
Just as Internet users can access member of the Society for Information clouds will also access extra capacity from
Amazon, Google, Barnes & Noble or any Management and vice president of tech- public providers when needed. During
Web site they wish to use from anywhere, nology at insurance brokerage William times of increased demand, the meta
Goodman wants hospital workers to be Gallagher Associates in Boston.“To me it operating system as described by Gartner
able to move among different devices and means outsourcing to a hosted provider. will automatically procure additional
have the same experience. I would not think of it in terms of cloud capacity from outside sources, and users
“You get the advantage of taking that computing to my own company. [Out- won’t necessarily know whether they are
entire experience and making it roam sourcing] relieves me of having to buy using computing capacity from inside or
without the nurse having to carry or push hardware, software and staff to support a outside the firewall.
anything,” he says.“They can move from particular solution.” While “cloud” might strike some as
device to device.” Analyst Theresa Lanowitz of Voke, a an overused buzzword, Kundra views
Goodman also says a cloud-based model strong proponent of using external clouds cloud computing as a necessary transi-
where applications and desktops are deliv- to reduce management costs, says building tion toward more flexible and adaptable
ered from a central data center will make internal clouds is too difficult for most IT computing architectures.
data more secure, because it’s not being shops. “I believe it’s the future,” Kundra says.
stored on individual client devices. “That is a cumbersome task,” she says. “It’s moving technology leaders away
“If we relocate that data to the data “One of the big benefits of cloud com- from just owning assets, deploying assets
center by virtualizing the desktop, we can puting is the fact that you have companies and maintaining assets to fundamentally
back it up, we can secure it, and we can out there who can offer up things in a changing the way services are delivered.”
provide that data to the user wherever
they are,” he says.
In the Washington, D.C., government,
Kundra came on board in March 2007 with
the goal of establishing a DC.gov cloud
that would blend services provided from
his own data center with external cloud
platforms like Google Apps. Washington
moved aggressively toward server virtual-
ization with VMware, and made sure it had
enough network bandwidth to support
applications hosted on DC.gov.
The move toward acting as an internal
hosting provider as well as accessing
applications outside the firewall required
an increased focus on security and user
credentials, Kundra says. But that was a
necessary part of giving users the same
kind of anytime, anywhere access to data
and applications they enjoy as consumers
of services in their personal lives.
“The line is blurred,” he says.“It used
to be you would come to work and only
work. The blurring started with mobile
technologies, BlackBerries, people doing
work anytime, anywhere.”

Sponsored by APC
www.apc.com 13
 EXECUTIVE GUIDE
Back to TOC

Section 2: Emerging technologies • • •

Google, Microsoft spark interest in modular data

centers, but benefits may be exaggerated
Experts question energy efficiency claims

I
By Jon Brodkin, Network World, 10/13/2008

Interest in modular data centers

is growing, fueled by high-profile
endorsements from Microsoft and
Google. But the model raises new
management concerns, and effi-
ciency claims may be exaggerated.
Modular, containerized data centers
being sold by vendors such as IBM, Sun
and Rackable Systems fit storage and
hundreds, sometimes thousands, of servers
into one large shipping container with its
own cooling system. Microsoft, using Rack-
able containers, is building a data center
outside Chicago with more than 150 con-
tainerized data centers, each holding 1,000
to 2,000 servers. Google, not to be outdone,
secured a patent last year for a modular
data center that includes “an intermodal
shipping container and computing systems
mounted within the container.”
Rackable Systems’ ICE Cube portable data center can be fitted
To hear some people tell it, a container- with as many as 22,400 processing cores in 2,800 servers.
ized data center is far easier to set up than
a traditional data center, easy to manage container part of its green strategy, says the according to Sams,“in almost all cases
and more power-efficient. It also should same efficiency can be achieved within they’re comparing a highly dense [con-
be easier to secure permits, depending on the four walls of a normal building. tainer] to a low-density [traditional data
local building regulations. Who wouldn’t IBM touts a “modular” approach to data center].”
want one? center construction, taking advantage Containers also eliminate one scal-
If a business has a choice between of standardized designs and predefined ability advantage related to cooling found
buying a shipping container full of servers, components, but that doesn’t have to be in traditional data centers, according to
and building a data center from the in a container.“We’re a huge supporter of Sams. Just as it’s more efficient to cool an
ground up, it’s a no-brainer, says Geoffrey modular. We’re a limited supporter of con- apartment complex with 100 living units
Noer, a vice president at Rackable, which tainer-based data centers,” says Steve Sams, than it is to cool 100 separate houses, it’s
sells the ICE Cube Modular Data Center. vice president of IBM Global Technology more cost-effective to cool a huge data
“We don’t believe there’s a good reason Services. center than many small ones, he says. Air
to go the traditional route the vast majority Containers are efficient because they conditioning systems for containerized
of the time,” he says. pack lots of servers into a small space, and data centers are locked inside, just like the
But that is not the consensus view by use standardized designs with modular servers and storage, making true scalability
any stretch of the imagination. Claims components, he says. But you can deploy impossible to achieve, he notes.
about efficiency are over-rated, according storage and servers with the same level of Gartner analyst Rakesh Kumar says
to some observers. density inside a building, he notes. it will take a bit of creative marketing
Even IBM, which offers a Portable Container vendors often tout 40% for vendors to convince customers that
Modular Data Center and calls the to 80% savings on cooling costs. But containers are inherently more efficient

Sponsored by APC
www.apc.com 14

Section 2: Emerging technologies • • •
than regular data centers. Gartner is still
analyzing the data, but as of now Kumar
says,“I don’t think energy consumption
will necessarily be an advantage.”
Finding buyers
That doesn’t mean there aren’t any
advantages, however. A container can
be up and running within two or three
months, eliminating lengthy building and
permitting times. But if you need an instant
boost in capacity, why not just go to a
hosting provider, Kumar asks.
“We don’t think it’s going to become a
mainstream solution,” he says.“We’re strug-
gling to find real benefits.”
Kumar sees the containers being more
suited to Internet-based,“hyper-scale”
companies such as Google, Amazon and
Microsoft. Containerized data centers offer
scalability in big chunks, if you’re willing to
buy more containers. But they don’t offer
scalability inside each container, once it
has been filled, he says.
Container vendors tout various benefits,
of course. Each container is almost fully
self-contained, Rackable’s Noer says.
Chilled water, power and networking are
the only things from the outside world
that must be connected to each one, he
says. Rackable containers, which can be
fitted with as many as 22,400 processing
cores in 2,800 servers, are water-tight,
fitted with locks, alarms and LoJack-like
tracking units. Sun’s Modular Data Center
can survive an earthquake — the company
made sure of that by testing it on one
of the world’s largest shake tables at the
University of California in San Diego.
A fully-equipped Rackable ICE Cube
costs several million dollars, mostly for the
servers themselves, Noer says. The con-
tainer pays for itself with lower electricity
costs due to an innovative Rackable design
that maximizes server density, Noer says.
But it’s still too early to tell whether
containerized data centers are the way of
the future.“We’re just at the cusp of broad
adoption,” Noer says.
Potential use cases for containers
include disaster recovery, remote locations
such as military bases, or big IT hosting
companies that would prefer not to build
brick-and-mortar data centers, Kumar says.
A TV crew that follows sporting events
may want a mobile data center, says Robert
Bunger, director of business development
for American Power Conversion. APC
doesn’t sell its portable data center, but
in 2004 it built one into a tractor-trailer
as a proof-of-concept. It was resilient.“We
pulled that trailer all over the country” for
demos, Bunger notes.
But APC isn’t seeing much demand,
except in limited cases. For example, a
business that needs an immediate capacity
upgrade but is also planning to move its
data center in a year might want a con-
tainer because it would be easier to move
than individual servers and storage boxes.
“I think it is key that the combination of
virtualization and distributed infrastructure
produce a container that can be out of service
without impacting the application as a whole.”
Lee Kirby, general manager, Lee Technologies
UC-San Diego bought two of Sun’s
Modular Data Centers. One goal is to
contain the cost of storing and processing
rapidly increasing amounts of data, says
Tom DeFanti, principal investigator of the
school’s GreenLight energy efficiency
research project. But it will take time to see
whether the container approach is more
efficient.“The whole idea is to create an
experiment to see if we can get more work
per watts,” DeFanti says.
The Modular Data Center is not as con-
venient to maintain as a regular computer
room, because there is so little space to
maneuver inside, he says.”But it seems to
me to be an extremely well-designed and
thought-out system,” DeFanti says.“It gives
us a way of dealing with the exploding
amount of scientific computing that we
need to do.”
Beware vendor lock-in
Before purchasing a containerized data
center, enterprises should consider several
issues related to their manageability and
usefulness. Vendors often want you to fill
the containers with only their servers,
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
Kumar notes. Besides limiting flexibility
at the time of purchase, this raises the
question of what happens when those
servers reach end-of-life. Will you need the
vendor to rip out the servers and put new
ones in, once again limiting your choice of
technology?
“At the moment, most vendors will fill
their containers only with their servers,”
Kumar says.
IBM, however, says it uses industry-
standard racks in its portable data center,
allowing customers to buy whatever
technology they like.
DeFanti said Sun’s Modular Data
Center allows him the flexibility to buy a
heterogeneous mix of servers and storage.
Rackable, though, steers customers toward
either its own servers or IBM BladeCenter
machines through a partnership with IBM.
“I think vendors are learning that
people want more flexibility,” DeFanti says.
Another consideration is failover capa-
bilities, says Lee Kirby, who provides site
assessments, data center designs and other
services as the general manager of Lee
Technologies. If one container goes down,
its work must be transferred to another.
Server virtualization will help provide this
failover capability, and also make it easier
to manage distributed containerized data
centers — an important consideration for
customers who want to distribute com-
puting power and have it reside as close to
users as possible, Kirby says.
“I think it is key that the combination
of virtualization and distributed infrastruc-
ture produce a container that can be out
of service without impacting the applica-
tion as a whole,” Kirby says.
15

Section 2: Emerging technologies • • •
The big data-center synch
With Fibre Channel over Ethernet
now a reality, enterprises prep for
data and storage convergence
F
By Dawn Bushaus, Network World, 01/26/2009
For the County of Los Angeles – with
more than 10 million residents, the
nation’s largest county – the tim-
ing couldn’t be better for the arrival
of Fibre Channel over Ethernet on
the data-center scene.
L.A. County has undertaken a server virtu-
alization initiative, a data-center redesign and
construction of a new data center scheduled
for completion in late 2011.“When we put
those three things together, we knew we
needed a new design that could accommo-
date them all,” says Jac Fagundo, the county’s
enterprise solutions architect.
Because it can unite data and storage
networks on a 10 Gigabit Ethernet fabric,
FCoE enabled Fagundo to turn that design
vision into a reality.
The plan for the new data center calls
for extending virtualization beyond hosting
servers to data and storage networks,
and includes an overhaul of the county’s
stovepipe approach to storage and server
deployments.“We realized we were going to
have to integrate the storage,” Fagundo says,
“so we decided, why not go a step further
and integrate the storage and IP networks in
the data center?”
When the county started looking at
FCoE as an option, it turned to Cisco, its
Ethernet switch vendor. It completed a
proof-of-concept trial using the Nexus
7000 and 5000 switches – switches that
represent Cisco’s biggest push yet into the
enterprise data center. The 5000 supports a
prestandard version of FCoE, while the 7000
will be upgraded with FCoE features once
the American National Standards Institute
finalizes the FCoE standards, probably
in late 2009. Cisco currently is the only
vendor offering FCoE support. Its chief Fibre
Channel competitor, Bro-
cade Communications,
is beta-testing an FCoE
switch. Juniper Networks
also is considering
adding FCoE capability to its data-center
switches – if customer demand warrants it,
the company says.
L.A. County is replacing eight Cisco Cata-
lyst 6500 Ethernet switches with two Nexus
7000 switches in its data-center core, and will
add a Nexus 5000 switch to the data-center
distribution network. The county expects to
have live traffic running through the switches
by spring.
“This is a multiyear project where the
platform features will evolve as we migrate
into it,” Fagundo says.“Our next-generation
data center, of which the Nexus switches and
FCoE are a part, is a design that we plan to
implement over the next five years.”
A technology for tough times
For large enterprises operating distinct
Fibre Channel storage-area networks (SAN)
and Ethernet data networks, FCoE’s potential
benefits and cost savings are promising,
even though the technology requires new
hardware.
“The rule of thumb is that during tight
economic times, incumbent technologies
do well and it’s not wise to introduce a new
technology - unless your proposition is one
of economic favor,” says Skip Jones, president
of the Fibre Channel Industry Association
(FCIA).“And that’s exactly what FCoE is.”
Jones has an obvious bias toward FCoE,
but Bob Laliberte, analyst at Enterprise
Strategy Group, agrees.“The economy will
certainly get more companies to look at
FCoE,” he says.“If vendors can show how
FCoE can reduce the cost of powering,
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
cooling and cabling, that will be a compel-
ling reason for companies to look at it.”
At a minimum, adopting FCoE lets a
company consolidate server hardware and
cabling at the rack level. That’s the first step
in moving to FCoE. Then more consolida-
tion can take place in the core switching
fabric, and that reduces costs even further.
Ultimately, storage arrays will support FCoE
natively (NetApp already has announced
such a product), making end-to-end FCoE
possible.
L.A. County’s Fagundo expects substantial
cost savings from using FCoE, although he
declined to share detailed estimates. He did
say, however, that power consumption alone
will be cut in half when the eight Cisco
Catalyst switches are consolidated into two
Nexus switches. In addition, FCoE will let the
county reduce server hardware and cabling
by using a converged network adapter
(CNA) to replace Ethernet network interface
cards and Fibre Channel host bus adapters.
The cost savings in infrastructure,
components, edge devices and cabling are
compelling reasons to consider FCoE, agrees
Ian Rousom, corporate data-center network
architect at defense contractor Lockheed
Martin in Bethesda, Md.“Cabling is often
overlooked, but you can potentially elimi-
nate one of two cable plants using FCoE.
If you consolidate racks, you eliminate the
biggest source of cable sprawl in the data
center.”
Cisco estimates that enterprises can expect
a 2% to 3% savings in power consumption
during FCoE’s rack-implementation phase,
16

Section 2: Emerging technologies • • •
and a total of about 7% savings once FCoE
is pushed into the data-center core.
Tall orders for FCoE
The promise of FCoE lies in a merged
data and storage network fabric, but
making the integration happen isn’t easy
and shouldn’t be done without careful
planning. Typically in an enterprise,
network and storage management teams
don’t work together. That has to change
with FCoE.
“Certainly at a minimum, collaboration
between Ethernet and Fibre Channel
departments will have to increase, and
merging the two would not be a bad idea,”
says Rousom, who is considering FCoE.
“The people who manage the infrastruc-
ture have to agree on policy, change
processes, troubleshooting policies and
the configuration management process.”
That could be a tall order in many
enterprises, FCIA’s Jones says.“Network and
storage administrators drink different kinds
of tea; they’re different breeds. There will
be some turf wars.”
L.A. County has dealt with those
differences by combining its data-center
storage and network groups under a single
management umbrella. Initially, the groups’
engineering staffs were concerned about
merging, Fagundo says.“Then they started
contributing to the design and proof of
concept, and now they’re happy about it.”
Beyond cultural issues, enterprises must
communicate with their server vendors to
find out whether their platforms support
CNAs as opposed to separate Ethernet
and Fibre Channel adapters, Rousom says.
IT managers also must consider security
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
ramifications.“If the Fibre Channel network
has a firewall or special security features,
the FCoE products may not offer those as
part of their feature sets,” he explains.
For now, FCoE is gaining traction with
such early adopters as L.A. County, where
unique circumstances have made the time
right to consider a big change. Many more
IT executives will take a more cautions
approach, however, as Rousom is doing.
“The fact that FCoE is still in the
standards process tells me that there is still
potential for change at the hardware level
before this is all complete,” Rousom says.
“That, more than anything, is preventing
me from buying it.”
Bushaus is a freelance technology writer
in the Chicago area. She can be reached at
dbushaus@mindspring.com.
17

Next-generation data centers:
opportunities abound
Section 3
Best practices
Seven tips for succeeding with virtualization
Experts share best practices for optimizing strategic
virtualization initiatives
A
By Denise Dubie, Network World, 10/20/2008
As server virtualization projects
gain scale and strategic value,
enterprise IT managers must move
quickly beyond tactical approaches
to achieve best results.
Consider these Gartner forecasts: More
than 4 million virtual machines will be
installed on x86 servers this year, and the
number of virtualized desktops could grow
from less than 5 million in 2007 to 660 mil-
lion by 2011. The popularity of virtualizing
x86 server and desktop resources has many
enterprise IT managers reassessing ways
to update already virtualized network and
storage resources, too.
Virtualization’s impact will spread
beyond technology changes to opera-
tional upheaval. Not only must enterprise
IT executives move from a tactical to a
strategic mindset but they also must shift
their thinking and adjust their processes
from purely physical to virtual.
“Enterprise IT managers are going to
have to start thinking virtual first and learn
how to make the case for virtualization
across IT disciplines,” says James Staten,
principal analyst at Forrester Research.
“This will demand they change processes.
Technologies can help, but if managers
don’t update their best practices to handle
virtual environments, nothing will get
easier.
Here enterprise IT managers and
industry watchers share best practices they
say will help companies seamlessly grow
from 30 to 3,000 virtual machines without
worry.
1. Approach virtualization
holistically
Companies considering standardizing
best practices for x86-based server virtual-
ization should think about how they plan
to incorporate desktop, application, storage
and network virtualization in the future.
IT has long suffered from a silo men-
tality, with technology expertise living
in closed clusters. The rapid adoption of
virtualization could exacerbate already
strained communications among such IT
domains as server, network, storage, security
and applications.
“This wave of virtualization has started
with one-off gains, but to approach the
technology strategically, IT managers need
to look to the technology as achieving
more than one goal across more than one
IT group,” says Andi Mann, research director
at Enterprise Management Associates.
To do that, an organization’s virtualiza-
tion advocates should champion the
technology by initiating discussions among
various IT groups and approaching vendors
with a broad set of requirements that
address short- and long-term goals.Vendors
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
with technologies in multiple areas, such as
servers and desktops, or with partnerships
across IT domains could help IT managers
better design their virtualization-adoption
road maps. More important, however, is
preventing virtualization implementations
from creating more problems via poor
communications or antiquated organiza-
tional charts, industry watchers say.
“With ITIL and other best-practice frame-
works, IT has become better at reaching
out to other groups, but the speed at which
things change in a virtual environment
could hinder that progress,” says Jasmine
Noel, a principal analyst at Ptak, Noel and
Associates.“IT’s job is to evolve with the
technology and adjust its best practices,
such as change management, to new
technologies like virtualization.”
2. Identify and inventory virtual
resources
Understanding the resources available
at any given time in a virtual environment
requires enterprise IT managers to enforce
strict processes from a virtual machine’s
birth through death.
Companies need a way to identify
virtual machines and other resources
throughout their life cycles, says Pete Lind-
strom, research director at Spire Security.
The type of virtual-machine tagging he
suggests would let IT managers “persistently
18

Section 3: Best practices • • •
identify virtual-machine instances over an
extended period of time,” and help to main-
tain an up-to-date record of the changes
and patches made to the original instance.
The process would provide performance
and security benefits because IT managers
could weed out problematic virtual
machines and keep an accurate inventory
of approved instances.
“The ability to track virtual machines
throughout their life cycles depends on a
more persistent identity scheme than is
needed in the physical world. IT needs to
know which virtual resources it created
and which ones seemed to appear over
time,” Lindstrom explains.“The virtual world
is so much more dynamic that IT will need
granular identities for virtual machines
and [network-access control] policies that
trigger when an unknown virtual machine
is in the environment. Rogue virtual
machines can happen on the client or the
hypervisor.”
For instance, using such tools as BMC
Software’s Topology Discovery, EMC’s Appli-
cation Discovery Manager or mValent’s
Integrity, an IT manager could perform an
ongoing discovery of the environment and
track how virtual machines have changed.
Manual efforts couldn’t keep pace with
the configuration changes that would
occur because of, say,VMware VMotion
or Microsoft Live Migration technologies.
“IT has to stay on top of a lot more data
in a much more dynamic environment,”
O’Donnell says.
3. Plan for capacity
Just because virtual machines are
faster to deploy than physical ones, the
task shouldn’t be taken lightly.“If you are
not careful, you can have a lot of virtual
machines that aren’t being used,” says Ed
Ward, senior technical analyst at Hasbro in
Pawtucket, R.I. He speaks from the experi-
ence of supporting 22 VMware ESX host
servers, 330 virtual machines, 100 worksta-
tions and 250 physical machines.
To prevent virtual-machine sprawl
and to curb spending for licenses and
power for unused machines, Ward says he
uses VKernel’s Capacity Analyzer virtual
appliance. It alerts him to all the virtual
machines in his environment, even those
he thought he had removed.
“There are cases in which you build a
virtual machine for test and then for some
reason it is not removed but rather it’s
still out there consuming resources, even
though it is serving no purpose,” Ward says.
“Knowing what we already have and plan-
ning our investments based on that helps.
We can reassign assets that have outlived
their initial purpose.”
When they create virtual machines, IT
managers also must plan for their deletion.
“Assign expiration dates to virtual machines
when they are allocated to a business
unit or for use with a specific application;
and when that date comes, validate the
need is no longer there and expire the
resource,” Forrester’s Staten says.“Park a
virtual machine for three months and if it
is no longer needed, archive and delete.
Archiving keeps options open without
draining storage resources or having the
virtual machine sitting out there consuming
compute resources.”
4. Marry the physical and virtual
IT managers must choose the applica-
tions supported by virtual environments
wisely, say experts, who warn that few if
any IT services will rely only on the virtual
infrastructure.
“While some environments could
support virtual-only clusters for testing,
the more common scenario would have,
for instance, two virtual elements and one
physical one supporting a single IT service,”
says Cameron Haight, a Gartner research
vice president.“IT still needs to correlate
performance metrics and understand the
profile of the service that spans the virtual
and physical infrastructures. Sometimes
people are lulled into a false sense of
security thinking the tools will tell them
what they need to know or just do [the
correlation] for them.”
IT managers should push their vendors
for reporting tools that not only show what’s
happening in the virtual realm but also
display the physical implications -- and
potentially the cause -- of an event. Detailed
views of both environments must be mar-
ried to correlate why events take place in
both realms.
For instance, if utilization on a host
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
server drops from 20% to 10%, it would be
helpful to know the change came about
because VMware Distributed Resource
Scheduler (DRS) moved virtual machines
to a new physical server, Haight says. In
addition, knowing when and where virtual
machines migrate can help prevent a
condition dubbed “VMotion sickness” from
cropping up in virtual environments. This
occurs when virtual machines move repeat-
edly across servers -- and bring problems
they might have from one server to the
next, Haight says. Proper reporting tools,
for example, could help an administrator
understand that a performance problem
is traveling with a virtual machine unbe-
known to DRS.
5. Eliminate virtual blind spots
The fluid environment created by
virtualization often includes blind spots.
“We monitor all physical traffic, and there is
no reason why we wouldn’t want to do the
same for the virtual traffic. It’s a huge risk
not knowing what is going on, especially
when the number of virtual servers is
double what you have for physical boxes,”
says Nick Portolese, senior manager of data
center operations at Nielsen Mobile in San
Francisco.
Portolese supports an environment with
about 30 VMware ESX servers and 500 to
550 virtual machines. Early on, he realized
he wasn’t comfortable with the amount
of network traffic he could monitor in his
virtual environment. Monitoring physical
network traffic is a must, but he found
the visibility into traffic within the virtual
environment was non-existent.
Start-up Altor Networks provided Por-
tolese with what he considered necessary
tools to track traffic in the entire environ-
ment. Altor’s Virtual Network Security
Analyzer (VNSA) views traffic at the virtual
-- not just the network -- switch layer. That
means inter-virtual-machine communica-
tions or even virtual desktop chatter won’t
be lost in transmission, the company says.
VNSA provides a comprehensive look at
the virtual network and analyzes traffic to
give network security managers a picture
of the top application talkers, most-used
protocols and aspects of virtualization
relevant to security. It’s a must-have for any
19
 EXECUTIVE GUIDE
Back to TOC

Section 3: Best practices • • •

virtual environment, Portolese says.
“We didn’t have anything to monitor the virtual switch layer, and
for me to try to monitor at the virtual port was very difficult. It was
impossible to tell which virtual machine is coming from where,”
Portolese explains.“You will get caught with major egg on your
face if you are silly enough to think you don’t have to monitor all
traffic on the network.”

6. Charge back for virtual resources

Companies with chargeback policies should apply the prac-
tice to the virtual realm, and those without a set process should
institute one before virtualization takes off.
Converting physical resources to virtual ones might seem like
a no-brainer to IT folks, who can appreciate the cost savings and
administration changes, but business units often worry that having
their application on a virtual server might affect performance
negatively. Even if a company’s structure doesn’t support the IT
chargeback model, business units might be more willing to get
on board with virtualization if they are aware of the related cost
savings, Forrester’s Staten says.
“IT can provide some transparency to the other departments
by showing them what they can gain by accepting a virtual server.
This includes lower costs, faster delivery against [service-level
agreements], better availability, more-secure disaster recovery and
the most important one -- [shorter time to delivery]. It will take six
weeks to get the physical server, but a virtual server will be over in
more like six hours,” Staten says.
In addition, chargeback policies would be an asset to IT groups
looking to regain some of their investment in virtualization. At
Hasbro, IT absorbs the cost of the technology while the rest of the
company takes advantage of its benefits, Ward says.“The cost of
physical machines comes out of the business department’s budget,
but the cost of virtual machines comes out of the IT budget,” he
says.

7. Capitalize on in-house talent

IT organizations also must update staff to take on virtualization.
Certification programs, such as the VMware Certified Professional
(VCP) and Microsoft’s Windows Server Virtualization, are available,
but in-house IT staff must weigh which skills they need and how
to train in them.“Certifications are rare, though I do have two VCPs
on my staff. Most IT professionals who are able to take the exam
and get certified would probably work in consulting,” says Robert
Jackson, director of infrastructure at Reliance Limited Partnership nating the knowledge across a team would make an organization
in Toronto. more secure and improve the virtualization implementation
With training costing as much as $5,000 per course, IT workers overall with fewer duplicated efforts and more streamlined
might not get budget approval. Gartner’s Haight recommends approaches.”
assembling a group of individuals from the entire IT organization In the absence of virtualization expertise, Linux proficiency can
into a center of excellence of sorts. That would enable the sharing help, Hasbro’s Ward says. VMware support staff seems to operate
of knowledge about virtualization throughout the organization. most comfortably with that open source operating system, he says.
“We surveyed IT managers about virtualization skills, and about In general, moving from pilot to production means increasing
one-quarter of respondents had a negative perspective about the staff for the daily care and feeding of a virtual environment,
being able to retain those skills in-house,” Haight says.“Dissemi- Ward says.“Tools can help, but they can’t replace people.”

Sponsored by APC
www.apc.com 20
 EXECUTIVE GUIDE
Back to TOC

Section 3: Best Practices • • •

Don’t let the thin-provisioning gotchas getcha

A how-to on getting the most out of this advanced
storage technology

N
By Sandra Gittlen, Network World, 01/26/2009

Next-generation storage is all

about dynamic allocation of
resources, and thin provisioning
can get that job done quickly and
easily – but not carefree.
As thin provisioning – also called dynamic
provisioning or flex volumes – becomes a
standard feature in virtual storage arrays,
IT executives and other experts warn that
dynamic resource allocation is not a one-size-
fits-all proposition. Applying the technology in
the wrong way could create a major disaster,
they caution.
“Vendors have made it so that IT teams
think,‘Thin provisioning is so easy, why
wouldn’t we use it?’ But some major thinking
has to go into how your storage network is
actually architected and deployed to benefit
from the technology,” says Noemi Greyzdorf,
research manager at IDC. IT director at New York media conglomerate an application reaching its threshold and
In traditional storage networks,IT has to IAC, speaking about one of the biggest miscon- storage being drained, and adjusts his alerts
project the amount of storage a particular appli- ceptions surrounding thin provisioning. accordingly. If the window is too close, he pads
cation will need over time,then cordon off that You’ve got to take the critical step of setting it.The goal is to allow sufficient time for adding
disk space.This means buying more hardware threshold alerts within your thin-provisioning capacity – and avoiding a disaster, he says.
than is needed immediately,as well as keeping tools because you’re allowing applications to By setting and fine-tuning his alerts,Yotko
poorly utilized disks spinning ceaselessly – a share resources,Yotko says. Otherwise, you can reports being able to realize utilization rates
waste of money and energy resources. max out your storage space, and that can lead of more than 80% among his array, a flip from
With thin provisioning, IT can keep storage to application shutdowns and lost productivity the less than 20% he had realized before thin
growth in check because it need not commit because users can’t access their data. provisioning.
physical disk space for the projected amount “You can get pretty close to your boundary,
of storage an application requires. Instead, IT fast, and that can lead to panicked calls No dumping here
relies on a pool of disk space that it draws from asking your vendor to rush you a bunch of Another common mistake IT teams make is
as the application needs more storage. Having disks.Alerting is an important aspect that we thinking that every application is a candidate
fewer idling disks means better capacity originally missed,”Yotko says. for thin provisioning, says Scott McCullough,
management, increased utilization, and lower Yotko has since integrated threshold alerts manager of technical operations at manufac-
power and cooling consumption. for IAC’s 3Par array into a central network turer Mine Safety Appliances in Pittsburgh.
management system, he says. Doing that lets “The only applications that can take advan-
Hold on a sec . . . him keep tabs on how myriad file, e-mail, tage of thin provisioning are those for which
IT executives shouldn’t let these ben- domain-controller and Web-application servers you can predict storage growth,” McCullough
efits, attractive as they are, blind them to the for 15 business units are handling the shared says. For that reason, he uses NetApp’s Provi-
technology’s requirements, early adopters say. resource pool.That pool supports more than sioning Manager to oversee resource pools for
You can’t just flip the switch on your storage 25TB of data, he adds. his Web servers, domain controllers and Oracle
pool and walk away, says Matthew Yotko, senior Yotko also calculates the time between database, but not the company’s high-volume

Sponsored by APC
www.apc.com 21
 EXECUTIVE GUIDE
Back to TOC

Section 3: Best Practices • • •

SQL Server. That server would quickly drain
any projected resource allotment, he says.
Before he thin-provisions an application,
McCullough studies its performance to make
sure it won’t endanger the pool.“It doesn’t
make sense to take up all your resources and
potentially starve other applications,” he says.
“You definitely need to be able to forecast or
trend the application’s trajectory,”IDC’s Greyzdorf
says. Biomedical and other science programs
can be particularly tricky for thin provisioning,
she notes, because they can start off needing
200GB of storage and quickly skyrocket to 3TB
with the addition of a single project.
Choosing the wrong applications to
thin-provision not only endangers your
entire storage pool, but also negates any
management and budget relief you might
gain otherwise. Done correctly, thin provi-
sioning should reduce the overall time spent
configuring and deploying storage arrays. If
applications continuously hit their thresholds,
however, and you’re forced to add capacity on
the fly, that benefit is quickly negated, costing
you in terms of personnel and budget.
This concern has ResCare rolling out
thin-provisioning piecemeal, says Daryl Walls,
manager of system administration at the Louis-
ville, healthcare support and services provider.
“We are cautious about our deployments.We
evaluate each implementation to see whether
it makes sense from an application, server and
storage point of view,” he says.
Once his applications have been thin-pro-
visioned,Walls closely monitors them to make
sure that usage patterns don’t change dramati-
cally. In the worst case, that would require them
to be removed from the pool.“A few times we’ve
underestimated, usage has crept up on us, and
we’ve received alerts saying,‘You’re at 70% to take responsibility in managing the way people Nutraceutical. Each time an application nears its
80% utilization,’”he says. In those instances, receive and use storage. Otherwise you wind threshold,Vance turns to Compellent Technolo-
IT teams must decide whether to expand the up wasting space, and that’s hard to clean up gies’ Storage Center software tools to analyze
application’s allotment, procure more resources after the fact,” Vance says. how the server used its storage space.“We then
or move the application off the system. For instance, being lax about monitoring decide whether it was used appropriately or if
the amount of space users and applications we need to tweak our policies,”he says.
What goes where are absorbing can lead to overspending Vance says he is a solid proponent of thin
Thin provisioning can wreak havoc on your on hardware and software, and necessitate provisioning, but he cautions his peers to stave
network if you don’t have proper allotment an increase in system management.This is off the complacency that automation can
policies in place, says Matt Vance, CIO at particularly concerning in Vance’s environ- bring on:“We can’t let the pendulum swing so
Nutraceutical, a health supplements company ment, where the principal driver for moving far toward automation that we forget to identify
in Park City, Utah. to virtualization and thin provisioning was where IT still has to be managing its resources.”
“IT has always managed and controlled the need to bring high-performance database
space utilization, but with thin provisioning you applications online quickly without breaking Gittlen is a freelance technology editor in
can get a false sense of security.We’ve found the bank on storage requirements. the greater Boston area. She can be reached at
that even with a resource pool, you still need to Reporting tools have become essential at sgittlen@verizon.net.

Sponsored by APC
www.apc.com 22

Section 3: Best Practices • • •
The challenge of managing mixed virtualized
Linux, Windows networks
Windows and virtualization are driving need for new
management standards, tools
By John Fontana, Network World, 10/27/2008
T
The sprawl of management
consoles, the proliferation of
data they provide and the rising
use of virtualization are adding
challenges to corporations look-
ing to more effectively manage
mixed Linux, Windows and cloud
environments.
Traditional standards are being tapped
in order to bridge the platform divide, and
new ones are being created to handle
technologies such as virtualization that
create physical platforms running one
technology but hosting virtual machines
running something completely different.
The goal is better visibility into what is
going right or wrong – and why – as com-
plexity rises on the computing landscape.
Some help is on the way. The Distributed
Management Task Force (DMTF) has begun
hammering out virtualization management
standards it hopes will show up in products
soon. Those standards will address interop-
erability, portability and virtual machine
life-cycle management, as well as incorpo-
rate time-honored management standards
such as the Common Information Model
(CIM).
Vendors such as Microsoft,VMware and
Citrix are on board with the DMTF and are
creating and marketing their own cross-
platform virtualization management tools
for x86 machines. Linux vendors, including
Novell and Red Hat, and traditional
management vendors such as HP also are
joining in.
To underscore the importance of
heterogeneous management, Microsoft
EXECUTIVE GUIDE
Back to TOC
is supporting Linux within its virtualiza- and from there you only start to see the
tion management tools slated to ship by stronger growth [of Linux],” says Ute Albert,
year-end rather than relying on third-party marketing manager of HP’s Insight manage-
partners. ment platform. HP is boosting its Linux
And the vendor has said it will inte- support with features HP already supports
grate the OpenPegasus Project, an open for Windows platforms, such as capacity
source implementation of the DMTF’s CIM planning.
and Web-based Enterprise Management Analyst firm the Enterprise Management
(WBEM) standards, so it can extend its Associates reports that use of Linux on
monitoring tools to other platforms. mainframes has grown 72% in the past two
The trend toward services is forcing IT years while x86 Linux growth hit 57%.
to think about management across systems In the trenches, users are moving to suck
that may have little in common, including the complexity out of their environments
the same LAN. Services are increasingly and make sense not only of individual
“We are starting to see IT put more mission-
critical applications on Linux and from there you
only start to see the stronger growth [of Linux].”
Ute Albert, marketing manager of HP’s Insight management platform
made up of numerous application compo- network and systems components but of
nents that can be running both internally composite services and how to aggregate
and externally, complicating efforts to data from multiple systems and feed results
oversee all the piece parts, their platforms back to administrators and notification
and their dependencies. systems.
The big four management vendors, BMC,
CA, HP and IBM, are handling the mixed- Console reduction
environment evolution by upgrading their At Johns Hopkins University, managers
monolithic platforms to better manage are trying to reduce “console sprawl” in a
Linux as its use grows. And a crop of next- management environment that stretches
tier vendors, start-ups and open source across 200 projects – many with their own
players are angling for a piece of the pie IT support in some nine research and
by providing tools that work alone, as well teaching divisions, as well as healthcare
as plug into the dominant management centers institutes and affiliated entities.
frameworks. Project leader pick their own applica-
“We are starting to see IT put more tions and platforms with about 90% to
mission-critical applications on Linux 95% running Windows and 5% to 10% on
Sponsored by APC
www.apc.com 23

Section 3: Best Practices • • •
Linux. There are also storage-area networks,
network devices, Oracle software, Red Hat,
VMware, EMC, IronPort e-mail relays, and
hardware from Dell, HP and IBM.
John Taylor, manager of the man-
agement and monitoring team, and
Jamie Bakert, systems architect in the
management and monitoring group, are
responsible for 15,000 desktops and 1,500
servers, nearly 50% of the university’s total
environment.
“Our challenge is we do not want to
create another support structure,” says
Taylor, who has standardized on Microsoft’s
System Center management tools
anchored by Operations Manager 2007 and
Configuration Manager 2007.
Because Taylor doesn’t control what
systems get rolled out, he is using Quest
Software’s Management Xtensions for
System Center to support non-Windows
infrastructure.
“Quest allows us to bring in anything
with a heart beat,” Bakert says.
And that allows for managing distributed
applications, which incorporate multiple
components on multiple platforms
“Microsoft has a limited scope of what
they are bringing into System Center at this
point,” he says.
For instance, Bakert uses Quest Xten-
sions to monitor IronPort relays that work
with Microsoft Exchange to ensure every-
thing in the e-mail service is monitored in
one tool.
The Quest tools also let Bakert store
security events on non-Windows machines
so he can report on both Windows and
non-Windows platforms, which helps with
collecting compliance data.
Taylor and Bakert also are beta testing
Microsoft’s System Center Service Manager,
slated to ship in early 2010, with hopes they
can reduce System Center consoles from
five to one. Eventually, Service Manager’s
configuration management database will
host data from Configuration Manager and
Operations Manager, as well as incorporate
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
ITIL, a set of best practices for IT services
management, and the Microsoft Operations
Framework.
Taylor and Bakert also are testing System
Center’s Virtual Machine Manager, which
will manage Windows, the VMware hyper-
visor and Suse Linux guest environments.
Virtualization getting
Microsoft ironically had the title as first
to support mixed hypervisor environments
because it was last to release a hypervisor
– Hyper-V.
Without the benefit of the in-develop-
ment Microsoft code,VMware, Novell,
Red Hat, HP and others are momentarily
playing catch-up on cross-platform man-
agement support.
Novell is using its February 2008 acquisi-
tion of PlateSpin to support management
across both physical and virtual environ-
ments. The company’s existing partnership
and interoperability agreement with Micro-
soft has yielded virtualization bundles and
24
 EXECUTIVE GUIDE
Back to TOC

Section 3: Best Practices • • •

the company’s acquisition of Managed Management Architecture for Server wrote to integrate Windows systems using
Objects will give IT admins and business Hardware (SMASH), used to unify data Microsoft’s native Windows Management
managers a unified view of how business center management, includes the SMASH Instrumentation.
services work across both physical and Server Management (SM) Command Line “We don’t provide the entire tool set
virtual environments. Protocol (CLP) specification, which simpli- you may want, but we at least take the
“In the data center we see that people fies management of heterogeneous servers time and energy out of providing the
are not saying consolidate [on a platform], in the data center. The Desktop and Mobile monitoring infrastructure,” Lilly says.Via
they are saying give me a universal remote,” Architecture for System Hardware (DASH) standards, GroundWork can plug into other
says Richard Whitehead, director of product provides standards-based Web services management tools such as service desk
marketing for data center solutions. management for desktop and mobile client applications.
Red Hat also is developing its portfolio. systems. Other open source management
Its February 2008 launch of the open resources include Open WS-Man, an XML
source oVirt Project has a stated goal Open source SOAP-based specification for management
of producing management products for Standards efforts are being comple- using Web services standards. The project,
mixed environments. mented by open source vendors who are which focuses on management of Linux
“The oVirt framework will be used to aligning their source-code flexibility with and Unix systems, is an open source imple-
control guests in a cloud environment, the interoperability trend. mentation of WS-Management, an industry
create pools of resources, create images, Upstarts such as GroundWork, Likewise, standard protocol managed by the DMTF.
deploy images, provision images and Hyperic, OpenQRM, Zenoss and Quest’s There are other WS-Man variations such as
manage the life cycle of those,” says Mike Big Brother platform are working the open the Java implementation called Wiseman.
Ferris, director of product strategy for the source route to build a united manage- “Interoperability is the end game,”
management business unit at Red Hat. ment front. DMTF’s Bumpus says.“You can have all the
HP has aligned its HP Insight Dynamics “We picked [tools] most people pick specs, but if you don’t have interoperability
– Virtual Server Environment (VSE) with when they use open source, and we pack- who cares.”
VMware and plans to add support for aged them together,” says Dave Lilly, CEO In today’s evolving data centers and
Microsoft’s Hyper-V in the next release, of GroundWork. The company’s package services revolution it turns out a lot of IT
according to HP’s Albert. In addition, HP includes 100 top open source projects, managers are beginning to care very much.
is increasing the feature set of its Linux including Nagios, Apache and NMap.
management and monitoring support. GroundWork also includes a plug-in it
And while the vendors work on their
tools, the DMTF is working on standards it
hopes will be as common as existing DMTF
standards CIM and WBEM.
The Virtualization Management Initia-
tive (VMAN) released by the DMTF in
September 2008 is designed to provide
interoperability and portability standards
for virtual computing. The initiative
includes the Open Virtualization Format
(OVF) for packaging up and deploying one
or more virtual machines to either Linux or
Windows platforms. Tools that are based on
VMAN will provide consistent deployment,
management and monitoring regardless of
the hypervisor deployed.
“The truth is we have been working on
this whole platform independence since
1998,” says Winston Bumpus, president of
the DMTF, in regard to the organization’s
goals.
Virtualization is only one of the DMTF’s
initiatives. The group has started its interop-
erability certification program around its
SMASH and DASH initiatives. The Systems

Sponsored by APC
www.apc.com 25

Next-generation data centers:
opportunities abound
Section 4
Inside the Data Center
The Google-ization of Bechtel
How the construction giant is transforming its IT operations
to emulate Internet leaders and embrace SaaS
I
By Carolyn Duffy Marsan, Network World, 10/29/2008
If you could build your IT sys-
tems and operation from scratch
today, would you recreate what
you have? That’s the question
Geir Ramleth, CIO of construc-
tion giant Bechtel, asked himself
several years ago.
The question – and the industry bench-
marking exercise that followed – prompted
Bechtel to transform its IT department
and model it after Internet front-runners
YouTube, Google, Amazon.com and
Salesforce.com. After all, these companies
have exploited the latest in network design,
server and storage virtualization to reach
new levels of efficiency in their IT opera-
tions. Ramleth wanted to mimic these
approaches as Bechtel turned itself into a
software-as-a-service (SaaS) provider for
internal users, subcontractors and business
partners.
After researching the Internet’s stron-
gest brands, Bechtel scrapped all of its
existing data centers and built three new
facilities that feature the latest in server
and storage virtualization.
Bechtel also designed a new Gigabit
Ethernet network with hubs at Internet
exchange points that it is managing itself
instead of using carriers. Now, Bechtel is
slashing its portfolio of software applica-
tions to simplify operations as well as the
end user experience.
Dubbed the Project Services Network,
Bechtel’s new strategy applies the SaaS
computing model internally to provide IT
services to 30,000 users, including 20,000
employees and eventually 10,000 subcon-
tractors and other business partners.
We operate “as a service provider to a
set of customers that are our own [con-
struction] projects,” Ramleth said.“Until we
can find business applications and SaaS
models for our industry, we will have to do
it ourselves, but we would like to operate
with the same thinking and operating
models as [SaaS providers] do.”
Nicholas Carr, author of several books
including “The Big Switch: Rewiring the
World from Edison to Google” which
chronicles a shift to the SaaS model, called
Bechtel’s strategy a smart move.
“For the largest enterprises, the very
first step into the Internet cloud may well
be exactly what Bechtel is doing: building
their own private cloud to try to get the
cost savings and flexibility of this new
model,” Carr says.“Large companies have
such enormous scale in their own IT
operations that the outside providers, the
true utility providers, just aren’t big enough
yet…to make them a better option.”
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
Carr predicts, however, that Bechtel’s
do-it-yourself SaaS strategy will be an
interim step until the company is able to
fully outsource its IT infrastructure. That
may take as long as 10 years, he adds.
“My guess is that over time -- and maybe
it will start with the HR system -- Bechtel
will look outside and start running some
aspects of its IT operations off of [SaaS]
sites,” Carr says.“Then its cloud will start to
blur with the greater Internet cloud.”
Bryan Doerr, CTO of utility computing
provider Savvis, says many enterprises like
Bechtel are interested in the SaaS model
for applications that don’t differentiate
them from their competition.
“The move to SaaS is simply a different
delivery model for an application that has
little to do with intellectual property or
innovation,” Doerr says.“Once you make
the decision to outsource to somebody
else’s software, the decision to host it
yourself or use a SaaS provider is about
economics…. For many enterprises,
licensing by end user seat is much more
efficient than licensing as a bulk package,
buying servers and storage and data center
space, and then training people to self-host
the application.”
Several business challenges are driving
Bechtel’s SaaS strategy.
Bechtel is a leading construction,
26

Section 4: Inside the Data Center • • •
engineering and project management firm
with 42,500 employees and 2007 revenue
of $27 billion. The privately held company
is working in more far-flung locations,
and it has difficulty finding and retraining
talented employees to work on its projects.
Bechtel’s employees are demanding
business software that is as intuitive as
popular Web sites. The company doesn’t
have time to train end users in software
applications, nor can it afford to maintain
hundreds of applications.
“We needed a different way of doing
applications and supporting them,”
Ramleth says.“We have more employees
coming into the organization, and we need
to get them up to speed fast.”
Another key business challenge is
protecting Bechtel’s intellectual property
when so many subcontractors and busi-
ness partners have access to its network
and data.
“A third of the people on our network
are non-Bechtel employees. That exposure
forms a security risk,” Ramleth says.
Bechtel started its transformation by
trying to figure out how to revamp its
software applications to operate more like
leading Web sites. But what Bechtel dis-
covered is that it had to fix the underlying
IT infrastructure -- including data centers
and networks -- before it could change its
applications.
“Not only do you have to solve the
IT architecture and the way you operate
it, but you have to make sure that IT is
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
accommodating Web applications that can
operate more in an Internet mode than in
an intranet mode,” Ramleth explains.
Perhaps most impressive is that Bechtel
is transforming its IT operations without
additional funding. Bechtel would not
release its annual budget for its Informa-
tion Systems & Technology group, but
the company said it has 1,150 full-time
employees in its IS&T group and 75 to 100
contractors.
“We’ve mostly paid for this by re-alloca-
tion of the budgets that we otherwise
would have used for refresh and mainte-
nance,” Ramleth says. We’re “doing a total
change of the traditional way of doing
things, and we have done it with very little,
if any, incremental funding.”
27

Section 4: Inside the Data Center • • •
Leader benchmarks
The transformation began
with Bechtel’s IS&T group
spending a year trying to figure
out how to drive faster adop-
tion of consumer technology
such as Google and Amazon.
com across the company.
“We asked ourselves: If we
started Bechtel today, would
we do IT the same way we are
doing it today? The answer was
no. If we had a clean slate, we
wouldn’t do it the way we were
doing it,” Ramleth says.
Ramleth decided to
benchmark Bechtel’s IT opera-
tion against leading Internet
companies launched in recent
years. He zeroed in on YouTube,
Google, Amazon.com and
Salesforce.com for comparison.
Bechtel’s IS&T staff studied
the available information about
how these Internet leaders run
their IT operations, and they
interviewed venture capitalists
with experience investing in
consumer applications.
Bechtel came up with
estimates for how much
money YouTube spends on
networking, Google on systems
administration, Amazon.com
on storage, and Salesforce.com
on software maintenance. What
Bechtel discovered is that its
own IS&T group was lagging
industry leaders.
“What we found were
tremendous discrepancies
between our metrics and what
these guys were dealing with,”
Ramleth says.“You can learn
a tremendous amount from
[companies] that have the
privilege of starting recently.”
When Bechtel researched
YouTube, it came to the
conclusion that YouTube must
be getting much less expen-
sive network rates because
otherwise it wouldn’t be able
to send 100 million video
streams a day for free. Bechtel
estimated that YouTube spent
$10 to $15 per megabit for
bandwidth, while Bechtel is
spending $500 per megabit for
its Internet-based VPN.
YouTube was “paying a frac-
tion of what we were paying,”
Ramleth says.“We learned
you have to be closer to the
high-bandwidth areas and not
haul the data away. We decided
we better bring the data to the
network, rather than bring the
network to the data.”
Next, Bechtel studied how
Google operated its servers.
Bechtel estimated that Google
Sponsored by APC
www.apc.com
used 12 system administrators
for every 200,000 servers, or
roughly 17,000 servers per
system administrator. Bechtel,
on the other hand, was oper-
ating with 1,000 servers per
system administrator.
“What we learned is that
you have to standardize like
crazy and simplify the environ-
ment,” Ramleth says.“Google
basically builds their own
servers by the thousands or
gets them built in a similar
fashion, and they run the same
software on it. So we had to get
more simplified and standard-
ized.”
Bechtel studied Amazon.
com and determined that
Amazon.com must have a
better storage strategy if it
is offering disk space for a
fraction of Bechtel’s internal
costs. While Amazon.com
was offering storage for 10
cents per gigabyte per month,
Bechtel’s internal rates in the
United States was $3.75 per
gigabyte.
Ramleth says the key to
reducing storage costs was not
only to simplify and virtualize
the storage environment, but
also to drive up utilization.
“Our average utilization
was 2.3%,” Ramleth says. With
virtualization,“we now expect
to have utilization in the 70% to
75% range.” However, he added
that the new virtualized storage
environment is “more complex
to operate.”
Bechtel turned to Sales-
force.com for its expertise in
running a single application
with millions of users. In
contrast, Bechtel operates 230
applications, and it runs 3.5
versions per application, which
means it maintains approxi-
mately 800 applications at any
given time.
“When you look at Sales-
EXECUTIVE GUIDE
Back to TOC
force.com, not only are they
running one application, but
they are running one version
and they are only running it
in one location,” Ramleth says.
“They upgrade that applica-
tion four times per year, and
they don’t disrupt the users by
having to retain them. Every
time we have a new version, we
have to retrain our users.”
With its benchmarking data
in hand, Bechtel decided to
revamp its IS&T operations
to model itself as closely as
possible after the SaaS model
pioneered by these four
Internet leaders.
“If you take the ideal world,
everything is done as a service:
computing, storage, software
and operations,” Ramleth
says.“It’s maybe the ultimate
goal…but if you start where all
the enterprises are today, that’s
a very long road to go.”
A trinity of data centers
Once Bechtel committed to
the SaaS model, the firm real-
ized it needed to revamp its
data centers to copy Google’s
standardized, virtualized
model.
Bechtel was operating seven
data centers worldwide, but in
2007 replaced those with three
new data centers, one each in
the United States, Europe and
Asia. The three data centers
have identical hardware from
HP, Cisco, Juniper and Riverbed.
On the software side, Bechtel
is using Microsoft and Citrix
packages.
“The hardware is the same,
the software is the same, and
the same organization [is]
managing all of them,” Ramleth
says.“It’s like one data center,
but it operates in three dif-
ferent locations.”
The new data centers have
virtualized servers with utiliza-
28

Section 4: Inside the Data Center • • •
tion of around 70%, Ramleth says. These
centers boast the most energy-efficient
design possible, so they are a fraction of
the size of the older data centers and use
significantly less electricity.
“In square footage, we’re down by a
factor of more than 10,” Ramleth says.
“Two-thirds of the power in a data center
is chilling, and if I don’t have to chill that
extra space…I get a dramatic reduction in
power needs.”
The three new data centers are opera-
tional, and Bechtel expects to close all the
older data centers by the end of 2009.
Ramleth says one of the hardest aspects
of the IS&T transition was closing data
centers upgraded as recently as 2005.
“Six of our data centers were relatively
modern. That was a tough thing. We
finished a [data center] consolidation
in 2005, and already in 2006 we started
talking about doing a re-do of our data
centers again. [Our IS&T staff] felt like
they hadn’t really gotten dry from the last
shower before they started getting wet
again,” Ramleth says.
Do-it-yourself networking
At the outset of this research project,
Bechtel was operating an IP VPN that it
had installed in 2003. To drive down the
cost of its network operations, Bechtel has
redesigned that network using YouTube’s
do-it-yourself model.
Bechtel has a Gigabit Ethernet ring con-
necting the three new data centers, with
dual paths for failover. Bechtel is buying
raw bandwidth from a variety of providers
-- Cox, AboveNet, Qwest, Level 3 and Sprint
-- but it is managing the network itself.
“We buy very little provisioned net-
working,” Ramleth says.“We do it ourselves
because…it’s less costly than buying from
others….We go to the Internet exchange
points, to the carrier hotels, where the
traffic terminates.”
So far, Bechtel has migrated the three
new data centers to the new network,
along with nine offices: San Francisco;
Glendale, Ariz.; Houston; Oak Ridge, Tenn.;
Frederick, Md.; London; New Delhi and
Brisbane.
The new network “is about the same
[cost] as what we paid before, but it offers
a heck of a lot more capacity,” Ramleth
says, adding that Bechtel is getting around
10 times more capacity for the same
amount of money.
Ramleth says the biggest cost-saving of
the new network design came from aggre-
gating network traffic at Internet exchange
points, which is what leading e-commerce
vendors do.
“We found that for the amount of traffic
and the amount of capacity that we put
it in, we could do it cheaper ourselves,”
Ramleth says.“This is not something for a
small or medium-sized enterprise, but we
found that we were big enough to be able
to do it ourselves.”
“Two-thirds of the power
in a data center is chilling,
and if I don’t have to chill
that extra space … I get
a dramatic reduction in
power needs.”
Geir Ramleth, CIO, Bechtel
Simple secure software
The next aspect of transforming
Bechtel’s IS&T operations is migrating its
applications to the new SaaS model.
Bechtel runs 230 software applications,
60% of which were developed internally.
Ramleth says he hopes to have 20 to 30 of
the firm’s key applications converted to
the new SaaS model within a year.
So far, Bechtel has a dozen applications
running in its new data centers, including
e-mail, a Web portal, a time record system,
and a workflow/document management
application.
Bechtel’s ERP applications, including
Oracle Financials for accounting and SAP
for human resources and payroll, will be
migrated to the new infrastructure by the
end of 2009, Ramleth says.
“We will be getting applications in
and getting them certified in this new
environment,” Ramleth says.“The ones
that we can’t totally certify, a few we will
let die….For the low usage, infrequently
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
used applications that are not as critical,
we are using Citrix solutions to solve some
of them so we don’t have to do too much
with them. We’re doing that as a bridge to
get from our past to our new infrastruc-
ture.”
Bechtel’s new portal is key to its SaaS
strategy and its goal of having software
applications that are as simple to use as
Google.
In its research, Bechtel found that 80%
of its end users want to get information
out of an application but don’t need to
understand how it works. With the new
Project Services Network Portal, Bechtel’s
end users can interact with applications
without requiring training.
Bechtel built the portal using Micro-
soft’s SharePoint software. Ramleth says the
portal “gives us consumerization and also
gives us the new security model.”
Ramleth likens Bechtel’s security
strategy to Amazon.com’s approach. With
Amazon.com, users can browse freely and
security is applied when a purchase is
made. Similarly, Bechtel is trying to create
Web applications that apply security only
when needed.
“We will apply different forms of secu-
rity based on what’s going on,” Ramleth
says.“By using the portal, we can limit
what you have access to by policy, by role,
in a way where people won’t necessarily
go in and find [our intellectual property]
without having the right to access it.”
Ramleth says the new portal and the
policy-based security model it provides
are among the top benefits that Bechtel
is gaining out of the IS&T transformation
effort. He says this benefit will be fully
realized when Bechtel’s business partners
are migrated to the portal, too.
The SaaS model allows us “to bring in a
different cost model that can afford us to
have a high-capacity, global, collaborative
work environment,” Ramleth says.
The risk for enterprises that don’t start
a SaaS migration strategy soon is that
their IT organizational structures will be a
competitive disadvantage, Ramleth warns.
“If they don’t start thinking about this
soon, the changeover in the future will be
really hard,” Ramleth says.
29

Section 4: Inside the Data Center • • •
Why San Diego city workers expect apps up
and running in 30 minutes or less
Major VMware project makes application deployment
a snap while easing the IT budget
D
By Jon Brodkin, Network World, 10/20/2008
Deploying a new application used
to be a month-long headache and
a budget-drainer at the San Diego
Data Processing Corp. Now the
process takes as little as 30 min-
utes and costs next to nothing,
thanks to the extensive use of
server virtualization.
The SDDPC, a private nonprofit that
handles IT for the San Diego municipal
government and its more than 10,000 city
workers, embraced virtualiza-
tion with a vengeance several
years ago -- and has been
reaping substantial rewards
ever since.
The first goal was server con-
solidation, says Rick Scherer, a
Unix systems administrator who
spearheaded the SDDPC virtu-
alization project. “We were just
like everyone else: We had tons
of x86 machines that were only
being 10% utilized,” he says. “I presented to
our directors that we could virtualize 20
machines into one box, get a better ROI and
save tons of money plus data center space.”
Easier application deployment followed,
which meant huge gains for business users,
he adds.
Before using VMware’s ESX Server, every
application upgrade or new deployment
meant the SDDPC had to buy -- then install
-- a new server. Such is the nature of the
Windows operating system, Scherer says.
“Anytime there was a new application, we
couldn’t put it on a box with another appli-
cation, because either those applications
wouldn’t work properly or the [applica-
tion] vendor didn’t support that.”
EXECUTIVE GUIDE
Back to TOC
For example, when the city needed as 35 virtual machines onto one physical
to upgrade a purchase-order system for server. Such density has allowed the
outside contractors, SDDPC had to push organization to power off 150 physical
the project out three to four weeks to get servers; it now runs 292 virtual machines
the infrastructure ready, Scherer says. The on 22 physical x86 servers -- leaving plenty
same three- to four-week wait was in store of room for expansion.“A lot of those hosts
when the organization needed to boost aren’t even being used; they’re just for
processing power for a Citrix Systems Pre- future growth,” Scherer says.
sentation Server deployment. Besides the SDDPC also uses Sun’s virtualization
annoyance, each new HP ProLiant server technology on Sun Sparc servers, and now
cost somewhere around $10,000, he says. runs 120 logical servers on 90 boxes.
When SDDPC started with server The goal is to virtualize as much as
virtualization, users were surprised at how possible:“We’ve set an initiative: For any
“We’ve set an initiative: For any new application
or service that needs to be deployed in our data
center, we’re going to do everything we can to
virtualize first.”
Rick Scherer, CIO, Unix systems administrator, SDDPC
speedily IT could turn up applications, new application or service that needs to
Scherer says.“But what’s funny is now that be deployed in our data center, we’re going
we’ve been doing it so long, they expect to do everything we can to virtualize first.
it. It has put a damper on management,” If there’s no way to virtualize it, we’ll look
he says. Users are disappointed now “if at physical hardware,” Scherer says, noting
they put a request in for a server and it’s that the organization also is aggressively
not up in a half-hour,” he adds. One of the moving the city’s existing applications, as
only things preventing further virtualiza- appropriate, to the virtual infrastructure.
tion right now is the time Scherer and his VMware remains the server-virtualiza-
colleagues must devote to day-to-day tasks tion tool of choice, even though products
and other projects. from rivals Citrix and Microsoft are now
available and cost quite a bit less. Using
VMware loyalty VMware, SDDPC pays $10,000 to virtualize
Before deploying server virtualization, a four-socket machine, but that one
the SDDPC had about 500 physical x86 physical host can support well over 20
machines, largely from HP. With VMware, virtual machines, Scherer says. If you buy
the organization can consolidate as many 20 physical servers at $10,000 a pop, you’re
Sponsored by APC
www.apc.com 30

Section 4: Inside the Data Center • • •
shelling out $200,000, he says.
Numerous management advantages,
disaster-recovery capabilities and security
also make the investment well worth it,
Scherer says. The SDDPC operates two
data centers and contracts with a co-
location vendor in Chicago for disaster
recovery. Initially, the organization included
only mission-critical applications in its
disaster-recovery plan, but it’s beginning
to account for many other applications
-- those it considers crucial though not
necessarily mission-critical -- because
virtualization makes it feasible to do so
from a cost perspective, he says.
In addition, SDDPC makes extensive
use of VMware’s VMotion, a live-migration
function that moves virtual machines from
one physical server to another without any
downtime. VMotion comes in handy for
balancing loads among servers, as well as
for routine maintenance, Scherer says.“We
can migrate virtual machines on a specific
host to others within the cluster; this allows
us to perform hardware and software
upgrades with zero downtime and no
impact to the customer,” he says.
As for the security benefits, if a virtual
server becomes compromised, it’s easy to
shut it down or isolate it into a separate
network group.“If we [want to] have a
Web zone, an application zone and a
database zone, we can accomplish that
with a lot less hardware. We’re virtualizing
our network as well. Instead of having
separate physical switches we have to pay
for, maintain and manage, all of it can be
done within the [VMware] ESX host just by
creating separate virtual switches,” Scherer
says.
SDDPC now is looking forward to its
next big project, Scherer says -- virtualizing
desktops. It’s testing several thin-client
devices to replace many of the city’s 8,500
desktops, and plans to use VMware’s Virtual
Desktop Manager to provision and manage
clients. This software “includes the ability
to create a work-from-home scenario. A
user can go home, not necessarily have
thin-client at home, but through a Web-site
connect to his desktop,” he says.“That
could potentially eliminate the need for
Citrix, which is a significant licensing cost
to the city every year.”
Nevertheless, desktop virtualization will
not happen immediately. The city refreshed
most of its desktops in the past year, so the
desktop virtualization project won’t kick
off for the next two or three years, Scherer
says.
Sticky points
Despite realizing benefits, Scherer has
run into some roadblocks and challenges
related to virtualization. His goal is to
virtualize nearly everything, but vendor
licensing setups are holding him back.
The SDDPC has virtualized Citrix’s applica-
tion-delivery software, SQL servers and
all its Web services on VMware boxes.
Vendor support issues, however, meant it
could run SAP software only on virtual
servers in its development environment,
and run Exchange nowhere on a virtual
infrastructure. SAP and Microsoft now sup-
port VMware, so Scherer will revisit those
earlier decisions, he says.“Until virtualiza-
tion is adopted by every single software
company, you’re going to run into those
issues,” he adds.
The potential for server overuse
is another issue. Because it’s so easy
to deploy new virtual machines, an
overzealous IT pro runs the risk of over-
Sponsored by APC
www.apc.com
EXECUTIVE GUIDE
Back to TOC
committing resources. It may be tempting
to allocate nearly 100% of a server’s
resources, but it’s better to leave room for
future growth.
In addition, virtualization introduces a
potential single point of failure, because
dozens of applications and virtual
machines may reside on one physical
server.“Your biggest risk is having a
hardware failure and then no place for
those virtual machines to run,” Scherer
says. That’s why it’s important to use such
disaster-recovery features as live migration,
and build out a strong network and storage
system to support virtual servers, he says.
The SDDPC relies on about 250
terabytes of NetApp storage, and uses
such features as data deduplication and
thin-provisioning to maximize storage
space and make sure each application has
enough storage dedicated to it. Devoting
too many physical and virtual machines to
one storage array can be problematic.
“That recently happened: We had
another application that wasn’t virtual-
ized and was on the same array as our
virtual-machine environment,” Scherer says.
“Resources spiked up and caused conten-
tion on our virtual-machine farm. Luckily,
we caught it in the early stage and moved
the data off, but it was definitely a lesson
learned,” he adds.
Beyond storage, IT pros who embrace
virtualization need to design full multipath
networks, Scherer says.“You want to make
sure that if a link fails, you have full redun-
dancy,” he says.“If all your virtual machines
are running on a data store that has only
one path and the link dies, that’s the same
thing as unplugging all your ESX hosts, and
you’re completely dead.”
31