Microsoft SQL Server 2005

Peter Koen
koen@kema.at
http://www.kema.at

Peter Koen

CEO

KEMA – Koen Electronic Media Agency

Consultant

Author

SQL Server Usergroup Austria

www.sqlexperts.at

MCP, MCAD, MCSD.NET, MCDBA, MCT

MVP – Visual Developer C#

CAI R/S, CASE R/S, IAT
2

1
Agenda

SQL Server – Where are we?

The 4th Wave

DBA Features

Development Features

BI Features

Brief History of Database

Use

Database use has exploded over the last
30 years

Most of us directly interact with a DBMS
many times a day

We’re transitioning from humans
interacting with databases to databases
interacting with databases

2
 First 3 Waves of Database

ERP

Automating Reducing Automating

Process Process Process
Steps Steps Steps

Example: Online Query & Example:

Payroll Reporting ERP, HR, CRM

Data Online Enterprise

Processing Processing Computing

1960 2000
5

Big Trend: “Born Digital” Data

100%
Percentage of Information

Word Processing
“Born Digital”

POS Data
Programming on Package
Punched Cards Tracking

0%

1960 2000
6

3
Big Trend: Schema

ERP Vendors Model

Enterprises
Schema Complexity

01 SAMPLE-REC.
03 SA-PAC-DEC-1 PIC 9(7)V99 COMP-3.
03 FILLER PIC X(5).
03 SA-PAC-DEC-2 PIC 999 COMP-3.
03 SA-PAC-DEC-3 PIC S9(3) COMP-3.
XML
COBOL Records
Mapping Files/Memory

Chen’s E-R Paper

1960 2000
7

SQL Server Yesterday & Today

SQL Server 6.0

First SQL Server release shipped solely by Microsoft

Key Features: Enterprise Manager, Replication, Server
Cursors

SQL Server 6.5

Shipped 10 months after 6.0

DTC Integration, New Replication Features

SQL Server 7.0

Major architectural release

Server Mostly rewritten – New QP, Storage Engine

Analysis Features

SQL Server 2000

More high scale engine features: Indexed Views,
Online utilities

Analysis Services, Data Mining, DTS
8

4
Key Observations

People’s time is precious

Cost of managing systems is HUGE!

Can consume CPU, Disk, & Bandwidth to
save people’s time

Allows people to concentrate on more
interesting tasks…

Data, Schema, Processing & Bandwidth

We have the capability to provide end-to-end
digital business (Autonomous Computing)

The ideal autonomous

computing database

Self-managing

Automatic configuration

Automatic, on-line, data maintenance

Easily programmed

Service programming may be deployed within the
database, (or in mid-tier)

Great messaging endpoint

Can directly plumb web services and process
messages via transactions

Must be both a “talker” and a “listener”

Great notification engine

Generates notifications on complex data changes
10

5
 • .NET CLR hosting • Comprehensive ETL platform
• Native XML technology • Real time analytics
• Integrated web services • Accessible, easy data mining
• Distributed application framework • Rich, integrated reporting

• Flexible, interoperable, scalable • 3 years in development

• Improved predictability • Multiple security reviews
• Self optimization and tuning • 1,000+ new and improved features
• Fast recovery and restore • Large private beta for early quality

The comprehensive database platform for rapidly building,

deploying and managing scalable, integrated solutions
11

New Features
.NET Framework
Common Language Runtime Integration
User-defined Aggregates
User-defined Data Types
User-defined Functions
SQL Server In-Proc Data Provider
Extended Triggers
Data Types
File Stream Storage Attribute
Managed SQL Types
New XML Datatype
SQL Server Engine
New Message Service Broker
HTTP Support (Native HTTP)
Database Tuning Advisor
Enhanced Read ahead & scan
Extended Indexes
Multiple Active Result Sets
Persisted Computed Columns
Queuing Support
Snapshot Isolation Level
Scale Up Partitioning
VIA support
NUMA support
Database Failure and Redundancy
Fail-over Clustering (up to 8 node)
Enhanced Multi-instance Support
Database Mirroring
Database Viewpoints
XML
XQUERY Support (Server & Mid Tier)
XML Data Manipulation Language
FOR XML Enhancements
XML Schema (XSD) Support
MSXML 6.0 (Native)
XQuery Designer
Database Maintenance
Backup and Restore Enhancements
Checksum Integrity Checks
Dedicated Administrator Connection
Dynamic AWE
Fast Recovery
Highly-available Upgrade
Online Index Operations
Online Restore
Parallel DBCC
Parallel Index Operations
Management Tools
MDX Query Editor
MDX Intellisense
T-SQL Intellisense
Version Control Support
XML/A
SQLCMD Command Line Tool
Performance Tuning
Profiler Enhancements
Profiling Analysis Services
Exportable Showplan
Exportable Deadlock Traces
Full-text Search
Indexing of XML Datatype
MDAC
Side by Side installation
Microsoft Installer base setup
Support for Active Directory Deployment
SQL Client .NET Data Provider
Server Cursor Support
Asynch
Security
All Permissions Grantable
Fine Grain Administration Rights
Separation of Users and Schema
Replication
Auto-tuning Replication Agents
Oracle Publication
Improved Blob Change Tracking
OLAP and Data Mining
Analysis Management Objects
Windows Integrated Backup and Restore
Web Services/XML for Analysis
DTS and DM Integration
Eight new DM algorithms
Auto Packaging and Deployment
Data Transformation Services
New Architecture (DTR + DTP)
Complex Control Flows
Control Flow Debugging
For Each Enumerations
Property Mappings
Full Data Flow Designer
Full DTS Control Flow Designer
Graphical Presentation of Pkg Execution
Immediate Mode and Project Mode
Package (Advanced) Deployment Tools
Custom Tasks and Transformations
Reporting Services
Multiple Output Formats
Parameters (Static, Dynamic, Hierarchical)
Bulk Delivery of Personalized Content
Support Multiple Data Sources
Sharepoint Support
Visual Design Tool
Charting, Sorting, Filtering, Drill-Through
Scheduling, Caching
Complete Scripting Engine
Scale Out architecture
XML Report Definition
Notification Services
12

6
 SQL Server Yukon Timeline

Beta 2 - Summer 2004

Feature complete

Very good quality

Engine perf ~90% of SQL 2K

Beta 3 – Q4 2004

RC Level quality and perf.

RTM – 2005

13

New for the DBA

Major SQL Server Subsystems

Configuration

Monitoring

Security

Development

Availability

Data Access Concurrency

Database Server Failure or Disaster

Database Maintenance and Operations

Manageability

Scalability and Reporting
14

7
 Fast Recovery
Restart or Restore

SQL Server 2000

Database is Available after Undo Completes

Redo Undo

Time Available

SQL Server 2005

Database is Available when Undo Begins

Redo Undo

Available
15

Database Mirroring

Fault Tolerant Virtual Database

Witness

Clients

Principal Mirror

16

8
Database Mirroring
How It Works

Transaction Shipping

Witness

Principal Mirror

Application

1 5
2
SQL Server SQL Server

2 4 3

17

New for the developer

Database Programming Evolution

New Application Frameworks

.NET Framework Integration

XML Support

Web Services Support

Data Access

18

9
New in SQL Server 2005

Three new application frameworks:

SQL Service Broker

SQL Server Notification Services

SQL Server Reporting Services

New server-side programming support

Stored procedures, triggers, & functions in
C++, C#, or VB

Server-side Abstract Data Types

Industry leading programming tools

19

SQL Service Broker

Inbound messages arrive on protocol pipe

Message is:

Authenticated

Dispatched to appropriate queue Service Program

Service Programs:

Pick up work from queue

Run in different context than
incoming message

May run inside or outside server
process

May send additional messages

Service Queue
Message
Message

Shared Private
Stuff State
20

10
 Notification Services Architecture
Subscription Management Application

Subscribers
Data Subscriber Devices Notifications External
Data Events Subscriptions External
Changes
Changes Delivery
Delivery

Event Provider Generator Distributor

File System Email
Watcher XSLT
SQL CF
XSLT File
Event
SQL Server Provider Match HttpExt
provider Host Rule .NET Alerts *
Custom CF
Custom EP Custom DP

21

Reporting Services Architecture

Browser Office Custom App

Report Control
Report Processing

Rendering Data Proc.

Web UI Security Services

Data Sources
(SQL, OLE DB, XML/A, (NT, Passport,
XML Web Service Interface Custom)
ODBC, Oracle, Custom)
Report Server

Report Processing

Data Processing Security

Scheduling and
Rendering
Delivery
Output Formats Delivery Targets
(HTML, Excel, (E-mail, SharePoint,
SQL Server Catalog Custom) 22
PDF, Custom)

11
 .NET Framework Integration
Key Features

Programming environment for Functions, Stored
Procedures, Triggers

User Defined Types, Aggregates

In-Proc Data Access (ADO.NET V2)

Symmetric data access

Mid-tier/data tier deployment decision

Security

Integration of SQL and CLR security

Three levels of code access security

Safe, External-Access (verifiable), Unsafe

Tight integration with Visual Studio

Authoring, debugging, deployment, & profiling
23

.NET Developer Experience

Build

Developer Visual Studio Assembly

Project

SELECT tax(salary)
FROM Employees
WHERE … CREATE ASSEMBLY
CREATE FUNCTION
CREATE TYPE
SQL Server
24

12
 Assembly & Class Loading
Harddisk
sys.assemblies

Execute

Load FX Validate
version assembly
Assembly
25

.NET Integration
Key Theme: Choice & Control

Choice of where to run logic

Database, for logic that runs close to data

Mid-tier, for logic that scales out

Symmetric programming model

Leverage skills mid-tier & server

Choice of programming language

C#, VB.NET, & Managed C++, for a safe, modern
execution environment

T-SQL continues to be supported & enhanced

Remains good choice for data-intensive procedures

Safe extended stored proc replacement
26

13
TSQL vs. C# - data
conversion
if @bin is null return null
declare @len int, @b tinyint, @lowbyte tinyint, @hibyte tinyint, @index int, @str nchar(2), @result nvarchar(4000)
set @len = datalength(@bin)
set @index = 1
set @result = '0x'
while @index <= @len
begin
set @b = substring(@bin, @index, 1)
set @index = @index + 1
set @lowbyte = @b & 0xF
set @hibyte = @b & 0xF0
if @hibyte > 0
set @hibyte = @hibyte / 0xF
set @result = @result +
((case
when @hibyte < 10 then convert(varchar(1), @hibyte)
when @hibyte = 10 then 'A'
when @hibyte = 11 then 'B'
when @hibyte = 12 then 'C'
when @hibyte = 13 then 'D'
when @hibyte = 14 then 'E'
when @hibyte = 15 then 'F'
else 'Z'
end)
if (value == null) return null; +
(case
StringBuilder sb = new StringBuilder(); when @lowbyte < 10 then convert(varchar(1), @lowbyte)
when @lowbyte = 10 then 'A'
foreach (byte b in value) when @lowbyte = 11 then 'B'
when @lowbyte = 12 then 'C'
sb.Append(b.ToString("X2")); when @lowbyte = 13 then 'D'
when @lowbyte = 14 then 'E'
return sb.ToString(); when @lowbyte = 15 then 'F'
else 'Z'
end))
end

27

TSQL vs. C# for relational task

SELECT * FROM T_CUSTOMERS WHERE C_CITY = @name

private static readonly SqlDefinition x_definition;

string query = “SELECT * FROM T_CUSTOMERS WHERE C_CITY = @name”;
SqlCommand cmd = new SqlCommand();
cmd.CommandText = query;
cmd.Parameters.Add(new SqlParameter("@name", SqlDbType.NVarChar, 128));
x_definition = new SqlDefinition(cmd);
SqlExecutionContext sec =
SqlContext.GetConnection().CreateExecutionContext(x_definition);
sec.SetSqlString(0, "ExecutePredefinedQuery");
SqlContext.GetPipe().ExecuteReader(sec);

28

14
Yukon XML Support

XML data type & XML index support

Unified XML & relational store

Both SQL & XQuery supported by same
industrial strength infrastructure

Leverages existing SQL engine & optimizer

XQuery with data modification extensions

XML views

Declarative Mapping Files

XML schema enforcement

Client access using ADO.NET & SOAP
29

XML Scenarios
Data Exchange…

XML data

Point-of-Sales Supplier

 Platform independent transport format

 Loosely-coupled systems
 B2B, B2C, work flow, …
30

15
XML Scenarios
Semi-structured storage…

First Name Last Name Address Phone XML data

type

XML Datatype

Loosely structured data

Data with a dynamic schema

XML Views

Mixed data – structured/unstructured

XML stores w/o relational support challenged 31

Native XML Store

XML Data Type

XML data type

Native SQL type

Use for column, variable or parameter
CREATE TABLE docs (id INT PRIMARY KEY, xDoc
XML NOT NULL)

Store un-typed or typed XML instances

Well-formed and validation checks

Methods on XML data type

Query(), value(), exist() & modify()

32

16
Native XML Store
XML Data Modification

Insert, update, & delete XQuery
extensions

XML sub-tree modification:

Add or delete XML sub-trees

Update values

Add a new section after section 2:
UPDATE docs SET xDoc::modify(
‘insert <section num=“2”>
<heading>Background</heading>
</section>
after /doc/section[@num=1]')
33

HTTP/SOAP Features

SQL & stored proc calls via HTTP/SOAP

Easy, standards based connectivity from Unix
platforms to SQL Server

Provides native HTTP listening

HTTP endpoint specifying URL, port, reqs

Publish WSDL for endpoints

Standard-based

SOAP 1.1 and 1.2, WSDL 1.1, inlined XSD

Windows and SQL authentication (SSL only)

Stored Proc can return result as DataSet

Compatible with mid-tier prog. model

34

17
Data Access
API Enhancements: ADO .NET V2

Multiple active result sets (MARS)

Object Persistence FX (ObjectSpaces)

Query notifications – for cache invalidation

Functional parity with ADO (ADOX, etc)

Server cursors (SqlResultSet)

Asynchronous client access

Bulk update, paging, and batching

XML Data type support

User Defined Type (UDT) support
35

SQL Server BI Platform

Reporting Services
Development Tools

Management Tools

Analysis Services
OLAP & Data Mining

Data Transformation
Services
ETL

SQL Server
Relational Engine

36

18
Analysis Services

Unified Dimensional Model

Integrating relational and OLAP views

Pro-active caching

Bringing the best of MOLAP to ROLAP

Advanced Business Intelligence

MDX scripts, translations, currency…

Web services

Native XML

37

Data Mining

Significant investment

Important, new algorithms

Association rules, time series, regression
trees, sequence clustering, neural nets, …

Embedded, integrated, complete

38

19
Microsoft Office System Integration

Office Document Formats

Excel, Word, outlook

Import reports from Access

HTML rendering

Interactive Charts, PivotTables, Spreadsheets

SharePoint Portal Server Integration

Report Web Parts

Report delivery to document libraries, lists

Report Library Template

Searchable

39

Questions, Discussions?

40

20