20181205-Regio Berlinbrandenburg Christiane Wellnitz Oracle Deutschland Step by Step Guide Autonomous

Autonomous Data Warehouse
Cloud Service (ADW)

Ein Data Warehouse, das wie von selbst läuft
Christiane Wellnitz
Architect for Cloud & IT-Technologies
ORACLE Deutschland B.V & Co. KG
Safe Harbor Statement
The following is intended to outline our general product direction. It is intended for
information purposes only, and may not be incorporated into any contract. It is not a
commitment to deliver any material, code, or functionality, and should not be relied upon
in making purchasing decisions. The development, release, timing and price of any
features or functionality described for Oracle's products may change and remains at the
sole discretion of Oracle Corporation. Fees apply for new Database product offerings.
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 2

Agenda
1 Introduction to Autonomous Data Warehouse

2 Getting started with Autonomous Data Warehouse
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Introduction to Autonomous Data Warehouse

Autonomous Data Warehouse Cloud
Was soll man eigentlich darüber

erzählen?
…es läuft einfach von selbst…
…und das ist das Gute daran…

Bevor wir beginnen…
Autonomous Data Warehouse Cloud Service

1 2
ADWC umfasst 2 wichtige Aspekte:
1. „Autonomous“
2. Cloud
Beide Aspekte ergänzen sich gegenseitig.
Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | Oracle Confidential – Restricted 6
Autonom ist etwas anderes als automatisch*
• Autonom bedeutet:
vollkommen selbständig agieren
• basierend auf „lernenden“ Regeln
Die Nutzer sagen nur noch
was sie wollen
* Automatisch hat die Oracle Datenbank immer schon Vieles gemacht
Automatisiert vs. Autonomous
Automatisiert Autonomous / Autark
• Automatisierung in einem Fahrzeug • Ein Fahrzeug fährt selbst
Cruise control – Keine Lenkrad-Nutzung
– Start-/Stopp-Automatic – Kein Gas geben oder Bremsen
– Tempomat – Man gibt nur das Ziel an und das Auto fährt
– Warnungen beim Spurwechsel dort hin
• Vereinfachte Datenbank- • Die Datenbank verwaltet sich selbst

Operationen: – Alle Features automatisch
– Man legt nur noch grob das Ziel fest
– Automatic storage management,
(Performance, Datenmenge … etc) und
– Automatic memory management, … es läuft
– Automatic degree of parallisme
Der Weg zur Autonomous Database
• Oracle hat kontinuierlich Database Automation vorangetrieben
Oracle Database 9i, 10g Oracle Database 11g, 12c

• Automatic Storage Management (ASM) • Automatic SQL Tuning
• Automatic Memory Management • Automatic Workload Replay
• Automatic DB Diagnostic Monitor (ADDM) • Automatic Capture of SQL Monitor
• Automatic Workload Repository (AWR) • Automatic Data Optimization
• Automatic Undo Tablespaces • Automatic Storage Indexes
• Automatic Segment Space Management • Automatic Columnar Cache
• Automatic Statistics Gathering • Automatic Diagnostic Framework
• Automatic Standby Management (Broker) • Automatic Refresh of Database Cloning
• Automatic Query Rewrite • Autonomous Health Framework

Was ist die Autonomous Database?
Autonomous Oracle
Database Cloud Secret
Sauce

Der Weg zur Autonomous Database
• Mit Engineered Systems,
stellt Oracle eine Automated Database Infrastructure

Wo wird die Autonomous Database verfügbar sein?
Cloud at Customer Public Cloud Service
Customer Data Center Oracle Cloud

Subscription Subscription
Oracle Managed Oracle Managed
• Easy
– Fully-managed, pre-configured and optimized for DW workloads
– Simply load data and run
• No need to define indexes, create partitions, etc.
• Fast
– Based on Exadata technology
• Elastic
– Instant scaling of compute or storage with no downtime

Fully-managed
• Oracle automates end-to-end
management of the data warehouse
– Provisioning new databases
– Growing/shrinking storage and/or compute
– Patching and upgrades
– Backup and recovery
• Full lifecycle managed using the service
console
– Alternatively, can be managed via command-
line interface or REST API

Fully-tuned
• “Load and go”
– Define tables, load data, run queries
• No tuning
• No special database expertise required
– Good performance out of the box
• Query using any business analytics tool
or cloud service
– Built-in SQL worksheet and notebook also
included

Architecture
Developer Tools Autonomous
Data WarehouseData Warehouse Cloud
Services
(EDWs, DW, departmental marts and sandboxes)
Service Management Built-in Access Tools
Oracle Exadata Cloud Service

SQL Developer
Oracle Database Cloud Service
Service Console SQL Worksheet Oracle ML
Express Cloud Service
Data Integration
Services
Oracle Data Integration
Cloud Service
DW Database
3rd Party DI on
Oracle Cloud Compute
3rd Party DI
on-premises

Getting Started with
Provisioning a database

Provisioning a Database
• Provisioning requires only 5 simple
questions:
– Database name?
– Data center?
– Number of CPUs?
– Storage capacity?
– Admin user password?
• New service created in <30 seconds
(regardless of size)
– Ready to connect

Demo
Provisioning a database

Einloggen in Oracle Cloud Infrastructure

Oracle Cloud Infrastructure Dashboard

Oracle Cloud Infrastructure Menue

Oracle Cloud Infrastructure ADW

Oracle Cloud Infrastructure Create ADW (1)






Oracle Cloud Infrastructure – Backup after some days

Connecting to the database

Connecting to the Database
• Securely connect using credential wallets via SQL*Net, JDBC, ODBC
• Wallet can be downloaded from the service console or using REST APIs

Demo
Downloading the credential wallet


Oracle Cloud Infrastructure Service Console ADW (1)

Oracle Cloud Infrastructure Service Console ADW (2)

Oracle Cloud Infrastructure – Wallet (1)



Demo
Connecting to the database using the wallet in SQL Developer





Database Service Names
• 3 pre-defined database services
– Choice of performance and concurrency
• HIGH
Example for a database with 16 OCPUs
– Highest resources, lowest concurrency
No of concurrent
– Queries run in parallel queries
Max idle time CPU shares
• MEDIUM HIGH 3 5 mins 4

– Less resources, higher concurrency
– Queries run in parallel MEDIUM 20 5 mins 2
• LOW LOW 32 1 hour 1

– Least resources, highest concurrency
– Queries run serially





Creating users

Simplified User Creation
• Simplified user creation via the new database role
– No need to specify anything except the password
– DWROLE includes all necessary privileges for a DW developer/user
create user sh identified by sh;

grant dwrole to sh;

Loading data

Data Loading Options
• Data loading via SQL*Net
– Suitable for small volumes of data
• SQL*Loader from local filesystem
• ETL scripts that use DML to insert/update data
• Data loading from Oracle Object Storage

– Preferred technique for large volumes of data
• Additionally enables data-sharing with other cloud services
– Stage data in Oracle Object Storage, then load into the database using new PL/SQL
APIs

Loading Data from Object Stores
• Source data from files on object stores for
data loading
– Oracle Object Store or AWS S3
– Any supported Oracle-loader file format ADWC
– Roadmap: any Hadoop file format
Object Storage

New Cloud API to Access Object Stores, DBMS_CLOUD
• New PL/SQL package for accessing files
in object stores
• No need to manually define external
tables for loading files
– Makes it easier to specify the format of the
source data

Loading Data from the Oracle Object Store
• Define your credentials for the object store
– Oracle Cloud Infrastructure Object Store username and Swift password required
• Credential stored in the database schema once and used for accessing the
object store for all loads
begin
dbms_cloud.create_credential(
credential_name => 'OBJ_STORE_CRED',
username => 'tenant1',
password => ’password'
);
end;
/

Loading Data from the Oracle Object Store
• Load data directly into the target table without any intermediate steps
• Data format in the source file easily specified as JSON
begin
dbms_cloud.copy_data(
table_name =>'CHANNELS',
credential_name =>'OBJ_STORE_CRED',
file_uri_list =>'https://swiftobjectstorage.us-ashburn-
1.oraclecloud.com/v1/dwcsdemo/DEMO_DATA/chan_v3.dat',
format => json_object('ignoremissingcolumns' value 'true',
'removequotes' value 'true')
);
end;
/

Troubleshooting Loads
• Load operations logged for troubleshooting and historical load tracking
– New table user/dba_load_operations
• Log and bad files accessible as tables
select table_name,status,rows_loaded,logfile_table,badfile_table
from user_load_operations;
TABLE_NAME STATUS ROWS_LOADED LOGFILE_TABLE BADFILE_TABLE

-------------------- --------- ----------- -------------------- --------------------
CHANNELS FAILED COPY$1_LOG COPY$1_BAD
CHANNELS COMPLETED 5 COPY$2_LOG COPY$2_BAD

Querying data on object stores

Querying Data on the Oracle Object Store
begin
dbms_cloud.create_external_table(
table_name =>'CHANNELS_EXT',
credential_name =>'OBJ_STORE_CRED',
file_uri_list => 'https://swiftobjectstorage.us-ashburn-
1.oraclecloud.com/v1/dwcsdemo/DEMO_DATA/chan_v3.dat',
format => json_object('ignoremissingcolumns' value 'true', 'removequotes' value
'true'),
column_list => 'CHANNEL_ID NUMBER,
CHANNEL_DESC VARCHAR2(20),
CHANNEL_CLASS VARCHAR2(20),
CHANNEL_CLASS_ID NUMBER,
CHANNEL_TOTAL VARCHAR2(13),
CHANNEL_TOTAL_ID NUMBER'
);
end;
/
select count(*) from channels_ext;

Web-Based SQL Worksheet and Notebooks

Built-in SQL Worksheet and Notebook
• Quickly start running queries with
built-in web-based notebooks
– No need to install a client query tool
• Initially supports SQL and PL/SQL
– More languages in the roadmap
• Based on Apache Zeppelin

Oracle Cloud Infrastructure – Create ML Users (1)


https://docs.oracle.com/en/cloud/paas/autonomous-data-warehouse-cloud/omlug/getting-started-oracle-machine-learning1.html#GUID-02BFC69B-B304-4D04-A0F3-370F0C537DD5
https://docs.oracle.com/en/cloud/paas/autonomous-data-warehouse-cloud/omlug/getting-started-oracle-machine-learning1.html#GUID-D52F0376-BA88-435A-94D0-E70F8BCD3388
https://docs.oracle.com/en/cloud/paas/autonomous-data-warehouse-cloud/omlug/getting-started-oracle-machine-learning1.html#GUID-13452045-6AF6-402B-83CC-C9AFBCEC30CA
https://oracle.github.io/learning-library/workshops/journey4-adwc/?page=LabGuide700.md
https://docs.oracle.com/en/cloud/paas/autonomous-data-warehouse-cloud/omlug/getting-started-oracle-machine-learning1.html#GUID-2AEC56A4-E751-48A3-AAA0-0659EDD639BA
https://docs.oracle.com/en/database/oracle/oracle-database/12.2/dmprg/data-mining-SQL.html#GUID-46E7420D-17D0-4683-B9F3-6AB42E9B1EB6
https://docs.oracle.com/en/database/oracle/oracle-database/12.2/dmcon/introductions.html#GUID-E9D74E57-FEE4-41B9-ACDE-C46B03C20278
https://oracle.github.io/learning-library/workshops/journey4-adwc/?page=LabGuide700.md
https://dgcameron.github.io/adwc4dev/workshops/adwc4dev/?version=Self-Guided&page=L300.md










Scaling your database

Scaling Your Database
• Scale your database on demand without tedious manual steps
– Independently scale compute or storage
– Resizing occurs instantly, fully online
– Memory, IO bandwidth, concurrency scales linearly with CPU
• Close your database to save money when not used
– Restart instantly

Demo
Scaling your database

Oracle Cloud Infrastructure – Scale Up/Down (1)





Administration and monitoring

Automated Database Configuration
• init.ora parameters
– Parameters optimized for DW workloads
– Memory, parallelism, sessions configured based on number of CPUs
– Users can modify a limited set of parameters, e.g. NLS settings
• Tablespace management
– Pre-defined data and temporary tablespaces
– Users cannot create/modify tablespaces
• Compression
– All tables compressed using Hybrid Columnar Compression
– Users cannot change compression method or but disable compression

Automated Database Configuration
• Optimizer stats gathering
– Stats gathered automatically during direct load operations
– Users can gather stats manually if they want
• Optimizer hints
– Hints ignored by default
– Users can enable hints explicitly
• Result cache configuration
– Result cache enabled by default for all queries

Monitoring
• Simplified monitoring using the web-based service console
• Historical and real-time performance charts
• Real-Time SQL Monitoring to monitor running and past SQL statements
• Historical data load monitoring

Demo
Monitoring

Oracle Autonomous Data Warehouse
Summary

• Easy
– Fully-managed, pre-configured and optimized for DW workloads
– Simply load data and run
• No need to define indexes, create partitions, etc.
• Fast
– Based on Exadata technology
• Elastic
– Instant scaling of compute or storage with no downtime

Cloud Plattformen für Data Warehousing Datenbanken
Exadata Cloud Service Autonomous Data Warehouse Cloud

Customizable: Benötigt einen DBA Autonomous: kein DBA
Core DW DB Funktionalität:
Volle Datenbank Funktionalität:
Ausschließlich Fokussiert auf DW und
alle Oracle Features
Einfachheit

Confidential – Oracle Internal
https://cloud.oracle.com/cost-estimator
https://cloud.oracle.com/cost-estimator
25 Tage / Monat
12 Stunden / Tag
12 OCPUs / Std.
8 TB Exadata Storage
Studie von IDC
http://idcdocserv.com/US43571317
Autonomous Data Warehouse Cloud: So geht's
https://apex.oracle.com/pls/apex/germancommunities/dbacommunity/tipp/6921/index.html

Oracle Autonomous Transaction Processing

Oracle Cloud Infrastructure ATP (1)

Oracle Cloud Infrastructure ATP (2)

Oracle Cloud Infrastructure Service Console ATP (1)



Oracle Cloud Infrastructure ATP Service Console (4)


Christiane42#








Oracle Cloud Infrastructure – Manuel Backup (1)

Oracle Cloud Infrastructure – Manuel Backup (2)

Oracle Cloud Infrastructure – Import (1)







SQLMaria
https://sqlmaria.com/


20181205-Regio Berlinbrandenburg Christiane Wellnitz Oracle Deutschland Step by Step Guide Autonomous

Uploaded by

Document Information

Copyright

Available Formats

Share this document

Share or Embed Document

Sharing Options

Did you find this document useful?

Is this content inappropriate?

Copyright:

Available Formats

20181205-Regio Berlinbrandenburg Christiane Wellnitz Oracle Deutschland Step by Step Guide Autonomous

Uploaded by

Copyright:

Available Formats

Autonomous Data Warehouse

Cloud Service (ADW)

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 2

1 Introduction to Autonomous Data Warehouse

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Was soll man eigentlich darüber

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Autonomous Data Warehouse Cloud Service

* Automatisch hat die Oracle Datenbank immer schon Vieles gemacht

• Vereinfachte Datenbank- • Die Datenbank verwaltet sich selbst

Oracle Database 9i, 10g Oracle Database 11g, 12c

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 9

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 10

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. | 11

Customer Data Center Oracle Cloud

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Oracle Exadata Cloud Service

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• MEDIUM HIGH 3 5 mins 4

• LOW LOW 32 1 hour 1

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

create user sh identified by sh;

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

• Data loading from Oracle Object Storage

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |

Copyright © 2017, Oracle and/or its affiliates. All rights reserved. |