2006/02/23

Oracle 10g Data Pump Say good bye to exp and imp (or not)!
Simon Pane

Agenda

Data Pump Overview Using Data Pump Demonstration Data Pump Test Cases

2

Data Pump Overview

3

What is Data Pump?

A replacement of the traditional export/import utilities?

The evolution of the traditional export/import utilities?
A completely new 10g utility serving a similar yet slightly different purpose?

4

can specify what to include independent  Versatile  Platform  Serial output slow / resumable  Cons  Comparatively  Can be network intensive filtering options (for example. from one tablespace to another) 5  Non-interruptible  Limited  Limited .Other Options for Moving Data  Traditional Export and Import  Pros  Easy to use – most DBAs have years of experience using these utilities – various options available.e. can exclude just VIEWS) remapping options (i.

Other Options for Moving Data  Transportable Tablespaces  Pros  Undoubtedly  Can the fastest way to move data support if the platform byte-order is the same use the traditional exp/imp or Data Pump to move meta-data  Cross-platform  Cons  Tablespaces  Not must be made read-only selective (must move the entire tablespace) is not possible (tablespace is read only when copied) physical reorganization is performed  Flashback  No  Datafile  Must sizes remain constant use RMAN to convert the datafile if migrating to a platform with a different byte-order (check V$TRANSPORTABLE_PLATFORM) 6 .

Other Options Used Less Frequently     Extraction to a flat file and loading using SQL Loader Direct copy using database links (SQL Plus COPY command) Oracle Streams 3rd Party data ETL or reorg tools 7 .

can import through a network link Track in v$session_longops Resumable (interruptible and restartable) 8 . 6. PL/SQL Interface – programmable A file is not actually required . 9. 7.Top 10 Reasons to Love DataPump 10. Similar look and feel to the old exp/imp Can filter on the full range of object types Can re-map datafiles and or tablespaces on import Estimates the export file size (space needed) Parallelizable Significantly faster than the traditional exp/imp 4. 3. 1. 8. 5. 2.

7.Top 10 Reasons Not to Love Data Pump 10. never the client Oracle directories are required in the DB to access the files Does not support COMMIT on imp or CONSISTENT on exp If constraints are violated on import. 1. 2. 5. Still generates redo (unlike direct path inserts) 9. 3. No equivalent to the STATISTICS option Cannot be used with sequential media such as tapes and pipes (not read/written serially) 4. Aggregation of exported data is not possible (sort only) Performance on the server Harder to tell what it’s doing at any given time 6. Only accesses files on the server. the load is discontinued 9 . 8.

Operation Fundamentals  Export/Import  These utilities would basically connect to the Oracle database via Oracle NET and run queries or DDL/DML  Processing of returned results and I/O operations were done on the client  Data Pump  The executables call PL/SQL APIs  Therefore processing is done on the database server  This can be an advantage or a disadvantage depending on the situation  “Self-Tuning”: no longer need to use BUFFER or RECORDSET 10 .

Export Operation exp.exe Network Oracle Database Export File(s) 11 .

Data Pump Export Operation expdp.exe Network Oracle Database Export File(s) 12 .

not the client Must have a DIRECTORY created in the Oracle database for I/O  Permissions for the userid connecting to the instance. not the schemas being exported or imported  Canceling the client process does not stop the job    Doesn’t automatically overwrite dump file if it already exists – returns an error instead Parameters (command line) are reported in the log file Exported objects order based on table size (descending) instead of alphabetically 13 .Key Differences   Dump and log files are on the server.

Multiple Interfaces 1. Enterprise Manager PL/SQL  Can be used independently but is difficult  All of these call the DBMS_DATAPUMP API  Uses Oracle Advanced Queuing  Uses DBMS_METADATA 14 . Command line utilities expdb and impdb  Similar to the familiar exp and imp in usage  Use HELP=Y for a list of commands  Oracle documentation provides a comparison table to exp/imp 2. 3.

Unload Mechanisms  Data Pump automatically chooses to unload data either using:  Direct path  External Tables (new driver called ORACLE_DATAPUMP)  Same “External Tables” mechanism that was introduced in Oracle9i  When will it use External tables:  When parallelism can be used  When the table contains a complex data type or structure that prevents direct path unloads A lot of tables fall under this situation – see Oracle documentation for a complete list  It doesn’t really matter to us which method is used 15 .

the master control table is written to the dump file and dropped from the database  Worker Processes  Performs the loading/unloading  Number of processes depends on the degree of parallelism (the PARALLEL option) 16 .Multiple Processes  Master Control Process  Spawns worker processes  Populates the master control table and log file  The master control table can be queried to track the job’s process  At the end of an export.

run impdp with the ATTACH= option  Example: impdp userid=system/oracle attach=JOB_01  Brings you back into “interactive-command” mode 17 .Detaching and Re-Attaching  Issuing “Ctrl-C” from the data pump import will detach  Import is running on the server so it will continue  Brings you into “interactive-command” mode  To re-attach.

New Views  DBA_DATAPUMP_JOBS and USER_DATABASE_JOBS  Identify all jobs regardless of their state  Identify any master tables not associated with an active job  DBA_DATAPUMP_SESSIONS  Identify user sessions that are attached to a job  Data pump sessions populate v$session_longops  Documentation says that it is 100% accurate for imports but testing proves otherwise!!! 18 .

and control jobs initiated by others  Perform schema.Security Considerations    Still uses the EXP_FULL_DATABASE and IMP_FULL_DATABASE A privileged user will have these two roles A privileged user can:  Export/import objects owned by other schemas  Export non-schema objects (metadata)  Attach to. datafile. monitor. and tablespace remapping  Similar to the traditional export/import  Supports label security  If exporting user has the EXEMPT ACCESS POLICY role 19 .

If the source table has statistics.” 20 . Statistics are always saved for tables.”  From Oracle documentation regarding data pump imports:  “A parameter comparable to STATISTICS is not needed. they are imported.Object Statistics  From Oracle documentation regarding data pump exports:  “A parameter comparable to STATISTICS is not needed.

Other Random Points     Can still use a parameter file and the PARFILE command line option Fully supports Automatic Storage Management (ASM) Can still flashback to a specified time or SCN Can still extract (or backup) DDL (meta data)  Using the SQLFILE option instead of the traditional INDEXFILE or SHOW options  Full support of LOBS 21 .

Using Data Pump 22 .

Directory created.write on directory dpump_demo to simon. SQL> 23 . Grant succeeded.Oracle Directory Objects  Must first create an Oracle directory object and give the user who will be performing the Data Pump activities permission to use it (or rely on defaults): SQL> create or replace directory dpump_demo as 'C:\temp'. SQL> grant read.

…]       FILESIZE=integer[B | K | M |G] (default=unlimited) FLASHBACK_SCN=scn_value FLASHBACK_TIME=“TO_TIMESTAMP(time_value)” FULL={Y | N} INCLUDE=object_type[:name_clause] [.Key Data Pump Export Parameters       CONTENT={ALL | DATA_ONLY | METADATA_ONLY} DIRECTORY=directory_object (default=DATA_PUMP_DIR) DUMPFILE=[directory_object:]file_name [.…] JOBNAME=jobname_string 24 .…] ESTIMATE={BLOCKS | STATISTICS} ESTIMATE_ONLY={Y | N} EXCLUDE=object_type[:name_clause] [.

Key Data Pump Export Parameters       LOGFILE=[directory_object:]file_name NOLOGFILE={Y | N} PARALLEL=integer (default=1) QUERY=[schema.…] 25 .…]  TABLESPACES=tablespace_name [.]table_name[:partition_name] [.][table_name:]query_clause SCHEMAS=schema_name [.…] TABLES=[schema_name.

FUNCTION.VIEW  Using a query clause  QUERY=emp:„“WHERE salary > 100000”‟ 26 .Data Pump Export Parameter Samples  Multiple dump files using a substitution variable (%U):  DUMPFILE=DP_DIR1:SCOTT_20060223_%U. FUNCTIONS and VIEWS:  INCLUDE=TABLE.dmp  Excluding indexes that start with “EMP”:  EXCLUDE=INDEX:“LIKE „EMP%‟”  Excluding the SCOTT schema from a FULL export:  EXCLUDE=SCHEMA:“=„SCOTT‟”  Mimicking the traditional CONSITENT parameter:  FLASHBACK_TIME=“TO_TIMESTAMP”  Exporting only TABLES.

Key Data Pump Import Parameters       CONTENT={ALL | DATA_ONLY | METADATA_ONLY} DIRECTORY=directory_object (default=DATA_PUMP_DIR) DUMPFILE=[directory_object:]file_name [.][table_name:]query_clause 27 .…]      JOBNAME=jobname_string LOGFILE=[directory_object:]file_name NOLOGFILE={Y | N} PARALLEL=integer (default=1) QUERY=[schema.…] EXCLUDE=object_type[:name_clause] [.…] FULL={Y | N} INCLUDE=object_type[:name_clause] [.

…] SKIP_UNUSABLE_INDEXES={Y | N}     SQLFILE=[directory_object:]file_name TABLE_EXISTS_ACTION={SKIP|APPEND|TRUNCATE|REPLACE} TABLES=[schema_name.…] TABLESPACES=tablespace_name [.Key Data Pump Import Parameters       REMAP_DATAFILE=source_datafile:target_datafile REMAP_SCHEMA=source_schema:target_schema REMAP_TABLESPACE=source_tablespace:target_tablespace REUSE_DATAFILES={Y | N} SCHEMAS=schema_name [.…] 28 .]table_name[:partition_name] [.

Interactive Mode Commands       ADD_FILE=[directory_object:]file_name [.…] CONTINUE_CLIENT EXIT_CLIENT FILESIZE=number KILL_JOB PARALLEL=integer    START_JOB STATUS STOP_JOB 29 .

Demonstration 30 .

dmp schemas=larry SELECT * FROM DBA_DATAPUMP_JOBS.dmp schemas=scott impdp system/oracle@ORA1020 dumpfile=scott.dmp schemas=SCOTT remap_schema=SCOTT:LARRY expdb system/oracle@ORA1020 dumpfile=larry. 31    .Exporting and Importing Sample Schemas  expdp system/oracle@ORA1020 dumpfile=scott.

Using Interactive Mode  Ctrl-C to detach from the current export   Export> status Export> stop_job  expdp system/oracle@ORA1020 attach=SYS_EXPORT_SCHEMA_01 Export> start_job Export> exit_client 32   .

Data Pump Test Cases 33 .

6  Actual data is not important  Schema objects  Tables: 218 (many empty tables)  Indexes: 1180 (SIEBEL is a heavily indexed application)  Schema size (from DBA_SEGMENTS)  Tables: 1255MB  Indexes: 148MB 34 .1.Test Scenario #1  Generated sample SIEBEL data  Brushed off the dust on some old SIEBEL data population scripts (circa 07/2000)  Designed for SIEBEL 6 on Oracle 8.

Export Performance Test Criteria   All work performed on laptop C681 SGA remains constant  SGA_TARGET=0  SGA_MAX_SIZE=256MB  BUFFER CACHE=152MB  SHARED_POOL=60MB    Mostly default parameters Not monitoring CPU utilization Performed 4 runs  Disregarded results from 1st run and averaged the other 3 35 .

sql >> expdp.log schemas='SIEBEL' directory=test_dir -s system/oracle @timestamp.dmp logfile=SIEBEL.sql >> exp.sql > expdp.exe userid=system/oracle@ORA1020 dumpfile=SIEBEL.log 36  sqlplus .log owner='SIEBEL' -s system/oracle @timestamp.log  sqlplus  expdp.exe  sqlplus -s system/oracle @timestamp.sql > exp.log  EXPDP  erase SIEBEL.log userid=system/oracle@ORA1020 file=SIEBEL.Export Scripts  EXP  sqlplus  exp.dmp -s system/oracle @timestamp.dmp log=SIEBEL.

Export Performance Test Results exp 6:02 N/A 965 MB expdp 3:18 1.201 GB 621MB Average Export Time Estimated File Size Actual File Size 37 .

sql > expdp.log fromuser='SIEBEL' touser='SCOTT' commit=y -s system/oracle @timestamp.log 38  sqlplus .sql >> exp.dmp logfile=SIEBEL.sql >> expdp.dmp log=SIEBEL.exe -s system/oracle @timestamp.Import Scripts  IMP  sqlplus  imp.log schemas='SIEBEL' directory=test_dir remap_schema=SIEBEL:SCOTT remap_tablespace=TOOLS:SCOTT_DATA -s system/oracle @timestamp.exe userid=system/oracle@ORA1020 dumpfile=SIEBEL.log  impdp.log  sqlplus  IMPDP  sqlplus -s system/oracle @timestamp.sql > exp.log userid=system/oracle@ORA1020 file=SIEBEL.

Import Performance Test Results imp Average Import Time Average Import Time (no indexes) Average Import Time (no rows) impdb 27:07 25:19 27:27  Database was in ARCHIVELOG mode    Destination tablespace and archived log destination were both on ASM drives Machine performance was degraded much more by impdb import No import tuning performed (only COMMIT=Y) 39 .

Test Scenario #2   Data taken from an actual CGI internal application Schema objects  Tables: 22  Indexes: 26  Schema size (from DBA_SEGMENTS)  Tables: 300MB  Indexes: 101MB 40 .

Export Scripts  EXP  sqlplus  exp.log 41  sqlplus .log userid=system/oracle@ORA1020 file=SCOTT.log  EXPDP  erase SIEBEL.sql >> exp.dmp logfile=SCOTT.dmp log=SCOTT.dmp -s system/oracle @timestamp.log schemas='SCOTT' directory=test_dir -s system/oracle @timestamp.sql > expdp.exe userid=system/oracle@ORA1020 dumpfile=SCOTT.log  sqlplus  expdp.exe  sqlplus -s system/oracle @timestamp.log owner='SCOTT' -s system/oracle @timestamp.sql > exp.sql >> expdp.

Export Performance Test Results exp 1:24 N/A 261MB expdp 1:32 290MB 233MB Average Export Time Estimated File Size Actual File Size 42 .

sql >> expdp.sql > expdp.log  IMPDP  sqlplus -s system/oracle @timestamp.exe userid=system/oracle@ORA1020 dumpfile=SCOTT.dmp logfile=SCOTT.Import Scripts  IMP  sqlplus  imp.log  impdp.dmp log=SCOTT.log schemas='SCOTT' directory=test_dir remap_schema=SCOTT:LARRY remap_tablespace=SCOTT_DATA:LARRY_DATA  sqlplus -s system/oracle @timestamp.log 43 .sql >> exp.sql > exp.log fromuser='SCOTT' touser='LARRY' commit=y  sqlplus -s system/oracle @timestamp.log userid=system/oracle@ORA1020 file=SCOTT.exe -s system/oracle @timestamp.

Import Performance Test Results imp 5:48 impdb 2:26 Average Import Time    Database was in NOARCHIVELOG mode Destination tablespace and archived log destination were both on ASM drives No import tuning performed (only COMMIT=Y) 44 .

Conclusions 45 .

Conclusions  Data Pump is an exciting new Oracle 10g tool that provides many benefits over the traditional export and import utilities Whether to use Data Pump. should a new file extension be used??? (. DBAs and developers should spend the time to learn how to use it Final thought: since Data Pump dump files and traditional export dump files are not compatible/interchangeable. or even the traditional exp/imp will depend on the situation Since the command line interface is easy to use and so similar to the traditional exp/imp.dpd) 46    .dmp vs . Transportable Tablespaces.

The End Comments. Questions ??? 47 .

Sign up to vote on this title
UsefulNot useful