Oracle import speed performance tips

Free Oracle Tips HTML Text

Search BC Oracle Sites

Oracle data load (import, SQL*Loader) speed tips
Oracle Tips by Burleson Consulting
July 1, 2003 - Revised March 12, 2006

Oracle Tips database_properties tips ORA 12899 Value too Large for Column ORA 27300 OS System De pendent Ope ration faile d with status ORA 01801 Date Format is too Long for Inte rnal Buffer

Home E-mail Us Oracle Articles

For professionals only: This is a expert-only overview of improving data pump import performance. For beginners, see here. Also see my notes on tuning and improving Oracle export (expdp) performance speed. Maximizing Oracle Import (impdp) Performance Regardless of which options were used when the data was exported, it has no influence on how the data is imported. For example, it is irrelevant to the import process whether it was a direct path export or not, since it is a plain export file, be it generated from direct or conventional means. Unfortunately, there is no direct option available for imports (only for export and SQL*loader). The import process has more tuning limitations than other utilities. The DBA should consider the following when trying to optimize import performance: Analyze once after the load - Set analyze=n and analyze with dbms_stats after the load has completed. Use Solid-state disk - For a fully-tuned import job, only faster devices can speed-up import rates. Many large companies used partitioned tables, and keep the current partition on SSD for fast imports. Increase recordlength - Many set recordlength to 64k, but it needs to be a multiple of your I/O chunk size and db_block_size (or your multiple block size, e.g. db_32k_block_size).

ORA-00824: Cannot Se t sga_targe t Due to Existing Inte rnal Se ttings

Oracle Training Oracle Tips Oracle Forum Class Catalog

Remote DBA Oracle Tuning Emergency 911 RAC Support Apps Support Analysis Design Implementation Oracle Support

SQL Tuning Security Oracle UNIX Oracle Linux Monitoring Remote support Remote plans Remote services Application Server Applications Oracle Forms Oracle Portal App Upgrades SQL Server Oracle Concepts Software Support Remote Support Development Implementation

Set commit=n – For tables that can afford not to commit until the end of the load, this option provides a significant performance increase. Larger tables may not be suitable for this option due to the required rollback/undo space. Dedicate a single, large rollback segment - Many professionals create a single large rollback segment and take all others offline during the import. Set indexes=n – Index creation can be postponed until after import completes, by specifying indexes=n. If indexes for the target table already exist at the time of execution, import performs index maintenance when data is inserted into the table. Setting indexes=n eliminates this maintenance overhead. You can also use the indexfile=filename parm to rebuild all the indexes once, after the data is loaded. When editing the indexfile, add the nologging and parallel keywords (where parallel degree = cpu_count-1). Set Sort parms - To speed-up index creation (as a separate import job), set sort_write_buffers=6, sort_write_buffer_size=64000, sort_direct_write=true, and sort_area_size large enough prevent disk sorts. Defer CBO stats - Using impdp with the parameter exclude=statistics will greatly improve the import speed, but statistics will need to be re-analyzed or imported later. Use the buffer parameter – By using a larger buffer setting, import can do more work before disk access is performed. Disable logging - You can also use the hidden parameter _disable_logging = true to reduce redo, but beware that the resulting import will be unrecoverable. Import Option commit=y commit=y buffer=64000 Elapsed Time (Seconds) 120 100 Time Reduction 17%

Consulting Staff Consulting Prices Help Wanted!

Oracle Posters

1 de 7

28/09/2011 22:55

: exp userid=system/manager full=y direct=y recordlength=65535 file=exp_full. 2 de 7 28/09/2011 22:55 . but in the hands of a well-trained DBA. Your exact performance gain depends upon the following factors: DB_BLOCK_SIZE the types of columns in your table your I/O layout (the drive receiving the export file should be separate from the disk drive where the database files reside) For example. etc. and the undocumented parameter _disable_logging=TRUE.log Of all of the performance techniques for database loading. Your database could suffer a corruption from which you can't recover. invoking a Direct path Export with a maximum I/O buffer of 64kb can improve the performance of the Export with almost 50%. Note this discussion where Oracle expert Mark Bobak notes that you cannot re-start a failed instance when using _disable_logging=true. recoverable backup: "The main things to consider about NOLOGGING: Supported feature of Oracle Works only with direct load insert Still maintains minimal logging.htm Oracle Books Oracle Scripts Ion Excel-DB commit=n buffer=30720 commit=N buffer = 64000 72 67 40% 44% Also see MOSC Note 155477. unsupported parameter all redo writes to log buffer still happen writes of redo buffer down to disk are disabled." All experienced DBA's know that You MUST get approval from Oracle Technical Support for all usage of undocumented parameters. All operations are no longer protected by redo on disk. the hidden parameters are often a Godsend. i. Take a full system backup right before bringing the system online.dmp if your instance dies.. (crash or shutdown abort) there's no data in the online redo logs to even do instance recovery. and Don Burleson notes that _disable_logging should only be used after taking a full. Take a dump. the most important is the nologging operation.log imp userid=system/manager full=y recordlength=65535 file=exp_full.dmp log=imp_full.1 on improving importing speed: You may be able to improve performance by increasing the value of the RECORDLENGTH parameter when you invoke a direct path Export. e. it's quite common in my world where you have super-tight maintenance windows and every second counts. The backup sandwich: Take a dump (full system backup) Load everything I can in PARALLEL NOLOGGONG mode (ONLY during a scheduled downtime window). Most DBA's call this high-speed maintenance a "backup sandwich". especially for speeding-up rebuilds during tight maintenance windows with limited downtime. space transactions. extent invalidation records for loaded extents.g. This is very bad.e. Contrast with _disable_logging=TRUE: Undocumented. Not limited to direct load insert. The big difference is. This can be achieved by specifying the additional Export parameters DIRECT and RECORDLENGTH.dba-oracle.Oracle import speed performance tips http://www.

in turn.30) table_name. 2. Monitoring import performance Monitoring Oracle Data Pump import performance is simple select substr(sql_text. Disable Indexes and Constraints. The bind array's size is equivalent to the number of rows it contains (rows=) times the maximum length of each row. For conventional data loads only.000 rows per second. an instance crash or a shutdown abort WILL trash your Oracle database.'into "'). Use Fixed Width Data.Manage segment header contention for parallel inserts . about 24 millions rows per minute. but NOLOGGING. Use Direct Path Loads . Use unrecoverable. There are certain cases.dba-oracle. there are several tuning approaches: a . trunc(rows_processed/((sysdate-to_date(first_load_time.'yyyy-mm-dd hh24:mi:ss'))*24*60. Maximizing SQL*Loader Performance Oracle SQL*Loader is flexible and offers many options that should be considered to maximize the speed of data loads. The direct path loader (direct=true) loads directly into the Oracle data files and creates blocks in Oracle database block format.'yyyy-mm-dd hh24:mi:ss'))*24*60)) rows_per_minute from sys.The conventional path loader essentially loads the data by using standard insert statements.htm Remember. Fixed width data format saves Oracle some processing when parsing the data.v_$sqlarea where sql_text like 'insert %into "%' and command_type = 2 and open_versions > 0. $ sqlldr control=first.sql must be executed. Multiple freelists add additional segment header blocks. Optimizing Oracle INSERT performance The fastest Oracle table insert rate I've ever seen was 400. 4. round((sysdate-to_date(first_load_time. 3. This option is available for direct path loads only.instr(sql_text. this option allows multiple SQL*Loader jobs to execute concurrently. 5. larger bind arrays limit the number of calls to the database and increase performance. The fact that SQL is not being issued makes the entire process much less taxing on the database. You can also use Automatic Segment Space Management (bitmap freelists) to support parallel DML. 7. The savings can be tremendous. Use ROWS=n to Commit Less Frequently. To prepare the database for direct path loads. removing the bottleneck. The unrecoverable option (unrecoverable load data) disables the writing of the data to the redo logs. however.Make sure to define multiple freelist (or freelist groups) to remove contention for the table header. disabling database archiving can increase performance considerably. in which direct path loads cannot be used (clustered tables). Speed of inserts is primarily a function of device lets examine tips and techniques to improve the performance of all types of data loading. 3 de 7 28/09/2011 22:55 .ctl parallel=true direct=true $ sqlldr control=second. the script $ORACLE_HOME/rdbms/admin/catldr. maximum parallel DML (which. a function of the number of CPU's and the layout of the disks) also factor into the equation. These include: 1. Please note that the UNRECOVERABLE option has been deprecated and replaced with the NOLOGGING option. 8. using _disable_logging=TRUE. While this may not be feasible in certain environments. rows_processed. using super-fast RAM disk (SSD). The size of the bind array is specified using the bindsize parameter. depending on the type of data and number of rows. When using standard SQL statements to load Oracle data tables.Oracle import speed performance tips http://www. unless you take a full backup after your maintenance. including data pump import (impdp). Disable Archiving During Load. Issuing fewer commits will enhance performance. Next.1) minutes.ctl parallel=true direct=true 6. For conventional data loads only. For conventional data loads only. sql*Loader and SQL INSERT DML. but ASSM has some limitations. Use a Larger Bind Array. Use Parallel Loads.sql. rows specifies the number of rows per commit. the disabling of indexes and constraints can greatly enhance the performance of SQL*Loader. Available with direct path data loads only.

which will allow Oracle to avoid almost all redo logging. not the VALUES clause.It's far faster to rebuild indexes after the data load. Mark Bobak notes that if you submit parallel jobs to insert against the table at the same time. pretty sure it was 32 CPU and 64GB RAM." Note: INSERT APPEND supports only the subquery syntax of the INSERT statement. the import speed is far greater using the newer Data Pump import utility (impdp) and imports run faster of faster servers. “The target box was an AIX 5L using LPARS. all at-once.Oracle import speed performance tips http://www. Also indexes will rebuild cleaner. insert /*+ append */ into customer (select xxx). so the data pump export/import was a matter of divide and conquer (i..e. Mike Ault notes in his book "Oracle Solid-State Disk Tuning".dba-oracle. Using SSD for insert tablespaces For databases that require high-speed loads. you ensure that Oracle always grabs "fresh" data blocks by raising the high-water-mark for the table. consider putting the table into NOLOGGING mode. an import load rate of over 75 gigabytes per hour. the Append mode is the default and you don't need to specify an APPEND hint. If the Oracle level processing for insert and update activities could be optimized for SSD. Mark Bobak notes "Also.I This 30% was due to the CPU overhead involved in the insert and update activities.e.” As we see.Parallelize the load . removing the benefit of parallel jobstreams. and some indexes were skipped/re-built later.e.Disable/drop indexes .com/oracle_tips_load_speed." Import Speed Benchmarks: Oracle guru Steve Callan notes that he has run parallel Data Pump import jobs to load 1. For this INSERT optimization. There were several distinct schemas. you reduce I/O because more rows fit onto a block before a "block full" condition (as set by PCTFREE) unlinks the block from the freelist. d . If you are doing parallel insert DML.You can invoke parallel DML (i. Also see: Optimizing Oracle INSERT SQL performance fast Oracle data load import sql*loader sqlldr data pump loading Oracle Undocumented Utilities Oracle9i RMAN Backup & Recovery OCP Preparation Class Improving OLAP Performance in Oracle Database 10g Release 1 Oracle performance collection using vmstat High Speed UNIX Oracle export import Turning the Tables on Disk I/O with Oracle Optimize Oracle import (data pump impdp) performance 4 de 7 28/09/2011 22:55 .By using the APPEND hint. e .8 terabytes (1. using the APPEND hint may cause serialization.APPEND into tables . 32k) blocksizes for the target table. significant performance gains might be realized during these activities. and with less I/O if they reside in a tablespace with a large block size. but that time was separate from the actual import.By defining large (i. Statistics gathering took a while. if you're going w/ APPEND. using the PARALLEL hint) to have multiple inserts into the same table.htm b . and the data pump version was about a third of that time.Use a large blocksize . stream several jobs/sessions). some shops define the insert table partition on solid-state disk (later moving it to platter disk). An old import version of this lasted right around three days. I think the largest chunk was around 900GB. a respectable 30% improvement in load speed: “In the SSD verses ATA benchmark the gains for insert and update processing as shown in the database loading and index build scenarios was a respectable 30%. make sure to define multiple freelists or freelist groups.800 gigabytes) in less than a day. c .

320.000 random IOPS 1. 1. 3 microseconds latency 391MB/s random sustained read or write per port (full duplex is 719MB/s). Most SSD vendors will provide a test RAM disk array for benchmark testing so that you can choose the vendor who offers the best price/performance ratio. This book includes scripts and tools to hypercharge Oracle 11g performance and you can buy it for 30% off directly from the publisher. many of the major hardware vendors (including Sun and EMC) are replacing slow disks with RAM-based disks.dba-oracle. Market Survey of SSD vendors for Oracle: There are many vendors who offer rack-mount solid-state disk that work with Oracle databases." As of June 2008. with 8 x 4Gbps FC ports aggregated throughput is approx 2. and Sun announced that all of their large servers will offer SSD. SearchStorage notes that SSD is will soon replace platter disks and that hundreds of SSD vendors may enter the market: "The number of vendors in this category could rise to several hundred in the next 3 years as enterprise users become more familiar with the benefits of this type of storage. As of June 2008. 5 de 7 28/09/2011 22:55 . here are the major SSD vendors for Oracle databases (vendors are listed alphabetically): Advanced Media Inc. EMC software MTron Pliant Technology SeaChange Solid Access Technologies Sun Microsystems Texas Memory Systems 2008 rack mount SSD Performance Statistics SearchStorage has done a comprehensive survey of rack mount SSD vendors.000 IOPS Violin Memory Violin 1010 RAM SSD PCIe Solid Access Technologies USSD 200FC RAM SSD Fibre Channel SAS SCSI Curtis HyperXCLR R1000 RAM SSD Fibre Channel Choosing the right SSD for Oracle When evaluating SSD for Oracle databases you need to consider performance (throughput and response time).000 IOPS 197MB/s sustained R/W transfer rate.htm Get the Complete Oracle Utility Information The landmark book "Advanced Oracle Utilities The Definitive Reference" contains over 600 pages of filled with valuable information on Oracle's secret utilities.000MB/s random sustained external throughput. and lists these SSD rack mount vendors. 35. 2008): manufacturer Texas Memory Systems model RamSan-400 technology interface RAM SSD Fibre Channel InfiniBand performance metrics and notes 3.400MB/s read. and the competitive market ensures that product offerings will continuously improve while prices fall.00MB/s write with ×4 with this showing the fastest rack-mount SSD devices (as of May 15. 400.Oracle import speed performance tips http://www. reliability (Mean Time Between failures) and TCO (total cost of ownership).000MB/s.

but after some usage the average read time becomes far slower than a hard drive. You can buy it direct from the publisher for 30%-off and get instant access to the code depot of Oracle tuning scripts.4 ms.0 ms. Burleson is the American Team 6 de 7 28/09/2011 22:55 . over 900 pages of BC's favorite tuning tips & scripts. The flash type SSD suffers from serious shortcomings. DRAM SSD vs.0 ms.Oracle import speed performance tips http://www.: for SSD support details. For Oracle. 1. it is important to note that there are two types of SSD.5 ms. you may enjoy the new book "Oracle Tuning: The Definitive Reference". namely a degradation of access speed over time. write speed BC was one of the earliest adopters of SSD for Oracle and we have been deploying SSD on Oracle database since 2005 and we have experienced SSD experts to help any Oracle shop evaluate whether SSD is right for your application. only rack-mounted DRAM SSD is acceptable for good performance: Avg. Just call 800-766-1884 or e-mail. BC experts can also help you choose the SSD that is best for your database.7 ms.htm Burleson Consulting does not partner with any SSD vendors and we provide independent advice in this constantly-changing market. 0. Flash SSD With all the talk about the Oracle “flash cache”. 94. Platter disk DRAM SSD Flash SSD If you like Oracle tuning. and only DRAM SSD is suitable for Oracle database storage.dba-oracle. Read speed 10.4 ms. 0. Avg. Flash SSD is 5 times faster than a platter disk. At first.

dba-oracle. Errata? Oracle technology is changing and we strive to update our BC Oracle support information. Verify experience! Anyone considering using the services of an Oracle support expert should independently investigate their credentials and experience. Oracle © is the registered trademark of Oracle Corporation. If you find an error or have a suggestion for improving our content.Oracle import speed performance tips http://www.2011 by Burleson Enterprises All rights All legitimate Oracle experts publish their Oracle qualifications. we would appreciate your feedback. Just e-mail: and include the URL for the page. Feel free to ask questions on our Oracle forum. Burleson Consulting The Oracle of Database Support Oracle Performance Tuning Remote DBA Services Copyright © 1996 . and not rely on advertisements and self-proclaimed expertise.htm Note: This Oracle documentation was created as a support and Oracle training reference for use by our DBA performance tuning consulting professionals. 7 de 7 28/09/2011 22:55 .

Sign up to vote on this title
UsefulNot useful