PL/SQL Bulk Collections in Oracle 9i and 10g

Kent Crotty

Burleson Consulting
October 13, 2006

AGENDA

Performance gains with Bulk Processing Array processing with BULK COLLECT and FORALL Oracle 10g FORALL improvements. Error handling.

Top Books on PL/SQL

Dr. Tim Hall Oracle ACE And Oracle’s ACE of the Year

John Garmany A fantastic beginner book on PL/SQL

Bulk Processing • Supercharge your PL/SQL code with BULK COLLECT and FORALL • Working at a table-level instead of the row-level • Simple and easy to use .

while. if.PL/SQL Code • Consists of two types of statements Procedural (declare. for …) SQL (select. begin. delete) • Oracle has two engines to process that information PL/SQL Engine SQL Engine • A Content Switch occurs each time the PL/SQL engine needs to execute a SQL statement • Switches are fast but large loops can cause performance delays . update. insert.

Context Switches Oracle Server Session PL/SQL Block PL/SQL Block PL/SQL Engine PL/SQL Block Procedural Statement Executor Data SQL Statement Executor SQL Engine SQL .

END update_price.2) ) IS CURSOR products_cur IS SELECT product_id. product_price FROM products WHERE product_type = product_type_in.product_id. multiplier_in IN number(2. .product_type%TYPE.PL/SQL Code Consider this procedure code CREATE OR REPLACE PROCEDURE update_price ( product_type_in IN product. END LOOP. BEGIN FOR prod_rec IN products_cur LOOP UPDATE products SET product_price = product_price * multiplier_in WHERE product_id = prod_rec.

PL/SQL Code FOR prod_rec IN products_cur LOOP UPDATE products SET product_price = product_price * multiplier_in WHERE product_id = prod_rec. END LOOP. there is going to be a conventional bind and a context switch! • Overhead for these statements can be large • But there is a solution – Bulk Collections .product_id. • For each iteration of this loop.

UPDATE. DELETE .Bulk Collection Categories • SELECT or FETCH statements BULK COLLECT INTO • Out-Bind binding RETURNING clause • In-Bind binding FORALL – INSERT.

SELECT / FETCH statements Data may be Bulk Collected/Fetched into: Table.column%TYPE Record of arrays Table%ROWTYPE Cursor%ROWTYPE Array of records Nested tables .

Table created. Commit complete. SQL> commit. . 4 end. 100000 loop 2 insert into products values (i.inserting 100000 records into the products table 1 for i in 1 . 'PROD'||to_char(i). 5 / PL/SQL procedure successfully completed.. 3 product_name varchar2(15). SQL> begin -.Products Table SQL> create table products ( 2 product_id number. 3 end loop.sysdate-1). 4 effective_date date ).

BULK COLLECT clause • • • • Used in a SELECT statement Binds the result set of the query to a collection Much less communication between the PL/SQL and SQL engines All variables in the INTO clause must be a collection .

END LOOP. DBMS_OUTPUT.BULK COLLECT SET SERVEROUTPUT ON DECLARE TYPE prod_tab IS TABLE OF products%ROWTYPE. Start_time := DBMS_UTILITY. products_tab(products_tab.last) := prod_rec.2 AND TRUNC(sysdate)) LOOP products_tab.count||’): ’||to_char(end_time-start_time)).get_time. END. products_tab prod_tab := prod_tab(). end_time := DBMS_UTILITY. SELECT * BULK COLLECT INTO products_tab FROM products WHERE effective_date BETWEEN sysdate .get_time. FOR prod_rec in (SELECT * FROM products WHERE effective_date BETWEEN sysdate .PUT_LINE(‘Bulk Collect (‘||products_tab. end_time number.get_time.extend.2 AND TRUNC(sysdate). start_time number.PUT_LINE(‘Conventional (‘||products_tab. BEGIN start_time := DBMS_UTILITY. DBMS_OUTPUT.count||’): ’||to_char(end_time-start_time)).get_time. end_time := DBMS_UTILITY. .

BULK COLLECT SQL> / Conventional (100000): 40 Bulk Collect (100000): 27 PL/SQL procedure successfully completed. Bulk Collect is quite a bit faster! .

OPEN products_data.get_time. end_time := DBMS_UTILITY. END. CLOSE products_data. IF products_data%NOTFOUND THEN products_tab. DBMS_OUTPUT.Fetch DECLARE TYPE prod_tab IS TABLE OF products%ROWTYPE. CLOSE products_data.count||’): ’||to_char(end_time-start_time)). FETCH products_data INTO products_tab(products_tab. . LOOP products_tab. Start_time := DBMS_UTILITY. END LOOP. BEGIN start_time := DBMS_UTILITY. OPEN products_data.get_time.last).extend. products_tab prod_tab := prod_tab().get_time. end_time := DBMS_UTILITY.count||’): ’||to_char(end_time-start_time)).get_time. DBMS_OUTPUT.BULK COLLECT – Explicit Cursor. start_time number. end_time number.PUT_LINE(‘Bulk Collect (‘||products_tab. CURSOR products_data IS SELECT * FROM products. FETCH products_data BULK COLLECT INTO products_tab.delete(products_tab.last).PUT_LINE(‘Conventional (‘||products_tab. EXIT. END IF.

BULK COLLECT SQL> / Conventional (100000): 117 Bulk Collect (100000): 14 PL/SQL procedure successfully completed. Bulk Collect is significantly faster – over 8 times! .

BULK COLLECT . we can now process the result set in chunks • Explicit cursors must be used with the LIMIT clause .LIMIT • Collections are arrays held in memory – massive collections can eat up all the memory • By using the LIMIT clause.

PUT_LINE('Processed '||to_char(products_tab. BEGIN Start_time := DBMS_UTILITY. end_time := DBMS_UTILITY. CURSOR products_data IS SELECT * FROM products. Result Set is limited to only 10000 rows – more memory efficient! Will the processing be slower because of the LIMIT? .get_time.LIMIT DECLARE TYPE prod_tab IS TABLE OF products%ROWTYPE. DBMS_OUTPUT. CLOSE products_data.get_time. end. products_tab prod_tab := prod_tab(). start_time number. LOOP FETCH products_data BULK COLLECT INTO products_tab LIMIT 10000. DBMS_OUTPUT.PUT_LINE('Bulk Collect: ‘||to_char(end_time-start_time)).BULK COLLECT – Explicit Cursor. end_time number. EXIT WHEN products_data%NOTFOUND. END LOOP.count)||' rows'). OPEN products_data.

Still over 8 times better than conventional! .BULK COLLECT – Explicit Cursor.LIMIT SQL> / Processed 10000 rows Processed 10000 rows Processed 10000 rows Processed 10000 rows Processed 10000 rows Processed 10000 rows Processed 10000 rows Processed 10000 rows Processed 10000 rows Processed 10000 rows Bulk Collect: 15 PL/SQL procedure successfully completed. but only slightly. Yes.

count||' rows'). products_tab prod_tab := prod_tab().PUT_LINE('Deleted Product Ids: '|| products_tab. SQL> / Deleted Product Ids: 80000 rows PL/SQL procedure successfully completed.product_id%TYPE.RETURNING • The RETURNING clause can be used to return specific columns after a DML statement • This is referred to as OUT-BINDING DECLARE TYPE prod_tab IS TABLE OF products. .BULK COLLECT . DBMS_OUTPUT. end. BEGIN DELETE FROM products WHERE product_id > 20000 RETURNING product_id BULK COLLECT INTO products_tab.

BULK COLLECT Use Considerations • • Use the LIMIT clause to manage memory requirements NO_DATA_FOUND will not be raised if no records are returned – check contents to make sure records are retrieved .

FORALL • We have seen BULK COLLECT with the SELECT statements • For the INSERT.BULK COLLECT . UPDATE and DELETE statements there is the FORALL statement • This is referred to as IN-BINDING .

product_id.first . start_time number. END. EXECUTE IMMEDIATE 'TRUNCATE TABLE products'.BULK COLLECT – INSERT .get_time.get_time. products_tab prod_tab := prod_tab().product_name. DBMS_OUTPUT.last LOOP INSERT INTO products (product_id.get_time. product_name. products_tab. END LOOP. COMMIT. end_time := DBMS_UTILITY.effective_date). Start_time := DBMS_UTILITY. end_time := DBMS_UTILITY.last INSERT INTO products VALUES products_tab(i). effective_date) VALUES (products_tab(i). Start_time := DBMS_UTILITY. products_tab(i)..PUT_LINE(‘Bulk Insert: ’||to_char(end_time-start_time)).get_time. end_time number.Populate a collection .FORALL DECLARE TYPE prod_tab IS TABLE OF products%ROWTYPE. FOR i in products_tab. BEGIN -. EXECUTE IMMEDIATE 'TRUNCATE TABLE products'. DBMS_OUTPUT. FORALL i in products_tab.first .PUT_LINE(‘Conventional Insert: ’||to_char(end_time-start_time)).. products_tab. .100000 rows SELECT * BULK COLLECT INTO products_tab FROM products. products_tab(i).

BULK COLLECT – INSERT . The Bulk Operation is considerably faster! .FORALL SQL> / Conventional Insert: 686 Bulk Insert: 22 PL/SQL procedure successfully completed.

END.get_time. TYPE prod_tab IS TABLE OF products%ROWTYPE. .extend.first .first .2 WHERE product_id = products_id_tab(i). 10000 LOOP products_id_tab.10000 rows FOR i in 1 .. DBMS_OUTPUT.last).last) := i+10.Populate a collection .2 WHERE product_id = products_tab(i). FORALL i in products_tab. END LOOP. END LOOP. Start_time := DBMS_UTILITY. products_tab(products_tab. Start_time := DBMS_UTILITY.get_time.extend.last LOOP UPDATE products SET ROW = products_tab(i) -. start_time number. DBMS_OUTPUT.product_id := i.PUT_LINE('Bulk Update: '||to_char(end_time-start_time)).. products_tab. FOR i in products_tab. products_tab.get_time.product_id. end_time := DBMS_UTILITY. end_time number. products_id_tab prod_id_tab := prod_id_tab().get_time.FORALL DECLARE TYPE prod_id_tab is TABLE OF products. products_tab. end_time := DBMS_UTILITY.BULK COLLECT – UPDATE .product_id%TYPE. products_id_tab(products_id_tab. products_tab prod_tab := prod_tab()..PUT_LINE('Conventional Update: '||to_char(end_time-start_time)). BEGIN -.last UPDATE products SET ROW = products_tab(i) – ROW available in 9.ROW available in 9.

BULK COLLECT – UPDATE . The Bulk Operation is again considerably faster! .FORALL SQL> / Conventional Update: 301 Bulk Update: 116 PL/SQL procedure successfully completed.

FORALL Use • • • • Only a single DML statement is allowed per FORALL In 9i. the binding array must be sequentially filled Use SAVE EXCEPTIONS to continue past errors SQL%BULK_ROWCOUNT returns the number of affected rows .

• Bulk Collects • Can be used with implicit or explicit cursors • Collection is always filled sequentially starting with 1 .The Finer Points • Use bulk bind techniques for recurring SQL statements in a PL/SQL loop. Prior successful DML statements are not rolled back. statement is rolled back. • Bulk bind rules: • • • • Can be used with any type of collection Collection subscripts cannot be expressions Collections should be densely filled If error.

New in 10g – FORALL Improvements • • • FORALL driving array no longer needs to be processed in sequential order The INDICES OF clause is used to reference the row numbers defined in another array The VALUES OF clause is used to reference the values defined in another array .

. END. TYPE prod_tab IS TABLE OF products%ROWTYPE. products_tab(1000). products_tab(100). products_id_tab(10) := TRUE.New in 10g – FORALL .effective_date := sysdate.INDICES DECLARE TYPE prod_id_tab is TABLE OF BOOLEAN INDEX BY PLS_INTEGER.effective_date := sysdate + 100. products_id_tab prod_id_tab := prod_id_tab(). products_tab prod_tab := prod_tab(). FORALL i IN INDICES OF products_id_tab UPDATE products SET ROW = products_tab(i) WHERE product_id = products_id_tab(i). BEGIN products_tab(10). products_id_tab(1000) := TRUE. products_id_tab(100) := TRUE.effective_date := sysdate + 10.

FORALL i IN VALUES OF products_id_tab UPDATE products SET ROW = products_tab(i) WHERE product_id = products_id_tab(i). END.effective_date := sysdate. BEGIN products_tab(10). .effective_date := sysdate + 100. TYPE prod_tab IS TABLE OF products%ROWTYPE. products_tab(1000).effective_date := sysdate + 10.New in 10g – FORALL . products_id_tab(100) := 10. products_id_tab(200) := 100. products_tab prod_tab := prod_tab().VALUES DECLARE TYPE prod_id_tab is TABLE OF BOOLEAN INDEX BY PLS_INTEGER. products_id_tab prod_id_tab := prod_id_tab(). products_tab(100). products_id_tab(300) := 1000.

FORALL – Error Handling • No more FULL rollback in case of an EXCEPTION • SAVE EXCEPTIONS clause • SQL%BULK_EXCEPTIONS – Collection of records • SQL%BULK_EXCEPTIONS(i).ERROR_INDEX – stores iteration “i” when exception is raised • SQL%BULK_EXCEPTIONS(i).ERROR_CODE – stores the Oracle error code • SQL%BULK_EXCEPTIONS.count – returns the count of the exceptions .

Context Switches – Conventional Binds Oracle Server Session PL/SQL Block PL/SQL Block PL/SQL Engine PL/SQL Block Procedural Statement Executor Data SQL Statement Executor SQL Engine SQL .

Context Switches – Bulk Binds Oracle Server Session PL/SQL Block PL/SQL Block SQL PL/SQL Engine PL/SQL Block Procedural Statement Executor Data SQL Statement Executor SQL Engine .