PL/SQL Webinars for Oracle Education

Collect Yourself: Optimize PL/SQL Code with Collections
Steven Feuerstein
PL/SQL Evangelist, Quest Software

Copyright 2000-2009 Steven Feuerstein - Page 1

How to benefit most from this session
 Watch, listen, focus on concepts and principles.  Download and use any of my the training materials:

PL/SQL Obsession

 Download and use any of my scripts (examples, performance scripts, reusable code) from the same location: the file. filename_from_demo_zip.sql  You have my permission to use all these materials to do internal trainings and build your own applications.
– But remember: they are not production ready. – Modify them to fit your needs and then test them!
Copyright 2000-2006 Steven Feuerstein - Page 2

What we will cover on collections
     Review of "foundation" features Indexing collections by strings Working with collections of collections MULTISET operators for nested tables Then we will apply collections (overview):
– Data caching – Bulk processing with FORALL and BULK COLLECT – Table functions and pipelined functions
Copyright 2000-2008 Steven Feuerstein - Page 3

What is a collection?
1 abc 2 def 3 sf 4 q


22 rrr

23 swq

 A collection is an "ordered group of elements, all of the same type." (PL/SQL User Guide and Reference)
– That's a very general definition; from collections, you can build queues, stacks, lists, sets, arrays. – Collections are single-dimensional and homogeneous, but you can emulate multi-dimensional structures.

 Collections are a critical feature in many of the newest and most important features of PL/SQL.
– Yet they are greatly underutilized by PL/SQL developers.
Copyright 2000-2006 Steven Feuerstein - Page 4

 Dramatically improve multi-row querying.Why use collections?  Generally. Copyright 2000-2006 Steven Feuerstein . inserting. random access cursors. Combined with BULK COLLECT and FORALL.. updating and deleting the contents of tables.Page 5 ..  Avoid mutating table trigger errors. to manipulate in-program-memory lists of information..  Serve up complex datasets of information to nonPL/SQL host environments using table functions. – Much faster than working through SQL.  Emulate bi-directional.

– Required for some features.Three Types of Collections  Associative arrays (aka index-by tables) – Can be used only in PL/SQL blocks. such as table functions – With Varrays. – Part of the object model in PL/SQL. allows you to access elements via arbitrary subscript values. Copyright 2000-2006 Steven Feuerstein . – Similar to hash tables in other languages. but also can be the datatype of a column in a relational table.Page 6 . you specify a maximum number of elements in the collection.  Nested tables and Varrays – Can be used in PL/SQL blocks. at time of definition.

147. – This range allows you to employ the row number as an intelligent key.483.About Associative Arrays  Unbounded.sql .sql Copyright 2000-2006 Steven Feuerstein .647. practically speaking.483. as is required in traditional 3GL arrays and VARRAYs.647 to 2. such as the primary key or unique index value.  Index values can be integers or strings (Oracle9i R2 and above). because AAs also are:  Sparse – Data does not have to be stored in consecutive rows.Page 7 collection_of_records. assoc_array_example.147. – Valid row numbers range from -2.

647.Page 8 nested_table_example. – Valid row numbers range from 1 to 2.  Can be defined as a schema level type and used as a relational table column type.483.  Part of object model. Copyright 2000-2006 Steven Feuerstein .  Is always dense initially. requiring initialization.147. but can become sparse after deletes.sql .About Nested Tables  No pre-defined limit on a nested table.

varray_example.  Is always dense. you can only remove elements from the end of a varray.Page 9 .About Varrays  Has a maximum size. requiring initialization.sql Copyright 2000-2006 Steven Feuerstein .  Can be defined as a schema level type and used as a relational table column type. – Can adjust the size at runtime in Oracle10g R2.  Part of object model. associated with its type.

. – If you need to specify a maximum size to your collection – Access the collection inside SQL (table functions.How to choose your collection type  Use associative arrays when you need to. columns in tables) – Want or need to perform high level set operations  Use varrays when you need to..Page 10 . columns in tables)..... – Work within PL/SQL code only – Sparsely fill and manipulate the collection – Take advantage of negative index values and string indexing  Use nested tables when you need to. Copyright 2000-2006 Steven Feuerstein . – Access the collection inside SQL (table functions.

– NEXT/PRIOR return the closest defined row after/before the specified row. number of elements allowed in a VARRAY.Page 11 .  Modify the contents of the collection – DELETE deletes one or more rows from collection. – FIRST/LAST return lowest/highest numbers of defined rows. – EXISTS returns TRUE if the specified row is defined. – EXTEND adds rows to a nested table or VARRAY. – TRIM removes rows from a VARRAY. – LIMIT tells you the max.Handy Collection Methods  Obtain information about the collection – COUNT returns number of rows currently defined in collection. Copyright 2000-2006 Steven Feuerstein .

 Use the NOCOPY hint to reduce overhead of passing collections in and out of program units. Think about how you need to manipulate the contents.  Try to read a row that doesn't exist. nocopy*.  Don't always fill collections sequentially.Page 12 .Useful reminders for PL/SQL collections  Memory for collections comes out of the PGA (Process Global Area) or UGA (User Global Area) – One per session. so a program using collections can consume a large amount of memory.* Copyright 2000-2006 Steven Feuerstein .  Encapsulate or hide details of collection management. and Oracle raises NO_DATA_FOUND.

Session 1 memory (PGA/UGA) Copyright 2000-2006 Steven Feuerstein .Page 13 Session 2 memory (PGA/UGA) Session 2 plsql_memory*. Large Pool calc_totals show_emps upd_salaries Session 1 emp_rec emp%rowtype.. tot_tab tottabtype.* ..PL/SQL in Shared Memory System Global Area (SGA) of RDBMS Instance Shared Pool Shared SQL Reserved Pool Pre-parsed Select * from emp Library cache Update emp Set sal=. emp_rec emp%rowtype. tot_tab tottabtype.

– From the PL/SQL perspective.sql Copyright 2000-2006 Steven Feuerstein .How PL/SQL uses the SGA. Static_Constant CONSTANT PLS_INTEGER := 42.  The User Global Area contains session-specific data that persists across server call boundaries – Package-level data  The Process Global Area contains session-specific data that is released when the current server call terminates.pkg plsql_memory_demo. PACKAGE Pkg is Nonstatic_Constant CONSTANT PLS_INTEGER := My_Sequence. – Local data plsql_memory. PGA and UGA  The SGA contains information that can be shared across schemas connected to the instance. END Pkg.Nextval. this is limited to package static constants.Page 14 .

 You can now define the index on your associative array to be: – Any sub-type derived from BINARY_INTEGER – VARCHAR2(n).Expanded indexing capabilities for associative arrays  Prior to Oracle9iR2..) Copyright 2000-2006 Steven Feuerstein . where n is between 1 and 32767 – %TYPE against a database column that is consistent with the above rules – A SUBTYPE against any of the above..  This means that you can now index on string values! (and concatenated indexes and. you could only index by BINARY_INTEGER.Page 15 .

INDEX BY VARCHAR2(64). because INTEGER is not a subtype of BINARY_INTEGER.Examples of New TYPE Variants  All of the following are now valid TYPE declarations in Oracle9i Release 2 – You cannot use %TYPE against an INTEGER column. INDEX BY NATURAL. DECLARE TYPE TYPE TYPE TYPE TYPE TYPE TYPE INDEX BY BINARY_INTEGER. array_t1 array_t2 array_t3 array_t4 array_t5 array_t6 array_t7 IS IS IS IS IS IS IS TABLE TABLE TABLE TABLE TABLE TABLE TABLE OF OF OF OF OF OF OF NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER NUMBER Copyright 2000-2006 Steven Feuerstein .last_name%TYPE. TYPE array_t8 IS TABLE OF NUMBER INDEX BY types_pkg. INDEX BY VARCHAR2(32767).subtype_t.Page 16 . INDEX BY PLS_INTEGER. INDEX BY employee. INDEX BY POSITIVE.

Copyright 2000-2006 Steven Feuerstein .sql .Page 17 assoc_array_perf. but you should keep this in mind: – The datatype returned by FIRST. NEXT and PRIOR methods is VARCHAR2. convert to a string assoc_array*. – The longer the string values.1 or -2**31 + 1). LAST.tst int_to_string_indexing.Working with string-indexed collections  The syntax is exactly the same.sql index.sql genaa.  If you are indexing by integer and find that your values are getting close to the limits (2**31 . the more time it takes Oracle to "hash" that string to the integer that is actually used as the index value.

Copyright 2000-2006 Steven Feuerstein .DO NOTHING ELSE add_variable_declaration. l_variables.  There are lots of ways to do this.COUNT LOOP If varname_already_used THEN -.* .Practical application for string indexing  I need to keep track of names used in my program. – Specifically. END IF. but string-indexed collections make it really easy! FOR indx IN 1 . So I need to make sure that I do not declare the same variable more than once. we generate test code and declare variables. END LOOP.Page 18 Without string indexing: string_tracker0. in Quest Code Tester.. mark_varname_as_used.

FUNCTION string_in_use ( value_in IN maxvarchar2_t ) RETURN BOOLEAN IS BEGIN RETURN g_names_used.EXISTS ( value_in ). CREATE OR REPLACE PACKAGE BODY string_tracker IS TYPE used_aat IS TABLE OF BOOLEAN INDEX BY maxvarchar2_t. END string_tracker. END string_in_use. PROCEDURE mark_as_used (value_in IN maxvarchar2_t) IS BEGIN g_names_used ( value_in ) := TRUE.* Copyright 2000-2006 Steven Feuerstein . g_names_used used_aat.The String Tracker package (V1)  First iteration: I only need to maintain one list of names. END mark_as_used. string_tracker1.Page 19 .

Oracle9i Multi-level Collections  Prior to Oracle9i. you could have collections of records or objects.  Now you can create collections that contain other collections and complex types. but only if all fields were scalars.Page 20 . – A collection containing another collection was not allowed. – Applies to all three types of collections.  The syntax is non-intuitive and resulting code can be quite complex. Copyright 2000-2006 Steven Feuerstein .

.Page 21 .. – What if I need to track multiple lists simultaneously or nested?  Let's extend the first version to support multiple lists by using a string-indexed. multilevel collection. Copyright 2000-2006 Steven Feuerstein ..String Tracker Version 2  The problem with String Tracker V1 is that it only supports a single list of strings. – A list of lists.

TYPE list_of_lists_aat IS TABLE OF used_aat INDEX BY maxvarchar2_t. PROCEDURE mark_as_used ( list_in IN maxvarchar2_t . END mark_as_used.Page 22 .The String Tracker package (V2) CREATE OR REPLACE PACKAGE BODY string_tracker IS TYPE used_aat IS TABLE OF BOOLEAN INDEX BY maxvarchar2_t. value_in IN maxvarchar2_t . g_list_of_lists list_of_lists_aat. END string_tracker.* Copyright 2000-2006 Steven Feuerstein . BEGIN g_list_of_lists ( list_in ) ( l_name) := TRUE. case_sensitive_in IN BOOLEAN DEFAULT FALSE ) IS l_name maxvarchar2_t := CASE case_sensitive_in WHEN TRUE THEN value_in ELSE UPPER ( value_in ) END. string_tracker3.

sql OTN: OverloadCheck Copyright 2000-2006 Steven Feuerstein .Page 23 .Other multi-level collection examples  Multi-level collections with intermediate records and objects. – Use the UTL_NLA package (10gR2) for complex matrix manipulation. multilevel_collections. ambig_overloading.*  Four-level nested collection used to track arguments for a program unit. multdim*.sql  Emulation of multi-dimensional arrays – No native support. – Automatically analyze ambiguous overloading. but can creates nested collections to get much the same effect.

add_new_parameter Copyright 2000-2006 Steven Feuerstein .Page 24 . cc_smartargs. – Work with and through functions to retrieve contents and procedures to set contents.and all data structures -.next_overloading cc_smartargs.behind small modules.pkb: cc_smartargs.  What' s a developer to do? – Hide complexity -.Encapsulate these complex structures!  When working with multi-level collections. you can easily and rapidly arrive at completely unreadable and un-maintainable code.

records. – Nested tables are “multisets.Oracle10g Nested Tables unveil their MULTISET-edness  Oracle10g introduces high-level set operations on nested tables (only). objects.” meaning that there is no inherent order to their elements and duplicates are significant. Copyright 2000-2008 Steven Feuerstein .Page 25 .  You can now… – Check for equality and inequality – Perform UNION. INTERSECT and MINUS operations – Check for and remove duplicates  Works with nested tables of scalars.

put_line ('Group 2 != Group 3'). END IF. END. DECLARE TYPE clientele IS TABLE OF VARCHAR2 (64).put_line ('Group 1 != Group 2'). Copyright 2000-2008 Steven Feuerstein . group2 clientele := clientele ('Customer 1'. group3 clientele := clientele ('Customer 3'. 'Customer 2'). BEGIN IF group1 = group2 THEN DBMS_OUTPUT.put_line ('Group 1 = Group 2').sql 10g_compare_old.and NULLs have the usual disruptive impact.Page 26 10g_compare.put_line ('Group 2 = Group 3'). END IF. IF group2 != group3 THEN DBMS_OUTPUT.Oracle10g Check for equality and inequality  Just use the basic operators…. ELSE DBMS_OUTPUT. ELSE DBMS_OUTPUT. 'Customer 3').sql 10g_compare2. 'Customer 1').sql . group1 clientele := clientele ('Customer 1'.

sql 10g*union*. SQL: UNION ALL SQL: UNION SQL: INTERSECT SQL: MINUS Copyright 2000-2008 Steven Feuerstein .Page 27 10g_setops. MINUS  Straightforward. END. INTERSECT.sql 10g_favorites.Oracle10g UNION. BEGIN our_favorites := my_favorites MULTISET UNION dad_favorites. with the MULTISET keyword.sql . our_favorites := my_favorites MULTISET UNION DISTINCT dad_favorites. our_favorites := my_favorites MULTISET INTERSECT dad_favorites.sql 10g_string_nt. our_favorites := dad_favorites MULTISET EXCEPT my_favorites.

my_favorites IS NOT A SET. p. and determine if you have a set of distinct values. p.show_favorites ('FULL SET'. 'Keep_it_simple distinct?').l (favorites_pkg.pkg Copyright 2000-2008 Steven Feuerstein .l (keep_it_simple IS A SET. p. 'Keep_it_simple NOT distinct?'). p.l (favorites_pkg. 10g_set. 'My favorites NOT distinct?').sql 10g_favorites. favorites_pkg.my_favorites). DECLARE keep_it_simple strings_nt := strings_nt (). favorites_pkg.l (keep_it_simple IS NOT A SET. favorites_pkg.show_favorites ( 'DISTINCT SET'. 'My favorites distinct?'). BEGIN keep_it_simple := SET (favorites_pkg. keep_it_simple).Oracle10g Distinct sets of values  Use the SET operator to work with distinct values.Page 28 .my_favorites IS A SET.my_favorites). END.

Collections vs. but they will use more memory. – GTTs consume SGA memory. which is their main advantage over collections. Global Temporary Tables  Global temporary tables cut down on the overhead of working with persistent tables. – And you can use the full power of SQL. global_temp_tab_vs_coll.Page 29 .  GTTs still require interaction with the SGA.sql Copyright 2000-2008 Steven Feuerstein .  So collections will still be faster.

simply to ensure that you know what is possible.Page 30 . Copyright 2000-2006 Steven Feuerstein .Applying Collections  Data caching using packaged data  Turbo-charged SQL with BULK COLLECT and FORALL  Table functions  I offer light coverage of these topics.

– Oracle11g Function Result Cache – Deterministic functions Page 31 . but it is not always the most efficient means. – Package data structures: PGA memory has less access overhead than SGA.  Options for caching data: – The SGA: Oracle does lots of caching for us.Data Caching Options  Why cache data? – Because it is static and therefore you want to avoid the performance overhead of retrieving that data over and over again.

– Usually a collection.* .  Why query information from the database (SGA) if that data does not change during your session? – Trivial example: the USER function – More interesting: static tables  Instead. load it up in a package variable! Page 32 Very simple example: thisuser. – It persists for the entire session.Packaged collection caching  Prior to Oracle 11g. to store multiple rows of data. the best caching option for PL/SQL programs involves declaring a package-level data structure.

pkg emplu.Page 33 emplu.tst 11g_emplu*. Data retrieved from cache Data returned to application Database Application PGA Function Application Requests Data Copyright 2000-2006 Steven Feuerstein .Data Caching with PL/SQL Tables First access Database Not in cache.* . Database is not needed. Request data from database Pass Data to Cache Data retrieved from cache Data returned to application Application PGA Function Application Requests Data Subsequent accesses Data found in cache.

employee_id. BEGIN FOR rec IN emp_cur LOOP adjust_compensation (rec. newsal_in).hire_date FROM employee WHERE department_id = dept_in.department_id%TYPE .newsal_in IN employee. END LOOP.salary%TYPE) IS CURSOR emp_cur IS SELECT employee_id. UPDATE employee SET salary = rec.Page 34 Row by row processing: elegant but inefficient .Turbo-charge SQL with bulk processing statements  Improve the performance of multi-row SQL operations by an order of magnitude or more with bulk/array processing in PL/SQL! CREATE OR REPLACE PROCEDURE upd_for_dept ( dept_in IN employee.salary.salary WHERE employee_id = rec. END upd_for_dept. Copyright 2000-2006 Steven Feuerstein .

list_of_emps.) IS BEGIN FORALL indx IN list_of_emps..Page 35 bulk_rowcount.FIRST . – Use SAVE EXCEPTIONS to continue past errors.LAST UPDATE employee SET salary = newsal_in WHERE employee_id = list_of_emps (indx).sql Copyright 2000-2006 Steven Feuerstein .Use the FORALL Bulk Bind Statement  Instead of executing repetitive. SQL%BULK_EXCEPTIONS...sql . – New cursor attributes: SQL%BULK_ROWCOUNT returns number of rows affected by each row in array.  Things to be aware of with FORALL: – You MUST know how to use collections to use this feature! – Only a single DML statement is allowed per FORALL. – Prior to Oracle10g. individual DML statements.. bulktiming. END. you can write your code like this: PROCEDURE upd_for_dept (..sql bulkexc. the binding array must be sequentially filled.

FOR indx IN 1 . Always check contents of collection to confirm that something was retrieved. starting with 1. Declare a collection of records to hold the queried data..COUNT LOOP process_employee (l_employees(indx)).sql WARNING! BULK COLLECT will not raise NO_DATA_FOUND if no rows are found. Fetch all rows into collection sequentially. Copyright 2000-2006 Steven Feuerstein . bulkcoll. .Use BULK COLLECT INTO for Queries DECLARE TYPE employees_aat IS TABLE OF employees%ROWTYPE INDEX BY BINARY_INTEGER. BEGIN SELECT * BULK COLLECT INTO l_employees FROM employees.Page 36 Iterate through the collection contents with a loop. END. END LOOP. l_employees employees_aat. l_employees.

LOOP FETCH emps_in_dept_cur BULK COLLECT INTO emps LIMIT 100. bulklimit.sql . Use the LIMIT clause with the INTO to manage the amount of memory used with the BULK COLLECT operation. END LOOP.Page 37 WARNING! BULK COLLECT will not raise NO_DATA_FOUND if no rows are found. process_emps (emps). TYPE emp_tt IS TABLE OF emps_in_dept_cur%ROWTYPE.Limit the number of rows returned by BULK COLLECT CREATE OR REPLACE PROCEDURE bulk_with_limit (deptno_in IN dept. BEGIN OPEN emps_in_dept_cur.deptno%TYPE) IS CURSOR emps_in_dept_cur IS SELECT * FROM emp WHERE deptno = deptno_in. Best to check contents of collection to confirm that something was retrieved.COUNT = 0. emps emp_tt. Copyright 2000-2006 Steven Feuerstein . EXIT WHEN emps. END bulk_with_limit.

and have it be treated as if it were a relational table. – Not everything can be done in SQL. you can now more easily transfer data from within PL/SQL to host environments.  Combined with REF CURSORs. works very smoothly with cursor variables Copyright 2000-2006 Steven Feuerstein .The Wonder Of Table Functions  A table function is a function that you can call in the FROM clause of a query.  Table functions allow you to perform arbitrarily complex transformations of data and then make that data available through a query. for example.Page 38 . – Java.

sql .. count_in IN INTEGER ) RETURN names_nt IS retval names_nt := names_nt ().EXTEND (count_in). 100)) names. END LOOP. and then call that function in the FROM clause. CREATE OR REPLACE FUNCTION lotsa_names ( base_name_in IN VARCHAR2.Page 39 SELECT column_value FROM TABLE ( lotsa_names ('Steven' . Steven 100 tabfunc_scalar. BEGIN retval. count_in LOOP retval (indx) := base_name_in || ' ' || indx. FOR indx IN 1 . COLUMN_VALUE -----------Steven 1 .Simple table function example  Return a list of names as a nested table... Copyright 2000-2006 Steven Feuerstein . RETURN retval. END lotsa_names.

trade_date DATE. – Example: transform one row in the stocktable to two rows in the tickertable. CREATE TABLE stocktable ( ticker VARCHAR2(20).sql Copyright 2000-2006 Steven Feuerstein .Page 40 . pricedate DATE. pricetype VARCHAR2(1). price NUMBER) / tabfunc_streaming. close_price NUMBER ) / CREATE TABLE tickertable ( ticker VARCHAR2(20).Streaming data with table functions  You can use table functions to "stream" data through several stages within a single SQL statement. open_price NUMBER.

transform each row of the stocktable into two rows in the tickertable. CREATE OR REPLACE PACKAGE refcur_pkg IS TYPE refcur_t IS REF CURSOR RETURN stocktable%ROWTYPE. / Copyright 2000-2006 Steven Feuerstein ..Streaming data with table functions . BEGIN INSERT INTO tickertable SELECT * FROM TABLE (stockpivot (CURSOR (SELECT * FROM stocktable)))..refcur_t) RETURN tickertypeset . END.2  In this example. END refcur_pkg.sql .Page 41 tabfunc_streaming. / CREATE OR REPLACE FUNCTION stockpivot (dataset refcur_pkg.

 Pipelined functions can be defined to support parallel execution.Page 42 . Copyright 2000-2006 Steven Feuerstein . asynchronous to termination of the function. CREATE FUNCTION StockPivot (p refcur_pkg. it is passed back to the calling process/query. – Iterative data processing allows multiple processes to work on that data simultaneously.Use pipelined functions to enhance performance. – As data is produced within the function.refcur_t) RETURN TickerTypeSet PIPELINED  Pipelined functions allow you to return data iteratively.

– Use a pipelined function to "serve up" data to the webpage and allow users to being viewing and browsing. use the PARALLEL_ENABLE clause to allow your pipelined function to participate fully in a parallelized query.  Improve speed of delivery of data to web pages. – Critical in data warehouse applications. – In Oracle9i Database Release 2 and above.Applications for pipelined functions  Execution functions in parallel. Copyright 2000-2006 Steven Feuerstein .Page 43 . even before the function has finished retrieving all of the data.

out_rec.nothing at all! Copyright 2000-2006 Steven Feuerstein . NULL).. CLOSE p.ticker. NULL. END LOOP.price := in_rec. EXIT WHEN p%NOTFOUND. tabfunc_setup.sql Pipe a row of data back to calling block or query RETURN.ticker := in_rec.openprice. END.Page 44 . PIPE ROW (out_rec).sql tabfunc_pipelined.refcur_t) RETURN tickertypeset PIPELINED IS out_rec tickertype := tickertype (NULL. out_rec. out_rec. in_rec p%ROWTYPE. RETURN.Piping rows out from a pipelined function Add PIPELINED keyword to header CREATE FUNCTION stockpivot (p refcur_pkg.pricetype := 'O'. BEGIN LOOP FETCH p INTO in_rec..

– Want to call a user defined function inside a query and execute it as part of a parallel query. – Need to pass back complex result sets of data through the SQL layer (a query)..  Consider using them when you..Table functions – Summary  Table functions offer significant new flexibility for PL/SQL developers.Page 45 . Copyright 2000-2006 Steven Feuerstein .

taking full advantage of new features. – Your code will get faster and in many cases much simpler than it might have been (though not always!). – From array processing to table functions. unless you use collections.  Today I offer this challenge: learn collections thoroughly and apply them throughout your backend code.Collections – don't start coding without them. Copyright 2000-2009 Steven Feuerstein .  It is impossible to write efficient. high quality PL/SQL code. collections are required.Page 46 . (Class id : 2347771)  Course Evaluation link https://eval.Some Useful 47 .com  Instructor Feedback link : Copyright 2000-2009 Steven Feuerstein .   E-Attendance link http://education.