Professional Documents
Culture Documents
SQL Loader Concepts
SQL Loader Concepts
SQL LOADER utility is used to load data from other data source into Oracle. For example, if
you have a table in FOXPRO, ACCESS or SYBASE or any other third party database, you can
use SQL Loader to load the data into Oracle Tables. SQL Loader will only read the data
from Flat files. So If you want to load the data from Foxpro or any other database, you
have to first convert that data into Delimited Format flat file or Fixed length format flat
file, and then use SQL loader to load the data into Oracle.
Following is procedure to load the data from Third Party Database into Oracle using SQL
Loader.
->Convert the Data into Flat file using third party database command.
->Create the Table Structure in Oracle Database using appropriate datatypes
->Write a Control File, describing how to interpret the flat file and options to load the
data.
->Execute SQL Loader utility specifying the control file in the command line argument
->To understand it better let us see the following case study.
---------------------------------------------------------------------------CASE STUDY (Loading Data from MS-ACCESS to Oracle):------------------------------------------------------------------------Suppose you have a table in MS-ACCESS by name EMP, running under Windows O/S,
with the following structure
EMPNO
NAME
SAL
JDATE
INTEGER
TEXT(50)
CURRENCY
DATE
This table contains some 10,000 rows. Now you want to load the data from this table
into an Oracle Table. Oracle Database is running in LINUX O/S.
Solution:-------------Steps:---------Start MS-Access and convert the table into comma delimited flat (popularly known as
csv) , by clicking on File/Save As menu. Let the delimited file name be emp.csv
1. Now transfer this file to Linux Server using FTP command
a. Go to Command Prompt in windows
b. At the command prompt type FTP followed by IP address of the server running Oracle.
FTP will then prompt you for username and password to connect to the Linux Server.
Supply a valid username and password of Oracle User in Linux
For example:-----------------C:\>ftp 200.200.100.111
Name: oracle
Password:oracle
FTP>
c. Now give PUT command to transfer file from current Windows machine to Linux
machine.
FTP>put
Local file:C:\>emp.csv
remote-file:/u01/oracle/emp.csv
File transferred in 0.29 Seconds
FTP>
d. Now after the file is transferred quit the FTP utility by typing bye command.
FTP>bye
Good-Bye
2. Now come the Linux Machine and create a table in Oracle with the same structure as
in MS-ACCESS by taking appropriate datatypes. For example, create a table like this
$sqlplus scott/tiger
SQL>CREATE TABLE emp (empno number(5),
name varchar2(50),
sal number(10,2),
jdate date);
3. After creating the table, you have to write a control file describing the actions which
SQL Loader should do. You can use any text editor to write the control file. Now let us
write a controlfile for our case study
$vi emp.ctl
1
LOAD DATA
INFILE
BADFILE
DISCARDFILE
/u01/oracle/emp.csv
/u01/oracle/emp.bad
/u01/oracle/emp.dsc
Notes:
----------(Do not write the line numbers, they are meant for explanation purpose)
1.
The LOAD DATA statement is required at the beginning of the control file.
2.
3.
Specifying BADFILE is optional. If you specify, then bad records found during
loading will be stored in this file.
4.
Specifying DISCARDFILE is optional. If you specify, then records which do not
meet a WHEN condition will be written to this file.
5.
1.
2.
3.
REPLACE: First deletes all the rows in the existing table and then, load rows.
4.
6.
This line indicates how the fields are separated in input file. Since in our case the
fields are separated by , so we have specified , as the terminating char for fields. You
can replace this by any char which is used to terminate fields. Some of the popularly use
terminating characters are semicolon ;, colon :, pipe | etc. TRAILING NULLCOLS
means if the last column is null then treat this as null value, otherwise, SQL LOADER will
treat the record as bad if the last column is null.
7.
In this line specify the columns of the target table. Note how do you specify
format for Date columns
4. After you have wrote the control file save it and then, call SQL Loader utility by typing
the following command
$sqlldr userid=scott/tiger control=emp.ctl log=emp.log
After you have executed the above command SQL Loader will shows you the output
describing how many rows it has loaded.
The LOG option of sqlldr specifies where the log file of this sql loader session should be
created. The log file contains all actions which SQL loader has performed i.e. how many
rows were loaded, how many were rejected and how much time is taken to load the rows
and etc. You have to view this file for any errors encountered while running SQL Loader.
CASE STUDY (Loading Data from Fixed Length file into Oracle):
----------------------------------------------------------------------------------Suppose we have a fixed length format file containing employees data, as shown below,
and wants to load this data into an Oracle table.
7782
7839
7934
7566
7499
7654
7658
7654
CLARK
KING
MILLER
JONES
ALLEN
MARTIN
CHAN
MARTIN
SOLUTION:
------------------Steps :-
-------------1.
First Open the file in a text editor and count the length of fields, for example in our
fixed length file, employee number is from 1st position to 4th position, employee name is
from 6th position to 15th position, Job name is from 17th position to 25th position.
Similarly other columns are also located.
2.
Create a table in Oracle, by any name, but should match columns specified in
fixed length file. In our case give the following command to create the table.
3.
NUMBER(3) );
After creating the table, now write a control file by using any text editor
$vi empfix.ctl
1)
LOAD DATA
2)
INFILE '/u01/oracle/fix.dat'
3)
4)
(empno
name
job
mgr
sal
5)
POSITION(01:04)
POSITION(06:15)
POSITION(17:25)
CHAR,
CHAR,
POSITION(27:30)
POSITION(32:39)
INTEGER EXTERNAL,
INTEGER EXTERNAL,
DECIMAL EXTERNAL,
comm
POSITION(41:48)
DECIMAL EXTERNAL,
deptno
POSITION(50:51)
INTEGER EXTERNAL)
Notes:
(Do not write the line numbers, they are meant for explanation purpose)
1.
The LOAD DATA statement is required at the beginning of the control file.
2.
The name of the file containing data follows the INFILE parameter.
3.
The INTO TABLE statement is required to identify the table to be loaded into.
4.
Lines 4 and 5 identify a column name and the location of the data in the datafile
to be loaded into that column. empno, name, job, and so on are names of columns in
table emp. The datatypes (INTEGER EXTERNAL, CHAR, DECIMAL EXTERNAL) identify the
datatype of data fields in the file, not of corresponding columns in the emp table.
5.
4.
After saving the control file now start SQL Loader utility by typing the following
command.
INTEGER EXTERNAL,
name
job
mgr
sal
POSITION(06:15)
POSITION(17:25)
CHAR,
CHAR,
POSITION(27:30)
POSITION(32:39)
INTEGER EXTERNAL,
DECIMAL EXTERNAL,
comm
POSITION(41:48)
DECIMAL EXTERNAL,
deptno
POSITION(50:51)
INTEGER EXTERNAL)
POSITION(06:15)
POSITION(17:25)
CHAR,
CHAR,
POSITION(27:30)
POSITION(32:39)
INTEGER EXTERNAL,
INTEGER EXTERNAL,
DECIMAL EXTERNAL,
comm
POSITION(41:48)
DECIMAL EXTERNAL,
deptno
POSITION(50:51)
INTEGER EXTERNAL)
i.e. it does not scan for free blocks before high water mark. Direct Path load is very fast
because
->Partial blocks are not used, so no reads are needed to find them, and fewer writes are
performed.
->SQL*Loader need not execute any SQL INSERT statements; therefore, the processing
load on the Oracle database is reduced.
->A direct path load calls on Oracle to lock tables and indexes at the start of the load
and releases them when the load is finished. A conventional path load calls Oracle once
for each array of rows to process a SQL INSERT statement.
->A direct path load uses multiblock asynchronous I/O for writes to the database files.
->During a direct path load, processes perform their own write I/O, instead of using
Oracle's buffer cache. This minimizes contention with other Oracle users.
Restrictions on Using Direct Path Loads:------------------------------------------------------The following conditions must be satisfied for you to use the direct path load method:
->Tables are not clustered.
->Tables to be loaded do not have any active transactions pending.
->Loading a parent table together with a child Table
->Loading BFILE columns