DATABASE

Management Systems
Lecture 5
Algorithms for Query Processing and
Optimization
- Prerequisites

Refer to Fundamentals of Database System by Elmasri & Navathe

4-5th chapter 15 or 6th chapter 18
Performance and Optimization: Prerequisites

• Database Architecture
• Working with External Data
– Error Handling
– Loading Data
– External Table

2
 Oracle Database Architecture

• An Oracle database
server consists of a
database and at least
one database
instance
• Database:
– Physical files
• Instance
– Process and Memory
Management
3
https://docs.oracle.com/en/database/oracle/oracle-database/21/cncpt/introduction-to-oracle-database.html
 Multitenant Architecture
• The multitenant architecture enables
an Oracle database to be a CDB
• Every Oracle database must contain or
be able to be contained by another
database. 
• CDBs
– contains one or more user-created PDBs
and application containers.
• PDBs
– a portable collection of schemas, schema
objects, and nonschema objects that
appears to an application as a separate
database.
• Application Containers
– An application container is an optional,
user-created container within a CDB that
stores data and metadata for one or more
applications.

4
 Sharding Architecture
• Oracle Sharding is a database
scaling technique based on
horizontal partitioning of data
across multiple PDBs. Applications
perceive the pool of PDBs as a
single logical database
• Benefit for OLTP:
– linear scalability
– fault containment
– geographical data distribution

5
 Oracle Database (1)
• A collection of data that is treated as a unit
• Consists of Data Files and System Files
• Datafiles: physical files that store the data
• Oracle uses the storage in logical structure

6
https://www.oracle.com/webfolder/technetwork/tutorials/architecture-diagrams/18/technical-architecture/database-technical-architecture.html
Oracle Database (2)
• System files are files that are used during the
operation on an Oracle Database
• Reside on database server
• Belong to the containers

7
 Oracle Instance

• A set of memory structures

that manage database files
• Always opens one and only
one database
• Consists of memory and
background structure
• When an instance is started,
Oracle Database allocates a
memory area called the 
system global area (SGA)
 and starts one or more 
background processes. 

8
Oracle Process
A process is a mechanism in an operating system
that can run a series of steps.
• Client Process:
– runs the application or Oracle tool code
• Oracle Process
– Server Process:
• Connects to the Oracle instance and is started
when a user establishes a session
– Background Process:
• Started when an Oracle instance is started

9
 Oracle Server Process

Dedicated Server Shared Server

10
 Oracle Background Process

• Process Monitor • Database Writer Process

Process (PMON) (DBW)
• Process Manager • Log Writer Process
(LGWR)
(PMAN)
• Checkpoint Process
• Listener Registration
(CKPT)
Process (LREG) • Manageability Monitor
• System Monitor Processes (MMON and
Process (SMON) MMNL)
• Recoverer Process
(RECO)
11
Database Writer (DBW)

• DBWn writes when:

– Checkpoint occurs
– Dirty buffers reach
threshold
– There are no free
buffers
– Timeout occurs

12
 Log Writer (LGWR)

• LGWR Writes when:

– A user commits a
transaction.
– An online redo log switch
 occurs.
– Three seconds have
passed since LGWR last
wrote.
– The redo log buffer is one-
third full or contains 1 MB
of buffered data.
– DBW must write modified
buffers to disk.
13
 Checkpoint (CKPT)

• Responsible for:
– Signaling DBWn at
checkpoints
– Updating datafile
headers with
checkpoint
information
– Updating control
files with checkpoint
information (SCN)

14
 Memory Structure
System Global Area (SGA)
• Allocated during instance
startup
• Shared by all server and
background process
• Fundamental for Oracle
Instance
Program Global Area (PGA)
• Allocated when the server
process is started
• Nonshared memory
region
• Fundamental for server
process only
15
System Global Area
• The SGA serves various purposes, including the
following:
– Maintaining internal data structures that are accessed by
many processes and threads concurrently
– Caching data blocks read from disk
– Buffering redo data before writing it to the online redo log
files
– Storing SQL execution plans
• The SGA is shared by the Oracle processes,
which include server processes and background
processes

16
SGA components
• Database Buffer Cache
• Redo Log Buffer
• Shared Pool
• Large Pool
• Java Pool
• Fixed SGA

17
 Database Buffer Cache
• The memory area that stores
copies of data blocks read from
data files
• Enables great performance gains
when you obtain and update data
• Managed through an LRU
algorithm
• All users concurrently connected
to a database instance share
access to the buffer cache.
• Has 3 states:
– Unused
– Clean
– Dirty

18
 Redo Log Buffer

• The redo log buffer is a

circular buffer in the SGA
that stores redo entries
describing changes made
to the database.
• A redo record is a data
structure that contains the
information necessary to
reconstruct, or redo,
changes made to the
database by DML or DDL
operations.
19
 Shared Pool

• Caches various types of

program data.
– Most recently executed SQL
statements
– Most recently used data
definitions
• For example, the shared
pool stores parsed SQL,
PL/SQL code, system
parameters, and 
data dictionary information.
• The shared pool is divided
into several subcomponents
20
Shared Pool Example

21
Oracle Architecture - Summary
• Oracle Server components
– Database
• Collection of physical files to store the data
• Use Logical structure to use the storage
• Consists of 4 common types:
– Permanent
– Temporary
– Undo
– System
– Instance
• Memory Management
– Shared (SGA) & Private (PGA)
• Process Management
– Client Process and Oracle Process (Server & background)

22
Working with External Data
• External data is data originated from other
resources
• External data access:
– Read externally
– Imported / Loaded internally
• Problems might occur during the reading
process
– Different format
– Low quality of data
– Error

23
External Data Error and Handling (1)
• Data Type
– ‘2’  is this Number of String?

• Data Format
– ’Aug 09 18 09:34:55 PM’  is this a standard format?

• Incompatible Format
– ’Aug 09 18 09:34:55.00 PM’  Unrecognized format
– $95.76  Number that is stored as a text

24
External Data Error and Handling (2)
• Missing Value
• Column Identifier
• Special Characters

25
External Data Error and Handling (3)
• Some preprocessing steps:
– Replace incorrect characters
– Replace the column identifier
– Reformat the data
– Fill the missing values

• Preprocessing step is necessary to ensure

complete restoration

26
Loading External Data
• Temporarily
– Using External Table
• Permanently
– SQL*Loader
– Oracle Export/Import Utility

27
 External Table
• External tables are created using the SQL
CREATE TABLE...ORGANIZATION EXTERNAL
statement.
• Read only
• Can be used in normal SQL statement
• The following attributes must be configured:
– TYPE
– DEFAULT DIRECTORY
– ACCESS PARAMETERS
– LOCATION
https://docs.oracle.com/en/database/oracle/oracle-database/12.2/sutil/oracle-external-tables-concepts.html
28
 External Table Example
CREATE DIRECTORY mydir AS 'C:/dataset/dms/';
CREATE TABLE extemp
( employeeid NUMBER(3),
firstname VARCHAR2(30),
lastname VARCHAR2(30),
email VARCHAR2(50),
phone VARCHAR2(20),
hire_date DATE
)
ORGANIZATION EXTERNAL
(TYPE ORACLE_LOADER
DEFAULT DIRECTORY mydir
ACCESS PARAMETERS
(RECORDS DELIMITED BY NEWLINE
SKIP 1
FIELDS TERMINATED BY ","
( employeeid CHAR(3),
firstname CHAR(30),
lastname CHAR(30),
email CHAR(50),
phone CHAR(20),
hire_date CHAR(10) date_format DATE mask "mm/dd/yyyy"
)
)
LOCATION ('extdata.csv') 29
)REJECT LIMIT 5;
External Table Example

30
 SQL*Loader

• SQL*Loader loads data

from external files into
tables of an Oracle
database.
• It has a powerful data
parsing engine that puts
little limitation on the
format of the data in the
data file
• You can use SQL*Loader
in two ways: with or
without a control file
31
https://docs.oracle.com/en/database/oracle/oracle-database/12.2/sutil/oracle-sql-loader-concepts.html
SQL*Loader Control File
• The control file is a text file written in a
language that SQL*Loader understands.
• The control file tells SQL*Loader where to
find the data, how to parse and interpret the
data, where to insert the data, and more.
• Three main sections:
– Session-wide information
– Table and field-list information
– Input data (optional section)

32
 SQL*Loader Control File Example

CREATE TABLE loademp -- Load from extdata.csv

( employeeid NUMBER(3), OPTIONS (SKIP=1)
LOAD DATA
firstname VARCHAR2(30),
INFILE 'extdata.csv'
lastname VARCHAR2(30),
BADFILE 'ext.bad'
email VARCHAR2(50), DISCARDFILE 'ext.dsc'
phone VARCHAR2(20), -- INSERT APPEND REPLACE TRUNCATE
hire_date DATE REPLACE INTO TABLE loademp
); FIELDS TERMINATED BY ","
TRAILING NULLCOLS
( employeeid,
firstname ,
lastname,
email,
phone,
hire_date DATE "MM/DD/YYYY"
)

33
SQL*Loader Example

34
SQL Developer - Import Utility

• Built-in GUI tool to export/import data to

existing table
• Provides user-friendly interface and step-by-
step data configuration
• Support multiple insertion methods

35
Insertion methods (1)
• Insert
– For new tables, the table will be created
• Insert Script
– DML INSERT
• External Table
– DDL for External Table
• Staging External Table
– DDL for External Table with DML
• SQL*Loader
– Script for SQL*Loader

36
Data Import Wizard

37
38
39
40
41
Summary
• Oracle Architecture for performance
• Work with external data source
– Read Only
• External Table
– Permanent
• SQL*Loader
• Import utility

42