You are on page 1of 36

Linux best practice

for Oracle software

Galit Koka Elad
Agenda
• Why are we here?
• How Oracle works on Linux:
- Memory management
- Files management
- Network management
Why are we here?
• Since oracle like Linux can be then best
thing in the market – but it takes a good
DBA and a good Sys Admin to do it!
Oracle RDBMS
• Oracle RDBMS is bases on 2 part:
- Instance: Memory and background
processes
PMON
- Database: files Instance
SMON

DBWn
Database
• All is being managed archivexx LGWR
archivexx More..
by Oracle RDBMS archivexx
Memory management
• The instance is based on OS Shared
memory, taken/freed as soon as the
instance starts/stop.
• Fixed size (can’t be resize while inst is up)
• Should be contiguous, better bases on ram.
• Referred as SGA (system global area).
• Can be seen with OS command: ipcs
Cont: Memory management..
• The shared memory in Linux Kernel
parameters:
- Shmmax – max size (bytes) for a shared memory segment
allowed on the system.
- Shmmni – max number of shared memory segments
allowed on the system (4096)
- Shmall – max number of shared memory pages in total that
can be allocated at the system. System pages*
• These parameters are limited by the OS
limitation (32/64bit).
Cont: Memory management..
• In addition to the SGA, each instance has PGA
(process global area).
• This memory unit is dynamic and taken/freed as
needed by the RDBMS clients.
• DBA can limit this memory unit (each and total)
• Should be based on RAM. When no RAM
available, Oracle will use its own I/O
• Oracle uses semaphores to handle processes
- Semaphores: semmsl semmns semopm semmni
- See ipcs –ls
Files management
• The Oracle database is based on 3 different
files types:
- Datafiles (as many as RDBMS version allows)
- Control files (at least 2)
- Log files (at least 2)
• All are binaries.
• All can’t be “touched” while DB is up.
• DB backup means copy of those files*

* Make sure you plan the backup with your DBA
Cont: Files management..
• Only DBWR process writes from SGA to
the data files. Can be up to 20
• Only one LGWR process writes to the
redo file.
• Redo logs are being written in cyclic way
• All clients reads from data files
Cont: Files management..
• Oracle supply two types of file systems:
- CFS
- ASM (since 10g)

• Other relevant options:
- Ext3
- Raw device
- Other vendors (Netpps, Veritas etc)
- XFS for Suse (SGI)
Cont: Files management..
• Fs.file-max – limit of open files each
process can have (65536)
• /etc/security/limits.conf: nofile (65536) for
shell limits (ulimit –u)
• Async IO and Aio-max-size (131072 / 1MB)
• Block size in FS level (CFS/ASM etc).
• IO schedular: CFQ, noop, …
Network management
• DB server listen for clients request
• Each request gets its on connection (port)
• Range of port values
can’t be controlled.
• Set:
net.ipv4.ip_local_port_range
1024 - 65000
Cont: Network management..
• Check send/received buffer default and
max size: (/etc/sysctl.conf)
- net.core.rmem_default=262144
- net.core.wmem_default=262144
- net.core.rmem_max=524288 (or 262144)
- net.core.wmem_max= 524288 (or 262144)
Cont:Memory management..
For better performance, configure Oracle to use large
buffer cache and reduce SAN/NAS access:
• Use VLM (32bit) and hugepages (for both 32/64bit)
• For 32bit systems, mount an in-memory file system
shmfs with the needed size.
• 32bit: In DB level configure:
USE_INDIRECT_DATA_BUFFERS=TRUE
• VLM-32bit, better not to pass the 16gb limit.
Memory: Huge pages
• Eliminated page table lookup overhead
• Decreased page table overhead - Faster
overall memory performance
• Not swappable
• Should be contiguous (startup).
• Relevant for both 32bit and 64bit systems
Memory: Huge pages
• Requires kernel configuration for sizing
and permissions (for non root users).
• Huge page size (cat /proc/meminfo)
- Red hat X86: 4mb
- Suse x86: 2mb
- Itanium (red hat & Suse): 256mb
• Vm.nr_hugepases * huge page size.
Summary
• Oracle RDBMS upon Linux requires
adjustment in and beyond the RDBMS level.
• While doing so, there are many variables that
need to be taken into account:
- Application needs
- Linux Kernel
- Oracle version and features
- Hardware
We as the Sys Admins and the infrastructure
DBAs should be aware to them.
Thank you

Galit.koka@oracle.com
Linux best practice
for Oracle software

Galit Koka Elad
Agenda
• Why are we here?
• How Oracle works on Linux:
- Memory management
- Files management
- Network management
Why are we here?
• Since oracle like Linux can be then best
thing in the market – but it takes a good
DBA and a good Sys Admin to do it!
Oracle RDBMS
• Oracle RDBMS is bases on 2 part:
- Instance: Memory and background
processes
PMON
- Database: files Instance
SMON

DBWn
Database
• All is being managed archivexx LGWR
archivexx More..
by Oracle RDBMS archivexx
Memory management
• The instance is based on OS Shared
memory, taken/freed as soon as the
instance starts/stop.
• Fixed size (can’t be resize while inst is up)
• Should be contiguous, better bases on ram.
• Referred as SGA (system global area).
• Can be seen with OS command: ipcs
Cont: Memory management..
• The shared memory in Linux Kernel
parameters:
- Shmmax – max size (bytes) for a shared memory segment
allowed on the system.
- Shmmni – max number of shared memory segments
allowed on the system (4096)
- Shmall – max number of shared memory pages in total that
can be allocated at the system. System pages*
• These parameters are limited by the OS
limitation (32/64bit).

Shmall: depends on page size (4k for x86, 16k for Itanium) and the•
needed shmmax. Should be at least shmmax/system page size
Cont: Memory management..
• In addition to the SGA, each instance has PGA
(process global area).
• This memory unit is dynamic and taken/freed as
needed by the RDBMS clients.
• DBA can limit this memory unit (each and total)
• Should be based on RAM. When no RAM
available, Oracle will use its own I/O
• Oracle uses semaphores to handle processes
- Semaphores: semmsl semmns semopm semmni
- See ipcs –ls

Oracle uses I/O on temporary tablespace
Files management
• The Oracle database is based on 3 different
files types:
- Datafiles (as many as RDBMS version allows)
- Control files (at least 2)
- Log files (at least 2)
• All are binaries.
• All can’t be “touched” while DB is up.
• DB backup means copy of those files*

* Make sure you plan the backup with your DBA
Cont: Files management..
• Only DBWR process writes from SGA to
the data files. Can be up to 20
• Only one LGWR process writes to the
redo file.
• Redo logs are being written in cyclic way
• All clients reads from data files

Dbwr: default is 1 or cpu_count/8 – whichever is greater
Cont: Files management..
• Oracle supply two types of file systems:
- CFS
- ASM (since 10g)

• Other relevant options:
- Ext3
- Raw device
- Other vendors (Netpps, Veritas etc)
- XFS for Suse (SGI)

.Verify last point
Cont: Files management..
• Fs.file-max – limit of open files each
process can have (65536)
• /etc/security/limits.conf: nofile (65536) for
shell limits (ulimit –u)
• Async IO and Aio-max-size (131072 / 1MB)
• Block size in FS level (CFS/ASM etc).
• IO schedular: CFQ, noop, …

For Async, make sure to link Oracle with it as wel.131072 is for OLTP. 1MB
is for warehouse
.ASM usually 1mb. CFS between 4k to 4mb in 4k multiple
Network management
• DB server listen for clients request
• Each request gets its on connection (port)
• Range of port values
can’t be controlled.
• Set:
net.ipv4.ip_local_port_range
1024 - 65000

net.ipv4.ip_local_port_range – set the range of local ports for outgoing connections change
.32768-61000 to 1024-65000
Cont: Network management..
• Check send/received buffer default and
max size: (/etc/sysctl.conf)
- net.core.rmem_default=262144
- net.core.wmem_default=262144
- net.core.rmem_max=524288 (or 262144)
- net.core.wmem_max= 524288 (or 262144)

The receive buffers are used by TCP and UDP to hold received data
until it is read by the application. The receive buffer cannot overflow
because the peer is not allowed to send data beyond the buffer size
window. This means that datagrams will be discarded if they don't fit in
the socket receive buffer. This could cause the sender to overwhelm
.the receiver
Cont:Memory management..
For better performance, configure Oracle to use large
buffer cache and reduce SAN/NAS access:
• Use VLM (32bit) and hugepages (for both 32/64bit)
• For 32bit systems, mount an in-memory file system
shmfs with the needed size.
• 32bit: In DB level configure:
USE_INDIRECT_DATA_BUFFERS=TRUE
• VLM-32bit, better not to pass the 16gb limit.

In 11g, we will be able to save query results in memory – great
.advantage that requires memory
Memory: Huge pages
• Eliminated page table lookup overhead
• Decreased page table overhead - Faster
overall memory performance
• Not swappable
• Should be contiguous (startup).
• Relevant for both 32bit and 64bit systems

Access speed increases by improving the likelihood that the virtual
memory address of the pages that oracle requires is already cached
•Not swappable: HugePages are not swappable. Therefore there is no
page-in/page-out mechanism overhead.HugePages are universally
regarded as pinned.
•Decreased page table overhead: Each page table entry can be as
large as 64k and if we are trying to handle 50GBof RAM, the pagetable
will be approximately 800MB in size which is practically will not fit in
880MB size lowmem (in 2.4 kernels - the page table is not necessarily
in lowmem in 2.6 kernels) considering the other uses of lowmem. When
95% of memory is accessed via 256MB hugepages, this can work with
a page table of approximately 40MB in total. See also Note 361468.1.

•Eliminated page table lookup overhead: Since the pages are not
subject to replacement, page table lookups are not required.

•Faster overall memory performance: On virtual memory systems each
memory operation is actually two abstract memory operations. Since
there are less number of pages to work on, the possible bottleneck on
page table access is clearly avoided.
Memory: Huge pages
• Requires kernel configuration for sizing
and permissions (for non root users).
• Huge page size (cat /proc/meminfo)
- Red hat X86: 4mb
- Suse x86: 2mb
- Itanium (red hat & Suse): 256mb
• Vm.nr_hugepases * huge page size.
Summary
• Oracle RDBMS upon Linux requires
adjustment in and beyond the RDBMS level.
• While doing so, there are many variables that
need to be taken into account:
- Application needs
- Linux Kernel
- Oracle version and features
- Hardware
We as the Sys Admins and the infrastructure
DBAs should be aware to them.
Thank you

Galit.koka@oracle.com