Document 1514705.1 https://support.oracle.com/epmos/faces/DocumentDisplay?_...

Copyright (c) 2020, Oracle. All rights reserved. Oracle Confidential.

Linux System Memory Utilization (Doc ID 1514705.1)

In this Document

Purpose
Questions and Answers
Q1. How to read 'free' command output.
A1. How to read 'free' command output.
A1.1 From the perspective of Operating System
A1.2 From the perspective of applications
A1.3 More accurate
Q2. How to forcefully free the cached memory.
Q3. How the cached memory improve the system performance.
Q4. How to get the memory usage by a specific process.
Q5. When hugepage is enabled.
References

APPLIES TO:

Linux OS - Version Oracle Linux 5.0 and later

Oracle Cloud Infrastructure - Version N/A and later
Information in this document applies to any platform.

PURPOSE

Some Linux user may be confused by the Linux memory utilize: the memory used by application and the memory reported by 'free'
command does not match.

QUESTIONS AND ANSWERS

Q1. How to read 'free' command output.

[root@localhost /]# free -tm
total used free shared buffers cached
Mem: 7880 6934 945 0 170 3631
-/+ buffers/cache: 3133 4747
Swap: 8191 0 8191
Total: 16072 6934 9137

Why this server have only 945MB memory left, which process consume so much memory?

A1. How to read 'free' command output.

We can see 5 lines from the 'free' command output, for memory usage, we concerned the second and third lines.

A1.1 From the perspective of Operating System

The second line was the memory utilize from the Operating System's perspective:

7880MB Total memory

6934MB Used memory, which included the buffers & cached.

170MB Memory used for buffers, which was relatively temporary storage
for raw disk blocks, shouldn't get tremendously large.

3631MB Memory used for cached, which was In-memory cache for files
read from the disk (the pagecache, as well as dentry cache, and
inode cache)

The buffer/cache will increase the I/O performance, they can be returned to OS once any application need more memory.

第1页 共4页 2020/8/18 11:32

Document 1514705.1 https://support.oracle.com/epmos/faces/DocumentDisplay?_...

Note: some recently used memory usually don't return to OS unless absolutely necessary:

[root@localhost /]# cat /proc/meminfo |grep Active:

Active: 2900528 kB

A1.2 From the perspective of applications

After excluding the 'buffers' and 'cached' from the 'used' memory, we get the memory utilization from the perspective of applications:

The free memory was:

free + buffers + cached:

945 + 170 + 3631 = 4747MB

The used memory was:

used - buffers - cached:

6934 - 170 - 3631 = 3133MB

That's the third line we see from the output:

3133MB used, and 4747MB free.

A1.3 More accurate

Though it's possible to free the cached memory (see Q2), but kernel would not reclaim the caches for recently used files, unless it's
necessary.
Freeing caches for recently used files may result in performance degradation (see Q3).

There's 2 fields in /proc/meminfo:

Active: the amount of buffer and cache memory the has been recently used, which is not usually reclaimed
Inactive: the amount of buffer and cache memory the has not been recently used, can be reclaimed by kernel

# cat /proc/meminfo
MemTotal: 8069496 kB
MemFree: 6369944 kB
Buffers: 341724 kB
Cached: 414800 kB
SwapCached: 0 kB
Active: 839852 kB
Inactive: 503996 kB
[SNIP]

So you may consider free memory as MemFree + Inactive for a system with good performance, while free + buffers + cached is the
bottom line.

Q2. How to forcefully free the cached memory.

Forcibly clearing caches should not be done unless specifically directed by Oracle Linux Support. This switch is a debugging/benchmarking feature, not supported for production use.
It can cause serious problems in production systems, including whole system hang or kernel crash. Cache memory is automatically freed as it is needed for other tasks, so forcing a
drop is never necessary in a normal environment.

However, if after it has been determined that a specific case does need the cache to be forcibly freed, follow the instructions below:

1) Flush the file system buffers

# sync; sync; sync

2) Drop the cached memory

According to Linux kernel document: writing to /proc/sys/vm/drop_caches will cause the kernel to drop clean caches, dentries and
inodes from memory, causing that memory to become free.

To free pagecache:

# echo 1 > /proc/sys/vm/drop_caches

To free dentries and inodes:

# echo 2 > /proc/sys/vm/drop_caches

To free pagecache, dentries and inodes:

第2页 共4页 2020/8/18 11:32

Document 1514705.1 https://support.oracle.com/epmos/faces/DocumentDisplay?_...

# echo 3 > /proc/sys/vm/drop_caches

Q3. How the cached memory improve the system performance.

Here's an example will show the difference.

The initial state:

[root@localhost /]$ free -m

total used free shared buffers cached
Mem: 7880 6427 1452 0 438 3321
-/+ buffers/cache: 2667 5213
Swap: 8191 0 8191

Create a new file of 200MB, we can also see the cached increased by 200MB, exactly the same size of the newly create file.

[root@localhost /]$ dd if=/dev/zero of=/tmp/test.img bs=1M count=200

200+0 records in
200+0 records out
209715200 bytes (210 MB) copied, 0.12154 s, 1.7 GB/s
[root@localhost /]$ free -m
total used free shared buffers cached
Mem: 7880 6612 1267 0 438 3512
-/+ buffers/cache: 2661 5219
Swap: 8191 0 8191

With the file cached, do a I/O test on it.

[root@localhost /]$ time cat /tmp/test.img >/dev/null

real 0m0.102s
user 0m0.000s
sys 0m0.038s

Drop the cached memory, so the file was not cached anymore.

[root@localhost /]# echo 3 > /proc/sys/vm/drop_caches

[root@localhost /]# free -m
total used free shared buffers cached
Mem: 7880 2843 5036 0 1 311
-/+ buffers/cache: 2529 5350
Swap: 8191 0 8191

Redo the test, the result was about 20 times slower than the cached.

[root@localhost /]# time cat /tmp/bigfile > /dev/null

real 0m2.016s
user 0m0.001s
sys 0m0.128s

Q4. How to get the memory usage by a specific process.

There isn't an accurate way to get the memory consumption for a specific process:

1) Linux would not load the entire resource (VSZ) into memory, but only part of them are residential on the physical memory (RSS).

For example, the Virtual memory size required by bash was 108500KB, but only 1864KB are residential on memory.

[root@localhost /]# ps aux | head -1; ps aux | grep 16222

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 16222 0.0 0.0 108500 1864 pts/13 S 11:33 0:00 -bash

2) Many libraries are shared between processes, they only consume 1 copy of the memory for many different processes.

For example, in the following output, ld-2.12.so, libc-2.12.so, libdl-2.12.so, and libnss_files-2.12.so are shared libraries, which may be
used by other processes as well.

[root@localhost /]# pmap -x 16222

16222: -bash
Address Kbytes RSS Dirty Mode Mapping
0000000000400000 848 572 0 r-x-- bash

第3页 共4页 2020/8/18 11:32

Document 1514705.1 https://support.oracle.com/epmos/faces/DocumentDisplay?_...

00000000006d3000 40 40 40 rw--- bash

00000000006dd000 20 20 20 rw--- [ anon ]
00000000008dc000 36 8 0 rw--- bash
00000000008e5000 380 252 252 rw--- [ anon ]
000000367d200000 128 108 0 r-x-- ld-2.12.so
000000367d41f000 4 4 4 r---- ld-2.12.so
000000367d420000 4 4 4 rw--- ld-2.12.so
000000367d421000 4 4 4 rw--- [ anon ]
000000367d600000 1572 568 0 r-x-- libc-2.12.so
000000367d789000 2048 0 0 ----- libc-2.12.so
000000367d989000 16 16 4 r---- libc-2.12.so
000000367d98d000 4 4 4 rw--- libc-2.12.so
000000367d98e000 20 16 16 rw--- [ anon ]
000000367de00000 8 8 0 r-x-- libdl-2.12.so
000000367de02000 2048 0 0 ----- libdl-2.12.so
000000367e002000 4 4 4 r---- libdl-2.12.so
000000367e003000 4 4 4 rw--- libdl-2.12.so
000000368e600000 116 60 0 r-x-- libtinfo.so.5.7
000000368e61d000 2048 0 0 ----- libtinfo.so.5.7
000000368e81d000 16 12 4 rw--- libtinfo.so.5.7
00007ffff1f40000 96836 60 0 r---- locale-archive
00007ffff7dd1000 48 24 0 r-x-- libnss_files-2.12.so
00007ffff7ddd000 2048 0 0 ----- libnss_files-2.12.so
00007ffff7fdd000 4 4 4 r---- libnss_files-2.12.so
00007ffff7fde000 4 4 4 rw--- libnss_files-2.12.so
00007ffff7fdf000 12 12 12 rw--- [ anon ]
00007ffff7ff4000 8 8 8 rw--- [ anon ]
00007ffff7ff6000 28 20 0 r--s- gconv-modules.cache
00007ffff7ffd000 4 4 4 rw--- [ anon ]
00007ffff7ffe000 4 4 0 r-x-- [ anon ]
00007ffffffde000 132 20 20 rw--- [ stack ]
ffffffffff600000 4 0 0 r-x-- [ anon ]
---------------- ------ ------ ------
total kB 108500 1864 412

Q5. When hugepage is enabled.

Please see the following note for hugepage:

HugePages on Linux: What It Is... and What It Is Not... (Doc ID 361323.1)

When hugepage is enabled, kernel reserved a specified amount memory for hugepage.
Only the applications support hugepage such as Oracle Database with proper configuration are able to make use of hugepage.
To verify the hugepage utilization:

# cat /proc/meminfo
[SNIP]
HugePages_Total: 1000
HugePages_Free: 600
HugePages_Rsvd: 400
HugePages_Surp: 0
Hugepagesize: 2048 kB
[SNIP]

In the above example, kernel reserved 1000 * 2048kB (HugePages_Total * Hugepagesize) for hugepage.
Part of the hugepage were unused and wasted, the amounts equal to (HugePages_Free - HugePages_Rsvd) * Hugepagesize,
That's (600 - 400) * 2048kb wasted.

It's very important not to waste hugepage, since most applications don't support it.

For Oracle Database, please follow this note for proper hugepage configuration.

Shell Script to Calculate Values Recommended Linux HugePages / HugeTLB Configuration (Doc ID 401749.1)

REFERENCES

https://www.kernel.org/doc/Documentation/sysctl/vm.txt
NOTE:1502301.1 - How to Check Whether a System is Under Memory Pressure
NOTE:1630754.1 - How to Calculate Memory Usage on Linux
Didn't find what you are looking for?

第4页 共4页 2020/8/18 11:32