Infrastructure Management

White Paper
ServiceCenter Shared Memory
White Paper

Shared Memory in ServiceCenter: Setup,

Explanations and Scalability

November 3, 2004

© Copyright 2004 Peregrine
Systems, Inc.
White Paper
PLEASE READ THE
FOLLOWING MESSAGE
CAREFULLY BEFORE
INSTALLING AND USING
THIS PRODUCT. THIS
PRODUCT IS COPYRIGHTED
PROPRIETARY MATERIAL
OF PEREGRINE SYSTEMS,
INC. (“PEREGRINE”). YOU
ACKNOWLEDGE AND
AGREE THAT YOUR USE OF
THIS PRODUCT IS SUBJECT
TO THE SOFTWARE
LICENSE AGREEMENT
BETWEEN YOU AND
PEREGRINE. BY
INSTALLING OR USING THIS
PRODUCT, YOU INDICATE
ACCEPTANCE OF AND
AGREE TO BE BOUND BY
THE TERMS AND
CONDITIONS OF THE
SOFTWARE LICENSE
AGREEMENT BETWEEN
YOU AND PEREGRINE. ANY
INSTALLATION, USE,
REPRODUCTION OR
MODIFICATION OF THIS
PRODUCT IN VIOLATION OF
THE TERMS OF THE
SOFTWARE LICENSE
AGREEMENT BETWEEN
YOU AND PEREGRINE IS
EXPRESSLY PROHIBITED.
Information contained in this
document is proprietary to
Peregrine Systems, Incorporated,
and may be used or disclosed
only with written permission
Infrastructure Management
from Peregrine Systems, Inc.
This book, or any part thereof,
may not be reproduced without
the prior written permission of
Peregrine Systems, Inc. This
document refers to numerous
products by their trade names. In
most, if not all, cases these
designations are claimed as
Trademarks or Registered
Trademarks by their respective
companies.
Peregrine Systems, AssetCenter,
AssetCenter Web, BI Portal,
Dashboard, Get-It, Peregrine
Mobile, and ServiceCenter are
registered trademarks of
Peregrine Systems, Inc. or its
subsidiaries.
Microsoft, Windows, Windows
NT, Windows 2000, SQL Server,
and names of other Microsoft
products referenced herein are
trademarks or registered
trademarks of Microsoft
Corporation. Oracle is a
registered trademark of Oracle
Corporation. zlib software
copyright © 1995-2004 Jean-
loup Gailly and Mark Adler
ServiceCenter includes
cryptographic software written
by Eric A. Young
(eay@cryptsoft.com).
ServiceCenter includes software
written by Tim J. Hudson
(tjh@cryptsoft.com). Spell
Checker copyright © 1995-2004
Wintertree Software Inc. Smack
software copyright © Jive
Software, 1998-2004. SVG

Viewer , Mozilla JavaScript-C
(SpiderMonkey), and Rhino
software Copyright © 1998-2004
White Paper
The Mozilla Organization.
OpenSSL software copyright
1998-2004 The OpenSSL
Project. All rights reserved.
MX4J software copyright ©
2001-2004 MX4J Team. All
rights reserved. JFreeChart
software © 2000-2004, Object
Refinery Limited. All rights
reserved. JDOM software
copyright © 2000 Brett
McLaughlin, Jason Hunter. All
rights reserved. LDAP,
OpenLDAP, Java, and the
Netscape Directory SDK
Copyright © 1995-2004 Sun
Microsystems, Inc. Japanese
Morphological Analyzer ©2004
Basis Technology Corp. The
Sentry Spelling-Checker Engine
Copyright © 2000 Wintertree
Software Inc. CoolMenu
software copyright © 2001
Thomas Brattli. All rights
reserved. Coroutine Software for
Java owned by Neva Object
Technology, Inc. and is protected
by US and international
copyright law. Crystal Reports
Pro and Crystal RTE software ©
2001 Crystal Decisions, Inc., All
rights reserved. Eclipse software
© Copyright 2000, 2004 IBM
Corporation and others. All
rights reserved. Copyright 2001-
2004 Kiran Kaja and Robert A.
van Engelen, Genivia Inc. All
rights reserved. Xtree copyright
2004 Emil A. Eklund. This
product includes software
Infrastructure Management
developed by the Indiana
University Extreme! Lab
(<http://www.extreme.indiana.ed
u/>).
The information in this document
is subject to change without
notice and does not represent a
commitment on the part of
Peregrine Systems, Inc. Contact
Peregrine Systems, Inc.,
Customer Support to verify the
date of the latest version of this
document. The names of
companies and individuals used
in the sample database and in
examples in the manuals are
fictitious and are intended to
illustrate the use of the software.
Any resemblance to actual
companies or individuals,
whether past or present, is purely
coincidental. If you need
technical support for this
product, or would like to request
documentation for a product for
which you are licensed, contact
Peregrine Systems, Inc.
Customer Support by email at
support@peregrine.com. If you
have comments or suggestions
about this documentation, contact
Peregrine Systems, Inc.
Technical Publications by email
at
doc_comments@peregrine.com.
This edition of the document
applies to version 6.0 of the
licensed program.
 Infrastructure Management
White Paper

Statement of the problem .............Error!

Bookmark not defined.
Introduction.......... Error! Bookmark not
defined.
Detailed introduction to the problem or
challenge..Error! Bookmark not defined.
Introduction.......................................... 5
Creation and Use of Shared Memory.. 5
Explanation of the various shared
memory types ........................................ 7
Calculating shared memory needs ...... 8
Shared memory reports........................ 9
IRExpert and Shared Memory .......... 13
Troubleshooting ................................. 14
 Infrastructure Management

Introduction
Shared memory can be divided into 3 areas:
White Paper

• User Blocks and user-related memory

• Memory related to Information Retrieval (IR) information
• Memory for caching system information

The amount of shared memory that ServiceCenter allocates upon startup is

based on the shared_memory parameter in the sc.ini file in the
ServiceCenter RUN directory. The ServiceCenter shared memory is then held
in virtual memory.

Creation and Use of Shared Memory

As its name implies Shared Memory is an area of storage that is shared by all
ServiceCenter processes on the server. For example, whenever a process
executes a RAD application, a check is performed to see if the application is
loaded into shared memory. If it is the process will execute the shared memory
version of the application and if not, the application will be retrieved from the
database and then loaded into shared memory. This behavior of sharing certain
data, like applications, dbdicts and formats between all users significantly
increases performance, as reading data from memory is faster than from disk.
This decreases the overall memory usage, as one copy of the application,
dbdict or format is shared between all users.

The first user / process that signs in to ServiceCenter creates the shared
memory in the size that is defined in the sc.ini (referenced in the sc.log file:
Creating Resources for system '12670' with key 0x6dd8a600). Every user /
process that logs in afterwards attaches to the shared memory that was created
for ServiceCenter (referenced in the sc.log file: Attaching to resources for
system '12670' with key 0x6dd8a600). SCDBUTIL will create or attach to
shared memory just like any other process. However some of the reports, like –
reportsem, -reportshm and –reportlic, do not create shared memory.

The following scenario shows what happens in shared memory as the first user
logs into ServiceCenter. The first thing the process determines is that the login
application needs to be executed. The process will check shared memory, find
that the login application is not present and retrieve the login application from
the database, loading it into shared memory. In reality each application panel
is loaded into shared memory as it gets executed, however, conceptually it is
easier to assume the entire application is loaded at once. The process will then
begin to execute the login application. It will determine that it needs to display
 Infrastructure Management

the login.prompt format. It will check shared memory for the format, find that
it is not present and retrieve the login.prompt format from the database and
load it into shared memory. Once the format is displayed and the user enters
their Name and Password the application will attempt to validate this
information against the operator file. The process will check shared memory
White Paper

for the operator dbdict, find that it is not present, retrieve it from the database
and load it into shared memory. This process will continue for all new
applications, formats and dbdicts encountered as the process continues
execution. The first user in many cases is responsible for loading most of the
contents of shared memory. Each subsequent user then benefits from this
work.

code dbdict
Shared
Login
Memory Operator

Format Login.
prompt

SC Database
SC Server
Code
Format Client
DBDict logging
in

The fact that the first user to use a format, dbdict etc. will load it into shared
memory explains that some steps take longer when executed for the very first
time. This information is held in shared memory even after all users have
logged out since it is assumed that they will eventually log back in and need
this information.

The only time this information is freed is when the underlying data object had
been modified. For example if a RAD application has changed, the copy of
code held in shared memory will be freed, as soon as the last user using this
application has logged out. Until then, the “old” and the “new” versions are
held in shared memory.
 Infrastructure Management

Customers have mentioned that we never free our shared memory. In a way,
this is true. Once the shared memory is created with the size specified in the
sc.ini, it will never be released back to the operating system until
ServiceCenter is terminated. As previously shown we do not routinely free
White Paper

applications, formats and dbdicts from shared memory. Over time as new
applications, formats and dbdicts are referenced the percentage of shared
memory in use will continue to grow.

To obtain a list of all things that are held in shared memory, you may run the
reportshm report from the ServiceCenter RUN directory. An example of the
output of this report can be found in the ServiceCenter Technical Reference
Guide.

Explanation of the various shared memory types

• User Blocks (every process has a user block of ~512 bytes, that
contains information like userid, device (e.g. system, express, windows
32…), license information, system monitor information, etc.

• Messages: Every message sent to a user is put into this section of

shared memory

• Resource locks, for example for a locked Problem Ticket that someone
edits, are held in the shared memory. Whenever the ‘show locks’
option in the system status is selected, the information on existing locks
is taken from the shared memory.

• Database Services: Amongst others information written to the

sc.alert.log is cached here

• Cache overhead, dbdicts cache, application cache: These areas hold

cached information for dbdicts, applications, operators etc.

• On systems that are converted to an RDBMS, the SQL descriptor cache

holds information necessary for the mapping etc.

• JOIN/ERD/TYPE cache: Formats, Links, Categories etc are held in this

part of the cache.

• Information for SCD systems is held in the Remote DBDICT cache and
the Remote Record Cache.
 Infrastructure Management

• String Type Cache

• IR Expert Cache: In IR Expert cache we hold both the term lists and
doc lists of IR terms used.
White Paper

• Publish and Subscribe: Area used to cache marquee and chart

information

• The 24x7 cache contains information about hot backups

(startlogging/stoplogging).

• The web cache is for web clients.

The per-user shared memory requirement is 512 bytes for the user block, plus
small amounts for locks and other overhead tasks. Total shared memory
required can be estimated on a per-user basis. It can only be estimated, and not
calculated with certainty, because the caching area of shared memory is not
directly related to the number of users in the system. Total shared memory
depends on the number of modules used, the number of formats, dbdict
records, links, formatcontrol, and code records accessed by the user base. The
estimated amount of shared memory per user is higher for a small user base,
because the cached information in the system is usually relatively static and not
dependent on the number of processes in the system.

Calculating shared memory needs

A conservative calculation of required shared memory uses 1 MB per process
for deployments with up to 1000 users, and ½ MB per process for deployments
with over 1000 users.

If Information Retrieval (IR) is used, the IR section of shared memory should

aggregate the size of all IR files that are used.

The default size allocated for IR is 30% of total shared memory. If this amount
does not match the recommended size of shared memory for IR, use the
ir_max_shared:<size_in_bytes> parameter to set shared memory
for IR.

Use the scenter –reportshm periodically to check the amount of free

space. Peregrine recommends never letting the amount of free space fall below
25%. If the shared memory report indicates that free space is always above
 Infrastructure Management

75%, you can change the shared_memory parameter in the sc.ini file to
a lower number. Run the scenter –reportlic report with the shared
memory report to get accurate information about the number of processes
running.
White Paperr

Licensed Active Inactive

Active Users: 25 0 0
Limited Users: 0

Note: Adjustments to the shared_memory parameter can be made only

when the ServiceCenter server is stopped.

These formulas offer a good baseline for most customers’ systems, though it
may be necessary to adjust the amount of shared memory. To obtain a very
accurate measurement of shared memory consumption, use the scenter –
reportshm command, which can be run from the RUN directory on your
ServiceCenter server. This scenter –reportshm command presents the
following data regarding current shared memory allocation:

Shared memory reports

C:\scs\sc601\RUN>scenter -reportshm
11/02/04 12:55:16 pid (3456) ServiceCenter
diagnostic report follows:

------ Shared Memory ------

Shared Memory Release 6.0

Current Size 32000000

Segment Allocation 393848

Large Block Allocation 247552

Unused Space 31358600 (97%)

Free Space 31585584 (98%)

Shared Memory Type Allocations Frees

Allocated
------------------ ----------- ----------- ---------
--
Not named 6 0
2384
 Infrastructure Management

User blocks 2 0
528
Messages 0 0
0
Resource locks 129 128
White Paper

64
Database Services 23 0
3968
Cache overhead 3 0
10272
Application cache 1 0
16
DBDICT cache 66 0
261024
SQL descriptor cache 0 0
0
Join/ERD/Type cache 340 0
44608
Remote DBDICT cache 0 0
0
Remote Record cache 0 0
0
String Type cache 28 0
2736
IR Expert cache 368 0
88256
Publish/Subscribe 2 0
48
24x7 cache 1 0
512
Web cache 0 0
0

Current Size is the amount of memory in bytes defined by the

shared_memory parameter in the sc.ini file.

Free Space is the amount of shared memory that is available for use by
processes. Free Space is created as blocks of unused space are allocated for
use. Free Space is shown as total bytes and as an overall percentage.

Unused Space represents shared memory that has not yet been allocated.
When ServiceCenter starts, the entire amount defined by the
shared_memory parameter is allocated, and additional blocks are allocated
as needed. Unused Space is shown in total bytes and as an overall percentage.
 Infrastructure Management

If you add up all the space that is allocated by type, it does not correlate with
the amount of unused space in the report. This is because the Unused space
indicates that amount of shared storage that has never been used and is
available for the next shared storage request no matter the size of the request.
White Paper

Therefore from the above report, a shared memory request for 4437504 bytes
could be satisfied. However, a request for a smaller amount (such as 16 bytes)
might not (and probably would not) be allocated from this area of unused
space. In addition to unused space there is also FREE space within shared
memory and a request for a small amount of memory would probably be
satisfied from this FREE space.

C:\scs\sc601\RUN>scenter –reportshm:1
11/02/04 12:55:50 pid (2288) ServiceCenter
diagnostic report follows:

------ Shared Memory ------

Shared Memory Release 6.0

Current Size 32000000

Segment Allocation 393848

Large Block Allocation 247552

Unused Space 31358600 (97%)

Free Space 31585584 (98%)

Shared Memory Type Allocations Frees

Allocated
------------------ ----------- ----------- ---------
--
Not named 6 0
2384
User blocks 2 0
528
Messages 0 0
0
Resource locks 129 128
64
Database Services 23 0
3968
 Infrastructure Management

Cache overhead 3 0
10272
Application cache 1 0
16
DBDICT cache 66 0
White Paper

261024
SQL descriptor cache 0 0
0
Join/ERD/Type cache 340 0
44608
Remote DBDICT cache 0 0
0
Remote Record cache 0 0
0
String Type cache 28 0
2736
IR Expert cache 368 0
88256
Publish/Subscribe 2 0
48
24x7 cache 1 0
512
Web cache 0 0
0
Big line 31752448
Big line (lowest) 31752448
Grain Total Total Current
Size Allocs Frees Usage
16 365 0 17.82%
32 46 30 1.56%
64 81 67 2.73%
128 152 17 52.73%
128 270 14 100.00%
256 14 0 10.94%
512 7 0 10.94%
1024 10 0 31.25%
2048 3 0 18.75%
4096 7 0 87.50%
8192 3 0 75.00%
8192 4 0 100.00%

Big 7 0 0%

0 unused 32768 byte segments

 Infrastructure Management

Shared memory in ServiceCenter is divided into blocks of various

granularities. The granularities are 16, 32, 64, 128, 256, 512, 1024, 2048,
4096, 8192, BIG_ALLOC. Any request for shared storage greater than 8192
bytes is satisfied from the BIG_ALLOC pool. Also the smaller size
White Paperr

granularities are not satisfied one at a time but come from a pool of similar
sized requests. For example, when a request is made for 16 bytes of shared
storage and there are no 16 byte block FREE, ServiceCenter will allocate a
32K byte block (from the BIG_ALLOC space) and divide that 32K area into
16 byte blocks. The first 16 byte block would be returned to the requestor and
the remaining 16 byte blocks would be on the free chain. A complete report of
these blocks can be obtained with a scenter -reportshm:1 command which
produces the following type of report: Notice that the granularities are
included in the report and an indication of how much FREE space exists in
each block.

IRExpert and Shared Memory

IRExpert makes extensive use of shared memory and usually represents the
largest allocation type. The default amount of shared memory used by IR is 30
% of the total size allocated for shared memory. This percentage can be
changed using the ir_max_shared parameter in the sc.ini file. The optimum
size of the cache is the same as the size of all ir.* files added together.

In general IR keeps track of all documents that contain a search word, e.g.:

window

PM1001

PM1002

PM1123

PM13568

The document list for a word is maintained or loaded into shared memory as it
is referenced. A second search on a word may be much faster, as the first
process who did the search may have loaded it into shared memory and the ir*
file does not have to be read from disk again.
 Infrastructure Management

If the IR shared memory area is full, it will remove the least recently used word
from shared memory.

Increasing the size of the shared memory will only have a performance impact
on IRExpert searches. It will NOT help with general system performance.
White Paper

Troubleshooting
As soon as less than 25 % of shared memory is unused, we recommend
increasing the amount of shared memory. Unused shared memory is the
memory space that has never been allocated. The real percentage of available
memory is always higher than what the unused % in reportshm is showing.
Using reportshm:1 will give more information about the shared memory
allocation. The shared memory may become fragmented and leaving at least
25 % free space is a precautionary measure so that fragmentation does not get
so bad that no new space can be allocated within the shared memory area.

The use of another parameter in the sc.ini file can ensure that the shared
memory allocation is kept in an area that won’t interfere with other
applications in use on the server or by the operating system. The parameter is
shared_memory_address. This can be used on all operating systems. On
Windows NT for example, if more than 210 MB of shared memory is used, it
may collide with another process’s shared memory.

Solaris is the most fastidious of all Unix systems. It will get a protection fault
if a process uses shared memory that may be assigned to another process.
Moving the shared_memory_address may move the shared memory far enough
away from other processes shared memory areas.