Professional Documents
Culture Documents
Best Practices For Running TEMENOS T24 On Microsoft SQL Server and Windows Server
Best Practices For Running TEMENOS T24 On Microsoft SQL Server and Windows Server
Abstract
TEMENOS T24 (T24) is a complete banking solution designed to meet the challenges faced by
financial institutions in today’s competitive market. T24 provides a single, real-time view of
clients across the entire enterprise, making it possible for banks to maximize returns and keep
costs down.
Microsoft® SQL Server® provides the ideal database platform for T24. By choosing the Microsoft
platform, T24 customers experience faster funds transfers, higher security-trade volumes, and
quicker close-of-business processes; T24 customers can also benefit from open, state-of-the-art
technologies to accelerate innovation, greatly increasing the speed and effectiveness with which
new products and services are created. Using Windows Server® 2008 R2 as the operating system
makes it possible for T24 to exceed performance standards in a scalable, reliable environment
that offers ease of management.
This white paper provides best practices for configuring and running TEMENOS T24 on Windows
Server and on the SQL Server database platform. Implementing these best practices can help
you avoid or minimize common problems and optimize the performance of your TEMENOS T24
implementation.
This document is provided “as-is.” Information and views expressed in this document, including URL and
other Internet Web site references, may change without notice. You bear the risk of using it.
This document does not provide you with any legal rights to any intellectual property in any Microsoft
product. You may copy and use this document for your internal, reference purposes.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
Table of Contents
1 OVERVIEW ....................................................................................................................................... 5
9 SUMMARY ..................................................................................................................................... 35
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
1 Overview
To help financial institutions face the challenges posed by today’s around-the-clock, global
marketplace, Temenos Group AG, the leading provider of integrated core banking systems,
offers TEMENOS T24 (T24). T24 pairs comprehensive and powerfully flexible business
functionality with the most advanced and scalable architecture available today.
T24 is built on an open architecture, and it uses established standards such as HTTP, XML, and
Java 2 Platform, Enterprise Edition (J2EE). The design of T24 supports multiple application
servers and provides horizontal scalability with true non-stop resilience.
The capabilities of T24 can be enhanced by the choice of an enterprise-ready database platform.
Customers running T24 on a Windows Server® operating system and Microsoft® SQL Server®
data management software benefit from a wide range of products and tools that can be used to
further improve the performance and extend the capabilities of the T24 banking solution.
Figure 1 shows a logical model of the interaction of the various services and components that
make up a T24 banking implementation. In this white paper, we focus on best practice guidance
for the database layer (in green).
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
This white paper, intended for database administrators (DBAs), describes optimizations that you
can make to the database tier and describes the database tier’s interaction with the application
tier to help ensure a successful deployment of T24 on SQL Server.
The paper first discusses best practices for configuring the database server and the application
servers and provides guidance for building scalability and high availability into the T24 banking
solution. The paper then looks at the options for disaster recovery. Best practices for reporting
are presented, and best practices for monitoring the performance of the database tier and for
system maintenance are discussed. The paper also provides appendices with step-by-step
guidance and extensive links for further information.
Implementing these best practices can help you optimize the performance of your TEMENOS
T24 implementation and can also help you avoid or minimize common problems.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
2 Best Practices for the Database Server
Following are some best practices you should use to configure your database server.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
2.1.3 Use a Dedicated Server
If you are mixing workloads on one physical computer, you should use virtual machines or
Windows/SQL Server mechanisms to control and monitor resource usage. Diagnosing
performance problems requires more effort and precision on a system with mixed workloads.
The table contains resource utilization results for several installation sizes based on the number
of customer accounts, i.e., 1 million (1M), 5 million (5M), 10 million (10M), and 25 million (25M)
accounts.
Accruals &
Transactions Accruals Accruals Accruals
Capitalizations
Workload
COB Total Time 0hr 52min 2hr 17min 1hr 56min 3hr 17min
IC.COB Time 0hr 13min 0hr 59min 0hr 54min 1hr 20min
Cores used if
9 17 32 137
CPU at 100%
24 Xeon
#Cores 24 Xeon 7460 32 Xeon 7560 64 Xeon 7560
7460
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
IOPS avg 660 970 2,000 7,000
These results are based on lab testing with a standardized workload; every application has
different functionality and different close-of-business processes. You should work with the
Temenos Performance and Sizing Team to properly size your system based on the expected
workload. Temenos also provides a Universal Performance Measurement (UPM) tool that can
be used to verify a hardware configuration before T24 is installed. In some cases, tests that
model your business processes as closely as possible may be necessary to confirm the server
sizes required to support your business scenario.
Following are best practices you should use when designing storage for your T24
implementation.
It is generally better to have a larger number of small drives than a smaller number of large
drives. If your storage area network (SAN) has multiple buses (sometimes called shelves), create
each LUN using drives from each bus to improve the internal bandwidth. Table 2 shows how to
choose drives from multiple buses to make up one LUN.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
LUN 1 LUN 2… …LUN 4
Bus 1 1 4 7 10 13 … 46
Bus 2 2 5 8 11 14 … 47
Bus 3 3 6 9 12 15 … 48
You should refer to the article Predeployment I/O Best Practices for procedures to test your I/O
subsystem performance.
Note: For an online transaction processing (OLTP) system the ideal latency values on a well-
tuned I/O subsystem are:
< 5 milliseconds (ms) for log (ideally 1 ms on arrays with cache)
< 20 ms for data on OLTP systems (ideally 10 ms or less)
In Windows Server® 2008 and Windows Server 2008 R2, the partition offset is set appropriately
by default for new storage. (Note that the offset of existing storage is not changed when
upgrading from Windows Server 2003.)
2.3.3 Set the File Allocation Unit Size and Stripe Size
When formatting the partition that will be used for SQL Server data files, you should use a 64-
kilobyte (KB) allocation unit size for data, logs, and the tempdb database.
The stripe size is also important to reach an optimal configuration. This is set in the SAN
management software, not through Windows Server. The following two equations can be used
to determine if you have an optimal configuration. Each should result in an integer value:
Partition_Offset ÷ Stripe_Unit_Size
Stripe_Unit_Size ÷ File_Allocation_Unit_Size
For details on managing disk partition sector alignment, see Disk Partition Alignment Best
Practices for SQL Server.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
2.3.4 Dedicate a High Percentage of SAN Cache to Writes
Most SANs have a large cache that can be split between a read cache and a write cache. SQL
Server provides read-ahead and read caching, and SQL Server does not benefit much from a SAN
read cache. If you have the option (within the constraints of SAN usage by applications), you
should dedicate 90% of the SAN cache to writes to improve write performance (a cache ratio of
10/90 read/write).
Note that virtually all SANs sold today have battery-backed cache. If this is not the case with
your SAN, you must disable write caching or risk losing committed data in the event of a power
outage.
1–12 1 E data
13–24 2 F data
25–36 3 G tempdb
37–48 4 H log
Table 3. Sample storage configuration for SQL Server.
You can also present storage to SQL Server through mount points, disk volumes that are
mounted as folders on other physical disks, without incurring a performance penalty.
For further information about storage design for SQL Server, see Storage Top 10 Best Practices.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
2.4.2 Enable Receive-Side Scaling
You should enable Receive-Side Scaling (RSS) on the SQL Server network interface card (NIC)
that is serving the application servers. This setting is found on the Advanced Property tab of the
network card. Also, be sure that offloading options are enabled. See the Microsoft® Developer
Network (MSDN®) articles Introduction to Receive-Side Scaling and Receive-Side Scaling
Enhancements in Windows Server 2008 for more information. If your NIC does not support
these options, consider replacing it with one that does.
You should configure the maximum number of RSS processors by setting the MaxNumRssCpus
registry key value to 8 on a computer system with 32 or more CPU cores. For computer systems
with less than 32 cores, use the default setting.
The RSS base CPU number (RssBaseCpu) is the CPU number of the first CPU that RSS can use.
RSS cannot use the CPUs that are numbered below the base CPU number. You should set
RssBaseCpu carefully so it does not overlap with the starting CPU.
Lab testing has shown good results with setting both registry key values to 8 (on a computer
system with more than 32 cores); this means that 8 RSS processors are used starting with core
number 8 to process network traffic.
Note: You should use the Windows RSS registry keys to configure these values instead of NIC
settings because NIC settings can be overridden by the Windows registry keys.
2.4.4 Use Multiple HBAs and Set the HBA Queue Depth
You should use at least two host bus adapters (HBAs) to provide redundancy and to increase
bandwidth between the SAN and SQL Server. The HBA cards should be set as load balanced and
configured to provide high availability among them. Connections between the SAN and server
are ideally 8 gigabyte (GB)/sec (but not less than 2 GB/sec).
The HBA queue depth may need to be increased if you have a small number of LUNs. A queue
depth value between 128 (if there are few LUNs) and 32 (if there are many LUNs) should be
considered. Note that this is a new recommendation; queues now default to “per LUN” rather
than “per target.” When the queue depth is set too low, you may get increasing latency and
less-than-expected throughput given the bandwidth between host/storage and the number of
disks in a particular configuration.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
2.5 Data, Log, tempdb, and Backup File Recommendations
The location of SQL Server files affects performance. You should use multiple files for each
filegroup supporting your databases, and use distinct LUNs for each of the data, tempdb, log,
and backup files. Using separate LUNs also helps you monitor the disks based on the type of use.
You should avoid using very large LUNs (several hundred GB or more) so that chkdsk does not
run for an excessive length of time if it is invoked by the operating system on startup.
You should pre-allocate enough space in the data files based on the initial size of the computer
system. You should monitor the database free space and if necessary extend each file
simultaneously so that all of the files have the same amount of free space. SQL Server optimizes
writes by spreading its write operations across the files based on the ratio of free space among
the files, so extending all files at once maintains this optimization.
You can leave the autogrowth setting on as an “insurance policy” so that SQL Server does not
stop when it runs out of space; however, do not rely on autogrowth to extend the database files
as a standard way of operating. While you should not allocate space for the data files in small
units, if you allocate in very large units during autogrowth, the application must wait (possibly
several minutes) while the space is allocated. Since you cannot control when autogrowth
engages, allocate only by the space needed for a few days of operations.
To avoid autogrowth operations on the transaction log file, monitor its size on a regular basis
and adjust it as necessary. As a starting point, you can use the 80-GB transaction log size for T24
installations with 10 million accounts.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
uses a read-committed snapshot isolation level as its default isolation level, which uses row
versioning. For more information, see Isolation Levels in the Database Engine.
You should then configure the SQL Server “max server memory (MB)” setting by taking the
amount of memory allocated to the database system and subtracting one GB for every four
cores (round up). This leaves the operating system with enough memory to work efficiently
without having to “grab” memory back from SQL Server. For example, if the server has 64 GB of
RAM and 24 cores, set the maximum memory to 58 GB (64 GB minus 6 [24 cores divided by 4]).
For detailed instructions, see How to reduce paging of buffer pool memory in the 64-bit version
of SQL Server on the Microsoft® Support site.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
2.7 Guidance for SQL Server Configuration Settings
Following is guidance for SQL Server configuration settings.
Page latch contention can be identified by examining the “SQL Server: Wait Statistics – Page
Latch waits” performance counter and querying the dynamic management view
sys.dm_os_wait_stats using this query:
To identify which tables and which pages experience latch contention, you can use the following
queries:
SELECT *
FROM sys.dm_db_index_operational_stats (DB_ID('T24'), NULL,
NULL, NULL)
ORDER BY [page_latch_wait_in_ms] DESC,
tree_page_latch_wait_in_ms DESC
and
SELECT * FROM sys.dm_os_waiting_tasks
WHERE wait_type LIKE 'PAGELATCH%'
Table 4 shows tables that have been identified as likely candidates for a lower fill factor setting
during lab testing:
FBNK_PL_C002 10% On
F_LOCKING 10% On
FBNK_EB_C004 50% On
FBNK_ACCT_ACTIVITY 50% On
FBNK_ACCOUNT 50% On
Table 4. Table candidates for lower fill factor.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
Note: This testing has been done with a standardized workload. Every application has different
functionality, and if high latch contention is observed, you must identify these candidates for
the application-specific workload profile.
For more information on the fill-factor option for indexes, see the article Fill Factor.
Before changing the recovery interval, you should consider its implication on the mean time to
recovery and recovery point objectives. Note that when using failover clustering, a longer
recovery interval also influences the failover time of the database instance.
For more information about the recovery interval option, see the article Recovery Interval
Option.
For more information about this SQL Server trace flag, see Microsoft Support Article 920093
For more Information, see Encrypting Connections to SQL Server and How to: Enable Encrypted
Connections to the Database Engine (SQL Server Configuration Manager).
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
3 Best Practices for the Application Server
Following are some best practices you should use for configuring your T24 application server.
You should leave the default value for the bulk factor parameter (currently 5), unless you have a
large installation with 10 million accounts or more. For such high-volume installations,
increasing the bulk factor may improve performance. Lab testing has demonstrated good results
using a bulk factor of 20 for deployments with more than 10 million accounts deployment and a
bulk factor of 40 for a deployment with 25 million accounts.
Each T24 table can have a maximum of 5 different handles (i.e., for the following T24
operations: READ, INSERT, UPDATE, DELETE, and SELECT). Each handle consumes a fixed amount
of memory (64 KB) and consumes additional memory needed for data cache.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
The T24 Handle Cache parameter (JEDI_XMLDRIVER_HANDLE_LIMIT) specifies the maximum
number of statement handles that can be cached. The default value (currently 500) should be
adjusted based on the available application server memory as shown in Table 4.
>= 64 GB 15000
Table 5. Handle cache parameter scale based on application server memory.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
4 Best Practices for Providing High
Availability
Failover clustering―a high-availability (HA) solution―can keep applications running in the event
of the failure of hardware components. Failover clustering is therefore recommended to provide
database-tier HA for SQL Server in a T24 deployment.
Typically, the data resides on a SAN with disks configured as RAID 10 (recommended) or RAID 5,
providing redundancy for the LUNs used by SQL Server. The SAN is connected to the servers via
two or more HBAs. The time to detect a fault and switch to the alternate node is typically less
than one minute. (Note that the time is dependent on the current transactional load and on the
configuration, such as the number of drives or other resources in the resource group.)
The cluster can also be used for planned maintenance. With SQL Server 2008, you can apply
service packs to inactive nodes, and then you can fail the active SQL Server group over to the
patched nodes. (Note that this results in a brief usage outage.) For more information on
applying service packs to a failover cluster instance, see SQL Server 2008 Failover Cluster Rolling
Patch and Service Pack Process.
T24 uses a “lock arbiter” called jDLS to maintain locks in a multi-application server setup. When
an application lock needs to be taken, a request is sent to the jDLS socket server. jDLS can be
installed:
In a resilient mode on two application servers.
On the database server if enough resources are available. (Note that approximately 20%
more CPU utilization on the database server is expected in this case.)
On a separate application server.
In a cluster configuration with jDLS installed on the database server, you can configure jDLS in
resilient mode and place jDLS on both database server nodes, with the primary jDLS being on
the active cluster node.
For general overview on the high-availability features of SQL Server, see High Availability—
Always On Technologies and High Availability with SQL Server 2008 R2.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
4.2 High Availability Recommendations for the Storage
As described in section Storage Design Recommendations, your storage solution should be a
fault-tolerant SAN solution made up of a redundant disk configuration such as RAID 10. The SAN
is still a single point of failure, but the majority of its components are redundant.
You should be sure to consider the disaster recovery options described in the section Best
Practices for Disaster Recovery to mitigate the effect of a total SAN failure.
The listener process spawns and monitors child processes, which process the transaction
requests. Should a child agent fail, the aborted transaction is resubmitted by the application and
is started on a new agent process. This in turn connects to the secondary database node that is
active after the failover, and the transaction is then processed as normal.
If an agent (tSA) fails or aborts because of an error or failover, then the tSM initiates a new
agent to replace the failed agent. The new agent resubmits the request, and this causes a
connection to be established to the secondary database node which lets the COB jobs restart
and continue.
If the error or failover occurs while the tSM is accessing tables in the database (e.g., JOB.LIST)
and is “connected” to the database server, the tSM also terminates, leaving no monitoring
process to automatically restart the tSAs. The tSM needs to be manually restarted, which then
restarts the tSAs letting the COB processes continue from where they left off. You cannot
automatically restart the tSM by design, because an assessment of the failure is required to
ensure that it’s a “real” disconnection rather than a temporary network interruption.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
5 Best Practices for Disaster Recovery
Disaster recovery (DR) provides alternate services in the event of a catastrophic physical
disruption of the infrastructure at the primary site. For protection from natural disasters such as
earthquakes, the DR site is frequently located far from the primary site. SQL Server provides
various technologies for transferring data changes from the primary database to a disaster
recovery site.
One of the main decisions that you need to make is whether or not you are willing to tolerate
potential data loss in the event of complete destruction of the primary site. Loss-less DR
methods require the use of synchronous replication to the DR site. This means that all writes to
the storage systems must complete at both sites before the transaction is committed, which can
increase the time to perform the transaction significantly, depending on the line speed.
The two primary methods of synchronous replication are SAN mirroring and database mirroring
in “high-safety” mode. SAN mirroring requires additional features provided by the SAN vendor.
Database mirroring is a feature included with SQL Server.
Note: Asynchronous SAN mirroring is not recommended because it is not possible to determine
if the mirror is synchronized after a disaster, and you cannot easily discover any differences.
Database mirroring can use any form of TCP connectivity between the two servers, but as with
SAN mirroring, a large bandwidth is important in providing responsive, low-latency service. One
option to mitigate the cost of the network between the primary site and the DR site is to use
two levels of DR sites:
One site is geographically close to the primary site (under 100 kilometers [km]) and uses
mirroring to maintain an exact mirror of the primary site.
A second site is geographically remote from the primary site and is synchronized using
one of the asynchronous methods that can use a lower, less-expensive bandwidth.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
For more information, see Synchronous Database Mirroring (High-Safety Mode).
The primary methods of asynchronous replication are asynchronous database mirroring, log
shipping, or transactional replication (server-to-server transactional replication or peer-to-peer
replication). With these methods, there is a window of a few seconds to a few minutes where
the transactions committed on the primary site are not yet committed on the DR site.
Note that when using asynchronous database mirroring or log shipping, the log from the
primary site cannot be applied to the DR site once the DR site has been used to store new
transactions in its new role as the primary database. Transactions that were not shipped to the
DR site before switching to the secondary database server are lost.
For more information, see Log Shipping Overview, Database Mirroring Overview, and
Transactional Replication Overview.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
6 Best Practices for Reporting
In the default implementation of T24, reports are generated from the same database as online
services. If you find that this has an unacceptable impact on online performance, you can shift
the reporting workload to another database server.
Transactional replication is recommended for maintaining a near real-time copy of the database
for reporting purposes. For reducing the data volume of the reporting database and network
utilization between the servers, it is also possible to replicate only a subset of the T24 tables that
are relevant for reporting. T24 system tables can be omitted from replication as they do not
contain business data. Temenos provides a list of system tables that do not need to be
replicated.
The reporting database may be configured for simple database recovery because it is not
updated by client transactions; this lets you skip the backup process, but it means that if the
database becomes corrupted, you need to re-initialize the subscription from the primary
backup.
There are alternatives that can be considered, including snapshots, log shipping, and database
mirroring, but these are generally not as effective as replication.
Snapshots give only point-in-time data (typically once per day). When you update the
snapshot, all connections to the previous snapshot are broken, providing a poor user
experience. Snapshots can be generated by the SAN or you can use database snapshots.
Log shipping or database mirroring keeps the target database in “recovery” mode, and it
cannot be accessed for reading except through a database snapshot. This again means
that it is only point-in-time data, and when updating, all previous connections are
broken.
Database mirroring can only mirror to one other server. You cannot mirror a database
to both the DR site and to a reporting site.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
7 Best Practices for Performance Tuning
Following is guidance for performance tuning for a T24/SQL Server implementation.
You can use the following query to identify indexes that have not been used since the last time
SQL Server was started:
SELECT
object_name(i.object_id) AS object_name,
i.name AS index_name
FROM sys.indexes i JOIN sys.objects o ON i.object_id =
o.object_id
WHERE o.type = 'U'
AND i.index_id NOT IN
(SELECT s.index_id
FROM sys.dm_db_index_usage_stats s
WHERE s.index_id = i.index_id
AND s.object_id = i.object_id
AND s.database_id = db_id())
ORDER BY object_name(i.object_id) ASC
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
You can use the following query to monitor the index usage on a regular basis:
SELECT
object_name(i.object_id) AS object_name,
i.name AS index_name, s.index_id,
user_seeks + user_scans + user_lookups AS user_reads,
system_seeks + system_scans + system_lookups AS
system_reads,
user_updates,
system_updates
FROM sys.dm_db_index_usage_stats s JOIN sys.indexes i
ON s.index_id = i.index_id AND s.object_id = i.object_id
WHERE s.database_id = db_id()
AND i.type <> 0
ORDER BY user_reads DESC
After completion of the COB, you should look at F_JOB_TIMES for long COB job times. When
you have identified the long-running jobs, search the content of the T24 log file &COMO& to
match the jQL queries corresponding to the long-running jobs. Compare these jQL queries to
the previously identified SQL Server statements to select queries that are good candidates
for optimization.
Online Processing:
For online processing, identify slow-running operations in the user interface (UI) based on
user feedback and try to reproduce them (on a test database system). Use the SQL Server
Profiler to capture the corresponding SQL Server queries. After identifying the SQL
statements, you should test them and measure the CPU time and I/Os. This can be done
using the sys.dm_exec_query_stats dynamic management view and the query
described in the previous section.
If you identify multiple long-running queries involved in the online operations, sort the
queries based on the average CPU time, and consider the number of executions of each
query. Optimizing a query which is executed very often is more important than optimizing
queries executed only few times.
2. Consider optimizations.
After identifying slow-running queries that are good candidates for optimization, consider
the following:
If there are queries on tables STMT_ENTRY, CATEG_ENTRY, or RE_CONSOLSPEC_ENTRY
using a “where” clause that is different from RECID = <value>, then these are most likely
jQL SELECT queries in a custom-developed code.
There should generally be no queries on these tables with a search condition using any
fields (other than the RECID). These are the biggest T24 tables, containing huge number
of records, and there is usually heavy activity on these tables.
The application logic and the jQL SELECT queries on these tables should be reconsidered
and, if possible, rewritten. For example, instead of using STMT_ENTRY, you can use the
table ACCT_ENT_TODAY in many cases. ACCT_ENT_TODAY is much smaller in size; the
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
key is the account number, and the rows in the table contain the key of the entries that
have been made into that account during the current business day.
If a query has a search condition on a single-value field, then consider scalar promotion
(see step 3).
An example of this type of query is:
SELECT t.RECID,t.XMLRECORD FROM F_HOLD_CONTROL t
WHERE
t.XMLRECORD.exist(N'/row/c2[.="NEW.LOAN.REPORT"]') = 1
If a query has a search condition on a specific multi-valued field (specific local reference
field), then consider scalar promotion (see step 3).
For example, the following query uses specifically the eighth value of a multi-value field
in the search condition:
SELECT t.RECID,t.XMLRECORD FROM FBNK_CARD_ISSUE t
WHERE t.XMLRECORD.exist(N'/row/c14[@m="8"][.=12345]')
= 1
Consider the number of promoted columns and indexes per table. Indexes improve the
performance of select queries, but also increase the time that is required to perform
modifications (i.e., inserts, updates, deletes) to the underlying table. Too many indexes
may degrade the overall performance. As a general rule, you should avoid creating more
than seven indexes on a table.
Do not create XML indexes on T24 XMLRECORD fields. The impact on transaction
latency is too high, and the benefit in query performance is usually not significant.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
Following are two examples of scalar promotion:
-- example 1
-- scalar promotion of single valued field
CREATE FUNCTION udf_HOLD_CONTROL_C2(@xmlrecord XML)
RETURNS nvarchar(35)
WITH SCHEMABINDING
BEGIN
RETURN @xmlrecord.value('(/row/c2/text())[1]',
'nvarchar(35)')
END
-- example 2
-- scalar promotion of specific multi-valued field
CREATE FUNCTION udf_CARD_ISSUE_CUSTOMER(@xmlrecord XML)
RETURNS integer
WITH SCHEMABINDING
BEGIN
RETURN
@xmlrecord.value('(/row/c14[@m="8"]/text())[1]',
'integer')
END
-- example 2
CREATE INDEX ix_CARD_ISSUE_CUSTOMER ON
FBNK_CARD_ISSUE(C14_8)
Once there is a promoted column for a specific field in the table, T24 automatically
translates the queries to use that column relational in the “where” clause if there is
a search condition on that field.
4. Verify optimizations.
Verify that the changes are successful and measure the impact of the optimizations.
For scalar promotion (promoted and indexed fields):
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
Verify the query translation.
Without scalar promotion, T24 uses a query syntax such as:
SELECT t.RECID,t.XMLRECORD
FROM FBNK_CARD_ISSUE t
WHERE t.XMLRECORD.exist(N'/row/c14[@m="8"][.=12345]')
= 1
The execution of this query usually uses a table scan to retrieve the results.
After promoting the field “c2”, T24 should translate the query as:
SELECT t.RECID,t.XMLRECORD
FROM FBNK_CARD_ISSUE t
WHERE c14_8 = 12345
In this case, index lookup on ix_CARD_ISSUE_CUSTOMER is used.
Prove that the index is used by reproducing the query and verifying the actual
execution plan. You can run the query in SQL Server Management Studio and
activate the icon “Include Actual Execution Plan” on the SQL Editor toolbar.
Alternatively, you can use the SET STATISTICS PROFILE ON statement to
display execution plan information.
Verify the performance of the query has improved.
After using the application for a period of time (e.g., couple of hours or days) or
after running COB, use the sys.dm_db_index_usage_stats dynamic
management view to verify the index usage. Consider the ratio between index reads
and index writes, keeping in mind that an index usually improves the performance
for read operations but slows down modifications (i.e., inserts, updates, deletes) at
the same time.
You can use the queries described in section Standard T24 Queries for this purpose.
You should monitor the index usage statistics on a regular basis.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
application-specific purposes in the application layer. In some cases, concat files can also be
used as a “logical index” in T24.
Once in production, you should create a performance baseline and monitor trends in resource
consumption against the performance baseline so that you can predict future bottlenecks and
determine when resource limitations might begin to impact performance as your customer base
grows. You should not worry about seeing short spikes in utilization or high consumption during
processes that are operating at an acceptable performance level.
Note that SQL Server 2008 R2 introduces application and multi-server management, which can
help you manage the T24 database environment more efficiently at scale, with visibility into
resource utilization for consolidation and improved efficiencies across the application lifecycle.
For more information, see SQL Server 2008 R2 - Application and Multi-Server Management.
Metric Predicts
Disk Bytes/sec for each LUN Need to spread data across more LUNs.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
8 Best Practices for System Maintenance
You should perform regular maintenance tasks to keep SQL Server running optimally. Your
maintenance schedule should include implementing a backup strategy, updating database
statistics, rebuilding and reorganizing indexes, and identifying and correcting excessive
fragmentation.
For more information about creating maintenance plans, see Appendix 1: Create a Maintenance
Plan to Back Up the Temenos Application Database and Appendix 2: Create a Maintenance Plan
to Back Up the System Databases.
The compression is achieved by specifying the WITH COMPRESSION clause in the BACKUP
command or by selecting compression the Options page in the Back Up Database dialog box.
There is also a global setting to compress all backups taken on a server instance by default. (This
setting is accessed by using the Database Settings tab in the Server Properties dialog box or by
running sp_configure with backup compression default set to 1.) The RESTORE command
automatically detects that a backup is compressed and decompresses the backup during the
restore operation.
For more information, see the article Tuning the Performance of Backup Compression in SQL
Server 2008 on SQLCAT.com.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
It is recommended to schedule the full backup to run after the COB processing and after any
index maintenance tasks (e.g., index reorganize or index rebuild).
If you implement peer-to-peer replication, you must also implement a backup schedule for the
target database(s). The distributor database operates in simple recovery mode, so it does not
need to be backed up.
For more information, see the article Considerations for Backing Up and Restoring System
Databases.
For more information, see the article Statistics Used by the Query Optimizer in Microsoft SQL
Server 2008. For step-by-step guidance for manually updating statistics using a maintenance job,
see Appendix 3: Update Statistics.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
To decide which defragmentation method to use, analyze the index to determine the degree of
fragmentation. By using the DMF sys.dm_db_index_physical_stats, you can detect
fragmentation in a specific index, all indexes on a table or indexed view, all indexes in a
database, or all indexes in all databases. Using this function, you have access to the
fragmentation levels available in defined columns at any given time.
Column Description
You can then use Table 7 as a guide to determine the best method to correct the fragmentation.
Note that very low levels of fragmentation (less than 10%) should not be addressed by either of
these commands because the benefit from removing such a small amount of fragmentation is
almost always vastly outweighed by the cost of reorganizing or rebuilding the index. The white
paper Microsoft SQL Server 2000 Index Defragmentation Best Practices can provide additional
guidance.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
The reorganize process uses minimal system resources. Also, reorganizing is automatically
performed online. The process does not hold long-term blocking locks; therefore, it does not
block running queries or updates.
The following methods can be used to rebuild clustered and non-clustered indexes:
ALTER INDEX with the REBUILD clause. This statement replaces the DBCC DBREINDEX
statement.
CREATE INDEX with the DROP_EXISTING clause.
Each method performs the same function, but there are advantages and disadvantages to
consider. See Reorganizing and Rebuilding Indexes for more information. Note, that the
clustered index cannot be rebuilt online if the underlying table contains XML data type.
For more information on applying updates, see Update Management TechCenter and SQL
Server 2008 failover cluster rolling patch and service pack process.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
9 Summary
TEMENOS T24, coupled with Microsoft SQL Server database software, provides financial
institutions with a complete banking solution. Using the best practices described in this white
paper can help optimize the performance of the database tier. The links in the section that
follows provide even more information.
For benchmarking results from a SQL Server/TEMENOS T24 implementation at the North Shore
Credit Union in British Columbia, Canada, see:
TEMENOS T24 Core Banking Optimized on Microsoft SQL Server Database Platform.
Following is a list of technical articles that can help you learn more about specific Windows and
SQL Server topics:
Windows Configuration:
Receive-Side Scaling
Introduction to Receive-Side Scaling
Receive-Side Scaling Enhancements in Windows Server 2008
Desktop Heap Overview
Desktop Heap, part 2
Storage Configuration:
Predeployment I/O Best Practices
Storage Top 10 Best Practices
DiskPart
Disk Partition Alignment Best Practices for SQL Server
SQL Server Configuration:
Isolation Levels in the Database Engine
Optimizing tempdb Performance
Configure Server Startup Options
Recovery Interval Option
Fill Factor
How to reduce paging of buffer pool memory in the 64-bit version of SQL Server
High Availability and Disaster Recovery:
High Availability with SQL Server 2008
Failover Clustering in Windows Server 2008 R2
SQL Server 2008 Failover Clustering
Database Mirroring Overview
Synchronous Database Mirroring (High-Safety Mode)
Database Mirroring Best Practices and Performance Considerations
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
Log Shipping Overview
Transactional Replication Overview
Best Practices for Replication Administration
Replication Security Best Practices
Peer-to-Peer Transactional Replication
Database Maintenance
Considerations for Backing Up and Restoring System Databases
Tuning the Performance of Backup Compression in SQL Server 2008
Best Practices for Recovering a Database to a Specific Recovery Point
Statistics Used by the Query Optimizer in Microsoft SQL Server 2008
Reorganizing and Rebuilding Indexes
SQL Server 2008 R2 - Application and Multi-Server Management
SQL Server 2008 failover cluster rolling patch and service pack process
Update Management TechCenter
Troubleshooting Performance Problems in SQL Server 2008
See the SQL Server Best Practices portal for technical white papers, the SQL Server Best
Practices Toolbox, Top 10 Lists, and other resources.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
10 Appendix 1: Create a Maintenance Plan to
Back Up the Temenos Database
It is important to create a maintenance plan to back up the Temenos database. The following
step-by-step instructions guide you through the process.
1. Open SQL Server Management Studio, expand the Management node, and then
expand the Management Plans node.
2. Right-click Maintenance Plans, click Maintenance Plan Wizard, and then type an
appropriate name for this Temenos application database backup plan.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
4. On the Select Maintenance Tasks dialog box, select Clean Up History, Back Up
Database (Full), Back Up Database (Transaction Log), and Maintenance Cleanup Task.
Click Next twice.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
b. On the Define Back Up Database (Full) Task dialog box, click on These
databases, and select the T24 user database in the Database(s) box. Click OK.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
c. On the Job Schedule Properties dialog box, select the frequency and duration of
full backups. It is recommended to schedule the full backup to be executed,
after the COB processing. Click OK.
Figure 9. Options to set the frequency and duration of transaction log backups.
f. On the Define Back Up Database (Transaction Log) Task dialog box, configure
the transaction log backup. Click Next.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
11 Appendix 2: Create a Maintenance Plan to
Back Up the System Databases
You should create a maintenance plan to back up the system databases. The following step-by-
step instructions guide you through the process.
1. Open SQL Server Management Studio, expand the Management node, and then
expand the Management Plans node.
2. Right-click Maintenance Plans, click Maintenance Plan Wizard, and then type an
appropriate name for this Temenos system database backup plan.
3. On the Select Plan Properties dialog box, click Single schedule for the entire plan or no
schedule, and click Change to select the frequency and duration of transaction log
backups. Click Next.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
4. On the Select Maintenance Tasks dialog box, select Clean Up History, Back Up
Database (Full), and Maintenance Cleanup Task. Click Next.
5. On the Select Maintenance Task Order dialog box, make sure that the tasks appear in
the order shown in Figure 15. Click Next.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
6. As you complete the Maintenance Plan Wizard, you should select options based on the
needs of your organization. The following figures show examples of options you may
want to select.
a. On the Define Back Up Database (Full) Task dialog box, select System
databases. Click OK.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
b. Define the full backup task by selecting from the various options. Click Next.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
c. Define the maintenance cleanup task by selecting from the various options.
Click Next.
d. Define the history cleanup task by selecting from the various options. Click Next.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
e. Select the report options you want. Click Next.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
12 Appendix 3: Update Statistics
You should create a maintenance plan to update statistics. The following step-by-step
instructions guide you through the process.
1. Open SQL Server Management Studio, expand the Management node, and then
expand the Management Plans node.
2. Right-click Maintenance Plans, and click Maintenance Plan Wizard.
3. On the Select Plan Properties dialog box, type an appropriate name for this statistics
update plan, and click Single schedule for the entire plan or no schedule. Click Change
to select the frequency and duration of statistics updates.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
4. Based on your organization’s needs, select the frequency and duration of the statistics
updates on the Job Schedule Properties – Update Temenos Statistics (FullScan) dialog
box. Click OK, and then click Next.
5. On the Select Maintenance Tasks dialog box, select only Update Statistics. Click Next.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
6. On the Select Maintenance Task Order dialog box, make no changes. Click Next.
7. On the Define Update Statistics Task dialog box, click on These databases, and select
the T24 user database in the Database(s) box. Click OK.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
8. On the Define Update Statistics Task dialog box, select Tables and views in the Object
box. Under Scan type, click Full scan. Click Next.
9. On the Select Report Options dialog box, select report options based on your
organization’s needs. Click Next.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
13 Appendix 4: Reorganize and Rebuild
Indexes
The following script samples automatically reorganize or rebuild all partitions in a database. The
first one reorganizes all indexes that have an average fragmentation between 10% and 80%. The
second script rebuilds all indexes with an average fragmentation greater than 80 percent.
Executing these scripts requires the VIEW DATABASE STATE permission. These examples use
DB_ID() instead of specifying particular database name.
Note that an error is generated if the current database has a compatibility level of 80 or less. To
resolve the error, replace DB_ID() with a valid database name. For more information about
database compatibility levels, see sp_dbcmptlevel (Transact-SQL).
-- Reorganize indexes
-- Ensure that a USE <databasename> statement has been executed
first.
SET NOCOUNT ON;
DECLARE @objectid int;
DECLARE @indexid int;
DECLARE @partitioncount bigint;
DECLARE @schemaname nvarchar(130);
DECLARE @objectname nvarchar(130);
DECLARE @indexname nvarchar(130);
DECLARE @partitionnum bigint;
DECLARE @partitions bigint;
DECLARE @frag float;
DECLARE @command nvarchar(4000);
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
DECLARE partitions CURSOR FOR SELECT * FROM #work_to_do;
IF @partitioncount > 1
SET @command = @command + N' PARTITION=' +
CAST(@partitionnum AS nvarchar(10));
EXEC (@command);
PRINT N'Executed: ' + @command;
END;
-- rebuild indexes
-- Ensure that a USE <databasename> statement has been executed
first.
SET NOCOUNT ON;
DECLARE @objectid int;
DECLARE @indexid int;
DECLARE @partitioncount bigint;
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
DECLARE @schemaname nvarchar(130);
DECLARE @objectname nvarchar(130);
DECLARE @indexname nvarchar(130);
DECLARE @partitionnum bigint;
DECLARE @partitions bigint;
DECLARE @frag float;
DECLARE @command nvarchar(4000);
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
SET @command = @command + N' PARTITION=' +
CAST(@partitionnum AS nvarchar(10));
EXEC (@command);
PRINT N'Executed: ' + @command;
END;
You should generally create a nightly job for index reorganization (using the 1 sample script) and
a weekly job for index rebuild (based on the 2 sample script). Ideally, the index maintenance
tasks (reorganization or rebuild) should be performed after COB processing and before a full
backup is executed.
To create a nightly or weekly job to perform the reorganizing or rebuilding, perform the
following steps:
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
2. Name the job so it can be easily identified as the reorganization or rebuilding index.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
4. Select the Advanced page, change the On Success action to Quit the job reporting
success, and click OK.
5. On the New Job Schedule dialog box, schedule the job to run nightly or weekly,
depending on the needs of your organization. Click OK.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
6. On the Job Properties – Temenos Index Reorg/Rebuild dialog box, configure the
notifications for the job based on your organization’s needs. (Note that database email
must be configured correctly for the notifications to work.) Click OK.
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server
14 Appendix 5: Installation Check List
To assure successful T24 installation, the following items should be fulfilled:
Configuration settings:
Windows Hotfix KB976700
Lock pages in memory privilege granted to SQL service account
Best Practice Guidance for Running TEMENOS T24 on Microsoft SQL Server and Windows Server