Professional Documents
Culture Documents
January 2011
www.bmc.com
If you have comments or suggestions about this documentation, contact Information Development by email at doc_feedback@bmc.com.
Copyright 20042011 BMC Software, Inc. BMC, BMC Software, and the BMC Software logo are the exclusive properties of BMC Software, Inc., are registered with the U.S. Patent and Trademark Office, and may be registered or pending registration in other countries. All other BMC trademarks, service marks, and logos may be registered or pending registration in the U.S. or in other countries. All other trademarks or registered trademarks are the property of their respective owners. IBM, Informix, DB2, and AIX are registered trademarks of International Business Machines Corporation in the United States, other countries, or both. Linux is the registered trademark of Linus Torvalds. UNIX is the registered trademark of The Open Group in the US and other countries. Oracle and Java are registered trademarks of Oracle and/or its affiliates. Other names may be trademarks of their respective owners. BMC Software considers information included in this documentation to be proprietary and confidential. Your use of this information is subject to the terms and conditions of the applicable End User License Agreement for the product and the proprietary and restricted rights notices included in this documentation.
Customer Support
You can obtain technical support by using the Support page on the BMC Software website or by contacting Customer Support by telephone or email. To expedite your inquiry, please see Before Contacting BMC Software.
Support Website
You can obtain technical support from BMC Software 24 hours a day, 7 days a week at http://www.bmc.com/support_home. From this website, you can
Read overviews about support services and programs that BMC Software offers. Find the most current information about BMC Software products. Search a database for problems similar to yours and possible solutions. Order or download product documentation. Report a problem or ask a question. Subscribe to receive email notices when new product versions are released. Find worldwide BMC Software support center locations and contact information, including email addresses, fax numbers, and telephone numbers.
Product information Product name Product version (release number) License number and password (trial or permanent)
Operating system and environment information Machine type Operating system type, version, and service pack System hardware configuration Serial numbers Related software (database, application, and communication) including type, version, and service pack or maintenance level
Sequence of events leading to the problem Commands and options that you used Messages received (and the time and date that you received them) Product error messages Messages from the operating system, such as file system full Messages from related software
White Paper
White Paper
Overview
High-load environments present special issues relating to performance, hardware capacity, system downtime, and so on. A high-load environment presents two major issues: System scalability is dependent upon the capacity of the hardware. If the existing hardware is at its limit, it needs to be replaced by bigger, more powerful hardware. New hardware is often much more expensive than existing hardware. The old hardware could only be used as a hot standby system. (A hot standby system is running and ready for immediate service. It can be switched into service manually or automatically upon detection of a failure in the active equipment.) The system needs to be available for as much time as possible. This can be challenging and often requires redundant systems. The backup system is often in hot standby mode and is only activated in the event of an outage. Only one system can be used in production.
Solution
The solution to both challenges is a technology commonly used in the web environmenta load balancer. You can use a hardware load balancer to improve the scalability and availability of AR System servers. A load balancer is a valuable component in building a scalable, highly available AR System infrastructure. Scalability is achieved through the ability to add AR System servers as demand and workload increase. High availability is achieved by configuring multiple AR System servers to handle client requests, and if one AR System server fails, other AR System servers in the group handle the additional workload. By creating an infrastructure that scales with workload and reduces downtime, you can maximize the return on your AR System investment.
Definition
A load balancer is a transparent network device that distributes the traffic from multiple clients to several servers, preventing AR System servers from becoming overloaded. Distributing workload among several AR System servers can lead to performance benefits and cost savings. Buying many smaller machines is far less expensive than paying for a single high-performance machine. Since load balancers redirect incoming traffic, they are sometimes referred to as redirectors. A load balancer also provides high availability through redundancy and fail-over protection. Fail-over is the process of moving operations from one server to another upon service or machine failure. If one AR System server becomes unavailable for software reasons or if the machine hardware fails, other AR System servers in the group (or cluster) can take over the workload. Users will not be aware of any problems, nor will they experience any downtime.
Most load balancers work on the TCP level of the network protocol and can therefore balance the load of many applications that use this protocol. Some examples include HTTP, FTP, and the AR System server. The load balancer is transparent to users, so the client application cannot see it and is unaware that the client requests are being load-balanced. You can think of the load balancer as a virtual system, or as a super AR System server in this case. The load balancer is given a virtual IP address and a DNS name, either of which is used by BMC Remedy User clients when connecting to the group of load-balanced AR System servers. Both the short and long DNS names must be resolvable. (Long DNS names are fully qualified with a domain.) When a client request is received, the load balancer passes the request to one of the actual AR System servers within the group. The chosen AR System server performs the work and sends the results to the client. This balancing of connections spreads out the client requests evenly and distributes the load.
NOTE
For performance reasons, the AR System API clients establish a connection with the AR System server and keep the connection until the session is terminated or the connection is interrupted.
Functionality
Load balancers forward data packets in the same manner as Network Address Translation (NAT) devices. The packets are forwarded at the TCP level with modified address information to the target system. To distribute client requests across each group of AR System servers, the load balancer can use various scheduling policies. The following two policies are the most common: Round RobinThis policy directs client requests to each AR System server, one at a time. Successive requests are routed to the next AR System server, then the next, and this process is repeated consecutively. Least ConnectionsThis policy directs client requests to the AR System server that has the fewest connections. For better throughput, most load balancers offer multiple network ports. This method avoids collisions between the incoming and outgoing routed network traffic. The load balancer also offers clients the ability to stick to one target system. This means that all requests from a single client are routed to the same system.
Functionality
White Paper
For versions 7.6.04 or later, BMC recommends configuring the load balancer that is located between the web servers and AR System servers without setting a sticky bit. In versions earlier than 7.6.04, BMC recommended setting the sticky bit to activate session affinity to route all connections from one web server to the same AR System server. For more information, see Scenarios for load balancing between the web servers and AR System servers on page 14 and the BMC Remedy Mid Tier Guide.
Configuration examples
The following section describes a number of common configuration examples. All configurations require that AR System servers have been properly configured. Verify that the following configuration steps have been completed before you review the examples:
1 All AR System servers share the same database. 2 All AR System servers have the same server name (server name alias), and they
is configured for.
5 External operations managed by the server group (such as Email Engine and
Flashboards) must be must be installed locally on the AR System servers that manage them.
The load-balancer IP address is mapped to the AR System server IP address. The server is configured to ignore the actual IP address recorded during Alert client registration.
7 Distributed Server Option (DSO) is configured for a load-balanced environment.
The index collection directory is accessible to all AR System servers in the group.
9 Clients are configured to use the virtual address and port of the load balancer. 10 Servers are declared to be server group members, and operation rankings are
configured.
Configuration examples
White Paper
In Figure 1-3, AR System server client traffic comes into the firewall on TCP port 3030 and is directed to the load balancer. The load balancer routes this traffic to one of the AR System servers listening on port 2020. As shown in the diagram, the port number for the virtual address can be different from the port numbers for the real AR System servers.
Figure 1-3: Load-balancer configuration with a firewall, a virtual AR System server, and real AR System servers
Configuring a load balancer with a firewall, web servers, and multiple AR System servers
In this example, client requests pass through a firewall and into the load balancer. The load balancer directs the web requests to three web servers. The web servers access the AR System servers to fulfill AR System requests. The three AR System servers share a single database, as shown in Figure 1-4.
10
Figure 1-4: Load-balancer configuration with a firewall, web servers, and AR System servers
Using the hosts file, the IP address of the AR System server needs to be resolved manually on each web server. This is necessary because all web servers reference the same AR System server name; however, each web server is linked to a different AR System server, and each AR System server has a different IP address. If the server name was resolved using a DNS server, this server name would resolve to the same IP address and all the web servers would communicate with the same AR System server. Therefore, each web server uses the hosts file, and manually resolves the AR System server name to the appropriate AR System server. On Windows platforms, the hosts file is located in the %WINDIR%\system32\ drivers\etc directory. On UNIX, the hosts file is located in the /etc directory. The hosts file on Web 1 should include the following line:
myarserver 11.40.35.47
The hosts file for Web 2 should include the following line:
myarserver 11.40.35.49
The hosts file for Web 3 should include the following line:
myarserver 11.40.35.51
In the preceding sample hosts files, myarserver is the AR System server name that all the web servers reference.
Configuring a load balancer with a firewall, web servers, a second load balancer, and multiple AR System servers
In this example, client requests pass through a firewall and into a load balancer. The first load balancer directs web requests to the web servers. Web server requests to the AR System servers are directed through a second load balancer. The three AR System servers share a single database, as shown in Figure 1-5.
Configuration examples
11
White Paper
Figure 1-5: Load-balancer configuration with a firewall, web servers, AR System servers, and two load balancers
NOTE
For versions 7.6.04 or later, BMC recommends configuring the load balancer that is located between the web servers and AR System servers without setting a sticky bit. In versions earlier than 7.6.04, BMC recommended setting the sticky bit to activate session affinity to route all connections from one web server to the same AR System server. For more information, see Scenarios for load balancing between the web servers and AR System servers on page 14 and the BMC Remedy Mid Tier Guide. This type of configuration can also be used with a WAN, DMZ, or LAN as shown in Figure 1-6. Client requests pass through the firewall and into the first load balancer. The load balancer routes the traffic to one of the web servers. AR System requests from the web servers pass through the first load balancer, then through the firewall, and finally to the second load balancer. The second load balancer then routes the request to one of the AR System servers in the group.
Figure 1-6: Load-balancer configuration with a WAN, a firewall, web servers, AR System servers, and two load balancers
12
For better throughput, such as might be required in a high-performance environment, you can add a second firewall, as shown in Figure 1-7. AR System server traffic from the web servers is routed through the second firewall. AR System server traffic from web servers follows a different route from that of incoming AR System client requests, thereby reducing the likelihood of a network bottleneck in the load balancer.
Figure 1-7: Load-balancer configuration with a WAN, two firewalls, web servers, AR System servers, and two load balancers
13
White Paper
Scenarios for load balancing between the web servers and AR System servers
For versions 7.6.04 or later, BMC recommends configuring the load balancer between the web servers and AR System servers without setting a sticky bit. This allows a session from any web server to be distributed to any AR System server. This section describes the configuration for a load balancer between the web servers and AR System servers without setting a sticky bit for version 7.6.04 or later. As a starting point, the configuration for setting a sticky bit for the load balancer for versions earlier than 7.6.04 is first discussed.
Load balancing between the web servers and AR System servers by setting a sticky bit
In versions earlier than 7.6.04, BMC recommended configuring the load balancer between the web servers and AR System servers by setting a sticky bit. In this configuration, the load balancer between the web servers and the AR System servers routed all connections from one web server to the same AR System server, resulting in session affinity. Figure 1-8 illustrates session affinity between the web servers (WS) and the AR System servers (ARS) when the load balancer (LB) is configured with a sticky bit. For example, session affinity is established between WS1 and ARS1, WS 2 and ARS2, and WS3 and ARS3. The load balancer routes all connections from WS1 to ARS1.
14
Figure 1-8: Load-balancer configured with the sticky bit between the web servers and AR System servers
WS1 LEGEND ARS: AR System server LB: Load balancer WS: Web server
ARS1
WS2 WS3
ARS2
ARS3
WS1
ARS1
WS2 WS3
ARS2
If a new AR System server (ARS3) is added to this AR System, ARS3 is never considered as available because WS1, WS2 and WS3 have already established session affinity to either ARS1 or ARS2 (Figure 1-10).
15
White Paper
Figure 1-10: Load-balancer configuration with a new AR System server added to three web servers and two AR System servers with session affinity established
WS1
ARS1
WS2 WS3
ARS2
ARS3 new
In version 7.6.04 or later, you can route connections from a web server to a new AR System server on the fly by selecting the Enable Lifespan setting on the Connection Settings page of the Mid Tier Configuration Tool (Figure 1-11). This setting enables load balancing without setting a sticky bit. For more information, see the BMC Remedy Mid Tier Guide.
Figure 1-11: Enable Lifespan setting on the Connection Settings page of the Mid Tier Configuration Tool
With Enable Lifespan selected, the load balancer distributes sessions from any web server (in this case, WS3) across the AR System server group according to its scheduling policy (round robin or least connections). The newly-available AR System server (ARS3) is considered within that distribution, as shown in Figure 1-12. Connections may or may not get routed to ARS3 depending on the scheduling policy.
16
Figure 1-12: Load-balancer configuration with a new AR System server added without setting a sticky bit for the load balancer
WS1
ARS1
WS2 WS3
ARS2
ARS3 new
WS1
ARS1
WS2 WS3
ARS2
ARS3 fails
When ARS3 recovers and is recognized as an active resource by the load balancer, it does not receive connections from any of the three web servers because session affinity has been established between the web servers and either ARS1 or ARS2 (Figure 1-10). To rebalance the servers in versions earlier than 7.6.04, you need to shut down the AR System servers, load balancer, and web servers and then restart them in the proper sequence.
17
White Paper
In version 7.6.04 or later, make sure the Enable Lifespan check box is selected on the Connection Settings page of the Mid Tier Configuration Tool (Figure 1-11). If a sticky bit is not set, the load balancer distributes sessions from any web server to any AR System server, including a recovered or new AR System server (Figure 1-12).
WS1 Idle sessions from any web server that were routed to AR3 will remain open
ARS1
WS2 WS3
ARS2
ARS3 fails
To avoid problems from idle connections in version 7.6.04 or later, you can configure the fields in the Connection Pool Settings section on the Connection Settings page in the Mid Tier Configuration Tool (Figure 1-15). The Idle Connections per Server setting allows you to limit the number of idle connections per server. The Connection Timeout field allows you to specify how long the connections exceeding that limit will remain open before being terminated. For more information, see the BMC Remedy Mid Tier Guide. Lowering the Idle Connections per Server value minimizes the number of idle connections that can stay open until their sessions ends. If the Idle Connections per Server field is set to 0, then the connection pool will be closed when all connections are closed.
18
Lowering the Connection Timeout value minimizes the time that idle sessionbased connections can stay connected before being closed, resulting in better load redistribution. The number of current idle connections is determined by whether the Connection Time or Connection Lifespan setting is first met.
Figure 1-15: Connection Pool Settings on the Connection Settings page of the Mid Tier Configuration Tool
Configuring the Connection Pool Settings allows idle sessions to be used again in a timely manner. The AR System server end users see no change in behavior because their connections are not dropped.
Load balancing between the clients and web servers without setting a sticky bit
If the web servers in your AR System were installed in a cluster (with fail-over enabled), then setting the sticky bit on the load balancer between the clients and web servers is not needed.
19
White Paper
20
circuit VLAN10 ip address 172.23.32.15 255.255.248.0 circuit VLAN20 ip address 172.23.41.5 255.255.255.0
!************************** SERVICE ************************** service www-server1 ip address 172.23.41.15 active service www-server2 ip address 172.23.41.16 active
!*************************** OWNER *************************** owner sample content rule1 add service www-server1 add service www-server2 vip address 172.23.33.95 active
For information about how to configure a load balancer with the Cisco CSS, see the Cisco Systems website at http://www.cisco.com. The following documents are relevant to load-balancer configuration: Basic CSS Load Balancing Configuration, Document ID 12557,
http://www.cisco.com/en/US/products/hw/contnetw/ps792/ products_configuration_example09186a008009438d.shtml
NOTE
Website addresses change frequently. If you have difficulty finding these documents, go to http://www.cisco.com and navigate to the Documentation pages.
21
White Paper
22