CommuniGate Pro Best Practices


June 11, 2007

“fake” domains for use in testing: Page 2 of 12 . Following is a simplified overview diagram of a CommuniGate Pro Dynamic Cluster: Testing Environment Prior to installing and configuring CommuniGate Pro. This would include the following: 1. DNS environment: a good. it is recommended to setup an environment for testing.Best Practices: Benchmarking Introduction This document is provided by CommuniGate Systems Services as a helpful Best Practices guide to benchmarking a CommuniGate Pro Dynamic Cluster. fast DNS server (such as BIND 9) providing internal-only.

at a minimum. free tool The benchmark manager. or others c. 2-4 is sufficient. 3. A Network File System (NFS) on a Network-Attached Storage (NAS) 2. and sinks can run on one box or many. A Cluster File System (CFS) on a Storage-Attached Network (SAN) System Layout Every CommuniGate Pro Frontend server performs all of its high-I/O operations in /var/CommuniGate. SMTP sink – the postfix smtp-sink is a good. This directory may be placed on a high-performance disk system for best performance.html Dynamic Cluster Overview The foundation building block of the CommuniGate Pro Dynamic Cluster is the “Shared File System” mounted on all CommuniGate Pro Backend Servers. Generally speaking. It is critical to note that this file system must be shared (read and written) by multiple servers concurrently.lan: your CGP “local domain” Remote. could also function as other roles b. Load generators: the test environment will include. Tuning: Depending on your test platform.Best Practices: Benchmarking • • Example. The Shared File System can be of one of two types: for relaying outbound messages to. including the load generators. the following systems for the benchmark load generators: a. If located Page 3 of 12 . The Shared File System represents one or more file systems which are simultaneously accessible by all Backend Servers. Benchmark manager – the main testing box.communigate. Please reference the following “Scalability” page of the CommuniGate Pro Guide for recommendations on tuning: http://www.lan: a remote domain. which is called the “BASE” directory. Improper DNS setup will greatly decrease your test results. some tuning may be required. 2. It is critical that the DNS zones used in these tests include proper Arecords and PTR-records for all systems in the benchmark environment. Load generator – SPECmail or mstone. load generators.

whether on NFS or when using a Cluster File System on a SAN. each system must have its own separate directory.Best Practices: Benchmarking anywhere except /var/CommuniGate. /var/CommuniGate CANNOT be shared across multiple systems – if locating /var/CommuniGate on the SAN. then /var/CommuniGate should be a symbolic link to the real location. Here are two example layouts then: Layout #1: Configuration directory on local disk. Every CommuniGate Pro Backend server uses two main directory locations: /var/CommuniGate /var/CommuniGate/SharedDomains /var/CommuniGate provides the configuration functions for each system. and is generally stored on the local disk (although. can be placed on the NAS or SAN for some highest-level performance requirements). SharedDomains on NAS/SAN: System1: /var/CommuniGate System1: /data (Shared File System mount) System1: /var/CommuniGate/SharedDomains -> /data/SharedDomains (symbolic link) System2: /var/CommuniGate System2: /data (Shared File System mount) System2: /var/CommuniGate/SharedDomains -> /data/SharedDomains (symbolic link) Layout #2: Both the configuration directory and the SharedDomains on NAS/SAN: System1: /data (Shared File System mount) System1: /var/CommuniGate -> /data/CommuniGate-system1 (symbolic link) System1: /var/CommuniGate/SharedDomains -> /data/SharedDomains (symbolic link) System2: /data (Shared File System mount) System2: /var/CommuniGate -> /data/CommuniGate-system2 (symbo2ic link) System1: /var/CommuniGate/SharedDomains -> /data/SharedDomains (symbolic link) Page 4 of 12 . A symbolic link should be used to link this location to other real directory locations. However. /var/CommuniGate/SharedDomains must always be located on the shared filesystem. All Backend servers must have access to this shared filesystem.

This domain name entry MUST match the Dynamic Cluster license. you want to configure your system something like the following steps.html Once installed. and should be created securely.example. Under “Settings” section: 4. All paths below are “clicks” within the CommuniGate Pro WebAdmin GUI. please remember to install all licenses on all servers. this may be filled in correctly by default. and once you’ve connected to the WebAdmin Interface. you will be prompted to enter a new “postmaster”” on each host (or similar. 5. for best performance. Domain Name Settings->General->Domain Name Set to “<hostname>.com”. you log into the WebAdmin Interface (GUI) on each system at this location: http://<hostname>:8010 or https://<hostname>:9010 Upon initial startup of CommuniGate Pro. such as “*. 6.example.Best Practices: Benchmarking Installation Please reference these basic installation instructions here for specifics on installing on your specific platform: http://www. if using a different domain). CommuniGate Pro Cluster Configuration Once you've gotten this far. This is important. Depending on your systems’ true host and domain names.communigate. Licenses Settings->General->Licenses Install all license keys on all CommuniGate Pro This password has complete administration rights to each system. Cluster IP Addresses Settings->General->Cluster Add all Frontend and Backend IPs on all hosts: Example: Cluster Members Page 5 of 12 . 9.communigate. Start all servers Page 6 of 12 .html For every server you are For Backends: #### Startup.168.1.1. Add a “Startup” file on each For Frontends: #### Startup.3] Backend Server Addresses: 192.168.3_____________________________ 192.1. Additional tuning recommendations for various operating systems for CommuniGate Pro are always available in the “Scalability” section of the CommuniGate Pro manual.Best Practices: Benchmarking This Server Cluster Address: On restart Active [first IP_____] [192. you will need to add this file (default location): /var/CommuniGate/ SUPPLPARAMS=”—ClusterBackend --DefaultStackSize 131072 \ --useNonBlockingSockets --closeStuckSockets \ --CreateTempFilesDirectly 10” ulimit –n 16384 #### end SUPPLPARAMS=”—-ClusterFrontend --DefaultStackSize 131072 \ --useNonBlockingSockets --closeStuckSockets \ --CreateTempFilesDirectly 10” ulimit –n 16384 #### end Startup.1_____________________________ 192.4_____________________________ ________________________________________ ________________________________________ Frontend Server Addresses: 192.1. available by default here: http://www. Stop the CommuniGate Pro servers on each ________________________________________ ________________________________________ 7.168.

and will take affect for all servers within the cluster. you can then easily login to the WebAdmin Interface for all your systems).d/CommuniGate start 10. then use the system links on the MONITORS->CLUSTER page. A Cluster Controller is required before any other servers can join the cluster: # /etc/init. If the restart was successful. 1. Cluster settings (as part of any “Cluster-Wide” setting) only need to be set on one server. Server-Wide and Cluster-Wide Client IPs Be sure to add your “local” IP ranges to the Cluster-Wide Client IPs Settings->Network->LANIPs->Cluster-Wide 2.Best Practices: Benchmarking Be sure to start at least one Backend server first. as the first Backend server automatically becomes the “Cluster Controller”. Queue Settings->Queue->Settings->Message Queue->Queue Foldering = 100 Settings->Queue->Settings->Message Enqueuer->Processors = 50 Settings->Queue->Settings->Message Dequeuer->Processors = 10 3. If you open the WebAdmin Interface on your Cluster Controller. This page will display the entire cluster: MONITORS->Cluster (Note: The Cluster monitoring page is also a great place to manage all of your systems from. the Cluster Controller will show in the log the rest of the service: http://<controller hostname>:8010 Monitor -> Logs -> Display Confirm that all systems join the cluster. DNS Resolver Page 7 of 12 . Tuning optimizations You must set these settings on all servers for most of these options.

and additionally can take advantage of single-instance message store (which especially benefits Enterprises. SMTP Receiving Settings->SMTP->Receiving->Channels=1000 Settings->SMTP->Receiving->Disconnect after: [5] errors and Deny Access for: [15 minutes] 6. 8. Educational facilities. Cluster-Wide Domain Defaults Users->Domain Defaults->Cluster-Wide->”Account Storage”>Foldering Method = Hashed 2 Levels Users->Domain Defaults->Cluster-Wide->”Account Storage”>Rename in Place =Yes Users->Domain Defaults->Cluster-Wide->”Account Storage”>Generate Index =Yes 9.different filesystems perform different with each. it would probably be best to set the HTTPU->Listener ports to listen on 80 for HTTP and 443 for HTTPS. LOCAL Settings->LOCAL->other=50 7. SMTP Sending Settings->SMTP->Sending->Channels=1000 Settings->SMTP->Sending->Channels/Host->Other=50 5. TextMailbox or MaildirMailbox One can use with two options . MaildirMailbox format uses more small files and many more filesystem metadata operations.Best Practices: Benchmarking Settings->Obscure->Domain Name Resolver->Concurrent Requests=25 4. and ASPs Page 8 of 12 . if testing the WebUser Interface. Access Settings->Access->POP=1000 Settings->Access->IMAP=1000 Settings->Access->PWD clients=50 # (set PWD channels to at least 5 * <number of systems in cluster>) Also note here. rather than the default 8100/9100.

SPECimap. TextMailbox format uses bigger mailbox files but these files are indexed for high performance. and SPECsip committees. The default is and a voting member of the SPECmail. which are configured in the “Settings->General->Other” settings page: Settings->General->Other->”TextMailbox Manager”>Concurrently used large buffers=”50” Settings->General->Other->”TextMailbox Manager”->Index Mailboxes larger than=”1024K” (1 Mbyte) Optionally: Objects->Domains->Account Defaults->Server-Wide->New Mailboxes=TextMailbox (or MailDirMailbox) Objects->Domains->Account Defaults->Cluster-Wide->New Mailboxes=TextMailbox (or MailDirMailbox) Domain Setup Finally. We provide some details of one of these. it is recommended to reduce the default mailbox index size and increase the concurrent large buffers. and voice services. here. A combination using MaildirMailbox in the INBOX and TextMailbox is also possible – this method can be easily scripted in provisioning scripts.spec. and proceed with adding accounts. Domain setup is managed in the WebAdmin Interface at the location: Users->Domains Be sure to create all your domains as “Clustered Domains”.Best Practices: Benchmarking where one message may be sent to thousands of recipients). Benchmark Tools There are many tools available for benchmarking and testing e-mail. and in most cases it is recommended. SPECmail2001. As Page 9 of 12 . For most Cluster File Systems and NFS. CommuniGate Systems is a member of the SPEC organization "The Standard Performance Evaluation Corporation. SPECmail2001 SPECmail2001 is a standardized benchmark developed by messaging vendors and research organizations that measures mail server performance using a realworld workload. add your clustered domain(s). When using the “TextMailbox” (default) format. calendaring. at http://www. TextMailbox format may perform better due to less overall metadata file operations and good indexing.

These results provide a good overview of what a large SPECmail test looks like: http://www.5-million users.spec. The purpose of using this application in validation testing is that the initialization and benchmark test stages generate significant mail delivery and access load on the mail Practices: Benchmarking well. Since 2004. CommuniGate Systems Professional Services has regularly tested most new implementations with SPECmail2001 to ensure that the system is Page 10 of 12 .html The CommuniGate Systems Services department has used the SPECmail2001 application to test CommuniGate Pro implementations prior to going live in production. beating all previous records by 67% or more at 2. CommuniGate Pro is the current world-record holder in the SPECmail2001 benchmark.

disk storage. The benchmark characterizes throughput and response time of a mailserver system under test with realistic network connections.sun. including but not limited to Linux. Generally speaking. The goal is to enable objective comparisons of mail server products. or stability issues Shared storage performance External authentication or performance problems with integrated applications Content filtering (anti-spam. These (minimum) two systems cannot be part of the CommuniGate Pro Dynamic Cluster. and the other acting as the SMTP sink for outbound delivery. Page 11 of 12 .2 or later).000 users. with an overall user count in the range of approximately 10.we highly recommend you download the latest version of Java from http://java. SPECmail2001 Overview from SPEC.Best Practices: Benchmarking prepared for deployment. routers. and that the hardware used is stable and capable of providing sufficient Other java-like implements (such as GNU Java) are not sufficient . it is capable of demonstrating that the initial design of the system is fundamentally solid. should be placed in a location to route connections through any relevant loadbalancers. at least two "load-generator" systems are required for SPECmail testing. and firewalls in order to verify not just the CommuniGate Pro system(s). SPECmail2001 is a standardized mail server benchmark designed to measure a system's ability to act as a mail server servicing email requests. While SPECmail2001 is not able to completely duplicate the 'real world' environment. and client workloads. Solaris. The SPECmail2001 application should be capable of running on any Sun Javasupported platform.000. The benchmark focuses on the ISP as opposed to Enterprise class of mail servers. one acting as the SMTP/POP load generator.000 to 1. and Mac OSX.4. and runs best on a relatively recent version of Sun Java (1. Windows. anti-virus) and plugin performance SPECmail Preparation SPECmail2001 is a Java-based application. but also the entire production environment. The types of problems which SPECmail2001 can discover includes the following: • • • • • • • Faulty or insufficient hardware DNS server errors or configuration problems Load Balancer mis-configuration Network performance. based on the Internet standard protocols SMTP and POP3. and for best results.

Best Practices: Benchmarking SPEC® and the benchmark name SPECmail® are registered trademarks of the Standard Performance Evaluation Corporation. Page 12 of 12 . visit http://www. 2005.spec.'s benchmark that measures a system's ability to act as a mail server servicing email requests. Competitive benchmark results stated above reflect results published on> as of August 15. based on the Internet standard protocols SMTP and POP3. For the latest SPECmail® benchmark <http://www. The comparison presented above is based on Standard Performance Evaluation

Sign up to vote on this title
UsefulNot useful