SugarCRM Performance Test Report

DOCUMENT INFORMATION :
Title : Performance Testing Report Scalability Test

Remarks Version Author Date of creation

: Draft version : 1.0 : : 26 Nov 2010

DOCUMENT HISTORY :
Version By 1.0 Prepared Date 26Nov2010 Checked By Date Authorized By Date

INDEX
Purpose.......................................................................................................................................................3 1. Introduction............................................................................................................................................4 1.1. Objective...............................................................................................................................................4
2

1.2. Application...........................................................................................................................................5 1.3. Reference Documents..........................................................................................................................5 2. Test Setup................................................................................................................................................6 2.1. Hardware.............................................................................................................................................6 2.2 Software................................................................................................................................................6 3. Test Scenario ..........................................................................................................................................7 3.1. Description...........................................................................................................................................7 3.2. Scenario Deployment...........................................................................................................................7 3.3. Results..................................................................................................................................................7 4. Recommendations.................................................................................................................................14 4.1. Recommendations..............................................................................................................................14

Executive Summary
Purpose
The purpose of this document is to report the execution results of 22 scenarios of Sugar CRM Application when subjected to Scalability testing. Objective

The objective of the test is to verify the performance of SugarCRM application when a concurrent user load of 220 Vusers are accessing the application and to see the response times for specific scenario at the client side and the utilization(CPU, Memory etc..) on the server side. Software Under Test SugarCRM 4.5.1G (Build 1003)

1. Introduction
1.1. Objective
The objective of the project is to verify the performance of the SugarCRM application for 10 concurrent users and to check the response times, Hits per second, Throughput for different transactions and resource usage of server on which the Sugar CRM application is deployed.

1.2.

Application
SugarCRM is an open source CRM application. The application uses Apache as web server and MySQL as backend database.

1.3.

Reference Documents
The below mentioned are the referred documents, Business flow_SugarCRM Document

2. Test Setup
Below mentioned are the Hardware and Software components used during the test execution

2.1.

Hardware
The hardware that was used for testing including machine names, configuration, and role of the machine (application server/web server/ DB server) is detailed in following tables SI.No
1

Machine name
IP Address 10.188.144.75

Configuration
CPU RAM 2140 @1.60GHZ 2GB

Purpose
Web server ,DB server

2.2

Software
Below mentioned are the lists of softwares used including the major and minor versions

Machine Name

S/W Installed SugarCRM, ApacheServer, MySQL Load Runner

Version SugarCRM Version: 4.5.1g(Build 1003) Apache Server Version: 5.5.17 MQSQL: 5.1 9.5

IP Address 10.188.144.75 IP Address 10.188.144.75

3. Test Scenario
3.1.

Description
The testing scenario consist of two business process in the CRM application as given below, Scenario - Calendar Navigation

Specific details about the business process can be found in the Performance Requirements document.

3.2.

Scenario Deployment
Scenario Name Calendar Navigation Business Flow 1) User Log into the Sugar CRM application. 2) Click on Calendar tab in SugarCRM homepage. 3) In the Shortcuts menu of the Calendar module, click Schedule Meeting. 4) You will get the next page enter the details and click on Save button. 5) User Logs out the application. % of total load 100 Total Load 10

Scenario ID

3.3.

Results
The performance test results of Sugar CRM application can be divided into Client side monitors and Server side monitors. The client side monitors are End-to-end transaction monitors - Provide end-user response times, hits per second, transactions per second. The server side monitors provide hardware, network and operating system performance metrics, such as CPU, memory and network throughput. Both the client side and server side monitors are discussed in the result analysis.

Client side monitors:

Load Vs Response Time

Transaction Response Time:

The Transaction Response time graph can be shown as the response time of transactions in milliseconds (y-axis) as a function of the total average time taken in the scenario (x-axis). The graph we have shown is bar chart to show the average response time per web page in sorted descending order. The pages at the left hand side are the pages taking more response time and pages towards right are the ones with low response time. The below graph represents the Maximum, Minimum & Average response times for each transaction,

Observations: The above graph represents that the transactions Login and Log out has more response time i.e. approx 82.379 sec & 59.802 sec. The transaction Schedule_meeting is having less response time of 18.578 sec.

Load Vs Hits per Second Hits per Second

The Hits per Second graph shows the rate at which web pages are being retrieved from the Web server (y-axis) as a function of the elapsed time in the scenario (x-axis). The below graph represents the number of hits per sec when maximum load of 10 Vusers applied during the test execution.

Observations: The above graph represents that during the test execution when load of 10 vusers is applied the hits/sec is very low. Hits per second for the 10 Vusers load is more when the absolute time is between 21:02:00 to 21:05:00.

Load Vs Time The Load Vs Time graph will show the number of users which are active and load is applied then gradually decreases with absolute time.

Observations:

10

The above graph shows that 10 Vuser steady load is applied between the time intervals from 0 to 21:03:50 and gradually it is decreasing.

Load Vs Success & Failed Transactions: The below graph represents the total number of transaction passed and failed for the load of 10 Vusers.

Observation: The graph shows the transactions Login_Transaction & vuser_init transaction failed for maximum of 32 times.

11

Transaction Name Action_Transaction Trans_Calendar_Navigation Trans_Login Trans_Logout Trans_Schedule_meeting vuser_end_Transaction vuser_init_Transaction

Fail 8 0 32 0 8 0 32

Pass 2 10 10 2 2 2 10

Server side monitors:

Load Vs Processor utilization (% Processor Time) The below graph represents Load against the resource usage (%Processor Time & %User Time) for SugarCRM application.

Load Vs Processor utilization (%Processor Time)

250 200 Number of Vusers 150 100 50 0
8: 50 8: :25 51 P 8: :55 M 53 P 8: :25 M 54 P 8: :55 M 56 P 8: :25 M 57 P 8: :55 M 59 P 9: :25 M 00 P 9: :55 M 02 P 9: :25 M 03 P 9: :55 M 05 P 9: :25 M 06 P 9: :55 M 08 P 9: :25 M 09 P :5 M 5 PM

120 100 80 60 40 20 0 Processor Time (%)

vusers % Processor Time

Absolute time (sec)

12

Observations: The above graph represents that the CPU utilization was very less in the initial state when there is no load applied. The CPU utilization was maximum i.e. 100% when the application was subjected to 220 users.

Load Vs Memory The below graph represents the Memory Usage against Load.

Load Vs Memory
250 Number of Vusers 200 150 100 50 0 8:50:25 PM 8:51:25 PM 8:52:25 PM 8:54:25 PM 8:55:25 PM 8:56:25 PM 8:57:25 PM 8:58:25 PM 8:59:25 PM 9:00:25 PM 9:01:25 PM 9:02:25 PM 9:03:25 PM 9:04:25 PM 9:05:25 PM 9:06:25 PM 9:07:25 PM 9:08:25 PM 9:09:25 PM 8:53:25 PM 1800 1600 Resource Usage (Megabytes) 1400 1200 1000 800 600 400 200 0

Absolute time (sec) vusers Available MBytes Committed Bytes

Observations: The graph represents in the initial state when the Vuser load is zero the committed & available bytes are in steady state and its stable over the period of time. The available Mbytes starts decreasing and Committed Bytes starts increasing when the Vusers starts hitting the server. Throughout the test execution 400 MB memory was available.

13

4. Recommendations
4.1.

Recommendations
The test should be run for a more number of Business flows.

14