CSC 422

Computer System Performance Evaluation

Performance analysis

Performance analysis as applied to experimental computer science and engineering

should be thought of as a combination of measurement interpretation and
communication of a computer system speed or size, sometimes referred to as its
capacity. It is important to recognise that we need not necessarily be dealing with the
entire systems. It is necessary to analyse only a small portion of the system
independent of the other components. For instance, you may be interested in
studying the performance of a certain computer systems network interface
independent of its size, memory or type of processor. One of the signs of an expert
computer performance analyst is that he or she can use interaction to determine the
performance effect due only to a particular component. One of the most important
part of computer analyst can be figuring out how to measure the necessary data. A
large dose of creativity may be needed to develop good measurement techniques
that affect the system as little as possible, while producing accurate and reproducing
result. After the necessary data have been gathered, the analyst must interpret the
result using appropriate techniques. Finally, even measurement interpreted in a
statistically appropriate fashion are of no practical use to anyone unless they are
communicated in a fair and consistent manner.

Common Goal of Performance Analysis

The goals of any analysis of the performance of a computer system or one of its
components will depend on a situation and its skills, interest and abilities of the
analyst. The following are the common goal of performance analysis

1. Compare alternatives: when purchasing a new computer system, you may be

confronted with several systems from which to choose, furthermore, you may
have several different options that may hinder cost and performance, such as
the size of the main memory, the number of processors, type of network
interface, size and number of disks drive, type of system software and so on.
The goal of the performance analysis task in this case is to provide
quantitative information about which configuration are best under specific
condition
2. Determine the impact of a feature: in designing new systems or in upgrading
existing systems, you often need to determine the impact of adding, or
removing a specific feature of the system. For instance, the designer of a new
processor may want to understand whether it makes sense to add an
 additional floating-point execution unit to the micro architecture, or whether
the size of the on-chip cache should be increased. This type pf analysis is
often referred to as before-and-after comparison. Since only one well defined
component of the system is changed.
3. System tuning: the goal of performance analysis in system tuning is to find the
set of parameter values that produces the best overall performance. In time
sharing OS, it is possible to control the number of processes that are allowed
to actively share the processor. The overall performance overseen by the
system users can be substantially impacted both by this number and by the
time quantum allocated to each process. Many other system parameters such
as this and network buffer cycle can also significantly impact the system
performance.
4. Identify relative performance: the performance of a computer system typically
has this only in the content of its performance relative to something else such
as another system or another configuration of the same system. The goal in
this situation may be to quantify the change in performance relative to history
of the previous generation of the system. Another goal may be to quantify the
performance relative to a custom expectation or to a competitor system.
5. Performance debugging: debugging a program for correct execution is a
fundamental breaking size of any application program. Once the program is
functionally correct, the performance analysis task becomes one of the
findings performance problem. i.e. the program now produces the correct
result but it may be much slower that desired. The goal of the performance
analyst at this point is to apply the appropriate tools and analysis techniques
to determine why the program is not meeting performance expectation. Once
the performance problem is identified, it is to be known that the result will
meet expectation.
6. Set expectation: users of computer systems may have some ideas of what the
capability of the next generation of a line of computer system should be. The
task of performance analyst in this case is to set the appropriate expectation
for what a system is actually capable of doing.

Solution to some techniques that are wrongly taken

When one is confronted with a performance analysis problem, there are 3

fundamental techniques to find the desired solution. These are

1. Measurement of existing systems

2. Simulation
3. Analytical modelling
Actual measurement generally provides the best result since given the necessary
measurement tools, no simplifying assumptions need to be made. A common goal of
performance analysis is to analyse how the performance of a system changes as
certain parameters are varied. Evaluating the performance impact of varying the
speed of the main memory system is simply not possible in most real systems.
Measuring some aspects of performance on an actual system can be very time
consuming and difficult. Thus, while measurement of real system may provide the
most compelling result, there are inherent difficulties and limitation produce a means
for other techniques. A simulation for a computer system is a program written to
model important features of the system being analysed. The cost of a simulation
includes both the time and effort required to write and debug the simulation
program, and the time required to execute the necessary simulation. The primary
limitation of a simulation-based analysis is that it is impossible to model every small
detail of the system being studied. Consequently, simplified assumptions are
required in order to make it possible to write the simulation program and allow it to
execute in a reasonable amount of time. This simplifies assumptions then limit the
accuracy of the result by lowering the fidelity of the model compared with how an
actual system will perform. Simulation enjoys tremendous popularity for computer
system analysis due to its high degree of flexibility, and its relative ease of
implementation. The 3rd technique is analytical modelling.

Analytical model is a mathematical description of the system compared with the

simulation measurement of a real machine, the result of an analytical model tends to
be more less believable and more less accurate. A simple analytical model can
provide some quick insight into the overall behaviour of the system or one of its
components. This insight can then be used to help focus a more detailed
measurement or simulation experiment. Analytical models are also useful in
providing at least a coarse level of validation of a simulation or measurement i.e
analytical model can help confirm whether the result produced by a simulator or the
value measured on a real system appear to be reasonable.

Summary of the solution techniques

1. Flexibility: this is an indication of how easy it is to change the system to study

different configuration. The cost corresponds to the time, efforts and money
necessary to perform the appropriate experiment using each technique.
2. Believability: The believability of a technique is high if a knowledgeable
individual has a high degree of confidence that the result produced using that
technique is likely to be correct in practise. It is much easier for someone to
believe that the execution time of a given application program will be within a
certain range when one can demonstrate it on an actual machine
3. Accuracy: indicates how closely the solution obtained when using that
technique corresponds to the result that will be obtained on a real system. The
choice of a solution technique depends on the problem being solved. One of
the skill that must be developed by a computer system performance analyst is
determining which tech is the most appropriate in a given situation.

Comparison of the performance Analysis solution Technique

Solution Technique

Characteristics Analytical Simulation Measurement

Modelling
Flexibility High High Low
Cost Low Medium High
Believability Low Medium High
Accuracy Low Medium High
1. Performance Dimension Order: that is 2 or more dimensions on
a performance instrument follow or closely follow each other
and both describe or relate to a similar quality. The evaluator
then rates the second dimension similarly to the first because of
their proximity.
2. Initial Impression (This is rating based on first impression):
Failing to recognize the most consistently demonstrated
behaviour during the entire evaluation period.
3. Spill over effect: This is allowing past performance evaluation
rating to unjustly influence current rating; cost performance
rating, good or bad, result in the similar rating for the current
period although demonstrate behaviour does not deserve the
rating good or bad.
4. Status effect: This is overrating clients in higher level job or jobs
held in high esteem and underrating clients in lower level jobs or
jobs held in low esteem.
5. Same as me: This is giving a rating higher than deserved because
the person has perceived qualities or characteristics similar to
those of the evaluator.
6. Different from me: This is giving a rating lower than deserved
because the person has quality or characteristics dissimilar to the
evaluator.