We go into some detail about these steps in further sections of this document. It is important also to
consider the data from more than one tool in the analysis process and to cross-check the output
between them. For example, the thread information produced by Glance/gpm should be consistent
with the thread data seen when a “kill –3 <pid>” is issued to the JVM process.
It is also very useful to identify a single “unit of work” or “user transaction” that has a completion
time and is easily measurable, in order to gauge progress in performance tuning. This might be the
number of transactions per second put through a collection of software.
High throughput levels are not always the source of a performance issue. Many production systems
run with a high volume of transactions flowing through them constantly. The real question for the
performance engineer is “Where are the bottlenecks in the process and how can they be acted on ?”
In order to find these bottlenecks, it is sometimes necessary to place a load on the system which
represents the type of loading at peak user levels, when the system will be very busy. There are tools
for achieving this high loading on web based applications, which are given in the Tools section at the
end.
avoid making an uninformed guess at the problem cause
It is very tempting, especially when a project team is under pressure to solve a problem, to make an
initial stab at the problem cause without analysis and measurement. This is to be avoided. More often
than not, the initial guess we make is misleading.
Instead, it is recommended that the performance data be collected from a variety of tools
(Glance/gpm, HPjmeter, HPjtune, sar, the JVM options –Xverbosegc and –Xeprof and others) and
once the data from these tools is analyzed and understood, then a reason can be given for the
symptoms seen. This analysis takes some time – and typically the first hours of the performance
engineer’s time is spent in gathering and analyzing this data before making a change to one of the
performance-influencing variables of the system.
It is very important to have a notebook or journal of some kind, into which all the changes one
makes are recorded as we carry them out with their resulting effect on performance. The
performance engineer will be tempted to make several changes at once to the system, a course of
action to be avoided. Keeping track of the changes applied in written form defends the engineer
against the confusion that can easily arise from multiple changes, multiple results and many outside
influencing factors.
testing a smaller representative sample of a big application
A standard troubleshooting technique in computer programming is to boil the problem down to the
smallest possible piece of code that exhibits the symptom. This is fraught with danger, however, in
performance analysis, as the small sample can frequently behave quite differently to the real
production system. It is recommended that as much as possible, do the performance analysis on the
© 2001 HEWLETT-PACKARD COMPANY 5
