performancewiki.com - automate performance testing with STAF and JMETER Page 3

performance / tuning tips. to the point.

del.icio.us performancewiki.com Latest Items

Automate Performance Testing With STAF and JMETER - Part 3 / 4

Step 3. Deploy Resource Monitoring Tools (Unix, Linux, and Windows)

One most important aspect of performance testing is to learn how your application behaves under heavy workload. If your application scales (simply uses more cpu, memory, network bandwidth, and disk io when load increases without slowing down), system resources will be increasingly used accordingly. If your application doesn't scale, well, it is to hang, to slow down, and to crash, while system resources may still be plenty free.

Of course, each operating system reports resource usage though native commands. For example, unix / linux has sar, vmstat, netstat, iostat, top, topas, etc. But those require human interaction - eye balls to monitor them - so they are not easy to automate.

The following tools are excellent resource monitoring programs that not only allow you to monitor live but also log to text files for post-processing. For Windows 2003 Server and later, it is called "perfmon", by Microsoft; for Unix and Linux, it is called "nmon", by IBM.

Perfmon: Built-in Windows 2003 Server. Usage and info at Monitoring performance from the command line.

nmon: IBM-developed free tool available for AIX and Linux. See nmon performance.

To start perfmon to configure a customized logging, type 'perfmon' at command windows prompt:

To start and stop perfmon remotely (for automation later), from a windows machine, do:


c:\logman start perf -s webserver15
		
...run your test...
		
c:\logman stop perf -s webserver15

Sample perfmon config screen:

(1) Create a "Counter Logs"; (2) Name it, here it is named 'perf'; (3) Set properties and add resources to monitor and log; (4) simply choose all counters and all instances so you will have all data to select a subset from at post-processing. And select text file format.

Sample perfmon log file in text format, used to extact averages in post-test processing. Labels for counters are in double-quotes, and so are their corresponding values. Each iteration (say every 15 seconds) logs one line:

nmon is easy to start, simply use command line (example assume aix 5.3, logs every 15 seconds, for 240 loops, or 1 hour):


% nmon_aix53 -f -s 15 -c 240

Sample nmon log file in text format, used to extact averages in post-test processing (example shows one iteration, labels for the colunmns are found in the beginning of each nmon log file, too much to list here):

ZZZZ,T0009,14:42:32,25-MAY-2006
CPU01,T0009,18.5,15.5,7.0,59.0
CPU02,T0009,18.0,17.3,5.1,59.6
CPU03,T0009,17.5,19.3,4.3,58.9
CPU04,T0009,17.7,18.3,6.1,57.9
CPU05,T0009,18.3,17.5,4.9,59.4
CPU06,T0009,19.3,16.5,5.5,58.6
CPU07,T0009,18.5,17.3,4.9,59.2
CPU08,T0009,19.5,14.7,4.7,61.2
CPU09,T0009,19.5,15.5,7.6,57.5
CPU10,T0009,17.7,18.0,9.0,55.4
CPU11,T0009,19.6,16.3,7.9,56.2
CPU12,T0009,18.9,14.4,7.5,59.2
CPU13,T0009,19.9,16.2,8.1,55.8
CPU14,T0009,19.8,15.6,7.7,57.0
CPU15,T0009,20.7,15.7,7.4,56.2
CPU16,T0009,19.8,15.6,7.0,57.6
CPU17,T0009,19.9,16.1,4.5,59.6
CPU18,T0009,20.5,14.8,4.1,60.7
CPU19,T0009,18.6,17.4,3.7,60.3
CPU20,T0009,19.6,18.6,2.6,59.2
CPU21,T0009,21.3,15.0,3.7,60.0
CPU22,T0009,20.6,14.7,5.2,59.6
CPU23,T0009,19.0,17.9,5.1,58.0
CPU24,T0009,20.6,15.3,4.5,59.7
CPU_ALL,T0009,19.3,16.4,5.7,58.6,,24
MEM,T0009,90.3,100.0,147874.0,384898.0,163840.0,385024.0
MEMNEW,T0009,7.2,0.3,2.3,90.3,5.5,3.6
MEMUSE,T0009,0.3,19.2,76.7,960,1088
PAGE,T0009,7460.2,347.1,1523.3,0.0,0.0,0.0,0.0,0.0
PROC,T0009,11.00,1.80,123637,365105,624,2388,0,1,118123,893
FILE,T0009,0,6462,0,3258353,6060205,0,0,0
NET,T0009,0.2,2378.5,0.0,0.3,0.0,0.0,
NETPACKET,T0009,2.7,7895.4,0.0,1.8,4579.1,0.0,
NETERROR,T0009,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,
IOADAPT,T0009,381.7,2083.4,224.4,0.0,0.0,0.0,0.0,4851.9,966.3,1434.6
JFSFILE,T0009,25.4,5.5,16.1,1.0,30.1,6.2,0.6,61.3,4.1,37.0,4.1,4.1
JFSINODE,T0009,0.5,0.0,1.0,0.0,0.0,0.9,0.1,0.6,0.0,0.0,0.0,0.0,0.0
DISKBUSY,T0009,0.1,43.5,3.4,0.0,36.4,0.0,0.0,18.1,9.8,0.0,0.0,0.0
DISKREAD,T0009,0.0,513.1,275.4,0.0,154.6,0.0,0.0,436.4,485.1,0.0
DISKWRITE,T0009,1.1,2082.6,50.6,0.0,430.5,0.0,0.0,183.9,68.2,0.0
DISKXFER,T0009,0.3,124.8,19.9,0.0,97.0,0.0,0.0,94.8,59.2,0.0,0.0
DISKBSIZE,T0009,4.0,20.8,16.4,0.0,6.0,0.0,0.0,6.5,9.3,0.0,0.0,0.0

With these logs setup and collected, you can simply use PERL to parse the columns, calculate averages, or even 90 percentile (remember PERL's calc_percentile(0.9,$data) function?), and display the data anyway you want, even in HTML (just add tags!).

Now we have tools to automatically capture CPU, Memory, Network, and Disk IO usage, next let's use STAF to tie everything together.

Page 1 Page 2 Page 3 Page 4