Advise on sar output
 

Hi,

I have a server running on Linux ES 3 and with applications such as squid, apache, named, rsync and plug-gw.

Upon generating sar and sar -q, the runq-sz are quite high but the CPU loading is low.

Can anyone advise is the server CPU bound???

Thank you.

sar

%user %nice %system %iowait %idle
7:05:01 0.25 0 0.51 12.91 86.33
7:10:01 0.27 0 0.36 0.34 99.03
7:15:01 0.28 0 0.48 0.02 99.21
7:20:01 0.37 0 0.66 0.24 98.73
7:25:01 0.44 0 0.95 0.3 98.31
7:30:01 0.35 0 0.84 0.02 98.79
7:35:01 0.49 0 1.22 0.3 97.98
7:40:01 0.54 0 1.26 0.27 97.92
7:45:01 0.56 0 1.86 0.12 97.46
7:50:01 0.7 0 2.49 0.35 96.46
7:55:01 0.87 0 3.15 0.24 95.74
8:00:01 0.78 0 3.89 0.18 95.15
8:05:01 1.1 0 5.15 0.38 93.37
8:10:01 1.13 0 5.89 0.28 92.71
8:15:01 1.09 0 5.57 0.14 93.2
8:20:01 1.21 0 6.59 0.26 91.94
8:25:01 1.29 0 6.81 0.29 91.61
8:30:11 1.51 0 7.73 0.18 90.59
8:35:11 1.38 0 7.55 0.44 90.62
8:40:01 1.17 0 7.03 0.14 91.66
8:40:11 4.38 0 23.19 0 72.43
8:45:11 1.32 0 7.62 0.36 90.71
8:50:11 1.37 0 7.94 0.31 90.38
8:55:11 1.23 0 7.71 0.13 90.93
9:00:01 1.24 0 7.46 0.11 91.19
9:00:11 7.54 0 25.46 5.66 61.34
9:05:11 1.39 0 9.55 0.3 88.75
9:10:11 1.34 0 7.53 0.1 91.04
9:15:11 1.32 0 8.62 0.42 89.64
9:20:11 3.56 0 8.57 0.35 87.52
9:25:11 26.5 0 8.8 0.19 64.51
9:30:11 26.59 0 8.54 0.14 64.73
9:35:11 26.59 0 9.05 0.24 64.13
9:40:11 26.57 0 9.56 0.19 63.67
9:45:11 26.48 0 8.72 0.2 64.61
9:50:11 26.28 0 7.3 0.23 66.18
9:55:11 26.29 0 8.12 0.09 65.49
10:00:11 26.48 0 9.57 0.22 63.73
10:05:21 26.43 0 8.84 0.12 64.61
10:10:21 26.22 0 7.08 0.05 66.66
10:15:21 26.25 0 6.37 0.06 67.31
10:20:21 26.23 0 6.94 0.18 66.65
10:25:21 26.25 0 8.6 0.16 64.99
10:30:21 26.74 0 10.9 0.15 62.21
10:35:21 26.7 0 8.76 0.12 64.41
10:40:21 26.72 0 10.86 0.02 62.41
10:45:21 26.51 0 10.35 0.14 63
10:50:21 26.38 0 9.81 0.34 63.47
10:55:21 26.69 0 9.89 0.04 63.39


sar -q
runq-sz plist-szldavg-1 ldavg-5 ldavg-15
7:05:01 3 119 0.04 0.12 0.06
7:10:01 1 119 0 0.03 0.03
7:15:01 2 124 0.04 0.03 0.01
7:20:01 11 135 0.01 0.02 0
7:25:01 3 115 0.04 0.05 0
7:30:01 2 118 0.03 0.03 0
7:35:01 4 123 0.05 0.03 0
7:40:01 7 126 0.06 0.04 0
7:45:01 2 122 0.08 0.04 0
7:50:01 2 128 0.34 0.15 0.04
7:55:01 2 125 0.13 0.15 0.07
8:00:01 8 144 0.25 0.18 0.1
8:05:01 2 125 0.18 0.18 0.1
8:10:01 2 136 0.17 0.2 0.13
8:15:01 10 149 0.06 0.16 0.13
8:20:01 8 146 0.43 0.29 0.19
8:25:01 4 133 0.28 0.29 0.2
8:30:11 1 125 0.46 0.34 0.24
8:35:11 0 132 0.54 0.34 0.26
8:40:01 10 148 0.41 0.31 0.26
8:40:11 0 130 0.42 0.31 0.27
8:45:11 1 124 0.44 0.34 0.28
8:50:11 0 118 0.71 0.38 0.3
8:55:11 0 138 0.5 0.34 0.29
9:00:01 5 151 0.7 0.41 0.31
9:00:11 1 133 0.89 0.46 0.33
9:05:11 1 121 0.91 0.52 0.37
9:10:11 0 120 0.33 0.37 0.35
9:15:11 0 125 0.52 0.39 0.36
9:20:11 3 127 1.15 0.53 0.41
9:25:11 2 131 1.39 1.05 0.66
9:30:11 1 124 1.56 1.23 0.83
9:35:11 2 131 1.54 1.33 0.97
9:40:11 1 119 1.52 1.39 1.08
9:45:11 1 123 1.37 1.35 1.14
9:50:11 2 117 1.51 1.35 1.19
9:55:11 2 118 1.55 1.35 1.21
10:00:11 1 118 1.58 1.37 1.24
10:05:21 3 123 1.35 1.32 1.24
10:10:21 1 123 1.41 1.32 1.25
10:15:21 1 120 1.35 1.32 1.27
10:20:21 2 127 1.41 1.33 1.28
10:25:21 2 122 1.76 1.44 1.32
10:30:21 2 121 1.39 1.45 1.36
10:35:21 2 135 1.44 1.44 1.37
10:40:21 2 119 1.49 1.41 1.37
10:45:21 2 119 1.63 1.46 1.38
10:50:21 1 125 1.37 1.39 1.36
10:55:21 1 122 1.41 1.35 1.35

The amount of processes waiting for runtime is actually quite low (I think you formatted your listings wrong) but there are a few spikes where it goes up to 10, 11 but only a few correlate with kernel CPU spikes (and exactly one point where runq-sz + system + iowait line up). At 9:25 things go bezerk as userland applications start to hog CPU and don't relinquish their hold. I don't think the machine is CPU-bound, I think the real problem starts because you put all network-intensive applications with different requirements () on one machine.

thank you for the advise.....will look into separating the various apps to other servers is possible.....