Stating the obvious
Presenting load testing results to tell the story
If you've watched A developer's guide to load testing, you will have seen the graph below already. It shows the results from load testing a website written in .NET with the Apache JMeter load testing tool. Along the bottom of the graph are the actions that make up the load test script, while the vertical axis is the response time in seconds. Each line represents the results from a different test run with a different number of concurrent users.
The interesting thing about this graph is that the average response time for any page request is just a few seconds, with the exception of the spike around a third of the way in. This spike is caused by the .NET web tier having to aggregate information from several calls to a backend system, where some resource locking bottlenecks the incoming requests. From analysing the load testing environment during the test run, we were able to ascertain where the hotspot was because (a) the CPU load for the web-tier dropped off when this call was taking place and (b) we could see the requests being queued up on the backend system. From a technical perspective then, load testing is a fantastic way to find and highlight bottlenecks in the overall architecture, providing an opportunity to present some real evidence about the performance degradation and where the hotspots are.
Furthermore, in this example there was a general feeling that this particular action was slow, but the evidence was only empirical. People had experienced this slowness but had never actually measured it before. Kevin made a comment during the last user group where he mentioned that the architect sits between the technical and business spaces, spanning the two and bringing them together. Raw results from a load test are one thing, but presenting the data in a way that makes sense to non-technical people can tell a completely different story. Sometimes your load testing results state the obvious, but sometimes they don't.
