One of my current projects involves me performance testing a third party Java system. In essence, it's a distributed/n-tier Java EE web application and we're hitting it with a high simulated load thanks to Apache JMeter. One of the things we found is that performance degraded quite dramatically as load increased, to the point where some requests could take anywhere up to a minute or two to be serviced. By plotting the raw results and calculating (for example) the 95% percentile, we were able to see that only a very small number of the requests suffered in this way.

After some investigation, the suspected cause of these high response times was "stop the world" garbage collection and we eventually proved this by monitoring the JVM during a test run. Thankfully, the new Java VMs include a ton of tuning options and I've found the following references particularly useful in the past.

• Ergonomics in the 5.0 Java Virtual Machine
• J2SE 5.0 Performance White Paper
• Java SE HotSpot at a Glance

So then, some advice :

1. Make sure that you define your non-functional requirements as early in the project as you can, specifying performance in the context of scalability. Once you've done this, you can focus on figuring out how to meet them.
2. Make sure that you test your key non-functional requirements as soon as possible, preferably with something like a reference architecture. Tuning your JVMs early lets you gauge how much more performance you might get with further tuning in the future.
3. Make sure that you document your JVM settings if you've had to tune them to meet your performance targets. The system I'm testing is a third party product and there's no way that the customer should even be running, what's basically a "real-time" system, on untuned JVMs.
4. If you need to start investigating performance problems, this article over at InfoQ (The Box: A Shortcut to finding Performance Bottlenecks) has some good tips for starting out on this route.
5. Ensure that your Java system is easy to monitor, using something like JMX, because this will really help you prove and diagnose any issues you might come across. More on that later in the week.

If you get the opportunity to do some performance testing then I highly recommend it. Assuring the non-functional requirements is a key part of an architect's role so it's great experience.

Responses[7]

Posted by Simon Brown on 05 November 2007 12:23:34 GMT #