One of my current projects involves me performance testing a third party Java system. In essence, it's a distributed/n-tier Java EE web application and we're hitting it with a high simulated load thanks to Apache JMeter. One of the things we found is that performance degraded quite dramatically as load increased, to the point where some requests could take anywhere up to a minute or two to be serviced. By plotting the raw results and calculating (for example) the 95% percentile, we were able to see that only a very small number of the requests suffered in this way.
After some investigation, the suspected cause of these high response times was "stop the world" garbage collection and we eventually proved this by monitoring the JVM during a test run. Thankfully, the new Java VMs include a ton of tuning options and I've found the following references particularly useful in the past.
So then, some advice :
If you get the opportunity to do some performance testing then I highly recommend it. Assuring the non-functional requirements is a key part of an architect's role so it's great experience.
Simon is an independent software development consultant specialising in software architecture; specifically technical leadership, communication and lightweight, pragmatic approaches to software architecture. He is the author of two books about software architecture; Software Architecture for Developers (a developer-friendly guide to software architecture, technical leadership and the balance with agility) and The Art of Visualising Software Architecture (a guide to communicating software architecture with sketches, diagrams and the C4 model). Simon lives in Jersey (the largest of the Channel Islands) and regularly speaks to audiences around the world. You can find Simon on Twitter at @simonbrown.