Coding the Architecture - scalability tag

Discussion from the May 2008 user group

Sat, 30 Aug 2008 06:57:27 GMT

Although this happened a few months ago, we recorded the discussion that followed our London user group where Moudud Ahmed talked about building a high volume, low latency system in Java. Here are some of the topics that we talked about.

Performance and scalability.
Measuring latency.
Domain-driven design.
BEA's real-time Java platform.
Writing applications with large heap sizes that can survive not doing garbage collection during the working day.

The audio levels are very low on the recording, but we think that the interesting discussion was worth rescuing and publishing. You can grab the discussion from here (mp3 format) ... just don't listen on a tube train! If you missed the user group, a streaming video of the presentation is also available.

NFRs for system replacements

Wed, 28 May 2008 10:35:00 GMT

As software architects, we tend to write about non-functional requirements a lot; particularly about how they should be defined and challenged because of the influence they have. One of the reasons for talking about NFRs a lot is that, in the majority, project teams don't give them the attention that they should. And here's a great example that's slightly different from the story where none of the NFRs were defined.

As is the case in most organisations, technology has a limited lifespan. Perhaps the business evolves, the technology becomes harder to maintain, etc. Whatever the reason, many organisations regularly look at replacing their existing systems with solutions that are shinier and more buzzword compliant.

In one of these cases, an organisation put together a 125 page specification for the replacement system detailing exactly what the system should do along with some details of the non-functional requirements. However, out of those 125 pages, only 1 of them featured the NFRs, and they can be summarised as follows.

Performance : the system must be fast.
Scalability : the system must be able to support 100 messages per second.
Availability : the system must be available 24x7.
Security : only certain individuals are permitted to access the system.

Credit where credit is due ... at least some thought was put into the non-functional requirements and conversations to explore those requirements aren't starting from scratch. But on the other hand, those requirements are meaningless!

I want to end this post on a positive note, so let me contrast this story with another replacement project that I've been involved with recently. In this instance, the team *had* thought about the non-functional requirements and had detailed definitions for what they should be. This is excellent, particularly considering that the non-functional requirements surrounding the current implementation are vague in nature. System replacements are an opportunity to revisit what's important, and that includes the non-functional aspects too.

Scalability Principles

Wed, 21 May 2008 13:12:25 GMT

At the simplest level, scalability is about doing more of something. This could be responding to more user requests, executing more work or handling more data. While designing software has its complexities, making that software capable of doing lots of work presents its own set of problems. This article presents some principles and guidelines for building scalable software systems.

London User Group - May 2008

Fri, 25 Apr 2008 13:30:52 GMT

Here are the details of the May London User Group.

Title : Building a high volume, low latency system in Java
Summary : This session will provide some insight into the architecture, design and development of a greenfield high volume, low latency trading and pricing system. Topics include :
- The architecture and design itself.
- How the early development of an executable reference architecture was used to help prove the low latency requirements.
- The structure of the team and how the architecture was shared between a team of architects.
- How mainstream technologies such as Java 5, Spring, JMS, and Oracle were used to successfully deliver a solution where other technologies like C++ might be the immediate first choice for low latency applications.
Speaker : Moudud Ahmed, Detica
Date : Tuesday, 6th May 2008
Time : 18:30-20:00
Location : Skills Matter, 1 Sekforde Street
Format : Presentation followed by a breakout for discussion, with further discussion in a local pub (The Crown).
Cost : Free, but registration is required.

Many thanks to Skills Matter for sponsoring the user group.

Podcast #2 : QCon revisited

Tue, 25 Mar 2008 10:23:20 GMT

As promised the 2nd CTA podcast is a roundtable discussion between some of the CTA contributors - namely Simon Brown, Sam Dalton and Kevin Seal. In this podcast we discuss some of the themes emerging from the recent QCon conference held in London and our views on those themes.

The podcast can be downloaded here or you can subscribe. Full show notes follow below:

1. Introduction

QCon Conference London detail can be found here.

2. Monitoring and Management (0:27)

Roll your own vs. "off the shelf" monitoring
Asynchronous monitoring
JXInsight
The need for an System Architect and an overall understanding of WHAT to monitor and a holistic view
Including Monitoring NFRs upfront in SAD
Monitoring the monitoring vs. best effort
eBay and 1.5TB of monitoring and logfile information per day

3. Rehashing Refactoring (11:56)

Refactoring now mainstream, now emphasizing at the micro-level
Code-debt - how can you measure it?
Refactoring is not something to be "saved up"
Lots of rearchitecture projects sold as refactoring
Sometimes starting again is a better solution than large scale refactoring
Why do things end up in such a mess so often?
Need to get back to writing code for humans to understand
Worry about emphasis on low-level code optimizations rather than architecture
Is refactoring its own worst enemy being referred to as a separate discipline to regular software development?
We are our own worst enemies - we (especially the architects) need to stand up for the quality of our code - stop allowing ourselves to be convinced to make shortcuts

4. The Return of the Architect? (25:08)

Thinking about performance, monitoring etc upfront
Terracotta, Coherence, Gigaspaces
Kevin's views on Terracotta
How much performance could you squeeze from a system in 2 weeks with code changes alone?
Real-time VMs and performance improvements
The benefits of continuous performance testing
Premature optimization
Understanding a system's deficiencies can be as valuable as fixing them

Performance tuning Java systems

Mon, 05 Nov 2007 12:23:34 GMT

One of my current projects involves me performance testing a third party Java system. In essence, it's a distributed/n-tier Java EE web application and we're hitting it with a high simulated load thanks to Apache JMeter. One of the things we found is that performance degraded quite dramatically as load increased, to the point where some requests could take anywhere up to a minute or two to be serviced. By plotting the raw results and calculating (for example) the 95% percentile, we were able to see that only a very small number of the requests suffered in this way.

After some investigation, the suspected cause of these high response times was "stop the world" garbage collection and we eventually proved this by monitoring the JVM during a test run. Thankfully, the new Java VMs include a ton of tuning options and I've found the following references particularly useful in the past.

So then, some advice :

Make sure that you define your non-functional requirements as early in the project as you can, specifying performance in the context of scalability. Once you've done this, you can focus on figuring out how to meet them.
Make sure that you test your key non-functional requirements as soon as possible, preferably with something like a reference architecture. Tuning your JVMs early lets you gauge how much more performance you might get with further tuning in the future.
Make sure that you document your JVM settings if you've had to tune them to meet your performance targets. The system I'm testing is a third party product and there's no way that the customer should even be running, what's basically a "real-time" system, on untuned JVMs.
If you need to start investigating performance problems, this article over at InfoQ (The Box: A Shortcut to finding Performance Bottlenecks) has some good tips for starting out on this route.
Ensure that your Java system is easy to monitor, using something like JMX, because this will really help you prove and diagnose any issues you might come across. More on that later in the week.

If you get the opportunity to do some performance testing then I highly recommend it. Assuring the non-functional requirements is a key part of an architect's role so it's great experience.

Separating the non-functionals

Tue, 02 Oct 2007 11:07:00 GMT

I was recently involved in a discussion where we were talking about the best way to increase the scalability of a software system, which is essentially a Java EE web application. The most obvious way to do this is to horizontally scale-out the application across multiple web servers (Tomcat, in this case), although I found it really interesting that the conversation soon turned towards topics such as the performance impact that session replication would have. Important as this is, we had moved on to talk about performance, availability, reliability and failover before finishing the scalability discussion. After all, there are some other options that are available.

My experience indicates that it's really easy to get sidetracked when dealing with non-functional requirements and this is just one such example. Yes, you do need to take a holistic view of your non-functional requirements, but you do also need to take each in turn and look at all of the available options. In this particular example, I had a very clear idea of the scalability requirements but not the availability and reliability requirements. For all I know, it might be okay if a user is forced to log back into the application in the event of a Tomcat node going down.

My advice is this - start by focussing on one thing at a time. On a related note, Think you know what scalability is? is a timely article that provides a good overview of what scalability is all about.

Scaling is much more than software

Tue, 24 Jan 2006 22:21:38 GMT

From Scaling is much more than software comes this great little snippet about one of the ways to solve problems with your architecture.

What was the solution? Re-evaluating the business logic and requirements with the customer.

Whenever I'm talking to aspiring software architects about the role, I always find myself emphasising and re-emphasising that an architect must take both the functional and the non-functional requirements into account when designing a system. This reminds me of the previous question about business analysis - it's certainly part of the role and, as an architect, you shouldn't be afraid to question the customer's expectations, particularly if you feel that they are going to adversely affect other aspects of the development. Architecture, and it's success, is about balance.