Coding the Architecture - architecture definition tag

Jitter

Tue, 06 Sep 2011 08:02:41 GMT

On CTA we often talk about non-functional requirements and how this can drive the architecture of a system. Most of these cover issues of desired response time and capacity (latency, throughput, storage etc) but I believe that Jitter is a metric which is either forgotten or unknown to some software engineers – even though it's essential for hardware engineers.

The most basic definition would be “variation in response”. In other words, the response, transmission or latency in a system will not be a constant. In many systems this variation goes unnoticed as it is small compared to the response itself. However I believe that as the systems we deal with become more performant and demanding, software engineers will need to understand, measure and tune this.

A quick web-search of jitter shows the term being used to cover performance degradations due to activities such as garbage collection or unexpected user actions. This seems to greatly annoy telecommunications and hardware engineers who would argue that these are predictable, system and user events which could just be turned off – although ignoring your users and asking for a machine with infinite memory may get you fired. They would argue that true Jitter is an unpredictable variation in response whose occurrence frequency follows a normal distribution. Like most effects that are normally distributed it is caused by cumulative random events, most of which are due to the switching actions. Personally, I think you should measure whatever makes sense in your system.

Jitter means that hard limits for latency can be statistically likely but not guaranteed. Specifications such as:

“The system should respond to action X within 200ms”.

Should be challenged and replaced with statements like:

“The systems should response to action X within 200ms for 95% of requests”

Of course we are making the implicit assumption that this is normally distributed and the system WILL, eventually, respond. You might want to explicitly state that all actions will be executed but stating a hard limit for all responses means that if you measure for long enough, you'll break your specification.

Jitter can be quite obvious on messaging based systems that contain many hops. This will also tend to have a nice, Gaussian distribution due to the cumulative random delays.

The best way to measure Jitter is to set up a standard test where you can sequentially fire a large number of requests into your system and measure the response time. However, rather than simply add these up and find an average we need to count how many responses are in a set of timing windows e.g. how many responses fall between 5ms-10ms, 10ms-15ms, 15ms-20ms etc. If we plot the number in each section we should see our expected Normal distribution.

Reducing Jitter is subtly different from reducing latency itself. You are trying to reduce the variation in the response times or 'squeeze' the response profile.

Often when you reduce latency you will reduce the Jitter proportionally however this is not always the case. For example, network hops are a common cause of latency but each hop will increase the Jitter as well – you are accumulating more random delays with each hop. If you change the physical architecture of your system to have more but quicker hops you will almost certainly increase the jitter (remember this is the variation in response and not the actual response) even if the average latency is lower. It IS important to measure it and monitor as the system evolves.

Simon is covering a number of related issues in his Skills Matter tutorial Load Testing for Developers.

Speaking at the Software Architect 2010 conference

Thu, 30 Sep 2010 19:12:00 GMT

Just a quick post to say that I'm speaking at the Software Architect 2010 conference in London next month. I'm running a pre-conference workshop, a post-conference workshop and two technical breakout sessions as follows.

Software Architecture in a Day (pre-conference workshop)

This one-day workshop is an interactive introduction to software architecture and what it means to be a software architect. It's aimed at software developers who are looking towards their first software architect role, developers who want to become more architecturally aware and software architects that are new to the role. We'll be asking and answering the following questions:

What is software architecture?

What is the role of a software architect?

How do you define software architecture?

How do you share software architecture?

How do you deliver software architecture?

Where do you start?

One of the hardest things about software development is being asked to come up with a design when all you're given is a set of requirements and a blank sheet of paper. Many software teams will dive straight into the code and while this can initially be very productive, the slippery slope of constant refactoring is awaiting those teams that haven't quite found a design that works. Often, a little forethought is all that's needed to get the development process heading in the right direction. So where do you start?

How do you design for non-functional requirements?

You may have heard people talking about "non-functional requirements", mentioning things like performance, scalability, availability, security, flexibility and so on. What do you do if you're asked to design software that exhibits these qualities and furthermore, how do you do it in a pragmatic way? Come along to find out about some of the approaches that you can adopt and some of the trade-offs you'll need to make.

Boxes & Lines (post-conference workshop)

Talking about software architecture will get you so far, but in reality it's all about experience and that's exactly what this workshop offers. Given a set of functional requirements, you'll be asked to identify the other architectural drivers before doing some design, choosing some technologies and drawing some boxes and lines. Join us if you want to know where to start with designing software and how to go about the software architecture definition process in a lightweight, structured and pragmatic way.

Feel free to stop me and say hello if you're at the conference; should be a great event.

Structure and guidelines, consistency and clarity

Mon, 02 Aug 2010 21:05:00 GMT

One of my key points about software architecture is that it introduces structure and guidelines into a software system, which in turn leads to consistency and clarity of the overall design. Basically, I'm saying that there are some real benefits from thinking about your design upfront. This isn't a particularly groundbreaking statement, but you'd be surprised how few teams spend some quality time to do this. As I mentioned in Estimating a software system, this thinking doesn't need to take very long at all.

If you take a simple lightweight approach, you can map out the major functional requirements and come up with a design down to the major components in a couple of days. Then, you can do some high-level estimating off the back of this design. There are a number of ways that you can map out the functional requirements, although I find that user stories are a good way to summarise the key requirements.

Thinking about your software design on your own is great, but making it a collaborative exercise is even better and comes highly recommended. We're doing this at the moment for another project where each of us on the team has specialisms in different technology aspects of the overall solution. The benefit of this is that we all know how the technology in our own areas works, but the details of how those technologies collaborate is often unknown. Also, we sometimes tend to assume what responsibilities each technology will take on and it's only by discussing the overall solution as a group that we realise we've either doubled up on something or assumed that somebody else was looking after it. Essentially, software design as a collaborative exercise allows everybody's vision to meet so that the end-result is consistent and clear. As I've written before, often some of the basic foundations are missed.

For me, explicitly thinking about the components that make up the software system introduces structure, which in turn can be used as a template for building that system. From this, you get a consistent approach to solving common problems and a thought out design with a clear architectural vision. A couple of days upfront can really make a big difference to the big picture.

Start with the big picture

Wed, 18 Nov 2009 08:05:57 GMT

One of the hardest things about the software architecture role is being asked to come up with a software architecture when all you're given is a set of requirements and a blank piece of paper. It's certainly one of the most fun parts of the role, but it can be a daunting prospect trying to figure out where to start and what to do. Many software teams dive straight into the code and this can initially be very productive but many teams soon start to venture down the slippery slope of constant refactoring while they try to find a design that works. Often, a little forethought is all that's needed to get the architecture definition process heading in the right direction. So where do you start?

Read the full essay...