Adding the fuel gauge

Getting feedback from our software

One of my talks at the recent DevWeek conference was about the pitfalls software architects face and I covered some of the problems associated with technology selection. Probably *the* biggest problem is vendor marketing and hype, with many project teams simply taking this at face value. Sometimes a piece of technology will do exactly what it says on the tin, but sometimes it won't. There are truly no silver bullets and every technology, large or small, has trade-offs. You've probably seen this yourself at some point. Vendors (open source and commercial) promising features that they haven't yet implemented through to bold claims about performance or scalability. Depending on your project context, these promises can often make or break your project.

One of the analogies that I made during the session was about the fuel consumption figures quoted by car manufacturers in their glossy brochures. Let's imagine that you need to travel from one side of the country to the other, work out the mileage and then buy or rent a car based upon the fuel consumption figures quoted in a brochure. The quoted consumption figures are usually based on some optimum conditions but real world figures will vary according to the way that you drive, the speed you drive, the ambient temperature, the gradient, the road surface and so on. Depending on all of this, you may or may not achieve your goal.

When we undertake a technology selection exercise, we'll typically evaluate candidates against a set of criteria and choose the one that we think best suits our needs before plugging it in to our projects. Not testing the technology before adoption is the same as driving a car across the country - you're relying on somebody else's claims and it might not go as expected. Literally, your mileage may vary!

Of course, the key difference is that you get a fuel gauge in a car that provides you with constant feedback of how much fuel remains in the tank. In addition, newer cars have onboard computers that can provide you with real-time consumption figures and estimate the mileage remaining. This is all information and it provides a way to monitor what is happening so that you can adjust (or fill up!) as necessary. Laptop batteries are the same. The manufacturers quote maximum battery life figures and while you might not get that in real world usage, you do get to see how much battery life is remaining.

With this in mind, it's worth thinking about why we don't usually add fuel gauges to our own software systems. These systems are usually composed of many complex technologies, each of which makes its own claims and has its own trade-offs. Yet we often deploy and run our systems as a black box. Often this will work but sometimes it won't. And worse still, without a fuel gauge you have no idea when your system will grind to a crawl or stop working completely.

Adding a monitoring capability is fairly easy to do and can give you important insight into the health of your software. For example, it might allow you to monitor how many database connections are being used, or how many messages are waiting to be processed, or how many worker threads are busy servicing user requests. Here are some thoughts on how to cater for monitoring in your architecture, and they're particularly relevant if you're building Java applications.

As with cars and laptops, there are benefits to be had by adding some simple feedback devices to our software systems. After all, wouldn't it be great if you could understand the health of your system and proactively deal with problems before they become major issues?

About the author

Simon is an independent consultant specializing in software architecture, and the author of Software Architecture for Developers (a developer-friendly guide to software architecture, technical leadership and the balance with agility). He’s also the creator of the C4 software architecture model and the founder of Structurizr, which is a collection of open source and commercial tooling to help software teams visualise, document and explore their software architecture.

You can find Simon on Twitter at @simonbrown ... see simonbrown.je for information about his speaking schedule, videos from past conferences and software architecture training.



Re: Adding the fuel gauge

The analogy's a bit stretched, and I'm not sure it's all that appropriate. The monitoring in my car and laptop is static yet appropriate because their features and usage are (relatively) static. In many cases it's our application's increased use that needs to be monitored: it turns out we now need a bus, not a car with good MPG. The monitoring that was applicable under initial usage is not suitable under future loads as the focus changes from, say, transactions per second to cost per transaction.

You're right, of course, that some monitoring is better than no monitoring. We shouldn't allow things to get worse (or unacceptably bad) but we also shouldn't let this trick us into accepting stagnation.

Re: Adding the fuel gauge

From a pure performance point of view, the analogy makes sense provided that you identify the key statistics that you want to monitor. Examples that spring to mind are response times for downstream systems, maybe memory usage and so on. In other words, things that might affect the end-user's experience (human or otherwise). Scalability though, as you say, is different although the two are inherently linked.

Software lacks any sort of physicality and I think it's this that causes many of the problems. If you agree to drive me somewhere and I turn up with 50 people, it's obvious that we needed a bus and that we've not communicated the requirements adequately. If 50 people logon to a website designed for a much smaller user base, the problems aren't so clear. At least *some* monitoring would bring this to our attention before the angry users start to phone up! ;-)

Re: Adding the fuel gauge

My concern with the analogy is more related to how the focus on features moves, say, from them being correct to being optimal then being cost-effective. A single static metric doesn't appeal to this particularly well, especially as new features and interfaces come into the mix.

Inappropriate monitoring (or metrics) may constrain you unnecessarily. If I want to grow from transporting 1 person to 50 people I don't want to try to maintain the mileage of my initial solution (a bicycle) while meeting that requirement.

Re: Adding the fuel gauge

How far the analogy can be stretched may be up for debate, but I think the gist of the article is clear: spending effort to meaningfully monitor a system is often essential. Unfortunately a software system can have a vast array of measurable criteria. Some might be obvious -- e.g. the number of messages on a queue; but some might be less obvious -- e.g. the average length of time a thread in a given pool takes to process a request. Having the foresight to apply instrumentation to the right areas of a large system before problems occur is a skill. When you get it wrong you are often stuck with the laborious task of attempting reproduction in an environment where you have more fuel gauges.

Re: Adding the fuel gauge

No doubt we agree that monitoring is critical. The phrase "fairly easy to do" in the article does not resonate with my own experiences attempting to interpret incumbent monitoring nor with attempts to create my own. At least, it's not easy to do well.

I would concede that it is easy to monitor a few simple metrics which will tell you when things are about to fall in a heap. With the system I work with it's not just a case of "filling 'er up" when the dashboard lights up, although there is often remedial action in an attempt to preempt the likely failure. I'd also expect a great deal of lamentation that there wasn't some lower-level monitoring to tell us why it was about to fall over. This is the skill that you refer to, often applied with hindsight.

Re: Adding the fuel gauge

Really surprised that these comments (and those of the preceding articles) have not covered Hyperic. I have used it in the past and have been pretty impressed. Some key features: If it hasn't appeared on your radar already, expect it to soon, as it has just been acquired by SpringSource!

Add a comment Send a TrackBack