Coding the Architecture - rannett

Project Naming

Sat, 01 Mar 2008 21:06:34 GMT

Whenever I've started work on a new project I've had an introduction along these lines:

Wizard feeds into Pluto, which then re-values. It broadcasts changes that are picked up by Puma and recorded by Halo.

but what they really mean is something like:

The Market-data System feeds into the Trade Calculation Engine, which then re-values. It broadcasts changes that are picked up by the Trade Display GUI and recorded by the Audit system.

The IT industry seems to have a peculiar habit of giving projects names that have no relation to what they actually do. Perhaps we feel the need to make a dull system sound more interesting ("We can make the accounts receivable project exciting by calling it Skydive") and this is fine if you have only one system. I deal with architectures that have dozens of subsystems and this can be a real problem by adding to the learning curve and confusing newcomers.

There is an argument that a system's name shouldn't have meaning as the scope of the system will probably grow and change throughout its life. If a project has a very specific name such as "stock-option pricer" it will be inaccurate once you add the ability to price bond-options. A few years down the line it could actually be much confusing than calling it Pluto.

It would be nice if you could change the name of a project/system but this can be very hard. Not because you can't re-factor code and change configuration but because of funding, politics and the users familiarity.

What is your preferred technique for naming projects and systems? Something funky, something meaningful or a compromise? If you are attending the user group meeting this Tuesday you can share your craziest project name with me!

Update - It's been suggested to me that in a SOA system you should name a service very exactly and when the name no longer matches then it's probably time to introduce a new service. Neat!

System Design and Reconciliation

Fri, 08 Feb 2008 22:06:20 GMT

Even those not working in Finance have probably heard about the intriguing events in a large investment bank and the HUGE losses that have occurred.

The interesting comment from an IT and architecture point of view is the following:

The trader responsible for the fraud had "in-depth knowledge of the control procedures resulting from this former employment in the middle-office", the bank said.

I'm sure the story and our perception of it will change as more facts emerge but we can be certain that the banks internal security and control systems failed.

Many security and control terms such as authentication, authorization and auditing are probably familiar what about reconciliation? Consider a simple stock-taking example. The number of items delivered to a warehouse minus the number of items sold should tell you how many are still there. If it's less, you may have a thief. ( If you've ever been involved in a stock-take you may have seen many missing items - they tend to be small and valuable.) Stock taking is a physical example of reconciliation i.e do all the numbers add up.

Should you be doing this in a system of your own? If you have developed an online store do you periodically make sure that the number of orders to your website equals the number of payments received and items sent? It's very common for a system designer to assume this will be the case, as the data normally flows from one component to the next, but consider what happens if someone sends messages to your delivery system directly. They bypass your ordering and payment systems so there is no matching order or payment. This is likely to be an insider that knows the system well and you can lose a lot of money as you send out goods without being paid. These kind of issues are increasingly likely as designers move away from monolithic systems to SOAs.

Apart from internal checks between your own systems you can reconcile with external ones. In securities trading, we usually get a list of trades from the markets and exchanges at the end of he day and reconcile against the banks internal list of trades. This should show if a trader has been placing trades they shouldn't have (or failing to place those they should).

It's important to make sure that the main system and the reconciliation system are separate otherwise an error or fraud in one could effect the other. (Reconciliation is often simple and can be done in something like a script).

Finally, you should remember that regression testing is actually a form of reconciliation between different versions of systems, so you may be doing some already!

Architectural Assumptions

Mon, 24 Dec 2007 12:44:59 GMT

Most of the systems I've worked on in the recent past have been latency rather than throughput orientated. However my current project is definitely throughput focused and scales horizontally rather than vertically (this is a simplification but basically correct).

This has lead to me making a few errors based on my incorrect assumptions. As you may have read I've been retrofitting performance metrics, we're trying to discover the level of overhead compared to 'real' work, so I've been running a single node on my machine to determine the outputs required for statistics. I wanted to write the results to a database and the DBA asked me to produce an estimate of the load in production before he would create my table structures. Cursing the formality of DBAs, I made some calculations based on the number of tasks and the metrics for each. The number I came up with was huge and the DBA almost coughed up his breakfast.

I was surprised by the number because I had failed to realise that my local test was NOT representative of the real system. If the application was vertically scaled then the quantity of metrics in production would be (roughly) the same as on my desktop PC but this application runs across hundreds of grid machines and each one would output these metrics.

I had just designed a distributed denial of service attack on our own database server. The solution was quite simple - I got the machines that aggregate the outputs to also aggregate the performance metrics. This reduced the load by a couple of orders of magnitude.

I think this is a good example of what happens when you fail to take into account the architecture of a system when writing code. All developers need to be aware of the architecure and how it effects the code they are writing. Does anyone else have similar experiences?