Modifying Open Services

Authentication and Auditing

There's been a huge push recently towards service oriented architectures - sharing services within an organisation with benefits such as reuse and making information consistent. Take a simple example such as a catalogue of products for a furniture company. As a shared and open service, all of the companies systems - Sales, Marketing, Support, Delivery and Billing applications - can use this information in an open and consistent way.

If a service is very open and easy to use (e.g. services operating via a RESTful interface) then there is a good chance that applications will use it in a way you didn't originally intend and probably by applications you don't even know about. This sounds great but you'll soon come across the issue that you've lost the ability to audit the current use and gauge the effect of any change. As an example let's say you want to add details for 'forest sustainability' to our furniture information. We add a block of xml to describe this and release. However an application that uses our service starts generating errors as it's not expecting this new information. (We could argue that it shouldn't do this but this is what happens in the real world.) Problems are more likely if you have to modify rather than add to your format. Changing an integer to a floating point number could cause strange issues.

You need to be able to get dependent applications to test with your new service before you release but who's using it and how are they using it? You can log the incoming requests to know what is being used but you don't know who is using it - so how do you know who has to test changes?

This is a problem I've been seeing recently and a solution is to use authentication even if you have no intention of restricting access. You can make the credentials easy to obtain but you need to make sure the users of your service are registered and provide sufficient contact information. Of course, actually getting the users to test and adapt to changes are another issue but at least they can't complain they weren't informed.

Has anyone else seen this issue and what were your solutions? Did you just 'publish and be damned' or end up introducing heavyweight process to control releases?

About the author

Robert Annett Robert works in financial services and has spent many years creating and maintaining trading systems. He knows far more about low latency data systems and garbage collection than is good for anyone. He likes to think of himself as a pragmatist who loves technology but uses what's appropriate rather than what's cool.

When not pouring over data connections or tormenting interviewees with circular reference questions, Robert can be found locked in his shed with an impressive collection of woodworking tools.

E-mail : robert.annett at

Re: Modifying Open Services

Proper versioning is the key to much of this. Once you're happy you've migrated your sanctioned users from a previous version you simply turn off or report a fault when someone tries to use a discontinued version. Indeed, this is often required as it's virtually impossible to cut over to your new version smoothly when you've got more than zero important consumers.

How you choose to do this is up to you and your environment. You could make the version number part of the URL or have the client specify the version they support as part of a handshake. The latter is perhaps akin to the authentication approach you outlined, but provides a bit more control over how you interact with the caller. For example, whether you return an integer or a floating point number.

I've used just such a versioning approach in the past. It took at least six months before we could even consider turning off the previous version as that was the earliest one client could schedule a release. However, what we struggled with most was convincing them to modify their software at all: why should they spend money upgrading their software? Ironically, the reason for the new version was not a new feature but that the Web Services standards had moved underneath us and what used to be valid no longer was (so an application server upgrade didn't support our old service).

Re: Modifying Open Services

You touch upon the problem of the proliferation of supported (or even unsupported) versions to try to keep all old clients going. I tend to view this as putting off the problem until tomorrow and my personal preference is always to take a smaller amount of pain now. I think you could easily write a book about the issues of upgrading services!

Re: Modifying Open Services

You're obviously talking about an environment where the consumers are beholden to you for gracing them with your service rather than you to them for using it.

The pain of telling a customer who pays half your bills and to whom you have a contractual obligation that they need to roll out updated client software across their organisation because you need to provide a new feature for a different customer is not worth contemplating.

They'll laugh at your pathetic stick so it pays to find a carrot to attach to the end before brandishing it. In the meantime, multiple versions.

Re: Modifying Open Services

Another option is to just not support old versions after a set date. Of course, it depends on who owns the service, whether they're paying for it, etc, etc. Incentives are another option ... you could make it worth the effort to switch to a new version early, perhaps by temporarily reducing costs or by providing some value-add features that aren't available currently.

All of this points to the fact that it's crucially important to explicitly think about your service interfaces up-front if they are going to be published outside of your own system boundaries.

Re: Modifying Open Services

I have experience with this situation within the internal enterprise of companies as well. Instead of using authentication as a means of tracking, we have required an extra field where the client must identify themselves and their component name. This seemed to work well as requests were rejected if the information was not supplied, and majority of the time any reasonable client fills them out properly.

Re: Modifying Open Services

This is a good idea and allows you to collect more relevant information than simply using HTTP authentication.

Re: Modifying Open Services

The truth I think lies in the fact that you can't keep all the citizens of your sevice galaxy happy. It pays to recognize this. In any case there are numerous advantages for gathering information about who your service users are from the get-go.

Applying the 80/20 rule here along side versioning seems a good fit. You can phase in your new version but the period of running both versions in parallel could be a nightmare depending on the nature of your services. Read-only services or those where the users are not capable of making globally accessible changes will be fairly straight-forward to support. Examples of this approach are some of the Google APIs.

Publicize the new version while advertizing the demise of the old version by a cut off date. This will work best if you have been tracking usage from day one and can send out service level alerts to users with information of this nature.

May the force be with you!

Add a comment Send a TrackBack