Robert works in financial services and has spent many years creating and maintaining trading systems. He knows far too much about low latency data systems and garbage collection than is good for anyone. If you find yourself in a pub with him then do NOT mention phrases like "mark and sweep" or "memory profiling" as you'll be stuck there for hours and, might, be forced to break an ashtray over your own head to get away from him.
Development with Complex Architectures
I know someone will say "write unit tests" and of course I do but there is a class of problems that you see when services interact that you will miss with unit tests. For example, the coordination of service states and circular, service call references.
In the past I have seen several possible solutions:
1) Don't bother and just use unit tests for the local code and rely on automated integration tests to pick up problems caused by services interacting. This can work well if this category of problem is rare and you make sure you have good functional test coverage (which you should anyway). This is a little fire and forget and isn't good for iterative development where you spot issues and modify the code - if you have a number of these issues and the integration tests run overnight you might have broken the run for a week...
2) Run everything across several developer machines. This allows you to use your development equipment but ties up several people and it's very easy for the versions to get out of sync between boxes (you have to deliver code to a shared branch and sync to it).
3) Buy hugely powerful boxes for all developers. Obviously I like to have a super machine but the cost can be huge (I once worked on a project with over 100 developers) and this can still be slow for testing as even the biggest desktop machine will struggle to bring up 4 gigs worth of services (loading all static data and constructing in memory structures etc) and the contention is often a killer.
4) Have continually running, shared services on server boxes that are started every morning from the nightly build. Therefore the developer only runs up services they have modified and tests against the shared services. It is particularly effective if services need a large quantity of static/cached data as this will only be loaded when the shared services are started. Although this works well for business code contained within a specific service, it is obviously less effective if you change some common, foundation code. Developers also have to be careful that tests don't interact in unexpected ways e.g. two developers changing prices for the same item. Also watch out for serialization/on-the-wire encoding issues between the latest development code and the services built that morning.
5) Use virtualization. All of the above solutions have issues and ideally we want the following:
a) Developer integration tests are sandboxed from other developers.
b) Tests are run against the latest code on all services.
c) Most of the work is not done on a developers machine but performed on powerful, shared server machines (at any one time very few developers will be running integration tests).
One way of achieving this is to run the tests in a sandboxed, virtual environment. Virtualisation allows you to create a number of virtual servers on a single physical machine. As far as both internal applications and external users are concerned these virtual machines are the same as physical ones. What we can do is to create an entire set of machines (that mirrors a production environment) on the shared integration servers for our developer to run their tests on. This is created specifically for this test and can be destroyed afterwards.
This allows a FULL integration test for the code but also the system configuration associated with it. The tests don't know they are running on a virtual system so should be functionally identical. Care needs to be taken if you are running non-functional tests, as you are not guaranteed the full processing power of the machine.
You will need to write a more complex deployment script but this should be a one off cost.
At this point I have to admit that I have never created a full, virtual deployment as described above - does anyone have experiences or issues/ideas they'd like to share?
Re: Development with Complex Architectures
This approach works fine when testing functional requirements. As to the non functional ones, I remain to be persuaded. Although you can try and play with the resource allocation to the VM(like more/less RAM, additional processor etc.), but the host box might decide to take some time and handle other requests limiting the running VM. Great to emulate latent environments, though ;)
Re: Development with Complex Architectures
Re: Development with Complex Architectures
This does beg the question as to whether a more intelligent system would load only a subset of possible data for testing purposes (assuming there are no NFR side effects to this).
This also begs the question as to what constitutes 'static' and I might blog about this soon :-)
