Coding the Architecture

Following the specification

Back in the days when J2EE app servers were the best thing since sliced bread, I remember having lots of discussions with people about the J2EE specifications and I don't mind admitting that it was always my goal to build a truly spec compliant application. For example, file access from the EJB tier was explicitly forbidden in the specification; because of reasons around portability, lack of file system transactionality, etc. Fast forward a couple of years, and I started to realise that a lot of this stuff didn't really matter. File access via the EJB tier actually did work and it was much simpler than writing JCA connectors. In addition, and in the same way that I have almost never swapped out a database during a development project, it didn't really matter if my J2EE application was portable or not. We'd develop on WebLogic/WebSphere/etc and we'd deploy on the same platform.

Earlier in the week I was talking to a group of people who were defining the architecture for a Java EE system and they started asking how they were going to access the file system from within their EJB tier. I'm curious ... what's the general feeling about specification compliance these days? Do you care whether your applications follow all of the recommended guidelines, or do you take a pragmatic view that just asks whether it works? How important is this to you as an architect?

London User Group - July 2008

The next London user group is Tuesday 15th July and it'll be a social event rather than the usual presentation/discussion session. It's summer and we've all been really busy recently, so we thought we'd take some time out to relax. We'll be at The Crown (just around the corner from Skills Matter) from about 6:30ish. Please feel free to come along and join the informal discussion.

Who should own the non-functional tests?

This is a follow-up to a post entitled Performance tuning Java systems that I made last year, which talked about some performance testing that I was doing for a customer. Last week, I went back to the same customer to make some small tweaks to the test scripts, and this very simple act highlighted something I thought was very interesting.

The key thing about the system being tested is that it's a customized version of an existing vendor product (a high volume, low latency trading platform), with the vendor being responsible for making all of the changes. At some point last year, my customer wanted to run some tests to ensure that the platform could grow with their business, and that's where I got involved. After a round of testing, environment reconfiguration and patches, the system more or less delivered on what it promised.

Fast-forward to the current day and I was bought back in to make some changes to the test scripts, because the vendor had made some changes to their web application. Since our performance tests were JMeter based, those changes had a knock-on effect on the tests. The thing I found interesting is that the performance test pack hadn't been handed over to the vendor. Regardless of the reasons, the benefits of doing this are fairly clear.

Ownership : although it's good that my customer wanted to proactively test some of the non-functional aspects of the software, I can't help feeling that the software vendor should have ownership of the test pack. My understanding is that they do little in the way of non-functional testing themselves, so this is an ideal opportunity to rectify this.
Continuous execution : if the vendor has ownership, they can include the test pack as a part of their integration/build/release process for continuous execution and early warning of performance degrading changes.
Accuracy : I like to think that we did a good job at putting together the performance testing scripts, but we don't have the inside knowledge of the system that the vendor has. Vendor ownership would ensure that the test scripts are as accurate and representative as possible.
Up to date : the test scripts are inherently tied to the AJAX web application, which means that they need to be kept in sync when changes are made to the user experience. Again, the vendor is best positioned to do this, and taking a more proactive approach would ensure that user experience changes don't degrade the non-functional characteristics of the software.

The overriding question here is why doesn't the vendor undertake their own performance testing? They *should* take responsibility for the non-functional characteristics of their software, particularly when they are making claims of how fast and scalable it is. But perhaps more interesting is that the vendor of another product in use in the same also doesn't do performance testing. Shouldn't we be asking more from commercial product vendors?