Coding the Architecture - role tag

Is VM size an architectural smell?

Wed, 16 May 2012 08:01:04 GMT

A couple of months ago I went to an excellent set of talks at LJC night at QCon. This not only inspired my entry on staleness but also made me think about some more architectural smells.

Gil Tene from Azul spoke about the maximum size of VMs we all use. A quick show of hands indicated that the maximum size of VM heap memory most of us ran was 2-4GB. Gil argued, quite convincingly, that this was due to legacy issues with garbage collection but new techniques should make this limit redundant.

I sort of agree... However, even if all the issues surrounding garbage collecting large VMs is solved, the 2-4GB limit is still a useful one.

In Architectural Smells I argued that there are architectural smells in the same way that there are code smells (aspects of a system that indicate but not guarantee a possible issue) and the example I gave was caching. I also think that VM size is an architectural smell.

When designing complex systems one of the most common design guidelines is to split differing responsibilities into specific services. Non-functional requirements may complicate this but most of us try to avoid having all functionality in a single service. Certain services are obviously distinct and can be isolated. However it's easy for functionality creep to result in a service gaining a lot of responsibilities and this often results in a larger run-time footprint.

During testing this shows itself as a service requiring a large VM - certainly in comparison to other services. I view this as an architectural smell and analyse the service to see if it should be split up. (Easier if you have some form of lightweight and iterative design process.)

This is very similar to the code smell of long method. It's rare that your code size or service size starts out by being too large but tends to happen over time. This is a good reason why an architect can't just throw it over the wall but needs to stay involved with the project. As developers add code to the system the design might need to change.

Whether you take an absolute size or relative size will depend on your system in question. Personally, I get concerned when a service is larger than 1GB unless its prime purpose is data storage and even then the data should probably be clustered across services rather than in a single block - but that's an argument for a different occasion.

Data Integrity and System Design

Fri, 15 Apr 2011 14:29:43 GMT

Having been involved in several upgrade projects over the last few years, one thing I've often noticed is the poor quality of data that can be present in a large and long running system. This can present problems for upgrading and usually means that you have to spend quite some time fixing the data first.

Upgrading is difficult and causes regression tests to fail as:

The new system may have more data checking and refuse the old data.

The new system may be more precise e.g. not rounding a number, taking a sign into account etc.

The new system may be using data that was not used before e.g. data entry staff not bothering to enter a product's weight into a form as 'it is never used'.

Years of copy and paste can leave a vast amount of junk that fail consistency checks.

After you have corrected the data for upgrade, the original system has much higher quality data and other issues and inconsistencies have been solved. In a recent system we also saw large performance improvements due to duplicate and junk data being removed. On another system we saved the operations staff may hours work a week as the data improvements meant a large number of post report corrections were no longer needed.

So why isn't this analysis done on a regular basis to help keep a system healthy? The main reason is simply that it's just too hard for the operations staff to do. Therefore when you're designing a system you should take this into account and enable these kinds of maintenance tasks. This involves reporting and having tools that can correct sets of problematic data.

Some things to consider:

How easy is it to identity and delete orphaned data i.e. If you can't get to some data is it required?

Can a user identity data that has not been used for a long time? Can they then archive it?

Can you identify identical or similar data? A common example is user information that differs only by capitalisation e.g. an address.

Can the user run arbitrary consistency checks that go beyond the database rules? E.g. I've recently written a tool to allow an operations manager to run xpaths over data to check for bad bookings.

Can the user bulk load sets of missing or corrected data?

Please don't rely on database tools to do this as your operational staff probably won't know how to use them and your DBAs don't understand the business domain to analyse the data. You need tools at the appropriate level for the appropriate people and consider the complete lifecycle of your product.

Software development is not a relay sport

Wed, 13 Apr 2011 21:20:58 GMT

Software teams that are smaller and/or agile tend to be staffed with people that are generalised specialists; people that have a core specialism along with more general knowledge and experience. In an ideal world, these cross-discipline team members would work together to run and deliver a software project, undertaking everything from requirements capture and architecture through to coding and deployment. Although many software teams strive to be self-organising, in the real world they tend to be larger, more chaotic and staffed only with specialists. These teams, therefore, tend to need and have somebody in a technical leadership role.

There are a lot of people out there calling themselves "solution architects" or "technical architects" (particularly in larger organisations) who define a software architecture and then throw their solutions over the wall to a separate design and development team. What's worse is that often nobody takes ownership of the architecture, and the original architect takes only a cursory glimpse at the progress from time to time.

Read the full essay...

Software architecture introduces control

Wed, 10 Nov 2010 21:08:13 GMT

A while back I wrote about how software architecture introduces structure and guidelines, consistency and clarity into software projects. When discussing this on the training course over the past few months, it's become clear to me that software architecture is also, in reality, about introducing control.

I interviewed a team leader today and when the conversation turned towards quality and specifically quality assurance, I was told that he was fortunate to work with a team that were very competent. It's a fair answer, but my follow-up was "how do you know that your team members are competent?".

For me, software architecture is about reducing the number of assumptions that are typically made on software projects, thereby reducing the number of ugly surprises that bite you further down the line. For most projects, benefits can probably be realised by introducing control and restraint, for example, to stop team members going off on tangents. Scrum does this through the introduction of sprint backlogs so that the team has focus on what they need to deliver during any given iteration. Likewise with software architecture; you need some degree of control in order to introduce structure and guidelines, consistency and clarity. For example, you can't have people writing database access code in your view components if you've designed an n-tier application in order to support some of your key non-functionals. Ditto things like ensuring that all of your components are stateless so that they can be scaled-out to cope with additional load. It's also about simply having a clear and consistent structure to your codebase; appropriately organising your code into packages, namespaces, components, layers, etc.

How much control do you need?

The real question that needs to be answered on any given software project is how much control needs to be introduced? At one end of the scale you have the dictatorial approach where nobody can make a decision for themselves versus the other end of the scale where nobody is getting any guidance whatsoever. I've seen both in real-world software projects and I've taken over projects where everybody on the team was basically left to their own devices. Introducing control on this sort of project is really really hard work but it needs to be done if the team are to have any chance of delivering a coherent piece of software that satisifies all of the original drivers.

So how much control do you introduce? My own answer is that it depends on a number of things...

Are the team experienced?
Have they worked together before?
Are the project requirements complex?
Are there major constraints that need to be taken into account?
What sort of discussions are happening on a daily basis?
etc

I've also noticed that different countries and cultures place different values on control. Some value control and the restraint that it brings whereas others value empowerment and motivation. There's no universally correct answer, but it is worth thinking about how much control is right for your own software project.

Speaking at the Software Architect 2010 conference

Thu, 30 Sep 2010 19:12:00 GMT

Just a quick post to say that I'm speaking at the Software Architect 2010 conference in London next month. I'm running a pre-conference workshop, a post-conference workshop and two technical breakout sessions as follows.

Software Architecture in a Day (pre-conference workshop)

This one-day workshop is an interactive introduction to software architecture and what it means to be a software architect. It's aimed at software developers who are looking towards their first software architect role, developers who want to become more architecturally aware and software architects that are new to the role. We'll be asking and answering the following questions:

What is software architecture?

What is the role of a software architect?

How do you define software architecture?

How do you share software architecture?

How do you deliver software architecture?

Where do you start?

One of the hardest things about software development is being asked to come up with a design when all you're given is a set of requirements and a blank sheet of paper. Many software teams will dive straight into the code and while this can initially be very productive, the slippery slope of constant refactoring is awaiting those teams that haven't quite found a design that works. Often, a little forethought is all that's needed to get the development process heading in the right direction. So where do you start?

How do you design for non-functional requirements?

You may have heard people talking about "non-functional requirements", mentioning things like performance, scalability, availability, security, flexibility and so on. What do you do if you're asked to design software that exhibits these qualities and furthermore, how do you do it in a pragmatic way? Come along to find out about some of the approaches that you can adopt and some of the trade-offs you'll need to make.

Boxes & Lines (post-conference workshop)

Talking about software architecture will get you so far, but in reality it's all about experience and that's exactly what this workshop offers. Given a set of functional requirements, you'll be asked to identify the other architectural drivers before doing some design, choosing some technologies and drawing some boxes and lines. Join us if you want to know where to start with designing software and how to go about the software architecture definition process in a lightweight, structured and pragmatic way.

Feel free to stop me and say hello if you're at the conference; should be a great event.

Pitfalls for software architects

Mon, 30 Mar 2009 08:50:00 GMT

Thanks to everybody that came along to my "Pitfalls for new software architects" talk at DevWeek 2009 last week ... I hope you found it a useful look at some of the traps that I've seen people (including myself!) fall into over the years. The pitfalls are structured into a number of different areas as shown in the diagram below.

Each of these areas is part of our view of what a hands-on software architect should do, and the talk itself compliments a session I delivered at Software Architect 2008 called "Why software projects fail". You can download the slides for both sessions below.

Pitfalls for new software architects (PDF, ~4MB)

Why software projects fail (PDF, ~6MB)

The role of the software architect (video, part 1)

Mon, 09 Mar 2009 09:49:00 GMT

Novantus (a Jersey based cinematic/media production company) and I have recorded a version of my recent BCS Jersey presentation entitled The role of the software architect in successful projects. This video, part 1 in a series of webisodes, introduces the session by presenting some of the reasons behind software project failure and why software architects typically have a bad reputation.

p.s. this presentation is more or less the same as Why Software Projects Fail, which I delivered last year at the "Software Architect 2008" conference.

Why aren't software architects held accountable?

Tue, 03 Mar 2009 10:14:00 GMT

Mike Walker has an interesting post comparing the architect roles across the building and software industries, entitled The Nemesis of Software Architecture. Comparisons aside, there's one point that I really wanted to pick up on.

4. Accountability Inherent - Building architects are accountable for there work when their specification fail while software architects are not. An example of this is the case of an architect stealing a design for the Freedom Tower or the example of MIT that sued well known architect for defective structures. There is clear accountability whereas I still haven't heard of someone getting sued for copy & paste...

I'm a big proponent of software architects having the responsibility and authority to ensure that the projects they are working on come to a successful conclusion, but it does raise an interesting question of why project sponsors don't typically hold software architects accountable. Perhaps it's because the dynamics and roles of everybody on the project team aren't well understood in most cases. For example, what's the working relationship between a project manager and an architect; or the developers and an architect? Or maybe it's because our agile approaches tend to favour sharing the responsibility throughout the entire team, which can end up with the project lacking a single coherent and consistent direction.

In order to address this, maybe we need to go back to basics. Why aren't software architects explicitly given accountability at the outset of a project and what incentive is there for them to accept it?

The role of the Software Architect in successful projects

Thu, 30 Oct 2008 21:26:00 GMT

Admittedly the "Coding the Architecture" audience for this is probably kind of small (hi Dave!), but I'm presenting a session at the Jersey BCS in a few weeks entitled "The role of the Software Architect in successful projects".

The IT industry has a love/hate relationship with the Software Architect role, with many organisations dismissing it because of their negative experiences of architects that dictate from ivory towers and aren't engaged with the actual task of building working software. This reputation is damaging the IT industry and inhibiting project success.

Things need to change. This session looks at some of the reasons why software projects fail and how a hands-on software architect can be invaluable in addressing these issues to drive the project to a successful conclusion. Software architects are here to help, not to hinder.

The session is aimed at anyone undertaking software development projects; including developers, architects and IT managers. It will take place on Thursday the 13th of November 2008, 12:00 in the Ouless Room at Jersey Museum. E-mail jersey.events at bcs.org to reserve your place.

p.s. if this sounds familiar, it's because it is

The role of a hands-on software architect

Wed, 10 Sep 2008 17:33:00 GMT

Becoming a software architect isn't something that simply happens overnight or with a promotion. It's a role, not a rank. It's an evolutionary process where you'll gradually gain the experience and confidence that you need to undertake the role. While the term "software developer" is fairly well understood, "software architect" isn't. This essay looks at the role I believe a good hands-on software architect should play on any software project.

Read the full essay in our book.