Mind the gap

Reducing the gap between software developers and software architects

Introduction

Our industry has a love/hate relationship with the software architect role, with many organisations dismissing it because of their negative experiences of architects that dictate from "ivory towers" and aren?t engaged with the actual task of building working software. This reputation is damaging the IT industry and inhibiting project success. Things need to change. This essay looks at the gap between software developers and software architects, offering some suggestions on how to reduce this gap and ensure projects are driven to a successful conclusion.

Developers focus on the low-level detail

If you're on a software development project at the moment, take a moment to look around at the rest of the team. How is your team structured? Does everybody have a well defined role and responsibilities? Who's looking after the bigger picture - things like performance, scalability, availability, security and so on?

We all dream about working on a team where everybody is equally highly experienced and is able to think about the project right from the code through to the architectural issues, but the real world just isn't like that. My experience suggests that most teams consist of a number of people with different skills and aspirations, some of who will be new to IT and others who have "been around the block a few times". Most of the teams I've worked with have been like this. I've also worked with some very, very good developers. Some of them are highly motivated by solving problems at the code level, adopting new technologies and getting a creative buzz out of the whole process. Others want to understand about the higher level architectural issues and the context in which they are working. In most of these teams, however, the developers typically only focussed on the low-level issues, whether this was through choice or through time pressures.

Developers typically focus on low-level issues

From my own experience, I know of a couple of systems that have been built by some very good developers. All of the latest language features have been used, the code has used interfaces to decouple dependencies, unit tests were automated and the codebase was structured and formatted to perfection. However, the system itself just didn't scale, with some serious performance problems arising when that system was eventually deployed into production.

Architects dictate from their "ivory towers"

Ivory tower architecture increases the gap between developers and architects

My view on this is that every software project should have some architectural guidance and, ideally, there should be a single person responsible for the technical quality of that project. It is possible for this responsibility to be spread throughout the team, but the team needs to be a manageable size and have good communication to make this a success.

Although architectural guidance is good, many projects tend to bring in an architect and stick them on a plateau above the rest of the team. In addition to immediately isolating the architect from the team, this just exaggerates the gap between the the architect and the development team.

A lot of people that I've spoken to don't like the perception that "an architect role is more advanced than a developer role". And I agree. It's not necessarily more advanced, it's just different. Some people view it as a step up from being a developer, and some view it as a step sideways. However you view it, a common view is that the architect role is about looking at the "bigger picture" to make sure everything comes together as expected. My own perspective on this is that the architect should be the one responsible for the technical quality of the solution and the person assuring that the chosen architecture satisfies the non-functional requirements and system qualities. After all, somebody has to do it.

Reducing the gap

Unfortunately, many software teams have an artificial gap between the development team and the architect (if they have an architect). This leads to several problems.

Important decisions fall between the gap because responsibilities aren't well defined.
The development team don't respect or follow the decisions made by the architect.
Project teams don't value architectural input and don't include an explicit architecture role in new project teams.
Projects eventually suffer because nobody has been tasked with looking at the bigger picture, which typically includes the non-functional requirements or system qualities.

The gap between developers and architects needs to be removed

Fortunately, there are some simple ways to address this problem, from both sides.

Architects	Developers
Be inclusive : Get the development team involved in the architecture definition process to help them understand the bigger picture and buy-in to the decisions you are making as a team. Get hands-on : If possible, get involved in some of the day-to-day development activities on the project to improve your understanding of how your architecture is being delivered. Depending on your role and team size this might not be possible, so look at other ways of retaining some low-level understanding of what's going on. Do rather than delegate : If you can't be involved in the hands-on tasks, try to make some time to be involved in those tasks that you might normally delegate to other team members such as design and code reviews. Having some understanding of how the software works will give you a better insight into how the development team are feeling about the architecture (e.g. whether they are ignoring it!) and it will provide you with valuable information that you can use to better shape/influence your architecture.	Understand the bigger picture : Taking some time out to understand the bigger picture will help you understand the context in which the architectural decisions are being made and enhances your understanding of the system as a whole. Challenge architectural decisions : With your new-found understanding, you now have the opportunity to challenge the architectural decisions being made. Architecture should be a collaborative process and not dictated by people that aren't necessarily engaged in the project day-to-day. If you see something that you don't understand or don't like, challenge it. Ask to be involved : Many projects have an architect that is responsible for the architecture and it's this person that usually undertakes all of the architecture work. If you're in this position and you want to get more involved, just ask.

Architects

Developers

Be inclusive : Get the development team involved in the architecture definition process to help them understand the bigger picture and buy-in to the decisions you are making as a team.
Get hands-on : If possible, get involved in some of the day-to-day development activities on the project to improve your understanding of how your architecture is being delivered. Depending on your role and team size this might not be possible, so look at other ways of retaining some low-level understanding of what's going on.
Do rather than delegate : If you can't be involved in the hands-on tasks, try to make some time to be involved in those tasks that you might normally delegate to other team members such as design and code reviews. Having some understanding of how the software works will give you a better insight into how the development team are feeling about the architecture (e.g. whether they are ignoring it!) and it will provide you with valuable information that you can use to better shape/influence your architecture.

Understand the bigger picture : Taking some time out to understand the bigger picture will help you understand the context in which the architectural decisions are being made and enhances your understanding of the system as a whole.
Challenge architectural decisions : With your new-found understanding, you now have the opportunity to challenge the architectural decisions being made. Architecture should be a collaborative process and not dictated by people that aren't necessarily engaged in the project day-to-day. If you see something that you don't understand or don't like, challenge it.
Ask to be involved : Many projects have an architect that is responsible for the architecture and it's this person that usually undertakes all of the architecture work. If you're in this position and you want to get more involved, just ask.

Your mileage may vary

What I've talked about here is easily applicable to small/medium project teams, but things do start to get complicated with larger teams. By implication, a larger team means a bigger project, and a bigger project means a bigger "bigger picture". I don't have all the answers to how an architect on such a project retains a hands-on involvement in the actual software development process. In fact, I've seen projects where this just isn't possible or feasible. One successful approach I have seen is like a "scrum of scrums"; where there is a team of architects and each takes responsibility for a small chunk of the overall architecture. Your mileage may vary, but the key point is that you should try to reduce the gap between the architect(s) and the development team.

Summary

Ensuring that the bigger picture isn't neglected is crucial for success and this typically falls squarely in the role of the architect. However, most software teams can benefit from reducing the artificial gap between developers and architects, with the gap itself being reducible from both sides. Developers can increase their architectural awareness, while architects can improve their collaboration with the rest of the team. Reducing the gap will help to ensure that the right decisions are made and that those decisions are being followed.