Some architecture needs to be done up front, and some doesn't
One of the major points of disagreement about software relates to how much up front design to do. People are very polarised as to when they should do design and how much they should do. From experience of working with software teams, the views basically break down like this.
- We need to do all of the software architecture up front, before we start coding features.
- Software architecture doesn't need to be done up front; we'll evolve it as we progress.
- Meh, we don't need to do software architecture, we have an excellent team.
These different views do raise an interesting question, how much architecture do you need to do up front?
It comes back to methodology
One of the key reasons for the disagreement can be found in how teams work, and specifically what sort of development methodology they are following. If you were to draw the major methodologies on a scale, you'd have something like the diagram below.
At one end of the scale you have waterfall that, in it's typical form, suggests big design up front. And at the other end you have the agile methods that, on the face of it, shy away from doing architecture. At this point it's worth saying that this isn't actually true. Agile doesn't say "don't do architecture", just as it doesn't say "don't produce any documentation". Agile is about sufficiency, moving fast, embracing change and delivering value. But since agile methods and proponents don't put much emphasis on the architectural aspects of agile projects, many people have misinterpreted this to mean "agile says don't do any architecture".
Sitting between the ends of the scale is the Rational Unified Process (RUP). Many RUP implementations are heavyweight monsters that have more in common with waterfall than they do with other iterative and incremental methods, but RUP can be scaled down to exhibit characteristics that let it take the centre ground on the scale. At its heart, RUP is a risk-driven methodology that basically says, "gather the majority of the key requirements, get the risky stuff out of the way, then iterate and increment". Done right, RUP projects can have a nice balance of up front design and evolutionary architecture.
You need to do "just enough"
My approach to up front architecture and design is that you need to do "just enough". If you say this to people they either think it's an inspirational breath of fresh air that fits in with all of their existing beliefs or they think it's a complete cop out! "Just enough" works as a guideline but it's vague and doesn't do much to help people assess how much is enough. If software architecture introduces structure and guidelines, leading to consistency and clarity, we can extend this further to say that you need to do just enough architecture to give you clarity and vision. In other words, do enough so that you know what your goal is and how you're going to achieve it. This is a better guideline, but it still doesn't provide any concrete advice.
Architecture represents the significant decisions
The key is that architecture represents the significant decisions, where significance is measured by cost of change. In other words, it's the stuff that's really expensive to modify and the stuff that you really do need to get right as early as possible. For example, qualities such as high performance, high scalability, high security and high availability generally need to be baked into the foundations early on because they are hard to retrofit into an existing codebase. It's also the stuff that you can't easily refactor in an afternoon; such as core technology choices, "architectural" patterns, core frameworks and so on.
Back to RUP for a second, and it uses the term "architecturally significant", advising that you should figure out what might be significant to your architecture and understand how you will resolve those forces. What might be significant? Well, it's anything that's costly to change, is complex (e.g. tricky non-functional requirements) or is new. In reality, these are the things with a higher than normal risk of consequences if you don't get them right. The significant elements are often subjective too, varying depending on the experience of the team. What you have here then is an approach to software development that lets you focus on what's risky in order to build a sufficient and robust set of foundations to move forward with. RUP is inherently a risk-driven methodology but the identification of architecturally significant elements and their corresponding risks is something that can be applied to all software projects, regardless of methodology. Some agile projects already do this by introducing an "iteration/sprint zero". Call it "risk-driven agile" if you like.
How much is just enough?
How much architecture do you need to do then? I would say that you need to do just enough in order to understand how the significant elements fit together (i.e. so you have a view of the big picture), mitigate the key risks, provide the foundations and create a vision to move forward with. Some architecture usually does need to be done up front, but some doesn't and can naturally evolve. Deciding where the line sits between mandatory and evolutionary design is the key.