Software development is not a relay sport
Successful delivery is not an implementation detail
Software teams that are smaller and/or agile tend to be staffed with people that are generalised specialists; people that have a core specialism along with more general knowledge and experience. In an ideal world, these cross-discipline team members would work together to run and deliver a software project, undertaking everything from requirements capture and architecture through to coding and deployment. Although many software teams strive to be self-organising, in the real world they tend to be larger, more chaotic and staffed only with specialists. These teams, therefore, tend to need and have somebody in a technical leadership role.
There are a lot of people out there calling themselves "solution architects" or "technical architects" (particularly in larger organisations) who define a software architecture and then throw their solutions over the wall to a separate design and development team. What's worse is that often nobody takes ownership of the architecture, and the original architect takes only a cursory glimpse at the progress from time to time.
Software Architecture for Developers coming to Oslo and London in May
A short note to say that Software Architecture for Developers is coming back to Oslo and London in May. What's it about? Well, in a nutshell, it's about the software architecture role, regardless of whether that role is undertaken by a single person or shared amongst everybody on a self-organising agile team.
Designing software given a vague set of requirements and a blank sheet of paper is a good skill to have, although not many people get to do this on a daily basis. Software Architecture for Developers is a two-day training course about "just enough" software architecture where you'll enjoy a mix of listening, discussing and practicing software architecture. Here's an example of how we practice.
Places for Oslo and London can be booked through Bouvet and Skills Matter respectively. Private courses are also available and can be run at your office (please e-mail me for more details).
Deliberate practice, effective sketches
Creating a collection of simple diagrams to communicate your vision
In Where are the software architects of tomorrow?, I talked about how most software developers don't get to take a blank sheet of paper and design software from scratch all that frequently. Related to this is the process of communicating those designs, either during (e.g. when collaborating) or after (e.g. when documenting) the architecture process. In concert with just enough architecture, my preference is for effective sketching rather than comprehensive UML models.
An initial set of sketches
Back at the start of March, I ran a tutorial at the QCon London 2011 conference entitled "Designing software, drawing pictures". After a short presentation to ensure a common starting point, everybody that came along was split into small groups and asked to design a software system given a brief set of requirements. Pretty much no guidance was given to the groups other than "choose some technology, design some software and draw some pictures to portray your vision". Here are the sort of pictures that were drawn.
You've probably seen pictures like this before. They're okay if you're explaining while you're drawing but they tend to show too much detail, mix abstractions and leave some things to your imagination.
A second attempt
To counter this, I tend to use a collection of simple diagrams, each of which focusses on a specific level of abstraction ... context, containers, components and classes. Back to the QCon tutorial and, after some additional guidance, the groups were asked to revise their diagrams. A short amount of time later and here are the sort of pictures created second time around.
Much better! Even without knowing anything about the software being designed the pictures look simpler, less cluttered and clearer. When you do know something about the software though, the new pictures allow you to get a feel for what's going on and understand the vision behind the design much more easily because each picture only shows a single level of abstraction.
You can see some more photos at http://www.codingthearchitecture.com/presentations/qconlondon2011-designing-software-drawing-pictures/ and chapter 3 of our book provides more information on the type of pictures that I draw. If you want to practice drawing effective sketches yourself, I'll be running a similar session at the GOTO conference in Copenhagen and we cover it on our Software Architecture for Developers training course.
Everybody is an architect, except when they're not
Single point of responsibility or shared amongst the team?
I went to a couple of great presentations on the "Software Craftsman" track at the QCon London 2011 conference last week that, together, provide a really easy way to explain how software architecture fits into modern software development teams.
Everybody is an architect
The first session, entitled Craft and Software Engineering by Glenn Vanderburg, explored the relationship and conflict between the craft and engineering of physical construction and software development. Really interesting session and I recommend downloading the slides or watching it if you get the opportunity. For me, one of the eureka moments was when he presented this picture from his Extreme Programming Annealed essay. Basically, it shows the scale at which the XP practices work.
I've highlighted collective ownership. Yes, it is about the code but collective ownership implies that everybody on the team needs at least some basic understanding of the "big picture". Think about your current project; could you jump into any part of the codebase and understand what was going on?
Imagine if you did have a team of experienced software developers that were all able to switch in and out of the big picture. A team of genuinely hands-on architects. That team would be amazing and all of the concerns you usually associate with software architecture (non-functionals, constraints, etc) would all get dealt with and nothing would slip through the gaps. From a technical perspective, this is a self-organising team. Think of them as a solid concrete dam blocking a fast flowing river, through which nothing can escape.
Except when they're not
My big problem with the whole self-organising team idea is that I rarely see it in practice. This could be a side-effect of working in a consulting environment in that my team always changes from project to project and I don't tend to spend more than a few months with any particular customer team. Or, I suspect that true self-organising teams are very few and far between. Striving to be self-organising is admirable, but most teams have bigger problems and the sort of scenario played out in my recent Software Project SOS session is indicative of the problems that I see.
And that brings me on to the second session, Team Leadership in the Age of Agile by the guitar playing Roy Osherove. Again, the slides are available to download. The session was about what you need to do in order to lead a software team, highlighting the fact that not all teams are equal. Roy categorises teams using a simple maturity model, which you can see described in more detail at The 3 maturity stages of a software team and how scrum fails to identify them. The three stages are: chaotic, learning/maturing and self-organising/matured. In essence, each maturity level requires a different approach to leadership.
As I said above, a team where everybody was an experienced software developer/architect would be amazing but this isn't something I've seen happen. Most projects don't have *anybody* on the team with experience of this "big picture" stuff and this is evidenced by codebases that don't make sense (big balls of mud), designs that are unclear, systems that are slow and so on. This type of situation is the one I see the most and, from a technical perspective, I recommend that *one* person on the team takes responsibility for the software architecture role. Roy also suggests that most teams are in the chaotic stage and, similarly, need more of direct leadership approach early on.
Whether you're talking about team leadership or technical leadership, the principle is the same. The chaotic team is like damming a fast flowing river with a bunch of twigs. It'll slow the water down for a very short amount of time but it soon becomes ineffective and you'll struggle just to keep stationary. A more direct leadership approach in these early stages will show you what you can't see and allow for some solid advice on which holes should be filled immediately.
Agile projects still need software architecture
Unfortunately, many teams view the "big picture" technical skills as an unnecessary evil rather than an essential complement, probably because they've been burnt by big design up front (BDUF) in the past. Some are also so focussed on the desire to be "agile" that other aspects of the software development process get neglected. Chaos rather than self-organisation ensues yet such teams challenge the need for a more direct leadership approach. After all, they're striving to be agile and having a single point of responsibility for the technical aspects of the project conflicts with their view of what an agile team should look like. This conflict tends to cause people to think that agile and architecture are opposing forces and that you can have one or the other but not both. It's not architecture that's the opposing force though, it's BDUF.
First things first then. Agile software projects still need architecture because all those tricky concerns around complex non-functionals and constraints don't go away. It's just the execution of the architecture role that differs.
As Roy said in his talk, a self-organising team doesn't need a dedicated ScrumMaster. Likewise, it doesn't need a dedicated software architect either. With collective code ownership, everybody needs to be able to work at the architecture level and so everybody *is* an architect. Teams that aren't at the self-organising stage will struggle, though, if they try to run too fast. Despite people's aspirations to be agile, collective code ownership and a distribution of the architecture role is likely to hinder chaotic teams rather than help them. Chaotic teams need a more direct leadership approach and they will benefit from a single point of responsibility for the technical aspects of the software project. In other words, they will benefit from a single person looking after the software architecture role. Ideally this person will coach others so that they too can help with this role.
One software architect or many? Single point of responsibility or shared amongst the team? Agile or not, the software architecture role exists. Only the context will tell you the right answer.
Load Testing for Developers
A one day training course to get you started load testing your web applications
I'm pleased to introduce Load Testing for Developers - a one day training course to get you started load testing your web applications.
Have you ever built a software system and your users have complained that it's too slow? I have; debugging live performance and scalability issues with business sponsors watching over your shoulder isn't fun! Load testing is an often forgotten and seemingly difficult task that many people shy away from but a basic level of load testing is often enough to give you confidence that you've satisfied expectations regarding performance and scalability. It's a natural extension to the software architecture, technical lead and software developer roles.
Can you tell if the following software systems will perform and scale acceptably?
No, of course you can't. You only guess that your design will work by looking at diagrams or source code, yet few people actually take the time to ask this question, let alone answer it. Testing provides confidence and this is what the course is about.
Load Testing for Developers is now available for private on-site delivery and public courses are being scheduled.
I need a price
Look at the bigger picture and collaborate, or fail
We all know the problems associated with fixed price, fixed scope contracts for software projects. Yes, there are alternative ways to run a software project, but sometimes you just need to quote a single price. Perhaps you need to secure internal budget or maybe you're a consulting company bidding for work. Either way, getting the cost wrong can have some huge consequences for everybody involved. Let's imagine that you *do* need to provide an estimate of the effort required to deliver some software. Sounds technical, let's ask the developers.
More often than not, asking software developers to produce estimates as the basis of a fixed price, fixed scope contract is a really bad idea. You see, we software developers tend to have a very technology-centric view of the world. We spend most of our days staring at code in our IDEs and we like to talk about things like classes and dependency injection and refactoring and automated testing and a whole bunch of other deeply technical stuff. There's nothing wrong this, and if you're not thinking about these sorts of things then you're probably in the wrong job.
Step back to see the bigger picture
The problem with software developers costing up software projects comes from not leaving that world behind and stepping back to see the bigger picture. If you're trying to cost up a project while having discussions about the number of fields on a user interface or whether the data should be stored as XML or broken out into a full relational model then you're going about the estimation process in the wrong way.
Sure, these things do become important at some point in time, but they probably don't make a great deal of difference in the grand scheme of things. What does? It's the major architectural drivers ... the functional requirements, the non-functional requirements, the constraints and the principles. Instead of thinking about the details, you need to be asking questions like...
- What are the major drivers for the software? What problems is it solving? If it's replacing an existing software system, what's wrong with the old one? What are the benefits delivered by the software?
- Are there any inter-system interfaces? What are they? Who owns them?
- Are there non-functional requirements (e.g. security, audit) or constraints (e.g. regulatory) that will prevent the system from going live?
- What technologies are already in use and what technology choices do we realistically have given the environment?
- Who will be operating, supporting and maintaining the system in the future? What sort of skills do they have?
- etc
These sorts of questions will help you comprehend the size and scale of the software project much more than understanding how many columns you need to store the data.
Most requirements documents are inadequate for estimation
Most of my career has been spent in a consulting environment and I've seen my fair share of requirements specifications over the years. Without doubt, I can say that none of them have been good enough to produce an estimate from. Look at it from the other side of the coin. Imagine you have a vision for a software project that you've been living and breathing for a few months. You think about it, you talk about it and you visualise it with colleagues. Then, in order to engage a development team, you open up Microsoft Word and start typing up a requirements document. The thing is, the reader probably doesn't have the same amount of context that you do. Those one-liners you've used to describe the major features are likely to seem ambiguous to people that haven't lived and breathed the vision like you have. Customer collaboration over contract negotiation anyone?
Put simply, it's unrealistic to expect somebody to produce a cost estimate from a few pages of "requirements", particularly since that document normally doesn't include all of the other essential information about the environmental constraints, policies, technologies, etc. If you find yourself in the position of having to provide a ballpark estimate, especially if it's going to be used as the basis for a fixed price contract, the best piece of advice I can give you is to collaborate. Grab a room with a whiteboard, invite some of the key stakeholders and use this as a forum to understand the vision. Oh, and don't forget to get those risks, issues, assumptions and dependencies out into the open.
Where are the software architects of tomorrow?
With many technical mentors disappearing, where do developers gain this experience?
Agile and software craftsmanship are two great examples of how we're striving to improve and push the software industry forward. We spend a lot of time talking about writing code, testing, tools, technologies and the all of the associated processes. And that makes a lot of sense. Let's not forget that the end-goal here is delivering benefit to people through software, and working software is key.
But we shouldn't forget that there are some other aspects of the software development process that few people genuinely have experience with. Think about how you would answer the following questions.
Collaborative design
Everybody's different ideas need to meet
Let's imagine that you've been tasked with building a 3-tier web application and you have a small team that includes people with specialisms in web technology, server-side programming and databases. From a resourcing point of view this is excellent because, together, you have experience across the entire stack. You shouldn't have any problems, right?
The effectiveness of the overall team comes down to a number of factors that include people's willingness to leave their egos at the door and focus on delivering the best solution given the context. Sometimes, though, individual specialisms can work against a team; simply through a lack of team-based experience or because ego gets in the way of the common goal. Ask the specialists in the team where a certain feature or component should go and you might get 3 totally different answers...
Tutorial @ QCon London 2011
Understand the drivers and collaborate to do just enough architecture
I'm running a tutorial at the upcoming QCon London conference entitled Designing software, drawing pictures. In essence, it's about the process of designing software if all you have is a blank sheet of paper and a set of vague requirements. Much of what we write about on this website is about software architecture ... defining structure, putting sufficient foundations in place, laying out the vision for a software system and then carrying that through the project to a successful conclusion. It's really just another angle to software craftsmanship and I firmly believe that software architecture needs to be done on most software projects, but of course it doesn't have to be about "big design up front". Even the most agile of projects can benefit from the introduction of the consistency and clarity that lightweight software architecture brings.
Why?
So why the tutorial? Well, it's a 1-day version of our Software Architecture for Developers training course where you'll be asked to design some software, choose some technologies and draw some diagrams to illustrate your design. Most people I've worked with, trained and coached don't tend to design a software system from scratch all that frequently (if at all) and it therefore follows that this is the area where most people lack experience. This tutorial is about providing developers with the opportunity to design something from scratch while receiving guidance about how to simplify the design and how to illustrate that design using a collection of simple pictures. After all, if we're losing our technical mentors to non-technical management positions, how do the software architects of tomorrow gain experience in this area?
You can find more details of the tutorial on the QCon London 2011 website. Join us if you want to know how to go about the software design process in a lightweight, structured and pragmatic way.
Just enough architecture
Some architecture needs to be done up front, and some doesn't
One of the major points of disagreement about software relates to how much up front design to do. People are very polarised as to when they should do design and how much they should do. From experience of working with software teams, the views basically break down like this.
- We need to do all of the software architecture up front, before we start coding features.
- Software architecture doesn't need to be done up front; we'll evolve it as we progress.
- Meh, we don't need to do software architecture, we have an excellent team.
These different views do raise an interesting question, how much architecture do you need to do up front?
