Coding the Architecture - agile tag

Is your development process about to be eroded?

Sun, 25 Jan 2009 14:35:42 GMT

The following quote from The Economist (Jan 3rd-9th) provides a reasonable analogy for what some development teams are experiencing as once free-flowing technology investment starts to dry up:

For the more curmudgeonly sort of older manager, the current recession is the joyful equivalent of hiding an alarm clock in a sleeping teenager's bedroom. Once again, the touchy-feely management fads that always spring up in times of plenty are being ditched in favour of more brutal command-and-control methods.

The article refers to how the relationship between old-school management and the net generation is shifting back in favour of the former's approach. While it wouldn't be correct to equate "touchy-feely" with "agile", some relationship is apparent.

I think this may ultimately be for our own good: a chance to experience how things might otherwise be done and to see whether our processes are really as perfect as we preach. Even Toyota, often cited by agile evangelists, is (understandably!) struggling, albeit far less than their american counterparts. Similarly, our software development processes will not make us immune from a downturn in business, however responsive they are to change.

As part of the interface between the business and the technology teams, architects may find themselves in an increasingly divisive position. On the one hand they may need to defend the progress (and investment) that has been made in coaching and process change. On the other, they may have to concede that lower-quality, shorter-term thinking may be required in difficult times. In either case, it would seem prudent to be proactively ensuring that your assumptions, requirements and, ultimately, the resultant process and architecture remain appropriate now that the business drivers may be shifting, if they haven't done so already.

How much architecture is enough?

Mon, 08 Sep 2008 16:01:00 GMT

Thanks to everybody that came along to our "in *our* brains" session last week where we tried to answer the question of how much architecture is enough. The event was part of Skills Matter's London Agile Month and it was a great turnout where we had some interesting disucssion ranging from what agile is all about through to how architecture tasks should be tackled during an agile project.

The set of slides that I used to set the scene can be downloaded from here (PDF, 0.9MB) and the video from the session can be watched from here (this includes the presentation and the discussion). Also, Alberto Brandolini has a really good blog entry, which covers the session along with some of his own thoughts that followed. See you next month!

How much architecture is enough?

Mon, 01 Sep 2008 10:11:25 GMT

This Wednesday, after the training course and as a part of Skills Matter's London Agile Month, we'll be running a session entitled "How much architecture is enough?". Although this is billed as "In the Brain of Simon Brown", it's actually going to be an "In *our* Brains" session where a few of us will be facilitating a group discussion.

As for how much architecture is enough, well we've touched upon this question before, particularly with respect to the level of design detail you should put in your architecture document and the need for just enough quality. While I don't think that software development is a relay sport, if you are planning on handing off your architecture, how much do you need to do to ensure that this exercise is successful? And on the subject of sharing architectures, how much should you share and are there more creative ways of efficiently delivering your message?

There's lots to talk about and if this stuff interests you, please sign up on the Skills Matter website. This will be the September London User Group and the session starts at 6:30pm.

Why do so many technology projects fail?

Fri, 23 Nov 2007 14:48:00 GMT

The Financial Times published an interesting article earlier in the week entitled Perspectives: Why do so many technology projects fail? (registration required) detailing a court action between BSkyB and EDS. In summary, it's about the failure to deliver a software project, how software projects aren't like construction projects, how a new way of working is needed, etc, etc. It's all stuff we've heard before and, as the article says, are lessons never learned?

Why do so many technology projects fail? Here are my thoughts, some of which overlap with the FT's article.

Iterative and agile techniques have revolutionized the way that software development is performed, but our industry needs to take a step back and look at the way in which software projects are engaged. Why, when you read about so many high profile big budget software failures, do businesses still initiate software projects with "we want this, tell me how much it will cost"? *We* know that they'll change their mind. *They* know that they'll change their mind. So let's change the engagement model, stop hiding behind fixed price contracts and work *together* to solve problems.
In my opinion, the view that software development is a commoditized skill is wrong. This is regardless of whether software is developed on-shore, off-shore, etc. Software development is a discipline that's part engineering and part art. If you want decent software built, don't treat it like a commodity. You wouldn't want anybody to build you a Ferrari, would you?
And finally, of course, there's the lack of a technical architect. Not all architects sit in ivory towers throwing unworkable "solutions" at development teams. Some of us like to work *in* the development team and eat our own dog food. Think of how many software projects you've seen that function correctly but are not fast enough, scalable enough or secure enough. That's why you need an architect.

I was chatting to Sam yesterday and he has some interesting thoughts too, which I'm hoping he'll write about one day. Something that we both agreed on though, is that the software industry generally needs to learn some lessons and wake up to some of the shortcomings that are so prevalent today.

Real Systems Develop Organically

Thu, 19 Jul 2007 08:55:37 GMT

Often software isn't designed and built like a bridge. It's not engineered and left in a fixed state. It's more like a city - growing from a small hamlet into a major city like London. Most of London wasn't planned, it just sort of happened. Interestingly, the bits that weren't planned are often better places to live and work than the bits that were. Usually because people built what they wanted, when they wanted it.

A 'typical' software product in financial services (I'm sure this will ring a bell in other businesses) starts out as a spreadsheet. A customer rings a consultant/trader/salesperson and asks for something unusual. Smelling a fat commission they price up the request (adding a huge 'spread') and the customer goes away happy. Often that's as far as it goes but sometimes another customer phones up asking for something similar, so they reuse the spreadsheet.

As the requests increase the spreadsheet gets more complex and other employees (traders on the desk) start using it too. Often the figures the spreadsheet produces are updated automatically and published onto a message bus like Triarch (finance messaging system), chat system or emailed. This in turn is used by other spreadsheets, VBA apps and the like.

Two years later IT is called in, possibly because "an important application failed yesterday". You discover an application with 100 users that's spread across multiple desktop machines. It all sort of hangs together, it's not documented and there is no backup policy. If one of the users running part of this application goes on holiday they have to tell someone their password so they can run it up. If they quit then... All the users love what it does, even if it's nasty to use, and don't want it to change. It generates several million in profit every year. It's now your job to stop this being a disaster waiting to happen while keeping the users happy. Sounds familiar?

To stretch my analogy to breaking point, this is similar to the problems the Victorians had in London when they wanted to supply water, sewage and power to old parts of the city. Do you try and provide the necessary infrastructure to what already exists or do you knock it down and replace it with something designed from scratch?

In theory you should be able to estimate the costs of both (don't forget to include any lost revenue, training time, mistakes with a new system, long term support costs etc) and choose the cheapest. In practice an organically grown IT system tends to have spread much more widely than you'd dream possible. On one occasion I've received an angry phone call from an IT manager in Zurich because a file was no longer being generated that he needed. The file appeared to be a side effect of a system I was replacing and was no longer needed - except his system was automatically doing an FTP for the file and using it as a risk input to a trading system. How was I to know it did that?

Unlike a physically growing thing like a city, it's just not easy to see what you affect. The strategy that seems to work best is to slowly replace systems a little at a time and run the replacement in parallel. You have to test that the new functionality works but also make sure you're not side-effecting something else entirely and unexpectedly. You may (or may not) be surprised at how many systems run for years in undead mode, with no one daring to turn them off. You need to avoid this as the system will become harder to support over time as those who know how it works leave. Disable the system to start with but make sure you can re-enable if you have to. Only remove it when it has been disabled for a reasonable period of time.

Organically grown systems are common and often successful. I'd advise not ripping them out but viewing them as an agile system that needs some process and refactoring. Does anyone else have similar war stories?

Speculative design

Thu, 09 Feb 2006 10:59:58 GMT

From Agile Answers comes the following paragraph (emphasis added), which is pretty timely given the conversations I've had recently.

Do the simplest thing for the task at hand

Originally a pushback against speculative development (particularly in code frameworks), this guideline is very easy to distort if you want to attack agile methods. Developing incrementally does not (in my opinion) mean taking your brain out before starting. I generally have an idea of where I'm going when developing, based on knowledge and experience. I use that mental roadmap to decide how to write the next test. Sometimes I'll skim through and try a quick implementation of part of a feature to see if it looks right (quick meaning minutes, not days). Then I'll write the tests that exercise my intended implementation, which will probably change it slightly to make it more testable. Code is clay, not marble. Or at least it should be.

For me, the logical extension to this is to talk about iterative and incremental development in general. Take a common component. With a traditional waterfall methodology, common components are typically planned, designed and built upfront before being baselined as complete and ready to use. While this works, problems can arise later on in the project when you realise that the common component doesn't quite support what you need or, worse, provides functionality that you're never going to need. This happens because you speculated what was needed too early.

Iterative and incremental development provides a way for you to "do the simplest thing for the task at hand", yet it's easy for teams to fall into the trap of doing speculative design because they like to have a fully planned out schedule before starting off on the project, which is contrary to the adaptive planning approach recommended for iterative/agile development. As Darren says, you don't take your brain out before starting an incremental project - you still need to know where you're going.

Those common components will get built, but they'll be built when they're needed and, as a result, they'll support exactly the features that are needed. If you do build common components outside of the context in which they will be used, what do you say when somebody asks you whether they're finished? How do you know when you're finished?

How long is a timebox?

Mon, 23 Jan 2006 16:52:26 GMT

One of the most common questions about timeboxing relates to the length of timeboxes. How long should they be and what factors influence this decision?

While there are many recommendations out there, some of which are strongly tied to the development process you've adopted (e.g. Scrum recommends a month), the factor I would like to talk about is the skill of the team. Or rather, how much relevant experience they have. By relevant experience, I mean experience with whichever development process, team structure, technologies, tools, etc that are in use on the project. Based upon this information, you should be in a good position to find the ideal length of timebox that suits your project.

For very experienced teams, a short duration (say 2 weeks) allows for some very aggressive timescales and some very quick feedback loops, assuming that you would plan to do a release at the end of each timebox. On the other hand, a longer duration (say 4-6 weeks) is more suited to projects that are larger, more architecturally challenging or where the team is on a steep learning curve with the process, technologies, tools, etc. From experience, problems arise when you put a team that's on a steep learning curve into very short timeboxes and expect them to deliver. It's just not realistic to expect them to take onboard lots of new technologies *and* expect them to deliver something that's production quality.

The ideal length of a timebox for any particular project will vary and deserves some careful thought. Of course, adopting an iterative approach means that you have the flexibility to tweak the timebox duration throughout the project based on your growing experience.

Here at The Pragmatic Architect, we're interested to see what you you have to say. What are experiences here? What timebox duration works for you and why?