Structure and guidelines, consistency and clarity
A couple of days upfront can really make a big difference to the big picture
One of my key points about software architecture is that it introduces structure and guidelines into a software system, which in turn leads to consistency and clarity of the overall design. Basically, I'm saying that there are some real benefits from thinking about your design upfront. This isn't a particularly groundbreaking statement, but you'd be surprised how few teams spend some quality time to do this. As I mentioned in Estimating a software system, this thinking doesn't need to take very long at all.
If you take a simple lightweight approach, you can map out the major functional requirements and come up with a design down to the major components in a couple of days. Then, you can do some high-level estimating off the back of this design. There are a number of ways that you can map out the functional requirements, although I find that user stories are a good way to summarise the key requirements.
Thinking about your software design on your own is great, but making it a collaborative exercise is even better and comes highly recommended. We're doing this at the moment for another project where each of us on the team has specialisms in different technology aspects of the overall solution. The benefit of this is that we all know how the technology in our own areas works, but the details of how those technologies collaborate is often unknown. Also, we sometimes tend to assume what responsibilities each technology will take on and it's only by discussing the overall solution as a group that we realise we've either doubled up on something or assumed that somebody else was looking after it. Essentially, software design as a collaborative exercise allows everybody's vision to meet so that the end-result is consistent and clear. As I've written before, often some of the basic foundations are missed.
For me, explicitly thinking about the components that make up the software system introduces structure, which in turn can be used as a template for building that system. From this, you get a consistent approach to solving common problems and a thought out design with a clear architectural vision. A couple of days upfront can really make a big difference to the big picture.
Do architects define the software development process?
Somebody has to!
There's some very interesting follow-up to a blog entry called If you’re an architect… on Edward Williams' blog and one of the questions that has just been posed is this.
Is defining the development process (agile, waterfall - XP/Scrum/DSDM/Lean etc) the responsibility of the technical architect or does it come under the enterprise architect / CIO / CTO's umbrella?
This may sound controversial, but I've met very few IT managers (project managers included) who understand that software doesn't have to be developed according to the waterfall model, and fewer still that have actively shaped projects to take advantage of those other approaches.
It's a really interesting question. As an architect on a software project, the overall development process is always something that *I* try to have some influence over. My reasons for this are (a) I've seen too many waterfall style projects fail and (b) you can use iterative and incremental approaches to help with many of the challenges you face as an architect. For example, rather than get freaked out by some complex requirements and head into analysis paralysis, why not structure the project in iterations and tackle those things first, perhaps by doing some lightweight architecture/design and writing some code that you can load test.
To be honest, I usually get involved in shaping the overall project because nobody else has considered it and the project is bumbling along in a fairly ad hoc, waterfall-ish fashion. I like to inject some structure into the way that software is built and the architectural drivers can really help to define what that structure should be. There are certainly some challenges to be overcome once you start getting involved in shaping the overall process though. These range from project managers that "just want a normal project plan" through to those that agree with your use of timeboxes but "still just want a normal project plan". These sorts of issues are relatively easy to overcome, but I do often find myself doing some just-in-time training about the different types of development process.
Should architects define the software development process? Who defines the software process on your projects?
How do you introduce software architecture?
It's easier to ask forgiveness than it is to get permission
I get my fair share of questions about software architecture; ranging from queries about the role through to "I have this problem, how would you design a solution"? This question, though, represents a fairly common situation but isn't one that I get asked a great deal. I'm paraphrasing a little, but essentially the question was:
I understand software architecture and I get the stuff that you talk about, but our team just doesn't have the time to do it because we're so busy coding our core product. Having said that, I know that we need some architecture because we don't have consistent approaches to solving problems, etc, etc. How do we introduce architecture?
From my point of view it's worth asking a few questions to understand the need for architecture.
- What problems is the lack of architecture causing now?
- What problems is the lack of architecture likely to cause in the future?
- Is there a risk that these problems will lead to more serious consequences (e.g. loss of reputation, business, customers, money, etc)?
In this particular case, after a short conversation and a little more exploring on my part, the reason that no architecture activities had been introduced was because:
Our managers won't give us time to do architecture. If we're doing architecture, we're not coding.
How much architecture should you do?
Okay, so one of the things that I tell people new to the architecture role is that they do need to dedicate some time to doing architecture work (the bigger picture stuff) but a balance needs to be struck between this and the regular day-to-day development activities. If you're coding all of the time then that bigger picture stuff doesn't get done. On the flip-side, spending too much time on "architecture" means that you don't ever get any coding done, and we all know that pretty diagrams are no use to end-users!
This is one of those questions that you can't really answer in a classroom based training course because it requires changes to the way that a software team works, and these can only really be made when you understand the full context of the team. On a more general note though, there are two ways that teams will change the way that they work.
Reactively changing
The majority of teams will only change the way that they work based upon bad things happening. In other words, they'll change if and only if there's a catalyst. This could be anything from a continuous string of failed system deployments or maybe something like a serious system failure. In these cases, the team knows something is wrong (probably because their management are giving them a hard time!) and they know that something needs to be done to fix the situation. This, unfortunately, appears to be the norm.
Proactively changing
Some teams (ironically, usually the better ones!) proactively seek to improve the way that they work. Nothing bad might have happened yet, but they can see that there's room for improvement to prevent the sort of situations I've just mentioned.
So how do you introduce software architecture?
Back to the original question then. In this case, it seemed to me that the person asking the question was doing so with a view to proactively improve their way of working. After all, if something bad had happened there probably would have been a push from "above" to get things fixed. Instead though, in effect, the team was asking permission to spend some time doing the architecture stuff but they weren't getting buy-in from their management. Perhaps their management didn't clearly understand the benefits of doing it or the consequences of not doing it. Either way, the team didn't achieve the desired result. Whenever I've been in this situation, I either take one of two approaches.
- Present in a very clear and concise way what the current situation is and what the issues, risks and consequences are if behaviours aren't changed. Typically this is something that you present to key decision makers, project sponsors or management. Once they understand the risks, they can decide whether mitigating those risks is worth the effort required to change behaviours.
- Be proactive and lead by example. For example, if I start working on a project that has no high-level technical documentation, I'll start to create it until other people see the benefits and start contributing. Automated unit testing is always a good example of this - people get it when they see it, but somebody needs to put the initial seeds in place.
Each approach tends to favour different situations, and again it depends on a number of factors. Coming back to the original question, it's possible that the first approach was used but either the message was weak or the management didn't think that mitigating the risks of not having any dedicated "architecture time" was worth the financial outlay. In this particular case, I would introduce software architecture through being proactive and leading by example. Find a problem (e.g. multiple approaches to dealing with configuration, no high-level documentation, a confusing component structure, etc) and just start to fix it. I'm not talking about downing tools and taking a few weeks out here! I'm talking about baby steps where you evolve the situation by breaking the problem down and addressing it a piece at a time. Take 30 minutes out from your day to focus on these sort of tasks and before you know it you've probably started to make a world of difference. Stealth is often the key. What they don't know won't hurt them and I find that "it's easier to ask forgiveness than it is to get permission".
Good code isn't enough
A free in-the-brain session at Skills Matter, London
A quick note to say that I'm presenting a free "In-the-brain" session at Skills Matter in London on the 8th of September.
Writing code is easy. Writing good code is still relatively easy if you know what you're doing. Yet delivering quality software on time and on budget is a whole different story! Some software teams are just happy continuing to work the way that they've always done whereas others will jump on every new process and technology that they can lay their hands on. Successful delivery is a goal that many software teams still strive for but find hard to achieve, often because they are distracted from focussing on what's important.
This session will look at why delivering a successful software project requires more than simply choosing some technology and throwing it together, showing how you can take the best bits from traditional and modern approaches to form a structured yet lightweight approach to software development. The session will cover the complete end-to-end software development process; from planning, gathering requirements and software architecture through to the effective use of source code control, automated unit testing, continuous integration and load testing. Good code isn't enough.
You can find more information and register on the Skills Matter website.
More layers = more complexity
Nothing is ever free
We had an interesting discussion on the course a couple of weeks ago that I thought was worth summarising here. One of the key functional requirements of the case study that we run through is that the system should be able to distribute data to a subset of users on the corporate LAN. Now there are 101 different ways to solve this problem, with one of the simplest being to allow the users to access the data via an internal web application. Since only a subset of the users within the organisation should be able to see the data, any solution would need some sort of authentication and authorisation on the data.
Given the buzz around Web 2.0, AJAX and RIA in recent times, one of the groups decided that it would be nice to allow the data to be accessed via a Silverlight application. They'd already thought about building an ASP.NET application but liked the possibilities offered by Silverlight (e.g. the ability to slice and dice the data interactively). Another driving factor for their decision was that the Silverlight client could be delivered "for free" in that it would take just as long as building an ASP.NET application. "For free" is a pretty bold claim, especially considering that they were effectively adding an extra architectural layer into their software system. I drew up the following summary of their design to illustrate the added complexity.
While I don't disagree that Silverlight applications aren't hard to build, the vital question they hadn't addressed was where the data was going to come from. As always, there are options; from accessing the database directly through to exposing some data services in a middle-tier. The group had already chosen Windows Communication Foundation (WCF) as the mechanism for exposing the data, but this led to yet further questions.
- What operations do you need to expose?
- Which technology binding do you use?
- How do you ensure that people can't plug in their own client and consume the services?
- ...
In the context of the case study, the third question is important. The data should only be accessible by a certain group of people and we really don't want to expose a WCF service that anybody with Visual Studio could consume. This led to discussion about the use of SSL to secure the service, but SSL only secures the transport layer to stop data being looked at in transit. In this case, some thought needs to be given to authentication/authorisation of the service itself.
Coming back to "it won't take longer than building an ASP.NET application" then. In this situation, the benefits brought by the additional Silverlight layer need to be considered alongside the additional complexity that's also been introduced. More moving parts means more work designing, developing, testing and deploying. Despite what it might say on the box, nothing is ever free and you need to evaluate the pros and cons of adding additional layers into a design, particularly if they result in communication between containers.
Where do you start? (video)
My session at the Skills Matter eXchange, London, May 2010
I've had a few people ask me whether there is a video to go with the slides from my "Where do you start?" talk. The answer is yes ... Skills Matter recorded the session that I presented at the Skills Matter eXchange in London a couple of months ago.
Feel free to get in touch if you have any questions.
Enterprise Software Developer training course
A new course, focussed on improving software development
I'm pleased to say that, from September, I'll be running a new training course called Enterprise Software Developer. Where Software Architecture for Developers focusses on the architecture and design elements of software projects, the new course expands upon this to cover the full project lifecycle.
"Enterprise Software Developer" is a four day practical training course about building software within an enterprise environment in a structured, lightweight and pragmatic way. It covers the complete end-to-end software development process; from planning, gathering requirements and software architecture through to the effective use of source code control, automated unit testing, continuous integration and load testing. Pragmatic enterprise software development is about taking the best bits from traditional and modern approaches, blending them together to form a structured yet lightweight approach to building software that's appropriate to the way that your organisation works. This is what the course is all about.
The course covers a number of different aspects of the software development process; from dealing with requirements through to the use of source code control, load testing and operational hand-over.
On the course, you'll learn how to...
- Choose the best development methodology for your project; whether that's agile, waterfall or something in-between.
- Gather and capture functional and non-functional requirements.
- Prioritise and estimate workload.
- Manage your software project and track progress.
- Define the software architecture for your solution.
- Use source code control and branching to manage change.
- Write automated unit and integration tests for confidence when refactoring and increased quality.
- Create an automated build script to manage release complexity and provide release consistency.
- Setup a continuous integration server to ensure continual quality control.
- Make your software production ready for release.
- Understand why configuration management is important and how to integrate it into your project.
- Write a load test script to undertake performance testing.
- Understand how to handover your system to an operational environment.
In addition to building software myself, I also undertake consulting engagements where I help software teams improve the way that they build software. I've been doing this for a number of years now and I'm confident in saying that while the above can really help to improve the way that teams build software, much of it still isn't regarded as "normal practice" for a large number of them. Some guidance in these areas really does go a long way. There are courses out there that look at automated unit testing, courses that teach TDD, courses that cover the use of agile techniques but not much that brings everything together. This is why I put the course together and, unlike Software Architecture for Developers, this course *does* include some coding and is offered in two flavours ... there's a .NET version and a Java version. Just let us know which you'd like to do when booking.
Estimating a software system
Start by decomposing the big picture
One of the things that we teach people on our Software Architecture for Developers training course is how to design software if all you have is a set of requirements and a blank sheet of paper. The approach that we present is based upon the way that we work ourselves, where we'll start with the big picture before breaking up the overall system into a number of different constructs.
I don't often talk about the requirements side of the software design process, but it's *really* important that some analysis is undertaken to understand exactly what needs to be built and why. I typically gather a first draft of the major requirements and, if the domain is new or complex, I'll additionally do some high-level domain modelling to flesh out the business concepts. Only then will I dive into the software design process, identifying the systems, containers and components.
The photos above summarise a software design exercise that I've been doing over the past couple of days. As it turns out, the business domain here *is* fairly complex but we needed to understand enough about it in order to be able to estimate how much it would cost to build a bespoke system to meet the requirements. It's been a very collaborative exercise where 3 of us have designed the system down to its components and services, using a CRC card approach. We've probably spent about a day looking into the business domain/requirements and the same designing the system. The project owes me £4.68 for the index cards and Blu-Tack that we used, but we've been able to use this approach to decompose the overall problem and put together some estimates as a result.
While you *can* estimate a software system top-down from the big picture, I find it easier and more accurate to decompose the overall problem into a number of logical components/services and estimate bottom-up. You don't need to do big design up-front, but some design up-front does have its benefits.
Where do you start?
Norwegian .NET User Group, 6pm, 24th August 2010
A quick note to say that I'm presenting a session called "Where do you start?" at the Norwegian .NET User Group (NNUG) on the 24th of August in Oslo.
Where do you start?
One of the hardest things about software development is being asked to come up with a design when all you're given is a set of requirements and a blank sheet of paper. Many software teams will dive straight into the code and while this can initially be very productive, the slippery slope of constant refactoring is awaiting those teams that haven't quite found a design that works. Often, a little forethought is all that's needed to get the development process heading in the right direction. So where do you start? This session will answer this question, presenting some simple techniques for tackling software architecture while dispelling the myths about the need for complex tools and big design up front.
I've presented this session a couple of times now, but this one will have a few tweaks and more of a .NET focus. You can find more information about this event on the NNUG site. I'm looking forward to the "geekbeer" after the session. :-)
Design-Build-Run
Or everything you should know about building software!
While in London last week I met up with Dave Ingram, author of Design-Build-Run. It's subtitled "Applied Practices and Principles for Production-Ready Software Development" but I like to think of it as "Everything you should know about building software". I've been working as a software developer since 1996 and during that time I've worked on a number of different projects for different customers. Some have been big, while others have been small. Some have been desktop applications, while others have been deployed on the web. Some have been very structured, while others have been less formal. I've had relative freedom on some and been restricted on others.
While the overall goal of all these projects was to develop software, each project brought its own challenges and experiences. One of the things that I most like about working in a consulting environment is that it can genuinely give you a very wide variety of work and this means that you tend to learn a lot in a comparatively short amount of time. You obviously get to learn lots of major things (new technologies, new ways to design software, etc) but there are a number of other things that you learn about too, particularly around some of the nuances of software development and the actual process of building software.
If you can imagine somebody sharing this sort of experience through a book about software development, then basically you have Design-Build-Run. Want to know about the sort of things that you should do in order to run a serious, high quality software project? No problem, chapter 3, Preparing for "Production", has the answers (including things like understanding architectural drivers and constraints). Want to know how to build software that operations staff can actually monitor? No problem, chapters 8 and 20 have information about logging, event logs, performance counters and so on. Want to know what sticky sessions are? Turn to page 207, where you'll also find information about a number of load balancing algorithms too. Want to know how to catch exceptions in your code and retry the transaction? Turn to page 550 where you'll find explanations of the various ways in which transactions can fail and some sample code that illustrates a "wait-retry" pattern.
The breadth of knowledge in this book is immense but the technical depth backing it up is there too; covering everything from planning and running projects through to designing, building and testing software. The book doesn't focus on particular technologies, frameworks or fashions; instead it explores and explains the important stuff that software developers really should know. The only niggles I have with the book are that it's pretty hefty at 650 pages and I think that some of the content may have worked better if it was collocated. For example, there's some excellent content about alerting and monitoring, but it's broken across a couple of chapters at opposite ends of the book. When you look at the breadth and depth of content in the book, these really are minor things though and the content is still very easy to find.
If you're a software developer looking for a way to fast-track your experience, you should definitely take a look at Dave's book. It really does contain everything that you should know about building software.
