Structuring the software design process
And avoiding complex and cluttered diagrams
I had a great time last week discussing software architecture across a mix of QCon, our software architecture training and the IASA session that I ran. I mentioned this earlier in the year, but we've enhanced our material around the architecture definition process to include much more guidance on how you go about actually designing software when all you have is a set of requirements and a blank sheet of paper. In addition to understanding the requirements (functional and non-functional), constraints and principles; it's really about putting some structure into the diagrams that you might draw during your initial agile modeling rather than drawing up a single very complex and cluttered picture that is hard to explain or understand.
I've already written about not needing a UML tool to undertake the software design process and I normally use either a whiteboard, flip chart or index cards, especially when I'm collaborating on the design with others. Tooling aside, here are a few essays that summarise some guidelines and my own approach to designing software.
- Start with the big picture - every picture should tell a different part of the same story, but where do you start?
- Architectural constructs - what are the building blocks?
- Systems - what is the system landscape?
- Containers - what are the executables that make up the system?
- Components - what are the major components and services?
- Interfaces - do you understand the architecturally significant interfaces?
I don't particularly want to define yet another software design process, but I do want to help people design software simply and effectively. While the guidelines don't replace the need to have a deep technical knowledge and a broad understanding of the available options plus their tradeoffs, they do help people to organise their thinking and design software. I do believe that *some* up-front design is necessary for most software systems but also that it should take hours rather than weeks. I'm sure we'll be refining this sort of content as the year progresses, but please do let us know if you have any feedback.
Architectural constructs
What are the building blocks?
The code for any software system is where most of the focus remains for the majority of the software development life cycle and this makes sense because the code is the ultimate deliverable. But if you had to explain to somebody how that system worked, would you start with the code?
Where do you start?
Slides from my IASA session are available to view and download
I had a fantastic time presenting and discussing software architecture at my IASA session called Where do you start? on Tuesday evening. We went through the things that you should do if tasked with designing a software system given a wish-list and a blank sheet of paper; which covered the requirements (both functional and non-functional), constraints and principles before looking at some guidelines for structuring the actual process of designing software.
The slides from the session are available to view online and download and some additional essays covering some of the same content will be published on the site very soon. Thanks again to Matt Deacon and Onalytica (who have stunning views across London) for organising and hosting the evening.
Upcoming events
Our plans for the next couple of weeks
Just a short post to provide a rundown of the events that we'll be speaking at over the next couple of weeks.
|
Kevin and I are presenting a one-day tutorial at QCon London 2010 called Software Architecture for Developers, which covers the majority of the content from our two-day training course without the practical case-study exercises. Following this, Kevin is presenting a session for the "Cool Stuff with Java" track about OSGi and industrial-strength Swing. |
|
I'm running our two-day training course at Skills Matter in London and, as I mentioned at the start of the year, this has been enhanced to include more content about the actual process of architecture, with a focus on coming up with a high-level software design in a few hours rather than a few weeks. You can preview the slides and there are a few places remaining if you want to come along. |
|
I'm running a session at the IASA UK Chapter that will look at where to start with designing a software system from a blank sheet of paper. The focus here is on pragmatic software design without the need for lots of process or expensive modelling tools. |
|
Finally, I'm running two sessions at DevWeek 2010; one called "Improving quality with an automated build process" (it has a .NET focus) and one called "A developer’s guide to load testing" that covers the basics for evaluating architectures where performance and/or scalability is important. |
It's going to be a busy couple of weeks, but I'm planning on spending some time in the audience at both QCon and DevWeek. If you want to meet or catch-up, feel free to send me an e-mail or a message via Twitter.
Evaluating enhancements
Software architecture is needed for enhancements too
This is a follow-up to Designing enhancements, where I briefly talked about how enhancements to existing software systems can benefit from the principles and practices of good software architecture, particularly where interfaces to other systems are involved.
Given the size of the enhancement project, we could have easily decided to deliver the whole lot in one go and in many ways this does seem the obvious thing to do. After all, it's only ~8 weeks of work for 2-3 people with a small scope and a limited complexity. But I decided against this and split the overall project into two distinct time-boxed phases as follows.
- The modification of one key use case, including integration with the third party end-to-end (happy scenario only, no exceptional scenarios).
- Delivery of everything else.
It wasn't the business logic associated with the interface between us and the third party that concerned me, nor was it the requirements around security or audit. No, for me, it was the actual connectivity between our application server and the Internet. As I said in the previous post, writing the code to send XML over HTTPS is easy but connecting out to the Internet from the architecture isn't something that's been done before. We knew that we'd need to open up firewall rules on the corporate LAN but this isn't something that I've seen done in this particular environment. For this reason, I wanted to deliver enough of the solution (the architecturally significant bits) to evaluate and prove that the connectivity would work as expected.
We delivered that first phase a couple of weeks ago now and initially the connectivity between the application server and the Internet *didn't* work! Everything was installed correctly and firewalls were configured, but we just couldn't find a direct route out of the network. Despite everybody saying that it wouldn't be a problem, it turned out that we'd need to send the traffic via an existing proxy server, which would only accept authenticated requests. A short while later we had a slightly modified version of our release with the appropriate proxy server configuration enabled and this allowed us to successfully see the outside world.
Although the release didn't go as expected, it was a success because we now have a much better understanding of the constraints placed upon our software architecture by the infrastructure and we found the side-effects much earlier than we would have done by going with a single release. We'd successfully identified the risks with the highest probability of happening and planned for them. Managing technical risk is one of the key aspects of the software architecture role, with early evaluation allowing you to prove that your software will work as expected when released. For new systems or enhancements to existing ones, understanding the big picture can and does contribute to the overall success of projects.
What is the big picture?
Ensuring that the code works toward your overall vision rather than against it
Whenever we talk about software architecture, undoubtedly we'll end up talking about the "big picture", but what exactly does this mean?
Designing enhancements
Software architecture is needed for enhancements too
We're enhancing one of the software systems that I'm involved with, where we're adding new functionality into some of the existing use cases. If you imagine that these use cases are implemented by the users working through a number of pages in a web application, we're basically modifying the final step across a number of them.
So then, the system is being modified from the user's perspective because they are going to see some changes in the way that they use the system. While this itself isn't architecturally significant, we are interfacing with a brand new system behind the scenes. Typically, system enhancement projects require a very light architecture touch but this one needs slightly more because of the nature of it.
Interfaces are usually one of the more risky elements of software projects, particularly when you're consuming an interface that you don't have control of, as is the case here. Furthermore we're doing something that hasn't been done in the existing architecture, which is interfacing with a third party over the Internet. From a coding perspective it's easy because we're just opening up a HTTPS connection and throwing some XML data down it. Where it starts to get tricky though is with our deployment infrastructure because our servers need to punch through the firewalls to reach the outside world.
Although an enhancement to an existing system, this is one of those projects that benefits from some architectural input. Rather than dive straight into the code, it's important that somebody steps back to look at the bigger picture elements such as:
- Design and how best to integrate the new functionality.
- The definition of the interface and whether it's synchronous or asynchronous, the protocol, the message format, etc.
- Confidentiality of sensitive user information and ensuring service credentials remain secure after deployment.
- How the changes affect the existing infrastructure from a security perspective.
- How the new functionality will meet existing audit requirements.
This is no different from what you'd do at the start of a software project, but it highlights that architecture is often necessary at other stages during a system's lifetime. Next time I'll discuss how we phased the delivery of this enhancement to evaluate our architectural decisions early.
Are you a software architect?
The line between software development and software architecture is a tricky one. Some people will tell you that it doesn't exist and that architecture is simply an extension of the design process undertaken by developers. Others will make out it's a massive gaping chasm that can only be crossed by lofty developers who believe you must always abstract your abstractions and not get bogged down by those pesky implementation details. As always, there's a pragmatic balance somewhere in the middle, but it does raise the interesting question of how you move from one to the other.
Software architecture: where do you start?
IASA UK chapter, 6pm, 9th March 2010
You may have seen this on Matt Deacon's blog already ... I'm running a session for the IASA UK chapter on the 9th of March in London.
Where do you start?
One of the hardest things about software development is being asked to come up with a design when all you're given is a set of requirements and a blank sheet of paper. Many software teams will dive straight into the code and while this can initially be very productive, the slippery slope of constant refactoring is awaiting those teams that haven't quite found a design that works. Often, a little forethought is all that's needed to get the development process heading in the right direction. So where do you start?
This session will answer this question, presenting some simple techniques for tackling software architecture while dispelling the myths about the need for complex tools and big design up front.
I'm planning the content for this session at the moment and it's going to be a mix of presentation, discussion and probably some group work to get you all drawing a few boxes and lines. We're going to cover a very very quick overview of what software architecture is all about before moving on to see what drives and influences it. Once this is done, we'll look at how to actually design software from a blank sheet of paper, focussing on some techniques to help you determine what components you need and how best to organise them. A nice side effect of the techniques we'll use to define architecture is that they can also be used to share architecture, so we'll wrap up by looking at how to document software architectures in a simple yet effective way.
You can find more information about this event and register here. I'm really looking forward to it and hope to see you there.
Designing Maintainable Systems
The Upgrade Process
I'm currently involved in a project to upgrade a third party piece of software and it's apparent that when the software was originally designed, the upgrade process was not considered. This became obvious when we totaled up the time required to perform, configure and post-release test the upgrade - it came to over three days of work. This was not even taking into account any rollback times (which is fortunately simplified these days by the use of virtualisation).
The software is used heavily from Monday to Friday so we wanted to upgrade over a weekend. The vendor suggested we perform an upgrade on a parallel system and then get the users to re-enter all the data into the new system that was missed - you can imagine how well that would have gone down. This would also mean trying to post-release, regression test two systems that are live, being used and not in sync.
Software almost always needs updating/upgrading (unless it's control software for a deep space probe!) The ability and consequence of upgrading should be considered as part of the design and development process. Questions to ask include:
- Can an upgrade be performed in parallel to a live, running system and how does a switchover occur?
- Will a system need to be taken down for any upgrades and for how long? How does this affect your Service Level Agreements?
- How easy will any upgrade be to rollback? Errors occur!
- Can you upgrade parts of the systems or does everything have to be done at once?
- What is the effect on any users? Will they need to log out first etc? Will they lose any work if they fail to follow your procedures?
- How easy will it be to test the upgraded system to determine success? Your notice of failure shouldn't be an angry user phone call.
Some simple tools can make all the difference. Most of my work is on financial applications and I like to run regression reports between systems for important points e.g. End-of-year. However it's often very difficult to get data out of systems to perform simple comparisons!
Sensible configuration management is often missing. If I've upgraded and configured new features in my pre-production environment I really shouldn't have to repeat the process from scratch in production. Manual processes are prone to errors and ideally once I've prepared for an upgrade I should just hit a 'go' button and sit back.
In my experience very few software developers are aware of IT Service Management (ITSM/ITIL). In particular we should be aware of the Change Management, Release Management and Configuration Management roles that support staff have. If you want to read about ITSM/ITIL then the wiki page is a good place to start.
Some of the processes of ITSM may strike agile developers as being heavy-weight but this doesn't stop you developing the system in an agile manner, it just means that it can be deployed within a formal environment.
An architect should be aware of how the software fits into the organisation. So remember that your ‘users’ aren't just the end users but also the support staff who'll be maintaining your system for the next ten years!
