Coding the Architecture

Is VM size an architectural smell?

Wed, 16 May 2012 08:01:04 GMT

A couple of months ago I went to an excellent set of talks at LJC night at QCon. This not only inspired my entry on staleness but also made me think about some more architectural smells.

Gil Tene from Azul spoke about the maximum size of VMs we all use. A quick show of hands indicated that the maximum size of VM heap memory most of us ran was 2-4GB. Gil argued, quite convincingly, that this was due to legacy issues with garbage collection but new techniques should make this limit redundant.

I sort of agree... However, even if all the issues surrounding garbage collecting large VMs is solved, the 2-4GB limit is still a useful one.

In Architectural Smells I argued that there are architectural smells in the same way that there are code smells (aspects of a system that indicate but not guarantee a possible issue) and the example I gave was caching. I also think that VM size is an architectural smell.

When designing complex systems one of the most common design guidelines is to split differing responsibilities into specific services. Non-functional requirements may complicate this but most of us try to avoid having all functionality in a single service. Certain services are obviously distinct and can be isolated. However it's easy for functionality creep to result in a service gaining a lot of responsibilities and this often results in a larger run-time footprint.

During testing this shows itself as a service requiring a large VM - certainly in comparison to other services. I view this as an architectural smell and analyse the service to see if it should be split up. (Easier if you have some form of lightweight and iterative design process.)

This is very similar to the code smell of long method. It's rare that your code size or service size starts out by being too large but tends to happen over time. This is a good reason why an architect can't just throw it over the wall but needs to stay involved with the project. As developers add code to the system the design might need to change.

Whether you take an absolute size or relative size will depend on your system in question. Personally, I get concerned when a service is larger than 1GB unless its prime purpose is data storage and even then the data should probably be clustered across services rather than in a single block - but that's an argument for a different occasion.

SharePoint projects need software architecture too

Tue, 15 May 2012 11:47:15 GMT

I was reading Requirements–we don’t need them! by Andrew Woodward of 21apps recently, which talks about the importance of having a visionary product owner on your SharePoint project to ensure that it delivers the right thing. That all makes sense and it's inspired me to write this blog entry ... SharePoint projects also need a healthy dose of software architecture.

If you're currently thinking "well, yes, duh!", that's awesome. Unfortunately, I've seen a number of SharePoint implementations where the basic tenets of software architecture have been forgotten or neglected. Here's a summary of why software architecture is important for SharePoint projects.

1. Many SharePoint implementations aren't just SharePoint

Many of the SharePoint solutions I've seen are not just simple implementations of the SharePoint product where end-users can create lists, share documents and collaborate. As with most software systems, they're a mix of new and legacy technologies, usually with complex integration points into other parts of the enterprise via web services and other integration techniques. Often, bespoke .NET code is also a part of the overall solution, either running inside or outside of SharePoint. If you don't take the "big picture" into account by understanding the environment and its constraints, there's a chance that you'll end up building the wrong thing or building something that doesn't work.

2. Non-functional requirements still apply to SharePoint solutions

Even if you're *not* writing any bespoke code as a part of your SharePoint solution, that doesn't mean you can ignore non-functional requirements. Performance, scalability, security, availability, disaster recovery, audit, monitoring, etc are all potentially applicable. I've seen SharePoint projects where the team has neglected to think about key non-functional requirements, even on public Internet-facing websites. As expected, the result was solutions that exhibited poor response times and/or severe security flaws (e.g. cross-site scripting). Often these issues were identified at a late stage in the (usually waterfall) project lifecycle.

3. SharePoint projects are complex and require technical vision too

Like any programming language, SharePoint is a complex platform and there are usually a number of different ways to solve a single problem. In order to get some consistency of approach and avoid chaos, SharePoint projects need strong technical leadership and the software architecture role is as applicable here as it is if you're writing a software system from scratch. If you've ever seen SharePoint projects where a seemingly chaotic team has eventually delivered something of a poor quality, you'll appreciate why this is important.

4. SharePoint solutions still need to be documented

With all of this complexity in place, I'm continually amazed to see SharePoint solutions that have no documentation. I'm not talking about hefty 200+ page documents here, but there should be at least *some* lightweight documentation that provides an overview of the solution. Some diagrams to show how the SharePoint solution works at a high-level are also useful, and something along these lines works well. Some *lightweight* documentation can be a great starting point for future support, maintenance and enhancement work; particularly if the project team changes or if the project is delivered under an outsourcing agreement.

Strong leadership and discipline aren't just for software development projects

If you're delivering software solutions then you need to make sure that you have at least one person undertaking the technical leadership role, looking after the things that I've highlighted above. If not, you're doing it wrong. As an aside, all of this applies to Microsoft Dynamics CRM projects too, especially if you're "just tacking on an Internet-facing ASP.NET website to expose CRM data via the Internet".

I mentioned all of this to somebody a while back and they replied "but SharePoint isn't software development". Regardless of whether it is or isn't software development, successful SharePoint projects need strong technical leadership and discipline. SharePoint projects need software architecture.

Software architecture track at GOTO Copenhagen 2012

Fri, 11 May 2012 06:47:27 GMT

It's not long now until the GOTO Copenhagen 2012 conference and I'm really honoured to be hosting the software architecture track. The vision for the track is to get people in the software architecture role to share their experiences of how they approach software projects. Here's the theme that Jasper Arildslund (Trifork) and I came up with.

In this track we relate the origins of the architect back to its roots of being a master-builder and ask why our industry has moved away from this in the real world. You will hear some stories from real master-builders as we take a look at how they approached the software architectures that they created.

We have a number of speakers covering hands-on software architecture from a number of different perspectives and I'm really looking forward to all of the talks. Here's the list...

1. Master-builder; the original generalising specialist? (Simon Brown)

Applying the building metaphor to software doesn't necessarily work, although in medieval times the people that architected buildings were the select few that made into the exclusive society of master-builders. The clue here is in the name and a master-builder really was a master of their craft. Once elevated to this status though, did the master-builder continue to build or was that task left to those less noble? Fast-forward several hundred years and it seems we're asking the same question. Join us as we look at how master-builders fit into our modern world of software development before looking at how my own role has evolved over the years. Read more...

2. A team, a system, some legacy and you (Eoin Woods)

Software architecture is often described as if it is something that is only undertaken (or relevant) when creating a new system from scratch. However in my career I’ve joined many more projects where a system existed already, than projects where I got to design a system from the beginning.

In these situations, it is paramount that you are a “master builder” rather than a designer of castles-in-the-air. A real system exists, it needs help quickly, there is far too much to do and there is no time to wish that things could have been done differently. Action is needed now!

In this talk I’ll describe some of my experiences working on the architecture of existing systems and the principles and techniques that I’ve used to be (mostly) successful in doing this. Read more...

3. Master-builders have rich conceptual models of software design (George Fairbanks)

How hard is it for an architect to be a master-programmer, or a programmer to be a master-architect? One might assume this is easy because both programmers and architects work with the same materials (software) and have the same goal: the design and construction of quality software. In reality, it is quite hard because programmers and architects focus on different aspects of the same software system, aspects that are intricately interconnected. To be a master-builder means understanding programming and architectural aspects separately and being able to reason through the interconnections.

The essential problem is that while we have good descriptions of individual concepts (e.g., encapsulation, design patterns, code indentation style, application frameworks, modules, layers, typing, callbacks, functional programming, quality attributes, domain modeling, code invariants, architectural styles) we lack a clear understanding of how all those concepts fit together and how they interact on real projects.

Until we can relate all these concepts, programmers and architects will only occasionally succeed in being master-builders. This talk presents a conceptual model that relates many of these design concepts and implementation concerns and is a first step towards a clear description of how they all fit together. Read more...

4. Functional Architecture (Phil Trelford)

Much has been made of the promise of functional programming to solve the pressing issues of concurrency woes as we move to a world of multi-core and distribute computing. In this talk I'll draw on my own commercial experience of designing and developing large enterprise applications across a range of industries discussing potential sweet spots for functional programming like domain modelling, domain specific languages and of course concurrency. Along with where object-orientated or a mixed approach may be more appropriate. Read more...

5. 15 Lessons from 15 Years as a Software Architect (Ingo Rammer)

In 1995, Ingo started working on some of the first Internet-based online shops, and he continues to work as a software architect more than a decade and a half later. Quite a bit has changed within these years, but a few things however seem to remain true, no matter what the current wave of technology brings us. In this session, Ingo will talk about his experiences as a software architect (and as a consultant to software architects) and about the lessons he has learned which caused projects to succeed - or to fail. After all: the best mistakes to learn from might be the ones you don’t have to make yourself. Read more...

And there's more

I'm also re-running the tutorial that I ran at QCon London 2012 called Are you a software architect?. See you in Copenhagen!

p.s. George and I will be repeating our talks on Friday at GOTO Amsterdam 2012 too

How do you design software?

Thu, 10 May 2012 09:26:57 GMT

I recently set a challenge ... describe how you design software in the time it takes you to have a coffee break. I've posed this same question to a few people myself and answering it is really tough. And that, I think, is kind of worrying. Related to this is my observation that teams now find it easy to visualise the software development process but not the actual software that they're building.

We used to have a reference point

If you look back 10 years or so, a number of teams were using the Rational Unified Process (RUP). Don't worry, I'm not about to start evangelising RUP but it does have a number of very useful elements and it gave software teams a reference point for the software design process. As a real-world example, I worked on a fairly large RUP project a while back and, as you can imagine, we spent a long time producing documentation and UML models in order to come up with the design for this particular enterprise Java system. I don't remember what the trigger was, but eventually we had some consultants from Rational come in to help us and they provided advice on which RUP products we should focus on in order to streamline the software design process.

As an aside, I met Uncle Bob for the first time when he was in London recently and we had a quick chat about this sort of stuff. The "boundary, controller, entity" approach that he mentions in his clean architecture talks is exactly the approach that the guys from Rational recommended that we take. In a nutshell, they gave us a set of simple steps for transforming use cases through to high-level software designs, which could then be implemented using enterprise Java.

Look back even further than this and you have all of the notations that preceded UML, data flow diagrams, entity relationship diagrams and higher level methods like SSADM. Again, you can consider these as reference points and guidance for the software design process.

Where is our reference point now?

Many of my thoughts about the current state of the software development industry are summed up in The frustrated architect so I won't repeat them here. What I will repeat though is that many software teams have thrown far too much away in their haste to adopt "agile". UML and documentation are typical examples of something that is being thrown away by software teams on a daily basis. After all, 3 months spent creating UML models is 3 months where the software (and value) isn't being delivered. That's all fine, but agility and, therefore moving fast, requires good communication yet many software teams now struggle to articulate what it is they are building and how they are building it.

If we can't articulate how we design software, we can't teach it

So how *are* teams designing software? They obviously *are* designing software because I've seen it! Occasionally I'll hear "we do something like domain-driven design". More usually though, teams tell me that they "draw some pictures on a whiteboard", to which my response is "how do you decide what to draw?".

If you've seen me speak, you're likely to have heard me talk about how the software architecture role should be about collaboration, coaching and mentoring. Why? Because we're losing the software architects of tomorrow and coaching is key to reversing this trend. But if we can't articulate how we design software, how are we going to teach this skill to others?

How much up front design is just enough?

Wed, 09 May 2012 19:22:41 GMT

I ran an interactive, gamestorming session a couple of weeks ago at Skills Matter to help us answer the question of "how much up front design is enough?". It was a lot of fun! Since the session was interactive, there was only a short amount of presentation but the slides are available to view online and the video can be viewed on the Skills Matter website.

More interestingly though are some write-ups of the session and I urge you to go and read the following blog entries if you want to see how we went about answering the question and the answers that we collectively came up with.

What is "just enough" design? by Paul Rozs
"Just Enough" upfront design by Richard Tappenden

A big thank you to everybody that came along and participated, particularly to Paul and Richard for taking the time to write-up the session.

Software architecture training in Jersey, June 2012

Wed, 09 May 2012 16:12:00 GMT

For a second year, I'm pleased to say that I'll be running Software Architecture for Developers in Jersey. If you want to get a feel for what we'll be covering during this two day training course, take a look at the following blog entries.

The frustrated architect (my talk of the same name is also available to watch online at InfoQ.com)
Photos from my QCon London 2012 tutorial (are you a software architect?)
Moving fast requires good communication
Deliberate practice, effective sketches

Here are the details...

Dates: Wednesday 27th June 2012 and Thursday 28th June 2012 (2 days)
Cost: £895 per person (includes a copy of Software Architecture for Developers - the book)
Trainer: Simon Brown
Location: Regus, St Helier, Jersey
Laptop required: No laptop is needed unless you want to use Evernote to take notes
More details can be found at www.softwarearchitecturefordevelopers.com
To book, please e-mail simon.brown at codingthearchitecture.com

If you're considering making the short journey over by air or sea, why not make a weekend of it and see what Jersey has to offer in the summer.

How do you design software?

Fri, 27 Apr 2012 08:36:00 GMT

A big thanks to everybody that came along to the Why do software projects fail? session last night, it was a really fun evening! The slides from my talk are available to download/view online.

As I said in my talk, here's my challenge to you. Next time you grab a coffee with your team, have a quick discussion about how *you* design software. By "design software", I mean taking a set of vague requirements with a blank sheet of paper and creating a software solution. Think about things like your process, approach, tools, techniques, etc. Can you explain how you design software within the time it takes to finish your coffee break?

Please do let me know how you get on. I'll post a follow-up next week...

Moving fast requires good communication

Thu, 12 Apr 2012 08:52:30 GMT

If you're working on an agile software development team at the moment, take a look around at your environment. Whether it's physical or virtual, there's likely to be a story wall or Kanban board in order to visualise the work remaining to be started, in progress and done.

We're adept at visualising our software development process

Why? Put simply, it's a fantastic way of introducing transparency into software projects because anybody can see, at a glance, a snapshot of the current progress at a high-level. Couple this with value stream mapping and you can start to design some pretty fancy looking Kanban boards to reflect that way that your team works. In a nutshell, we've become pretty adept at visualising our software development process.

But we've forgotten how to visualise the software that we're building

However, it seems we may have forgotten how to visualise the actual software that we're building. I'm not talking about post-project documentation here (that's a story for another day), I'm talking about communicating "the big picture". Why is this important? Well, if you want to ensure that everybody is contributing to the same end-goal, you need to be able to communicate the vision of what it is you're building. And if you want agility and the ability to move fast, you need to do this effectively and efficiently. You can argue about whether notations such as UML offer an effective way to communicate software architecture or not, but that's often irrelevant as a starting point because many teams have already thrown out UML or simply don't know it. What's the result? Usually boxes and lines diagrams like the sketches below.

Boxes and lines diagrams *can* work very well, but there are many pitfalls associated with sketching software architecture in this way. My approach is to use a collection of simple diagrams, each showing a different level of abstraction.

All of my guidance around doing this is going to be enhanced and included in my software architecture e-book but if you can't wait, you can always join me at Skills Matter in London (23rd April) or Bouvet in Oslo (26th April) for our Software Architecture for Developers training course.

The frustrated architect

Wed, 14 Mar 2012 11:27:00 GMT

The IT industry is either taking giant leaps ahead or it's in deep turmoil. On the one hand we're pushing forward, reinventing the way that we build software and striving for craftsmanship at every turn. On the other though, we're continually forgetting the good of the past and software teams are still screwing up on an alarmingly regular basis.

Software architecture plays a pivotal role in the delivery of successful software yet it's frustratingly neglected by many teams. Whether performed by one person or shared amongst the team, the software architecture role exists on even the most agile of teams yet the balance of up front and evolutionary thinking often reflects aspiration rather than reality.

Software architecture has a bad reputation

I tend to get one of two responses if I introduce myself as a software architect. Either people think it's really cool and want to know more or they give me a look that says "I want to talk to somebody that actually writes software, not a box drawing hand-waver". The software architecture role has a bad reputation within the IT industry and it's not hard to see where this has come from.

The thought of "software architecture" conjures up visions of ivory tower architects doing big design up front and handing over huge UML models or 200 page Microsoft Word documents to an unsuspecting development team as if they were the second leg of a relay race. And that's assuming the architect actually gets involved in designing software of course. Many people seem to think that creating a Microsoft PowerPoint presentation with a slide containing a big box labelled "Enterprise Service Bus" *is* software design. Oh, and we mustn't forget the obligatory narrative about "ROI" and "TCO" that will undoubtedly accompany the presentation.

Many organisations have an interesting take on software development as a whole too. For example, they've seen the potential cost savings that offshoring can bring and therefore see the coding part of the software development process as being something of a commodity. The result tends to be that local developers are pushed into the "higher value" software architecture jobs with an expectation that all coding will be undertaken by somebody else. In many cases this only exaggerates the disconnect between software design and software development, with people often being pushed into a role that they are not prepared for. These same organisations tend to see software architecture as a rank rather than a *role*.

Agile aspirations

Agile might be over ten years old, but it's still the shiny new kid in town and many software teams have aspirations of "becoming agile". Agile undoubtedly has a number of benefits but it isn't necessarily the silver bullet that everybody wants you to believe it is. As with everything in the IT industry, there's a large degree of evangelism and hype surrounding it. Start a new software project today and it's all about self-organising teams, automated acceptance testing, continuous delivery, retrospectives, Kanban boards, emergent design and a whole host of other buzzwords that you've probably heard of. This is fantastic but often teams tend to throw the baby out with the bath water in their haste to adopt all of these cool practices. "Non-functional requirements" not sounding cool isn't a reason to neglect them.

What's all this old-fashioned software architecture stuff anyway? Many software teams seem to think that they don't need software architects, throwing around terms like "self-organising team", "YAGNI", "evolutionary architecture" and "last responsible moment" instead. If they do need an architect, they'll probably be on the lookout for an "agile architect". I'm not entirely sure what this term actually means, but I assume that it has something to do with using post-it notes instead of UML or doing TDD instead of drawing pictures. That is, assuming they get past the notion of only using a very high level system metaphor and don't use "emergent design" as an excuse for foolishly hoping for the best.

So you think you're an architect?

It also turns out there are a number of people in the industry claiming to be software architects whereas they're actually doing something else entirely. I can forgive people misrepresenting themselves as an "Enterprise Architect" when they're actually doing hands-on software architecture within a large enterprise. The terminology in our industry *is* often confusing after all.

But what about those people that tend to exaggerate the truth about the role they play on software teams? Such irresponsible architects are usually tasked with being the technical leader yet fail to cover the basics. I've seen public facing websites go into a user acceptance testing environment with a number of basic security problems, a lack of basic performance testing, basic functionality problems, broken hyperlinks and a complete lack of documentation. And that was just my external view of the software, who knows what the code looked like. If you're undertaking the software architecture role and you're delivering stuff like this, you're doing it wrong. This *isn't* software architecture, it's also foolishly hoping for the best.

From frustration and beyond

Admittedly not all software teams are like this but what I've presented here isn't a "straw man" either. Unfortunately many organisations do actually work this way so the reputation that software architecture has shouldn't come as any surprise.

If we really do want to succeed as an industry, we need to get over our fascination with shiny new things and starting asking some questions. Does agile need architecture or does architecture actually need agile? Have we forgotten more about good software design than we've learnt in recent years? Is foolishly hoping for the best sufficient for the demanding software systems we're building today? Does any of this matter if we're not fostering the software architects of tomorrow? How do we move from frustration to serenity?

If this strikes a chord and your view of software architecture is more like the role we discussed at QCon London, you can view the slides and video from my "The Frustrated Architect" talk, which I'll be repeating at the DevWeek conference in London on the 29th of March. My book also has some answers. ;-)

Photos from my QCon London 2012 tutorial

Tue, 13 Mar 2012 13:48:00 GMT

I ran a tutorial at the QCon London 2012 conference last week where we focussed on the software architecture role. Throughout the day I asked the audience a number of questions, the answers to which I summarised on the flipchart at the front. Here are the questions and answers...

Is software architecture important?

What does the software architecture role include?

Should software architects code?

What soft skills are needed to undertake the software architecture role?

Who might be interested in your software?

What pitfalls are there for people new to the software architecture role?

Interesting that the software architecture role we discuss is so different from what you typically find in many organisations, huh? :-)

Thanks to everybody who came along!