Is VM size an architectural smell?

When do you revisit the structure of your services?

A couple of months ago I went to an excellent set of talks at LJC night at QCon. This not only inspired my entry on staleness but also made me think about some more architectural smells.

Gil Tene from Azul spoke about the maximum size of VMs we all use. A quick show of hands indicated that the maximum size of VM heap memory most of us ran was 2-4GB. Gil argued, quite convincingly, that this was due to legacy issues with garbage collection but new techniques should make this limit redundant.

I sort of agree... However, even if all the issues surrounding garbage collecting large VMs is solved, the 2-4GB limit is still a useful one.

In Architectural Smells I argued that there are architectural smells in the same way that there are code smells (aspects of a system that indicate but not guarantee a possible issue) and the example I gave was caching. I also think that VM size is an architectural smell.

When designing complex systems one of the most common design guidelines is to split differing responsibilities into specific services. Non-functional requirements may complicate this but most of us try to avoid having all functionality in a single service. Certain services are obviously distinct and can be isolated. However it's easy for functionality creep to result in a service gaining a lot of responsibilities and this often results in a larger run-time footprint.

During testing this shows itself as a service requiring a large VM - certainly in comparison to other services. I view this as an architectural smell and analyse the service to see if it should be split up. (Easier if you have some form of lightweight and iterative design process.)

This is very similar to the code smell of long method. It's rare that your code size or service size starts out by being too large but tends to happen over time. This is a good reason why an architect can't just throw it over the wall but needs to stay involved with the project. As developers add code to the system the design might need to change.

Whether you take an absolute size or relative size will depend on your system in question. Personally, I get concerned when a service is larger than 1GB unless its prime purpose is data storage and even then the data should probably be clustered across services rather than in a single block - but that's an argument for a different occasion.

About the author

Robert Annett Robert works in financial services and has spent many years creating and maintaining trading systems. He knows far more about low latency data systems and garbage collection than is good for anyone. He likes to think of himself as a pragmatist who loves technology but uses what's appropriate rather than what's cool.

When not pouring over data connections or tormenting interviewees with circular reference questions, Robert can be found locked in his shed with an impressive collection of woodworking tools.

E-mail : robert.annett at codingthearchitecture.com


Re: Is VM size an architectural smell?

It depends on the service you are offering... in an internet scale system, where storing data in the JVM is never a good idea, I'd say anything requiring more than 128MB heap is an architectural smell. A little off topic, I once inherited a service that had a "memory leak", and had to be restarted every day. So, I looked at the maximum heap size, it was 24GB. I changed it to 512MB, and the problem went away. The memory leak was that the JVM was never garbage collecting, but rather just increasing its heap size until the OS refused, and then it threw out of memory errors. Back to the topic, heap size I've found is often programmers giving it more "just to be safe".

Re: Is VM size an architectural smell?

I think programmers just giving a JVM a large initial and max heap "just to be safe" is a smell in its own right!

Is VM size an architectural smell?

No. Is insufficient info.

Re: Is VM size an architectural smell?

The heap size may be a faint whiff. However, if there's a problem it'll most likely stink the place up in the form of genuine impact to your system's behaviour.

Sensible metric capture, even simple GC logging, will typically show issues fairly readily. They will also continue to demonstrate whether all is well for the lifetime of your software.

A similar smell, in my experience, is setting GC parameters at all (logging aside). A reasonably well designed and written app will typically be pretty amenable to GC ergonomics. YMMV, but I've seen these properties used most often in an attempt to keep a struggling JVM on its feet.

@James Roper: need to 24GB -> 512MB: isn't this a JVM issue?

>> James said: I once inherited a service that had a "memory leak", and had to be restarted every day. So, I looked at the maximum heap size, it was 24GB. I changed it to 512MB, and the problem went away. The memory leak was that the JVM was never garbage collecting, but rather just increasing its heap size until the OS refused, and then it threw out of memory errors.

To me, this looks more of a JVM design (or coding) issue. Why not subscribe to the OS notification that the memory is low, and do a collection then? Even if low-memory notification is not supported, then do a collection before returning the OOM condition to the app.
There is enough information for the VM to do the right thing, and not require human intervention.

Add a comment Send a TrackBack