Legacy code is something we all have to deal with at one point or another. It's all around us. Even if we find it scary we have to learn how to handle it without being petrified in the process. This article will look at some techniques that I have used to tackle legacy code. It's a collection of things that have helped me go from feeling scared and apprehensive when working in legacy code to feeling challenged.

Before we dive in I'll start with defining what legacy code is. I've done so before on this blog. Since then I've learnt a much simpler and more precise way to describe it. Legacy code is all production code which makes you scared when you have to change it.

But what is production code? Production code is any code that provides the business with value. This includes, but is not limited to, code that executes in production, test code, automation code and any other code that helps the business do it's business.

So here we are then, with the legacy code. How can we go about doing this scary job without causing to much pain to the customer, who probably doesn't want to be disturbed at all? What should we think about? What patterns and tools would be helpful?

When the customer asks us to work on a legacy system we need to ask them why. Why does the customer want us to muck around in this old thing in the first place? What is the expected value of the change? Is it possible to deliver this value to the customer? Can we deliver this value in a better way? Can we get an of the shelf product and replace the legacy system with? Should we make a system rewrite instead? (this one should be approached with utter care)

The answers to these questions will not just question the need for change. It will also help guide you in your work. It will give you the indications you need to know how ambitious you should be. When you start on the path of reworking an application through slow, steady improvements rather then a big rewrite you will find that ambition level is important. It is what will guide you when you decide how much effort you should spend on improvements. What is the expected lifetime of the system you are working on?

Once you have the answers it's time to figure out where to start. The most obvious place to start is of cause where you stand. This is usually an approach that works when you know pretty well what needs to be done and where. If you are about to add a feature to an already existing part of a system or fix a well known issue then you should start where you stand.

When you are not quite sure where the problem is or when you need to find out where the real problem hot spots in your system are there is another tool that can be really useful. It is called Offender Profiling. The technique was create by Adam Tornhill and is presented in his book "Your code as a crime scene". He also has several talks (for example here and here) on the subject. Adam combines many years experience as a programmer with theories from forensic psychology. This have allowed him to think outside the box and be able to find patterns hidden in our version control system's (VCS) history data.

By using the history data from your VCS, combined with some very simple complexity metrics, this will allow you to find the most likely candidates for issues in your code base. The inspiration comes from a forensic technique called offender profiling.

Adam has in his book combined various other psychology concepts with VCS data which will help organisations make better use of their software. But that is for another time.

Now that we know where to start we have to understand what the code does. This is usually the biggest reason for fear in legacy code. That we don't quite understand how the code works. What does the requirements for this code look like. How do I know I haven't broken anything when I think I'm done? There are several different ways to attack this. We will look at four different patterns, starting with test.

First out is a way to mine requirements from executing code using tests. The tests are called Specification tests.They are pretty much the same thing as the tests we create when working with Behaviour Driven Development. What we aim to do is to understand how the system we are about to change should work by writing black box tests. To do so we use Gherkin's Given When Then syntax. We then run the tests on a running system. This whole process should ideally be fully automated and executable from the command line. But if that's not possible, just manually be able to run repeatable regression tests is a huge benefit. There are a couple of different frameworks to help with this. For Java and Ruby there is Cucumber, for .Net SpecFlow, JavaScript has Karma. There are many other frameworks available as well and for most other languages. You can also use tools like Selenium to execute your specification directly on the UI. Even if you are executing your tests directly against the graphical user interface and the cost of maintenance seems high this is a good investment. If this is an old application the UI will likely not change much. Also, you will get automated feedback on stuff the user will notice. If you can, sit down with users and watch how they use the UI. Then write the tests based on your observation.

With some test in place to make sure we don't have regression from the clients perspective it's time to start understanding the architecture and the code. We will look at two of Michael M. Feather's patterns from his book "Working effectively with legacy code" (WELC). We will also look at how we can use Simon Brown's C4 model for architecture to understand the architecture of an existing system.

The first of Michael Feathers patterns is Conversational Scrutiny (WELC p. 224). When you sit with an extra hairy bit of code like a long method or complex class, you often talk with one or more colleagues to get some help understanding it. The conversation that ensues is well worth paying attention to. It will often contain the abstractions that are missing from the code. The abstractions and the words that would have made the code easier to understand. The conversation in it self is of cause the key to understanding what the code does. But also make sure to listen carefully for the hidden abstractions required to make improvements. They tend to be embedded in the reoccurring words used during the conversation.

What is also important to remember is that a mess often asks for more messiness. Even a good team of developers have a scary tendency to keep adding messy code to messy sections of the code base. When scrutinising the conversation and noting the missing abstractions, don't forget to add them in the code. Don't forget to continuously improve the code.

The second of the patterns from Michael Feathers is Telling the Story of the System (WELC p. 216). This is a technique used to understand how submodules and modules work together. Each participant is responsible for telling the others the story of e's module. The story should be as brief as possible, preferably just a few words. It should also be told as if the audience knows nothing of the system.

By doing the exercise we become aware of what responsibility belongs where, as well as where it is currently located. It is a great way to understand how the system should be re-structured as well as how it currently hangs together.

Simon Brown, author of Software Architecture for Developers, have described a way to explain software architectures he calls the C4 model. This is a very powerful way to illustrate what a system architecture looks like. Used correctly on a legacy system it's almost embarrassing how bad most architectures are designed. The tldr; of it is that you have four types of diagrams. Each describes a model of the system. At the top level we have a Context diagram. This describes how the system fits into the rest of the world. One level down we have a Container diagram. This will show which "containers" the system is made up of. A containers is a deployable unit. This can be an IIS or JEE server, a database, a Docker Container or something else that is deployable. At the third level down we have the Component diagrams. They will describe the different components in some detail and how they interact with other components. At the bottom is the Class diagram. Class diagrams are sometimes very illustrative but often to detailed and can be left out.

The audience for C4 is mainly developers. The diagrams are there to help understand how the high level architecture is structured. When the model is true to the actual system it is also a very good place to start when looking to change the current architecture into something more fitting.

Now that we have, hopefully, understood how the system works, it's time to look at some patterns for how to make changes without fear. Or at least as fearless as we can be.

Let's start with a pattern from Martin Fowler's bliki called Preparatory Refactoring. This is a practice we should be doing all the time. Actually, the failure to do so is the main reason to why we have problematic legacy code in the first place. The basic idea is pretty simple. You have a new feature which does not fit into the code where it's supposed to. To make sure you don't create a mess when plugging it in you first refactor the existing code. You are done with the refactoring when the new feature fits in as if it had always been there. The effects is that the code is cleaner and more suitably structured to the requirements it is there to serve now.

When doing this in a legacy code base you end up adding loads of test. Remember, you can't do refactoring without covering tests, then it's not refactoring. Dan North and Michael Feathers came up with a name of this practice of adding tests to a legacy system in order to be able to change a part of it. They called it Light the Forest.

Imagine you have been asked to chop down a tree in the depths of a large dark forest. You have problems seeing your hand in front of you. The sensible thing to do is to light your way in, cutting a path and mounting torches, so that you can find your way back again.

When diving in to the depths of a legacy code base you are well advised to do the same. Lighting your way with tests. We call these tests assumption tests. They prove that assumptions we make on our journey are correct and stay correct.

As you work with your legacy code base and apply Light the forest over and over again you will start to see patterns in the forest. It's like seams starts to appear in the system. Along these seams you can start splitting the system up into components which are possible to test individually. Michael Feathers calls this the Seams Model. His definition of a seam is:

“A seam is a place where you can alter behaviour in your program without editing in that place.” (WELC p. 31)

The seams are places where you can inject tests, mocks, stubs and other things. It is also places where you can start breaking up the system into smaller more understandable pieces.

To break the system apart at the seams we use what is called Dependency Breaking Techniques. Michael Feather describes several in his book (WELC). Dependency breaking is often common refactorings made more difficult by the legacy code base. In statically typed languages such as Java and C# mocking frameworks can help allot when introducing tests and breaking dependencies. I don't generally use mocking frameworks when creating new code. But in legacy systems they can be a real life saver.

Another way to understand what happens when you want to break out a certain dependency is to pass in a null reference. The test harness will then quickly tell you if the specific parameter is used by throwing null pointers when the code under test tries to use the dependency.

If mocking or passing nulls doesn't work a possible way is subclassing. By subclassing a dependency, or some times even the SUT (System Under Test), you can stub away especially troublesome parts whilst using the real code in others. You can either create a specific test double or let the test class extend the dependency directly. This technique allows you to understand the SUT in a very intimate way. It will usually also make the code uglier. This is OK as long as it is done in a controlled manner and is removed as soon as possible.

Another thing that tends to be lacking in legacy systems is good names. Helpful names, providing good abstraction, is what make code changeable, flexible and ultimately non-legacy. It is one of the key aspects of a good code base. So when you work in a legacy system it is important to continuously find and add the missing names and abstractions to the code. As we know naming can be very hard. The conversational patterns presented earlier can be of great help in this work, as is the conversation that happens when pair programming.

Something Kent Beck said a couple of years back that helps allot when it come to names as well is: If you can't come up with a good name, then come up with a really bad one. A good name will come to you at some point in the future and the really bad name will poke you in the eye so you remember to replace it.

What good names tends to do in the long run is to help create good abstractions. This can be such a simple thing as extracting a class for an ID. If the ID is just a string or number it's hard to find where it's used. When extracted to a value object you are bound to find places where certain simple operations are preformed on the ID. Such as an equals operation. In some places there will be a null check, in others not. A common problem for bugs and a headache when changes are to be made. If ID instead becomes a class all this functionality finds a home. And all of a sudden ID's are not longer some random string in the system. It's a thing that we can reason about, not just on the whiteboard but also in code.

When you have found the names, and the classes or functions you want to create, you can use a number of common refactorings to extract them. Some are: Compose Method, Method as a Comment, Extract Method and Extract Class.

This article only scratches the surface of the subject legacy code. But I do hope it has tickled your curiosity enough to continue explore. A good place to start is the literature I've mentioned. Especially Michael Feathers and Adam Tornhill.

I would love to get your feedback, thoughts and ideas on the article and the subject in general. Did I miss something? Is there something that's not correct? Do you disagree? Or do you have furthers insights, thought and ideas you'd like to share? Please do post a comment.

The article is derived from a talk I do about tackling legacy code. If you are interested in this you are welcome to get in touch.

 

I have been at many places, in many teams, where no-one has ownership of the build scripts. It’s an area where a company can bleed a vast amount of time and money without realising it. It  may even be what makes senior member leave in the end due to frustration with to much inaction.

What is unfortunate is that it is relatively simple to fix. Sure, there are risks associated with any legacy clean up. But getting the build to work is far simpler then many other tasks in a legacy codebase. I find that the main reason it is not done is that the team can't communicate the value to management (including product owners). Therefor it gets low priority and ends up at the bottom of the back log.

I have always found that one of the best ways to communicate value to management is in money. By putting forward a business case with a clear RoI (return of investment). When it comes to build scripts this is a pretty easy task. First, ask everyone on the team who uses the build scripts to keep track of how much time they spend doing build related stuff. How much time do they spend working around a broken or non-functional build. Try to count only time which is spent unnecessarily. Once you have enough data to know what the broken builds costs on a weekly/monthly/quarterly (which ever suites your needs) basis it is time to look at how much it will cost to fix the build. I suspect the ratio will be quite scary. The difference is how much you loose each week/month/quarter. Hopefully this should get managements attention and priority.

Now that fixing the build is top in the backlog we need to make sure it doesn't fall back into despair again. This is where ownership comes into play. In a well functioning self organising team this is never an issue. Everyone owns the build. Everyone makes sure the build is just as snappy as everything else.

But If you don't work in senior or self organising team? Who should be responsible? A good place to start is the team. Perhaps someone feels very strongly for it. Passion goes a long way. Granted, you can't have just one who owns the build. It is knowledge which is far to important not to share between at least two. But with one lead it's usually easy to follow. Make sure to pair often to distribute knowledge.

If it's difficult to find someone who wants to take responsibility the team should make sure to pass it around. Share the pain to make it lesser for everyone. The main responsibility will however always fall on the architect. A working build is part of the non functional requirements that makes up an architects main concerns. How will you as the architect be able to guarantee that the solution you are the master builder of will ever reach it's intended user or customer if it cannot be built?

Now, go and fix that build and remove the pain. It is so worth it. And remember, with a working build you'll have so much more time to code, and that is what's really fun!

For some reason I could not get Karma and Angular to work with Jasmine on my machine and there was precious little debug help. So instead I decided to try out QUnit as the test runner inside Karma. This required some puzzling together of different blogs and other instructions so I though I'd putt it here for future reference.

Fist of all I created a new project directory. This is where all commands are executed from and all paths starts from.

Next we will need the libraries Angular and Karma-qunit. Lets start with Angular. Download it from the website. I choose the zip which includes all the Angular files. Expand the zip into lib/angular

To install Karam I use npm. Run the following command:

This will install karma-qunit into the project folder. I prefer this to using a global version since it happens all to often that tools break when upgraded and having them locally means that I can control which version is used for each project. The drawback is that in order to run karma you need to issue the command ./node_modules/karma/bin/karma every time you need to run it. To make life easier for your self you can add an alias for Karma like so:

Put the line at the bottom of your zshrc or bash_profile to ensure it's always loaded.

Next we need to configure Karma so that it can find the source code and libraries. This will also allow us to configure which browsers should be captured during testing. To do this we will use Karmas builtin init function. Run the command karma init and follow the instruction on the screen. Look at the config file below to for guidance when answering the questions.

Now that we have a configuration file there are a couple of things that may need changing. Open the file karma.conf.js in a text editor and compare it to the example config above. I made the following changes to mine:

  • In the file section there are two exploded values, I.E. values in {}. When specifying a file pattern to Karma you can either use a string or expand it to set some properties. Since we will not change the files we serve from Angular there is no need to set watchers on them which is why then are exploded. The lines below should be first in the list (make sure that the same file pattern is not in the list twice).

  • Make sure the following files are in the exclude list:
  • If you want to see more output from the running tests you can sett the logLevel value to config.LOG_DEBUG.

You should now be able to start Karma from the root of your project. When it starts it will report an error:

Firefox 26.0.0 (Mac OS X 10.9): Executed 0 of 0 ERROR (0.235 secs / 0 secs)
This is as it should be. There are no tests available and this is reported as an error. You can now leave karam running. It will detect changes to the files in js/ and test/ and rerun any tests it finds.

Now lets add our first test. To make sure the test runner works with Karma we'll start with a "non test". Since I will be testing a file called controllers.js when I write the real tests I'll add the test to the file test/controllerTests.js.

When I save the file Karma detects the change and runs the test. This should show up in your Karma console:

Firefox 26.0.0 (Mac OS X 10.9): Executed 1 of 1 SUCCESS (0.651 secs / 0.001 secs)

Lets add some real test code instead. I am going to create an application called freecycleApp. It will have a controller called NodeController. To load this into the test there is some scaffolding required so I'll add the following to test/controllerTest.js:

The module call will allow you to set up the scaffolding required to launch your Angular controller. The injector will load the application, the ctrl is your controller and the scope is the data object passed to the controller.

The first test added to test/controllerTest.js looks like this:

The test will fail, complaining that there is no module called freecycleApp so we better create it. The production code goes into the file js/controller.js.

This will create the angular module freecycleApp and the controller NodeController, which returns a list of nodes.

This should set you up to start test driving your Angular development using QUnit. Have fun!

Legacy Code is important. Not just to us, as programmers, and the companies we happen to be working for. It has much larger implications then that. A couple of years ago I heard in a podcast that software has a 98% penetration rate on our everyday lives (I can unfortunately not find the reference now). The figure was for the northern hemisphere but the south is catching up fast. This means that our legacy as programmers, our legacy code, has an impact on pretty much every part of the society we live in. It affects us all every day of our lives. At home, at work, going in-between and pretty much everywhere else. I think we often forget this when we hack away at the task at hand. I certainly often do. We should take credit for all the great things we are able to create. But we also have to take responsibility for the bad. It is up to us to ensure that we create great software.

Because of this I think it's important to analyse how we deal with legacy code. Can we make sure that the code we leave behind does not become a burdon, a liability, but an asset? If we can, then how can we do this? And if we can't, then how can we ensure that cost is as low as possible?

Before I dig in to this subject I will try to define what I mean with legacy code. It's a term that is used by many and in many different contexts. Looking around online I find Wikipedias definition:

"Legacy code is source code that relates to a no-longer supported or manufactured operating system or other computer technology."

Another well known definition is Michael M. Feathers's from his book Working Effectively with Legacy Code:

"To me, legacy code is simply code without tests."

These two are in a way two ends of a spectra, where Wikipedias definition is rather weak compared to how we use the term in our profession and Feathers definition is rather harsh, considering that most programmers don't test drive there work and therefor create legacy as soon as they sit down at the keyboard.

There are a couple of other interesting pages on the subject that I have stepped over in my search for a good definition, one being the C2 wiki page, which I found the most inspirational, the other is /*Programmers*/ which discusses the subject in quite some detail.

My definition is:

Any part of a system, or whole system, that is hard or impossible to change when the requirements for it changes in such a way that the code needs changing to continue to serve it's users in the best possible way.

This means that code becomes legacy as soon as the context in which the code functions in changes and the code cannot keep up with the change in an easy way. What I mean with the context is anything that effects the code, this includes users requirements, regulations and the technical environment as well as anything else that affects the codes function. Easy to change is when the difficulty of change is proportionate to the perceived impact of the change. Some things are harder to change, for example a storage solution. Others are simpler, for example UI widgets or fields.

This makes all legacy code a bad legacy. That is how we use the term in our every day work. Therefor we also have to try to minimise the legacy code we create.

I have found two possible ways to deal with this. One is test first or TDD. The other is to ensure that each component, each function, is small enough and decoupled enough to be thrown away in it's entirety when it needs changing. I'll look closer at each in turn.

Test first or TDD is a way to work with the low level code as well as with higher level requirements by writing tests first. We have all heard the debate about this. I think this way of working is the only way to ensure that all aspects of importance in the code are tested in such a way that we can ensure no regression happens when we change them. This is, as far as I know, the only way that we can keep a system agile and easy to change. In other words the only way that we can keep a system from becoming legacy. Higher level tests are bound to requirements, lower level tests are bound to implementation on a class and method level.

What I have also found is that in such cases, sometimes when the tests are to hard to change to fit new requirements it is possible to throw them and the accompanying code away. The tests provides the documentation needed to improve the subsystem to work according to the new and old requirements. And the high level tests will catch functional regression. For this to work properly everyone has to realise that all code is design, every single line, and that everyone is responsible for the design.

I know that not everyone think they can do Test first. And perhaps it is not required as long as all requirements are covered in functional, BDD style, tests. In my experience as a Test first person this is not true but perhaps I fail on the more general level since I have the safety net further down and closer to the implementation.

The other way to avoid costly legacy is to make sure that each piece of legacy is so small that it is disposable when requirements change. I realised this quite recently when understanding Micro Service Architectures. The way I understand this is that the concept of a "class" is pretty much a service. Sure, the classes we normally create are much smaller but the higher level classes that we aggregate to provide a function of the system, the "public" interface, can be defines as a service. They are then bound together using a message or service buss instead of the normal object messaging used in classic class oriented systems. This means that it's an external orchestrator that provides the system of functions to the outside world. Tests are still essential and I believe they should be written first. But test in this context are functional tests which verifies the services functions as provided to the buss, and other services. Here BDD and similar methods are excellent. And the boundaries are made very clear. It also means that one service can be written in one language and another in another language. When a service needs to change it is quite possible to throw it away and create a new one. Chances are it's even better when reworked a second or third time.

The risk with this approach is of cause that the orchestration becomes so complex that the legacy is introduced on higher level instead. It is however often easier to mange complexity on higher levels then further down.

There are of cause situations when code does not have to be all squeaky clean. PoCs, pilots and other early incarnations of systems are often put together in a hurry and can often be thrown away as a learning experience. The problem with legacy arises when we risk loosing important functionality. So here it is our responsibility not to give in to pressure from others to let system with to much and badly controlled technical debt go live. This is not easy but I think we have to in order to build a reputation as professionals.

I hope to get your comments on the subject so please share your thoughts below or in your own blog with a ping back.

I got the opportunity to go to Øredev last week. Thanks to Martin Stenlund, the CEO at the consultancy Diversify where I work, I got a full three day pass. I’ll try to summarise here what I found most valuable.

There were a couple of talks about Agile and how to bring Agile forward that I found very refreshing. They kind of go below the surface, the process, method and so on and look at the underlying values that can empower us if we only let them. There was the Mob Programming talk by Woody Zuill and also Implementing Programmer Anarchy by Fred George. Both of them focused on what can happen when you let the team get the jobb done and get out of the way. As Zuill put it it’s not about making great software, it’s about creating an environment where great software is inevitable. Much of this is about competence, both from the business side and from the programmers side. If either is not advanced enough to work in this way it will fail. But we can still strive to educate each other and our self to become knowledgeable enough to work like this. If you have the opportunity I warmly recommend you to see the talk by Fred George, it has unfortunately not been published online yet. You can see the Mob Programming talk here.

The most surprising talk I attended was a talk on performance testing by Scott Barber. The reason I was surprised was that I had planned to go to see a different talk, about Netflix architecture, but went to the wrong room. I think that was a strike on luck, the talk was an enlightenment on how we can do performance better, and much easier. What I brought with me was the thought that performance testing is something that should be done from the very start to the very end on every level. Scott pushed it as far as down to the unit level. Measure how long each operation takes by logging start and end time. You can then use the data to quickly pinpoint where and when a bottleneck was introduced. My first though when he said this was to write a new JUnit runner which logs start end end time for each test. This can be applied on all levels where JUnit is the test framework, which in Java is almost everywhere. If you have the opportunity to see this guys talk it’s worth the time. He is a good presenter with a very strong and contagious passion for performance.

I of cause watched Adam Petersen Tornhill’s talk Code as a Crime scene. He puts forward an interesting argument on how to track bug hotspots, places in dire need of fixing, using forensic physiology. What it really boils down to is to look at what changes the most at the same time, and has the most code in it. This is probably the place which will have the most issues. He also introduces a tool he has written which will help do the investigation by analysing SCM history. It’s available at Github. There is more to it then this and the talk is available online so please do watch it.

The last session I will mention here was J.B Rainsberger’s talk Extrem Personal Finance. In his talk he goes through how he managed to become fanacially independent without winning the lottery. What stuck with me was a method to calculate how much of my life I spend to be able to do something. As an example he mentions that his habitual coffee on the way to work cost him 23 minutes of his life each morning. Was it worth it? It is also well worth a watch and available online.

There was of cause several other good sessions. Some I missed and some I saw. But above are the sessions that have had the most impact on me and which will stick with me. All in all it was a very good conference and I am very grateful that I had the opportunity to go.