Oh boy. This is one of those subjects – reasonable people can have valid, cogent and totally incompatible views. Here’s what I’ve learnt over the past 15 years.

Software developers are mostly interested in writing software; our motivation is basically to make things work. It helps if those things are intrinsically interesting or cool – games, genetic algorithms, high traffic websites. We like knowing what problems we have to solve, and what constraints and priorities we have to work in. The better developers tend to be fairly impatient with repetitive work – we want to work on the cool problems, not writing lots of boiler plate or manually packaging a deployment file. We’re often annoyed at waste – which is why we often dislike change, because it forces us to discard work we’ve already done. A lot of the time, our work requires concentrated attention – “flow” is often mentioned.

In this light, I think there are 3 major aspects to software developers’ productivity.

Defining the problem

Firstly, they need to never run out of problems to solve; the better they understand those problems, the more likely it is that they’ll get on with the work. Well understood doesn’t necessary mean “documented”, or “UML diagram” – though this can really help! In a lot of projects, it takes a while for the team to work out what they’re doing, and productivity in those early days can be very low, despite hard work, because the developer spends time solving the wrong problem.

I remember working on a reporting application many years ago; we were working on a report which would provide insight into the financial performance of everyone in the company, which involved a lot of complicated business logic, and our database machine was a little underpowered. I took on the task of performance optimization – tuning the SQL queries, recommending new index schemes etc. I thought this was a clearly understood problem – I just had to take the queries and make them faster. I worked hard for a week or so, and tuned a dozen or so complex stored procedures, taking our nightly batch time from hours to minutes. However, it turned out – of course – that the majority of the stored procedures were changing dramatically to reflect the business logic rules my colleague was working on. Because we had a horrible source code control system, and I had locked out the files, my colleague had no way of seeing my optimizations while he was working; it turned out that we had so many incompatible changes, we had to discard my work. I’d worked a week, but effectively, my productivity was worse than if I’d stayed in bed – at least that way, we wouldn’t have had to work out what our conflicts were.

So, a “well understood” problem is not just one where the requirements are clear, it’s also important to understand how the problem fits in the wider picture, both architecturally and within the process. This means that a “top down” approach, where architects, designers and project managers hand down requirements to the team, can be inefficient, because it’s hard to avoid conflicts and incompatibilities by looking at a Gantt chart or UML diagram. I much prefer small (5 – 10 people) teams where everyone works together to solve the problem; to run bigger projects, you find ways of gluing those smaller teams together.

Tools, process and the environment

Joel Spolsky writes eloquently on the topic of tools and offices for software developers; Peopleware covers this ground too. I’ve never worked in a private office as a software developer – but I have pretty decent noise cancelling headphones, and a Spotify account. However, nowadays, the biggest tooling issues – in my view – are not so much the quality of your IDE, or the speed of your hard drive. I don’t think there are any bad IDEs, and even the cheapest PC you can buy these days has more than enough memory and disk space (though, yes, I do like me a nice SSD).

Far more important is the question “how long does it take for a new piece of code to get to the customer?”. This is usually more about the process than an individual developer’s workstation – processes like Continuous Integration and Continuous Delivery are becoming mainstream, and we’re investing in automated testing tools and BDD.

All these processes make a developer more productive – not just in the financial sense (it takes less time for the work to lead to value), but also because they tend to dramatically reduce the risk of rework. By getting your work through a testing framework within minutes of completing it, you get very quick feedback on whether it’s right. By pushing work through the entire deployment pipeline every time you commit something, you know very rapidly whether your work will be deployable.

This takes a bit of upfront investment, but platforms like Heroku and Appharbor work brilliantly for web applications.

Expectations

Finally, the expectations you place on a developer tend to have big impact on their productivity. For instance, if you say “get this done next week at all costs”, you are likely to get it next week – but at whatever quality level fits in that time frame. If you say “make sure this scales to a million users”, you will get exactly that – though possibly at the expense of complexity that you weren’t expecting.

A friend of mine worked on a project years ago which was supposed to build a “Human resource management” system (this was before these were off the shelf packages). The project had a bunch of constraints and priorities as usual – time, budget, quality, performance, scalability etc. However, because the application was designed for a multinational, flexibility was a key issue – different countries have different legal requirements, data requirements, workflows etc. The project stakeholder told the team that he wanted to use the system all around the world – so they saw flexibility as one of their major priorities.

I met my friend for a drink about a year after they’d started the project, and he told me what they’d achieved – a framework which supported complex, polymorphous data representation. In Visual Basic 3 (yes, it’s that long ago). The framework was pretty smart – it could draw data entry forms for those complex, polymorphic data entities, using “convention over configuration”. It could cope with multiple database engines, and was able to handle multiple languages and character sets. They’d written a really cool tool to create and manage the complex, polymorphic data entities, with a “Microsoft Office” like workspace, and a custom toolbar. He felt the project had achieved lots, and was really proud of the cool stuff they’d been doing.

A month later, the project was disbanded – the project stakeholder got fed up with the project, as it hadn’t delivered any Human Resource related functionality, just technical infrastructure stuff. He’d got an external consultant to look at the project and the code, and the feedback was pretty damning – the complexity of the framework made it unusable for anyone other than the original team, and some of the code was highly suspect.

I never forgot this episode – and nowadays, I make sure that I always establish every expectation, even the ones that aren’t very sexy. Maintainability, extensibility, flexibility and code quality don’t always have to be perfect, but everyone needs to agree when you’re building something throw-away, and when it has to last. Performance and scalability aren’t always huge priorities – but again, no surprises! Resilience and managability often seem unnecessary to developers, but can be a huge pain once the product goes live.

I often like to hand our project stakeholders post it notes (3 for all of the expectations we think apply), and ask them to place them against a list of expectations – usually, they balance out pretty evenly. I don’t think I’ve ever worked on a project where a single aspect was even twice as important as the others, usually, the balance is that one aspect gets four post its and the others all get 3. It is quite common for one or two aspects to be de-prioritized – “we don’t care about maintainability, this is a throw-away application”, or “don’t worry about performance so much, we’ll throw hardware at it”.

What’s all that got to do with productivity? It’s very easy for developers to get seduced by solving a cool, complex, technical problem to meet one of those expectations, and for that to take a whole lot of time out of the schedule. If that problem is really important, that’s a great use of their time. However, it can also lead to “gold plating”, which by definition harms productivity.

Updated: