There are already too many memes about testing in production. Image by WikiImages/Pixabay

Shifting Testing Right: Testing in Production

It was only a few years ago that any mention of “testing in production” made anyone involved in software engineering to dry heave. Nevertheless, it still happened. Software Engineers would test in Production quite often, but just not tell anyone. This seemingly reckless code deployment strategy caused a lot of problems and caused anyone involved in Software Development to baulk at those 4 words — “we test in Production”.

But times have changed. These days there is an abundance of tools that monitor, log, analyse data, automate critical error-prone parts of deployments, allow releases to Production to be broken up into more manageable chunks. There is also the DevOps ideology — that brings all of the painful parts of a software release into joyous harmony. And with all this harmonious joy it allows us to do things like testing in production.

However, the name is not really what it suggests. Testing still needs to happen at every stage of a product’s lifecycle. It doesn’t mean that it is ok for no testing to happen — far from it — it is just a new way of looking at an old problem. This is Shifting Testing Right.

Testing: In Production

Sometimes testing does occur in Production environments. No one wants to admit this. Even your Sales team know this, probably Marketing, and definitely the Engineers - but it’s a secret as it scares customers. Customers love using software that is 100% bug-free. Rather than the arguing and the futile persuasion that happens regarding this issue of 100% coverage, it is a lot better to just come to terms with the fact that it is likely that this number cannot be achieved, and figure out a solution instead. It is more worthwhile to mitigate failure through the use of continuous monitoring. This allows for better coverage and is more flexible in covering edge cases. When code is in Production it comes into contact with more systems and the code actually gets used by real customers’ — rather than a team of engineers. The data sets that get consumed by the…