Do Your Tests Wobble?

Image by delo from Pixabay

“The green reed which bends in the wind is stronger than the mighty oak which breaks in a storm.” 

― Confucius 

I was asked to pair Blog with Lena Weiberg - much to my shock and delight. This was the perfect
opportunity to hear another point of view on something, and share it with the wider community. 

~

Part of being a good tester, to me, is making sure I stay aware of the trends in development. Knowing 
how the team is working can help get comments and questions to them at the proper times, and has 
the information that can be acted on. In this quest, I attended a recent online conversation where the 
topic was primarily on the size of 'steps' taken in software development: making sure each step takes 
a reasonable amount of time, that it leaves the system in a state that isn't worse than it was before the 
change, and that could - if needed - be deployed at this point. 

The concept of “wobble”  (I'm not sure who originated the term)- how much the entire system is going 
to react to a change in one area – was presented, and caught my attention. Thankfully, this was near 
a break time, so I had a few moments to explore what this means to a team. And found several ways 
that this might change how they look at both ‘done’ and stability in a system. Does this fit the model 
they are using? And is the model wrong in this way?

And I wonder how much this concept has shaped the 'normal' path software goes through. Is this one 
of the reasons that testing has traditionally been placed at the end of the process?  Either by those 
writing the code itself, or as a chance to make sure that a release has what it needs to be able to get 
into the user’s hands?

I will use the example from the quote above: a clump of reeds with many points of contact – intentional 
and otherwise – is close to a flurry of tiny changes. This could be produced by having any change 
instantly sent out to everyone. So a button could shift colors several times in one day, or flip into a 
different position – as a deliberately extreme example. None of these are going to ‘break’ the system, 
but would certainly irritate me!

Or it could be more like an older, stiff tree: reacting quickly to a potentially damaging threat just isn’t 
possible. Large changes that appear with infrequent updates happen, but it is a shock to the user's 
workflow  They may have to puzzle out where the information is now accessed – and if the frustration 
of finding and relearning the ‘tree’ is large enough, you may lose that user, or a lot of them!

I mulled this over for a few hours, and got this clarified in my mind. When a system changes during 
coding, it should be easy to see how those changes influence nearby code, as well as the code it 
touches. This is a normal thing for many developers: it allows them a chance to make sure that the 
system isn't going to collapse. A change of something like a variable name will cause waves across 
the application - but this will settle out unless there is a conflict. Changing the way data is collected or 
displayed may cause more reaction, and might lead to a cleaner code base. This, to me, is an area 
of little wobbles: things are deliberately changing, but we minimize the wobble to avoid future work. 

Deliberately causing a large wobble is something that should be thought over carefully. Making sure 
the team knows of potential questions and issues, and a way forward to resolve them opens a chance 
to take time to look at the entire section. The amount of influence on the system as a whole is one 
consideration for how carefully you need to look. Changes like this most often are without problems, 
but they may infrequently cause damage, or at least confusion. And since the code has changed, 
there are the test and automated checks to update, and add to - no better time to make sure that this 
section of the code has more coverage than it did. 

A bunch of small changes can add up to a lot of 'tech debt'. It is very easy to lose focus, in this 
situation, on the larger change we are wanting to make. Changing the name of a variable (a legitimate 
change, and a 'step' of the proper size) shouldn't set off a flurry of tests. The 'flow' of the development 
would be disrupted, and cause even more wobble. But always waiting until the very end of the process 
might cause large wobbles, if a functionality is missing or changed beyond recognition.

Our tests themselves suffer from wobble. Exploratory testing, in some ways, seems to be built on the 
concept of wobble, and where it will take us. The tools we use to help find the errors – even 
something as simple as a set of tester notes – can suffer from changes that might appear simple. 
Those changes may have an effect on more-distant parts of the application, and don't get captured 
in the original set of notes on an area. Future testing and checking might be incomplete, and adds 
to the potential of missing things and ‘tech debt’.

Our developers have a focus on getting through the application for the specific use they see. Users, 
however, will very likely not follow this path, at points - or at all. Letting the system wobble may save 
hours of coding updates, even if it seems to be needless duplication in finding alternate ways to get to 
information. Ensuring that all of these ways work to make the user’s experience easier is not always 
the priority, but it should at least be thought about. This helpful wobble may delight someone who has 
been frustrated by a less flexible “happy path’ that other solutions offer.

Wobble, by itself, isn't bad. Reeds bend in wind, and the resistance they gain makes them stronger. 
Allowing the system to wobble a bit will help reinforce and guide future changes, rather than allowing 
the system to be inflexible. Change will happen, and the whole team is responsible for making sure 
that this is a positive thing.