Test the solution
Never in the history of prediction have so many people got it so wrong.
Testing the immediate solution
Sometimes you either won’t need or won’t have time to test the solution you have come up with, but most of the time you will want to do so, if only to have a degree of confidence that you have actually cracked the problem.
Ideally, you will test the solution in an ‘off-line’ way first: for example, when you have tied a pair of tights around the pulleys in your car in place of the broken fan belt, you would usually start the engine and run it up to see that it is
... before you close the bonnet and try driving off.
Then, when you drive off, you would probably drive relatively slowly, watching the alternator warning light for at least a short while before you relax and conclude that you have a decent, if temporary, solution to the problem. So you have carried out a simple two-stage test: a dry run and a controlled real-time test.
Similarly, in the real world of more complex business problems, you will want to test that your new luggage handling system works before you actually try it on the travelling public on the busiest flying day of the year.
Given that problems and their solutions come in all different shapes and sizes, you may want to use a risk assessment approach (see also the topic on Risk Management) to help you decide how to test your solution.
The risk, in this instance, being that the solution fails to solve the problem.
I have learned the novice can often see things that the expert overlooks.
In risk assessment, you first consider the impact of the risk: will it be barely noticed (at one end of the scale) or utterly disastrous (at the other end)? Then you consider the likelihood of that risk happening.
In this instance, there is going to be a degree of value judgement: if the solution looks perfect or if it comes from an unimpeachable source, you will have reasonable confidence that the likelihood of disaster is low; if it is a totally new, innovative, off-the-wall solution or comes from a source with little credibility, you may feel that it is more of a gamble. (Here, you need to be very wary, because solutions proposed by ‘experts’ may be disastrously wrong, while those that come from novices may be brilliant.
Similarly we need to be wary of solutions imported from other places, as discussed under Wheel reinventation avoidance.
If the impact of the solution failing would be high, then you will want to carry out further, and more exhaustive, tests to ensure that the probability of the solution failing is lower.
For example, if the problem were the losses of luggage and the solution was high tech, costly, and potentially affected 50,000 pieces of personal baggage, you would want more insurance than crossed fingers on the day your solution was put into operation.
If the impact of the solution failing would be low, then you might carry out a relatively cursory test, just to make sure that you don’t end up with egg on your face.
For example, if the problem was a single jamming printer on a network that had 15 similar printers, it wouldn’t be the end of the world if it didn’t work immediately. To test any solution, therefore, you might choose to
- Have an offline trial
- Slowly ramp up the pressure on your solution to just above ‘normal’ operating level
- Run your solution past focus groups to test reactions
- Run some sort of computer simulation
- Run some sort of ‘real life’ simulation (using staff or volunteers)
- Have a ‘dry’ run, rather like a test lap on a racing circuit
- Have a limited release, say, to a test market.
The dangers of not testing
Taking time and money to test a solution is often painful: egos are under test as well as the solution; the test is taking up potentially valuable time and money, and meanwhile the competition may be catching up with you, especially as secrets are so hard to keep these days!
However, testing is often vital, as it prevents us from creating whole new problems for ourselves and our organisations. Think how differently things might have turned out if the Titanic, or the UK poll tax, or the Ford Edsel had had more exhaustive tests.
A well-known department store had a consistent problem of image in their shoe departments: single shoes were left strewn all over the shelves and floor after customers tried them on. The ‘solution’ imposed by head office merchandising managers was that shoes were to be fixed together in pairs with those little plastic straps.
Downstream impact: a customer trying on the shoes couldn’t walk in them as they were hobbled together with a three centimetre, almost unbreakable, ankle-cuff; so shoe sales dropped disastrously.
Testing the longer-term effects of the solution
We need to test the immediate effectiveness of the solution, but we also need to test the longer-term, downstream impact of the solution. What effect might this solution have on other things that we do or on other people who are connected? These aspects are often ignored in the relief and excitement of having found the solution to a tricky problem; it is, however, vital that we look at the downstream impact; when a computer manufacturer ‘solves’ a technical problem, it may leave all their existing customers with now obsolescent machines, when a finance department solves a process problem, it may leave customer facing parts of the business unable to carry on working.
The security department at a Merchant Bank had an urgent problem with preventing unauthorised access of non-staff personnel into supposedly secure areas of the bank premises. Their solution was a card key turnstile, which prevented non-cardholders from passing through.
Downstream impact: they installed the turnstile over the weekend, but forgot that the staff didn’t have the cards and it took about two weeks for them to be issued to everyone... in the meantime the gates had to be locked open to allow people to get to work!
We can consider the longer-term effects of the solution in a controlled way by reversing the cause-and-effect analysis tool and adding stakeholder consideration:
We need to ask ourselves what effect this solution will have on the methods we use at present, the machinery we have, our manpower and systems. Then we need to ask what effect this solution will have on our staff, customers, suppliers and other stakeholders.
Using cause-and-effect analysis
A utility company had a problem with billing, as they often received inaccurate readings.
- They identified that one cause of this was the inability of the meter readers to see the meters, which were usually in dark, inaccessible places.
- They concluded that the solution to the problem was to provide torches to the meter readers.
- Downstream this meant that they had to supply batteries for the torches:
- These had to come from another department, which had to indent for the funds in advance
- They also had to find a supplier of torches and batteries
- And they needed to hold a stock, which meant they needed storage space.