There are a lot of times when during our testing we come across a bug that’s weird in terms of occurrence. Issues that occur in some special condition, which does not seem logical at all. But hey the issue is valid. How do we justify it? We would have to probe and isolate the issue in such a way that we find a pattern or a value that causes this issue. On similar lines, I came across this unique (weird) issue in the automobile industry. The story is as follows (for ppl who want the link: http://www.bairnet.org/footnote/981001pontiac.htm)
A complaint received by the Pontiac Division of General Motors:
“This is the second time I have written you, and I don’t blame you for not answering me, because I kind of sounded crazy, but it is a fact that we have a tradition in our family of ice cream for dessert after dinner each night. But the kind of ice cream varies so, every night, after we’ve eaten, the whole family votes on which kind of ice cream we should have and I drive down to the store to get it.
“It’s also a fact that I recently purchased a new Pontiac and since then my trips to the store have created a problem. You see, every time I buy vanilla ice cream, when I start back from the store my car won’t start. If I get any other kind of ice cream, the car starts just fine.
“I want you to know I’m serious about this question, no matter how silly it sounds: ‘What is there about a Pontiac that makes it not start when I get vanilla ice cream, and easy to start whenever I get any other kind?'”
The Pontiac President was understandably skeptical about the letter, but sent an engineer to check it out anyway. The latter was surprised to be greeted by a successful, obviously well-educated man in a fine neighborhood. He had arranged to meet the man just after dinner time so the two hopped into the car and drove to the ice cream store. It was vanilla ice cream that night and, sure enough, after they came back to the car, it wouldn’t start.
The engineer returned for three more nights. The first night, the man got chocolate. The car started. The second night, he got strawberry. The car started. The third night he ordered vanilla.
The car failed to start.
Now the engineer, being a logical man, refused to believe that this man’s car was allergic to vanilla ice cream. He arranged, therefore, to continue his visits for as long as it took to solve the problem. And toward this end he began to take notes: he jotted down all sorts of data, time of day, type of gas used, time to drive back and forth, etc.
In a short time, he had a clue: the man took less time to buy vanilla than any other flavor. Why? The answer was in the layout of the store. Vanilla, being the most popular flavor, was in a separate case at the front of the store for quick pickup. All the other flavors were kept in the back of the store at a different counter where it took considerably longer to find the flavor and get checked out.
Now the question for the engineer was why the car wouldn’t start when it took less time. Once time became the problem — not the vanilla ice cream — the engineer quickly came up with the answer: vapor lock. It was happening every night, but the extra time taken to get the other flavors allowed the engine to cool down sufficiently to start. When the man got vanilla, the engine was still too hot for the vapor lock to dissipate.
Moral of the story: Even insane-looking problems are sometimes real.
What I liked most in this case is the way the engineer looked at all factors from details of the car to the details of where the flavor of ice-cream was located in the store. There would be people who could overlook the latter detail, but that was THE key to solving the Vanilla ice-cream mystery. 🙂
So what are the ways:
- Reproduce the issue as many times and check for different conditions like environment, background applications, time of the issue (you never know what an AM and PM could do). Basically think out of the box!
- Once you have a pattern, try to put a logical link to the issue and get the technical reason for the same. (sounds simple but not easy)
- Google! It is always good to check the online resources where others might have encountered similar kind of bugs and read through their findings.
- Try to record the sequence of events that cause the issue and play it back. A visual recap of events always helps in triggering an idea. Reproducing is different from this. Here you are only watching the sequence and that could get something running.
- Talk to people around you even if they don’t have an idea about your work. Believe me sometimes they are of best help. 🙂
There was a time when I was known for finding bugs on my Mac to the extent that developers wanted the Mac to be removed from the test team :-). The frustrating part was that only the Mac that I had could reproduce the issue and not any other machine that we had in the team. A lot of times we couldn’t find the exact reason but in others it depended on the system configuration and order of executing the steps. Today when I think of it, I regret not having probed further to find out the actual cause.
It definitely is not easy to find the reason behind weird issues such as above, but in addition to the frustration involved it adds knowledge to one’s testing and probing skills. So the next time you find a weird issue step back and see how best you can find the cause and solution for it.