Friday, September 18, 2009

Making the testing better

Despite unit testing has formal metrics that can prove at least minimal level of thoroughness; many organizations struggle to uncover measurement that can help telling how much of unit testing is enough. The minimal level of testing can be measured in terms of code coverage. Code coverage can be as weak as statement to as strong as du-paths. But this is not my goal to explain each of those. My goal is to help those who try finding an alternative measure of unit testing quality.

One of ideas is measuring it post-factum. Well it may be too late to know the truth. Nonetheless this knowledge can be used in the following project to reduce number of defects of a specific sort. I mean the feedback that you receive from testing.

Software is being developed in several stages. Even in Agile, there are stages; but they are shorter and tightly intertwined. Defects injected at a given stage can be found at that stage or at the following ones. Depending on how many defects injected at the stage escaped quality control, we can draw our conclusion about testing quality.

For example, we have created a unit and performed unit tests. In result of testing we have found and fixed 16 issues. But later when a unit comes to Integration and System testing, we discover additional 4 issues attributed to unit. This means that defect removal efficiency of unit testing for that module was:

DRE = (1-4/(16+4))*100%=80%

To enable this measurement you need to analyze issues found at later stages of development. If you have a defect tracking system (and I hope you do) then you can simply introduce the field that will denote to what of development stages a defect can be attributed. The biggest problem here is having defects found at unit testing stage filed somehow. Many developers see it as an additional unneeded paperwork. So, if you are going to implement this metric be prepared to the resistance. You may decrease the pain by claiming only brief information on unit testing defects to be filed (summary only). There are tools that can be integrated into development IDE to soothe this pain for developers.

Other testing stages' effectiveness can be measured the same way. All you need is to collect this information continuously. Two or three projects later you will have plenty of interesting information to think of:

- Why is this module's unit testing is lower in effectiveness than others?
- Why developer A usually produces higher effectiveness than others? Can the latter share the experience to the team?
- Who is responsible for most of the issues down the road? What can you do about it?

Hope this helps! :)

Wednesday, September 9, 2009

Management means no friendship?

Not so long ago I partook in a long discussion related to whether or not a manager can have friends among his subordinates. As I remember, the vast majority people in management role believe there is no problem having friendly relationships with your teammates.

But I remember another very successful manager (a top manager) who kept saying "you should not be a friend to your people", "you should not go to lunch with them". Since then I keep replaying this through my mind and all the time it responds with "It is wrong", so I cannot adopt this thinking.

To me, building unnatural barriers is counterproductive. Open communication is a key to effective professional relationships. People are smart enough to pick up the signs of too artificial behavior. They will mind a reason behind it and will start doing the same. So they will never be open with you.

The same top manager once told me "why you cannot speak to me frankly?" How could I if I did not trust him because of his ways of managing people?! That was kind of paradox that we could not solve. I guess he had same problems with others. I heard it many times that people preferred to talk to him what he wanted to hear, not what they thought. No need to say that he was not able to manage the team effectively. So, despite of his experience I would say that he was wrong. Friendly open relationships help managers doing the job. It does not free people up of the responsibility; it rather imposes the responsibility on them. Subordinates tend to feel badly when they did not do things right, when they couldn't keep the promises not only to a manager, but to a friend as well.

Of course, this way of management works not for all of people you will encounter in your managerial career. As I always liked to say "In management there are always several ways to do the same. The way that you like is not the only one. So, keep learning."

It may also depend on the culture. If you have anything to say on this matter I would gladly hear that.

What do you all think? Long to seeing your comments…

Wednesday, September 2, 2009

What makes a Good tester?

Everyone can easily say a good tester from a not as good one. A good tester finds more quality defects. But what makes one to be good tester? What is the difference that helps identifying important sound bugs faster?

First is technique. A good tester knows what the weak points about the application are. If it accepts a text for the input and produces a tagged text as the output, then most likely the issues will start popping up if you feed it a large volume of text. If it ate it successfully then change the text to something developer would not expect for the input. This is a sort of destructive thinking that helped my son breaking seemingly unbreakable toys when he was from 3 to 5. Thinking in categories "what else people usually omit?" helps finding more defects faster.

Second difference is the feeling of urgency. A good tester will explore entire functionality of the application rather than focusing on a small part of it. In result, the number of defects and the severity of defects is usually higher than quantity and quality of defects produced by testers bogged down into details of a specific dialogue. Likewise, a tester who focused on functionality will produce more quality issues than a person who is too deep into UI testing. Small issues reported early in the testing when application is full or crashes may make developers and project manager go mad. You are very likely to hear: "What are you thinking about?" So, better start from beginning and focus on how system works. Of course they may be exceptions. For example, it may be not reasonable to test a part of functionality and get back to it later. Anyway, if it does not cost too much I would rather recommend focusing on what matters now.

Third difference is level of expertise. A good tester can quickly imagine in which way a new functionality can affect the existing one. Knowledge about system functional and structural dependencies can help finding integration issues. Such issues usually have heavy consequence, so they are rated as critical.

Forth difference is seeing it all as a whole. A good tester always finds out how the functionality is supposed to be used and tries it on realistic tasks. There is no need to remind what use case or end-to-end testing is. This is all natural for a highly skilled testing professional to think of what real user is supposed to do with a system. It also helps finding usability issues. Recommendations on how to make system better are always welcome!

I wish all testers around would be good ones. Hopefully this article will help some of the readers to become one :)