Despite unit testing has formal metrics that can prove at least minimal level of thoroughness; many organizations struggle to uncover measurement that can help telling how much of unit testing is enough. The minimal level of testing can be measured in terms of code coverage. Code coverage can be as weak as statement to as strong as du-paths. But this is not my goal to explain each of those. My goal is to help those who try finding an alternative measure of unit testing quality.
One of ideas is measuring it post-factum. Well it may be too late to know the truth. Nonetheless this knowledge can be used in the following project to reduce number of defects of a specific sort. I mean the feedback that you receive from testing.
Software is being developed in several stages. Even in Agile, there are stages; but they are shorter and tightly intertwined. Defects injected at a given stage can be found at that stage or at the following ones. Depending on how many defects injected at the stage escaped quality control, we can draw our conclusion about testing quality.
For example, we have created a unit and performed unit tests. In result of testing we have found and fixed 16 issues. But later when a unit comes to Integration and System testing, we discover additional 4 issues attributed to unit. This means that defect removal efficiency of unit testing for that module was:
DRE = (1-4/(16+4))*100%=80%
To enable this measurement you need to analyze issues found at later stages of development. If you have a defect tracking system (and I hope you do) then you can simply introduce the field that will denote to what of development stages a defect can be attributed. The biggest problem here is having defects found at unit testing stage filed somehow. Many developers see it as an additional unneeded paperwork. So, if you are going to implement this metric be prepared to the resistance. You may decrease the pain by claiming only brief information on unit testing defects to be filed (summary only). There are tools that can be integrated into development IDE to soothe this pain for developers.
Other testing stages' effectiveness can be measured the same way. All you need is to collect this information continuously. Two or three projects later you will have plenty of interesting information to think of:
- Why is this module's unit testing is lower in effectiveness than others?
- Why developer A usually produces higher effectiveness than others? Can the latter share the experience to the team?
- Who is responsible for most of the issues down the road? What can you do about it?
Hope this helps! :)