Yesterday at work I ran some code coverage metrics for the test suite we have. I learned that there's a substantial percentage of the code that isn't being covered. What I'd like to do is correlate that information with my knowledge of what feature aren't yet supposed to be covered. We have one large project in particular that isn't ready for testing, and I expect it accounts for a lot of the uncovered code. So I'd like to know what percentage of the uncovered code is due to that feature.
One idea that occurred to me would be to try to correlate call graph information with code coverage information. For example, maybe a base class grew a new method in order to support a new feature, and that method is overridden by a dozen subclasses. If that feature simply isn't getting tested, it would be nice to be able to "roll up" all those method implementations, and think of them as a single method that isn't getting executed. Then you could also roll up all the callers of that method, and all the methods it calls. Obviously if you go far enough up the call stack, then you're no longer talking about a particular feature, but it would still be useful to be able to see the entire uncovered code tree as a single entity.
A coworker mentioned that the idea that was "screaming out at him" was aspect oriented programming. If the coverage metrics could be categorized by aspect, that would clearly make it easier to understand the degree to which each aspect is covered. Assuming that aspects correlate with features, that would give you feature coverage. The only problem with this is that I've never yet worked anywhere that used AOP.
Posted on July 13, 2004 08:33 AM
More testing articles