Code Coverage for C++

If you need to test a codebase, one of the first things you should consider is getting code coverage metrics, so that you know what is currently being tested and what isn't. This is one of the first tasks I set myself when I started my new job, three weeks ago.

At first I tried to use gcov, which comes standard with the gcc compiler. The first problem that I ran into with gcov is that it doesn't support shared libraries. So I spent a day or two convincing my Makefiles to build static binaries and working around the resulting link errors. Unfortunately, as soon as I ran the static binaries, they crashed due to memory corruption. This was with gcc 3.3.2. At this point, I looked for other alternatives.

There are a plethora of code coverage tools for Java, but very few for C++.

I couldn't use PurifyPlus because they don't support code coverage analysis for Linux. Also, my previous experience with Purify suggested that they really want to quit supporting UNIX altogether.

On SourceForge I found a project called covtool, but I didn't bother trying it out because its mailing list only had one question, dated October 2002, claiming that it doesn't work. That message never even received a reply.

Then I found Bullseye Coverage, and my search was over. It took me only a few hours to install Bullseye, build the code, run the tests, and get some useful coverage metrics. I had only two problems: one was a bug in our code that gcc hadn't noticed before, and the other was a bug in Bullseye's handling of an obscure aspect of C++ (the instantiation of a temporary object defined in the condition of an if statement when the class of the object has no default constructor). I filed a bug report with Bullseye on Friday evening, and they had fixed the bug by Monday.

The first thing I liked about Bullseye was how easy it was to get it working. It literally only took two or three hours, including the time to rebuild the code.

The next thing that impressed me were the summary reports -- building similar summaries using gprof would have taken me a day or two of Perl hacking.

Once I got used to the summary reports, I came to appreciate Bullseye's focus on condition/decision coverage, instead of just line coverage. It gives you a much better idea how much coverage you've achieved in a qualitative sense, as opposed to the merely quantitative statistics that you get from measuring line coverage.

Another thing that I like about Bullseye is that it can output its results in CSV format. I've already used this capability to write a Perl script that generates hyperlinked and colorized reports, for easier browsing.

Also, because Bullseye uses source-to-source instrumentation, it should be possible to use it in conjunction with ccache.

Posted on March 16, 2004 10:36 AM
More testing articles

Comments

Hi,

besides Bullseye Coverage there is an other very good code coverage tool for C and C++ (also availble for all targets in embedded systems): CTC++ from Testwell in Finland.
Product infos are available from http://www.testwell.fi or http://www.verifysoft.net/en_ctcpp.html

Klaus

Posted by: Klaus Lambertz at May 12, 2004 08:47 AM

The compile process with BullseyeCoverage with my g++2.9.6 seemed endless:-(, any idea?

Posted by: XieWei at February 9, 2006 09:53 PM

Is it free/open source?

Posted by: Zhang sf at September 7, 2006 07:14 AM

Zhang, no, it isn't. It's relatively cheap though, if you're associated with a company.

Posted by: Kim at September 7, 2006 06:31 PM

How about the price? And another question, we want to a Unit Test for our software. And we write some UT cases for it by using CppUnit an auto-testing framework. Now, we want to know, our test cases code coverage for the software(code). Can the BullseyeCoverage or CTC++ do this job? If the tools can be run together with CppUnit case running? We work in GNU Linux. Another question, we, in some possibility, maybe use a c++ compiler for xscale.

Posted by: Zhang sf at September 8, 2006 03:44 AM

Zhang, for more accurate responses to your questions, I encourage you to check out the Bullseye website. Specifically, the purchase, and platform pages.

When I puchased it, Bullseye cost $800 for the first year, and $100/year after that.

It should work with any unit test framework, since it recompiles the product code.

Posted by: Kim at September 8, 2006 03:21 PM

Kim, Thanks a lot.

Posted by: Zhang sf at September 11, 2006 05:58 AM

Any chance you still have the Perl script you referenced in this article lying around? I'm evaluating C++ code coverage systems for my project, and one thing we hoped to accomplish is having our automated build system run coverage tests and produce results accessible from a web browser. I could write the script myself of course, but if a perfectly round wheel already exists I see no need to craft a new one. :)

Thanks!

Posted by: David at November 1, 2007 04:04 PM

My experience with bullseye was extremely positive. we use visual studia and the integration was so simple, even a dummy like me could get it working in minutes. The output is also very easy to read with summaries etc

I attempted to trial the testwell stuff, but after speaking with someone there, I didnt hear anything back from them about trial license etc.

Posted by: roger balakrishnan at May 9, 2008 02:02 PM

My experience with bullseye was extremely positive. we use visual studia and the integration was so simple, even a dummy like me could get it working in minutes. The output is also very easy to read with summaries etc

I attempted to trial the testwell stuff, but after speaking with someone there, I didnt hear anything back from them about trial license etc.

Posted by: roger balakrishnan at May 9, 2008 02:03 PM

Does anyone here have any experience using Coverage Validator by Software Verify: http://www.softwareverify.com/cpp/coverage/index.html

I am trying to explore C++ coverage analysis tools that have no conflicts with Visual Studio.

Bullseye looks promising from the posts...How about Coverage Validator...any comments.

TIA :)

Posted by: Evian at May 15, 2008 07:21 PM

Hi,
Can i re-insatll the evaluation copy of Bulls Eye Software on the same PC,After the completion of evaluation period.

Posted by: Gaurav at November 11, 2008 12:56 AM

how to install bullseye code covereage tool on unix

Posted by: sudeep at January 16, 2009 06:34 AM

Hi ,

Anyone knows how to use coverage Broswer in Bull's Eye code coverage . My code is in Unix box and i can run and see the coverage in UNix box , but anyone knows how can i use Coverage Browser to see the result in windows .

Posted by: Shilpa at February 5, 2009 04:53 AM

Semantic Designs offers a test coverage tool that handles a variety of C++ dialects: ANSI, GNU, MSVC 6 and MS Studio 2005. It also offers similar test coverage tools for other languages, including C#, Java, PHP and COBOL.

http://www.semanticdesigns.com/Products/TestCoverage/CppTestCoverage.html

Posted by: Ira Baxter at July 21, 2009 05:32 PM

My experience with bullseye was extremely positive. we use visual studia and the integration was so simple, even a dummy like me could get it working in minutes. The output is also very easy to read with summaries etc.

Posted by: Club Penguin Cheats at September 24, 2009 08:19 PM

As humans, we're simply born into one of these dog training classes: chief or follower. As these couple types are extremes of one another, you may simply be a chief, or a buff, tiny is there of any fashion in between these puppy training.Some might contest though, that you could be both at one time, the chief to numerous and a buff to one.There are more teach dog tricks to outlook out for to review if or not you're a chief or a follower. The bulk all over individual aspect is the abilities to generate determinations not simply for yourself, but in supplement for remnant as prolonged as dog behavior training.Take this covering for example; each post-graduate learner is rewarded a cubicle each, which doesn't draw seal for loose though. Most would just announce no just because it doesn't draw seal free to wholesale dog supplies. The actual justification behind their tenacity is the portion that none wants to act on away from the crowd.When one or couple of them creates a tenacity to receive one cubicle in any dog supplies, remnant in the past prolonged followed not far behind. One later another, the cubicles are being filled. Sadly, there just isn't passable room for everyone to wholesale pet supplies. Even as learners with very competent standings and privileges, not all are competent to take find how to lose weight and generate stern life correcting determinations on their own. A buff would perpetually have determinations organised for them, perpetually staring to lose weight to play-act first after the time arises. If you're one of those any person who essentially have a darling to move out with the accumulation, it's come seal time to commence correcting and take find fault for once. belly fat is a yardstick journalist for a many cluster of themes and subjects. He in supplement drafts on braun bathe and transform and braun bathe and charge.

Posted by: dogtraining at October 21, 2009 09:01 PM

There is a tool called Trucov which builds upon Gcov, producing cleaner coverage reports and also outputs the control flow graphs with coverage information.

The homepage is http://code.google.com/p/trucov/.

The wiki page is http://en.wikipedia.org/wiki/Trucov.

Posted by: at October 22, 2009 10:31 PM

Bullseye uses source-to-source instrumentation, it should be possible to use it in conjunction with ccache.

Posted by: free games at October 25, 2009 04:06 PM

Love this man. Great stuff, great prices, free shipping. Email him if you cant find what you are looking for and he will look for you. Good customer service.louis vuittonSomeone who is knowledgeable in louis vuitton.Another guy from hong Kong (Largest louis vuitton second hand city, I believe) he has great stuff at fair prices but his shipping is insane

Posted by: louis vuitton at October 29, 2009 09:10 AM

Love this man. Great stuff, great prices, free shipping. Email him if you cant find what you are looking for and he will look for you. Good customer service.louis vuittonSomeone who is knowledgeable in louis vuitton.Another guy from hong Kong (Largest louis vuitton second hand city, I believe) he has great stuff at fair prices but his shipping is insane

Posted by: louis vuitton at October 29, 2009 09:10 AM

Love this man. Great stuff, great prices, free shipping. Email him if you cant find what you are looking for and he will look for you. Good customer service.louis vuittonSomeone who is knowledgeable in louis vuitton.Another guy from hong Kong (Largest louis vuitton second hand city, I believe) he has great stuff at fair prices but his shipping is insane

Posted by: louis vuitton at October 29, 2009 09:10 AM

Good day! Thx for your great post and Im thinking about how to introduce my ugg boots to you cos Im not sure if u like this. Many people who live in the cold area like the ugg boots, especially the north-europe. Our uggshare.co.uk supply the cheapest classic cardy ugg boots, class tall ugg boots, classic short ugg boots and other series of uggs, hope my words didn’t trouble u, cy!

Posted by: ugg boots at November 11, 2009 10:04 PM

Eliminating the need for the extra trouble to wear ugg boots uk
ugg boots saleuggs, in the extreme sports, look for happiness in life, with UGG Classic boots and nike sb showed off your tall body, do not worry about being laughed at, dancing new style ugg australia and nike sb dunk . The flavor of life that is colorful, there are Christian Louboutincheap nike dunksUGG cardy boots、and nike sb for sale also used fear to creep away money from your pocket Mody ? Love beauty, you might even love UGG Classic tall bootsUGG Nightfall bootsUGG Classic short bootsnew nike sb。There are an infinite high power drivers of popular front ugg boots saleugg saleuggsnike dunkdiscount ugg bootsnike dunk sb, our website also have UGG Nightfall bootsbutton ugg bootsbailey button uggugg button bootsbailey button bootsnike dunk midnike dunk highChristian Louboutin Boots, pick your favorite, do not hesitate!

Posted by: ugg boots sale at November 12, 2009 04:08 AM

Louis Vuitton , commonly referred to as Discount Louis Vuitton and Louis Vuitton Backpack, or sometimes shortened to Louis Vuitton Sale has become one of the most Louis Vuitton Sale Agendas luxurybrands Louis Vuitton Store.

Posted by: 1 at November 17, 2009 01:38 AM

lxjuan08ed hardyis a famous ed hardy store which sell directly ed hardy clothing, shoes, boots, swim suit and other cheap ed hardy.

Browse through our catalogue of hundreds of ed hardy clothing,Ed hardy Shirt,Ed hardy ... Buy Ed Hardy today for a gift, for a friend, or for yourself.

ED HARDY Online Shop,Ed Hardy Store by Christian Audigier,coach handbags,rolex watches.We supply cheapest Newest ed hardy discount 2009 Commodity New styles.


ed hardy sale

Posted by: ed hardy at November 18, 2009 08:20 PM

We are the best online sales for the china wholesale . Here you can have a large of choices of kinds Ugg Boots,Converse Shoes,Timberland Boots,puma shoes,Nike Shox Shoes ,Nike Dunk SB Shoes,Nike Air Max,Links Of London,Tiffany Jewelry,Dior Handbags?,jimmy choo handbags ,Cartier Watches, 8GB Mp4 Players,Bluetooth Car DVDs. All our cheap online cheap goods are high quality and original packages, and best service. We offer our customers the best service, 7 days arrive at your door.Enjoy your easy and happy shopping with us.

Posted by: cheap goods sale. at November 19, 2009 10:08 PM

Do you like sports? If you like it,the NIke Air Max TN and Nike Air Max 360 is a good choice for you.And now the there is a good news,CheapNike Air Max running Shoes is waiting for you.I think The New Nike Air MAX 2009 Running Shoes makes you great comfortable.

Posted by: Nike Air Max shoes at November 20, 2009 03:34 AM

People all over the world know the abercrombie and fitch,but not everyone really knows how fashion the abercrombie is,hollister is the Legend maker. Everybody wears the hollister clothing would be the abercrombie mens and the abercrombie womens, if you want know you can search the Ruehl No.925 or abercrombie outlet in the www.google.com .

Posted by: fitch at November 21, 2009 03:18 AM

Do you think I can stay timberland boots uk to become nothing to you ? Do you think I am a timberland boot automaton ? --a discount timberland boots machine without feelings ? and can bear classic timberland boots to have my morsel of bread snatched from my lips, and my drop of kids timberland boots living water dashed from my cup? Do you think, because I am timberland mens shoes poor, obscure, plain, and little, I am soulless and heartless? You think timberland 6 inch boots wrong!--I have as much timberland custom boots soul as you,--and full as much timberland roll top boots heart! And if God had timberland womens shoes gifted me with some beauty and much wealth, I should have made it as hard for you to leave me, as it is now for me to leave you. I am not talking to you now through the medium of timberland hiking boots custom, conventionalities, nor even of mortal flesh.

Posted by: ugg boots at November 22, 2009 11:06 AM

Your article is write very well, I like it very much ~
ghd Hair Straightener
ghd Hair Straighteners
ghd straighteners
GHD
hair straighteners
bose in ear headphones
bose on ear headphones
bose headphones
nike sb dunks
nike dunks
I wish you have a wonderful day!Thank you.

Posted by: ghd straighteners at November 22, 2009 08:51 PM

Timeberland boots on sale now! Do you want moreTimeberland boots information?Just move your mouse and press it to our Timberland shoes online store.

Posted by: Timberland uk at November 23, 2009 01:25 AM

Laptop Battery Laptop Battery Laptop Batteries
Laptop Batteries discount laptop battery
discount laptop battery
notebook battery notebook battery
computer battery computer battery
replacement laptop battery replacement laptop battery
notebook batteries notebook batteries

Posted by: Laptop Battery at November 24, 2009 10:01 PM

chaoying nfl jerseys
Troy Polamalu Jersey
Hines Ward Jersey

Posted by: jerseysleague at November 29, 2009 04:26 AM

Paul Smith has been collaborating with a few different organizations recently and his latest venture sees him teaming up with Japanese bicycle saddle, Paul Smith bag, on a limited edition leather saddle. The release features Smith’s signature stripes where only 20 will be available at Paul Smith Clothing store in Japan and 8 at the Floral Street store in London. Get on it quick

Posted by: dfasf at November 29, 2009 11:26 AM

It is no doubt that Replica Louis Vuitton Speedy 30 enjoys highest reputation in the world. Louis Vuitton Speedy 30, purses, shoes, watches, clothes, and belts successfully seize loving heart of a majority of people. Speedy 30 is filled with romantic and fantasy, and it is a symbol of noble and high quality of life for gucci ever! Today, let us talk about Tivoli PM.belts have different styles and colors, such as Monogram series belts, Graffiti series belts, both men and women love them crazily. All of them match with various leisure clothes, formal business suits, and beautiful clothes very well. Whenever you wear a Louis Vuitton belt to make your public appearance, you will be naturally concerned and admired by every body. When we talk of high quality, there is no other company to compare with Louis Vuitton Replica Speedy 30. All Louis Vuitton belts adopt top quality cow leather, high quality hard ware, and the color will never be faded. The belts are so durable, and you can wear them to match your clothes for several years. Also, you can clearly see an obvious LV logo as a buckle of each belt.

Posted by: fanqin at December 1, 2009 01:25 AM

In order to welcome the approaching Christmas and to thank for the customer,there are amazing ugg boots sale promotions on our company.Once you buy one of ugg 30th anniversary,ugg nightfall,ugg coquette,ugg lo pro button or ugg knightsbridge other ugg boots on our website, we will give you a gift for free. http://www.uggboots4buy.com/

Posted by: ugg knightsbridge at December 1, 2009 03:57 AM

If a man join together to pursuit knowledge, no one can take it from him.cheap nfl jersys Books are to human beings which as nba jerseys sale emory to the individuanl nhl Jerseys.We cannot change anything mlb jerseys unless we learn and accept it, Damn does not liberate it, it oppresses.A classic adidas jerseys book which people praise but don't read.A man dies still if he has done nothing, as one who has done football child jerseys much.Education is something which remains ofter one has forgotten everything football jerseys he learned in school.Education is something remains ofter one has forgotten everything he learned in school.http://www.nfljerseymlb.com/

Posted by: discount nfl jersey at December 1, 2009 03:57 AM

ugg boots sale at a time when large heat has set off a winter 08/09 to Ewha.Outlines the

warm wool winter fashion degree to wool crocheted wool ugg boots uk to join after the

body does not stimulate your skin soft, soft comfort UP. Fashion can not without
ugg boots presence in Winter.and ugg boots have many series.such as:
ugg classic
UGG Bailey Button boots
ugg nightfall
ugg kids

Posted by: uggboots at December 2, 2009 08:31 PM
Post a comment









Remember info?




Prove you're human. Type "human":