I want to tell you about this cool little tool for haskell, called QuickCheck. What QuickCheck does is test invariants against randomly-generated test cases. An invariant is simply a haskell function that takes an input value, and returns true if the invariant is satisfied, or false if it isn't. You then pass this invariant to QuickCheck, and it generates random input values, and checks to see if any of them break the invariant.
Here's a simple example, from the QuickCheck documentation:
prop_RevRev xs = reverse (reverse xs) == xs where types = xs::[Int]
This invariant is asserting that if you take a list and reverse it twice, you end up with the same list. To test the invariant, you simply pass it to the quickCheck function:
Main> quickCheck prop_RevRev OK, passed 100 tests.
Here's an invariant from my inverted index project. It was directly inspired by the above example. It states that if you take an Index and invert it twice, you end up with the original Index.
prop_InvInv idx key = idxLookup cidx key == idxLookup inverted2 key
where inverted2 = idxInvert (idxInvert cidx)
cidx = canonicalize idx
types = idx::Index Int Int
I've been using QuickCheck to test my haskell inverted index, and I'm very impressed. It has discovered several corner cases that I didn't think of and wouldn't have thought to test when writing test cases by hand.
For example, I was originally using lists to represent sets. Because the lists were really sets, their order didn't matter. But when I asserted to QuickCheck that two "sets" were supposed to be equal, it quickly informed me that they were actually in a different order. That's why I wrote the "canonicalize" function that you see in the above example.
I'm of two minds about the canonicalize function. On the one hand, it was a pointless exercise just so that QuickCheck wouldn't complain about irrelevant differences. But on the other hand, I've had more than a few unanticipated irrelevant differences cause embarassing bugs once my software starts getting used by other people.
QuickCheck is a very different kind of testing from the usual unit testing. I'm not sure whether I would recommend it as a complete replacement for explicit unit tests, but I get the strong feeling that it will find bugs that explicit unit tests would overlook. Perhaps its strongest feature is that it forces you to think about your invariants much more clearly.
My current feeling is that QuickCheck definitely fits with haskell's strong static typing mindset. It basically extends the checking to runtime values, and therefore brings with it all the rigor that static checking implies. And since the point of writing an inverted index in haskell is to fully understand the haskell mindset, I've decided to use QuickCheck as much as I can.
I wonder, though, whether starting with haskell will strongly influence the results of the shootout. Haskell will flush out a lot of fuzzy thinking that I might otherwise have gotten away with in a less strict language, like erlang or ruby. Therefore the code I write in erlang and ruby will probably have fewer latent bugs, but may take longer to write than it otherwise would have.
Posted on May 8, 2003 09:37 PM
More languages articles