Simple Quickcheck implementation for Python

It’s hard to compete with Haskell’s QuickCheck when it comes to having random test cases generated for you from properties. QuickCheck in Haskell is aided by the fact that Haskell has a very rich type system. I wanted something similar when writing tests in Python, but Python is dynamically typed. Fortunately it’s not too much trouble, using decorators, to annotate methods in Python. Here’s an example using the simple library qc.py that I wrote:

from qc import forall, a_unicode, a_list, a_character

@forall(tries=1000, i=a_unicode, l=a_list, c=a_character)
def test_simple(i, l, c):
    assert type(i) == unicode
    assert type(l) == list
    assert type(c) == unicode
    assert len(c) == 1

The “tries” argument defines the number of test cases that will be generated (when not specified it defaults to 100). a_unicode, a_list and a_character are all functions that return thunks (which, now that I think about it, is just a fancy word for a closure with no arguments) that are forced to evaluate each time a test is performed.

To see what data trips up the test I use nose with the -d argument:

$ cat simple_test.py
from qc import forall, an_integer

@forall(i=an_integer(low=0, high=20))
def test_fail(i):
    assert i > 10

$ nosetests -d simple_test.py
F
======================================================================
FAIL: simple_test.test_fail
----------------------------------------------------------------------
Traceback (most recent call last):
...
AssertionError:
>>  assert 1 > 10

Nose displays the random value that caused the test to fail.

Dictionaries, integers, unicode strings and anything else you can dream up can be easily supported:

$ cat simple_test.py
from qc import forall, a_dict, an_integer, a_unicode, a_list
import random

@forall(tries=2,
        d=a_dict(item_generator=an_integer,
                 value_generator=a_list(
                   size=(0,10),
                   item_generator=lambda: random.choice(['monkey', 'fox', 'llama']))))
def test_dict(d):
    for key, u_list in d.iteritems():
        assert type(key) == int
        for item in u_list:
            assert type(item) == str

forall.verbose = True

$ nosetests simple_test.py -s -d
{'d': {26: ['llama', 'fox', 'fox', 'fox', 'fox', 'monkey']}}
{'d': {59: []}}
.
----------------------------------------------------------------------
Ran 1 test in 0.001s
OK

I used qc.py to assert that my Hangeul library works across all Korean characters. This test asserts that vowel, padchim, lead and join all work as expected:

@forall(character=a_unicode(minunicode=ord(u'가'), maxunicode=ord(u'힣'),size=(1,1)))
def test_join_randomly(character):
    assert join(lead(character), vowel(character), padchim(character)) == character

qc.py should be test framework neutral (since it uses asserts and just wraps the test method in a loop). I’ve only tested it with nose. I found peckcheck when searching for a Python QuickCheck implementation but thought it could be better done in a test framework agnostic way with decorators. I later found Paycheck (another QuickCheck implementation in Python that uses wrappers) after I had written qc. After a brief look it seems like the way that qc uses thunks might make it easier to support injecting arbitrary data types.