Unittest in detail

It is important that you are aware of how Unittest works in detail.

Examples

1st example

Take a look at this simple test case that creates a test suite consisting of tests that create a list of objects stored as class members.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
>>> from htf import TestLoader, TestRunner, TestCase

>>> class Test(TestCase):
...     def __init__(self, methodName='runTest'):
...         print("__init__(%i, %s)" % (id(self), methodName))
...         TestCase.__init__(self, methodName=methodName)
...
...     def setUp(self):
...         print("setup(%i)" % id(self))
...         TestCase.setUp(self)
...
...     def tearDown(self):
...         print("tearDown(%i)" % id(self))
...         TestCase.tearDown(self)
...
...     def __del__(self):
...         print("__del__(%i)" % id(self))
...
...     def test_garbage(self):
...         print("creating objects")
...         self.objects = [object() for _ in range(10000)]
...
>>> N = 3
>>> tests = ["__main__.Test"] * N
>>> suite = TestLoader().loadTestsFromNames(tests)
__init__(48444208, test_garbage)
__init__(48444624, test_garbage)
__init__(48444976, test_garbage)
>>> result = TestRunner(title="Garbage tests!", verbosity=2).run(suite)
test_garbage (__main__.Test) ...
setup(48444208)
creating objects
tearDown(48444208)
ok

Statistics:
-----------
Tests run: 1
Successes: 1

test_garbage (__main__.Test) ...
setup(48444624)
creating objects
tearDown(48444624)
ok

Statistics:
-----------
Tests run: 2
Successes: 2

test_garbage (__main__.Test) ...
setup(48444976)
creating objects
tearDown(48444976)
ok

Statistics:
-----------
Tests run: 3
Successes: 3


----------------------------------------------------------------------
Ran 3 tests in 0.008s

OK

So what happens in detail?

In line 24 a list of test names is created.

In line 25 a test suite is created. A test suite is an instance of unittest.TestSuite being a composite of other test suites or instances of htf.TestCase. That means 10000 instances of htf.TestCase before one single test is run (lines 26 .. 28). For every test method one instance of htf.TestCase exists with the member _testMethodName set to the name of the test method (lines 26 .. 28).

In line 29 the test suite is run with a htf.TestRunner. The test runner first creates a test result. Than it iterates over the test suite to find all tests. Every test is then run in a way that the test result captures the results of the tests.

Running a test is done with calling setUp() (lines 31, 42 and 53), <_testMethod>() (lines 32, 43 and 54) and tearDown() (lines 33, 44 and 55) in this order. <_testMethodName >() names the current test method.

Every test run creates 10000 instances of object that are referenced by the test case instance.

All test case instances live until suite dies because there are references into them and even into the created objects. That means that all class members live until the test suite dies. You should keep that in mind when writing tests and creating test suites.

2nd example

This is another example to make clear how htf.TestCase is instantiated.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
>>> import unittest
>>> from htf import TestLoader, TestRunner, TestCase

>>> class Test(TestCase):
...     def __init__(self, methodName='runTest'):
...         print("__init__(%i, %s)" % (id(self), methodName))
...         TestCase.__init__(self, methodName=methodName)
...
...     def test_foo(self):
...         print("foo")
...
...     def test_bar(self):
...         print("bar")
>>> suite = TestLoader().loadTestsFromTestCase(Test)
__init__(48735792, test_bar)
__init__(48734576, test_foo)

In line 14 a test suite is created. Test.__init__ is called twice here. One time for test_foo (line 15) and another time for test_bar (line 16).

Rules for writing good tests

To write good tests with good scalability you have to follow some simple rules:

  • Use stack variables not class variables whenever possible: Do not use self.variable = value but variable = member. These variables will not allocate any memory after execution.

  • Overwrite setUp() and tearDown(): If you need additional class members for your tests they should be created in setUp() method. Don’t forget to call TestCase.setUp(self) to call the super class’ method, too. The allocated memory should be deallocated explicitly in the tearDown() method. To do that use the builtin method del (line 11) e.g.. Don’t forget to call TestCase.tearDown(self) to call the super class’ method, too.

     1
     2
     3
     4
     5
     6
     7
     8
     9
    10
    11
    12
    13
    14
    import unittest
    from htf import TestRunner, TestCase
    
    class Test(TestCase):
        def setUp(self):
            self.variable = 1
            TestCase.setUp(self)
    
        def tearDown(self):
            del self.variable
            TestCase.tearDown(self)
    
        def test_foobar(self):
            print("foobar")