Is it possible to make a sub-test "expected failure" in Python unittest?

Question

In Python's unit test framework it is possible to declare that a test is expected to fail using the expectedFailure decorator. I use 'expected failures' to denote bugs in the implementation that need to be fixed at some later time, but are not critical. However, the expcetedFailure decorator applies to the whole test. I was wondering if there is a way to declare that a sub-test is expected to fail?

For example, consider the following test for some is_prime routine that does not declare 2 as a prime number (perhaps because it first filters out all even numbers):

class NumbersTest(unittest.TestCase):
    def test_primes(self):
        for i in range(2, 13):
            with self.subTest(i=i):
                if i in [2, 3, 5, 7, 11]:
                    self.assertTrue(is_prime(i))
                else:
                    self.assertFalse(is_prime(i))

The test obviously fails for i=2 because of the bug in is_prime, but as I am sure that for the now I will never call is_prime on 2, I would not make a big fuss out of it for the moment. I would like to have a means to declare that only the sub test for i=2 is expected to fail. If I decorate the entire test_primes with @expectedFailure then the routine does not get tested at all, so if someone changes the implementation and breaks its functionality it will not be noticed.

One possiblity is to use skipTest instead:

class NumbersTest(unittest.TestCase):
    def test_primes(self):
        for i in range(2, 13):
            with self.subTest(i=i):
                if i == 2:
                    self.skipTest("`is_prime` does not handle 2 correctly")
                if i in [2, 3, 5, 7, 11]:
                    self.assertTrue(is_prime(i))
                else:
                    self.assertFalse(is_prime(i))

But I am not too happy with skipping the test, as a test should normally be skipped if it cannot be performed (e.g., because of unavailable resources, etc.). In this case nothing prohibits us from running the test for i==2, it only fails and the failure is a known bug.

Answer 1

I realize this is kind of old, but if you can refactor the test to a method that accepts inputs, you can achieve this with minimal duplication

class NumbersTest(unittest.TestCase):
    def test_primes(self):
        for i in range(2, 13):
            with self.subTest(i=i):
                if i in [2, 3, 5, 7, 11]:
                    self.assertTrue(is_prime(i))
                else:
                    self.assertFalse(is_prime(i))

becomes

class NumbersTest(unittest.TestCase):
    def _test_primes(self, i, expectation):
       with self.subTest(i=i, expectation=expectation):
             self.assertEqual(is_prime(i), expectation)

    @unittest.expectedFailure
    def test_failing_primes(self):
         values = [2]
         for i in values:
             self._test_primes(i, False)

    def test_primes(self):
         for i in range(3, 13):
            if i in [3, 5, 7, 11]:
                self._test_primes(i, True)
            else:
                self._test_primes(i, False)