Unit tests and assertions (was: [Exceptional C++ Style] Item 18 - Virtuality)

[Ric Parkin]

On Sun, 9 Jan 2005 17:23:40 +0000, Kevlin Henney <kevlin at curbralan.com>  
wrote:

> In message <018a01c4f64b$1e6fccc0$0300000a at pc>, Terje Slettebø  
> <tslettebo at broadpark.no> writes
>>> From: "Kevlin Henney" <kevlin at curbralan.com>
>>
>>> In addition, introducing such a DbC framework is generally a waste of
>>> effort. Although it is based on good intentions, it is mostly ritual
>>> rather than time well invested. A set of unit tests will be a much
>>> clearer guide to the contract and the interface. Wading through an NVI
>>> version of a contract is more than a little tedious, especially when  
>>> the
>>> notion of contract enforcement is based on assertions: all that
>>> infrastructure for something that isn't even compiled in.
>>
>> Unless I misunderstand you, aren't unit tests and asserts rather
>> complementary to each other? The asserts checks
>> preconditions/postconditions/invariants, while you may use unit tests to
>> check that they indeed work, i.e. exercise the asserts.
>
> When a contract is phrased so that it does not describe what happens  
> when the normal terms of a contract are violated, the assumed behaviour  
> is undefined, ie the program may terminate, an exception may be thrown,  
> the called function need never terminate, everything may continue as  
> normal, etc. A contract phrased like this could be checked (or ignored)  
> quite legitimately through 'assert' (as in the <cassert> macro).  
> However, there is nothing for a test case to assert in such a case.
>
> Where a contract also publishes the specific quality of failure for  
> transgression of some or all of its normal parts, because such  
> behaviours are published in terms of definite measurable outcomes, they  
> may be tested for.

I don't think it's quite so one-or-the-other - my DBC and Test libraries  
integerate to the extent that the DBC checks will throw exceptions when  
being tested, and the tests check that said exceptions are in fact thrown.

So even though a precondition failure results in UB, the tests can  
actually tie this down and check for it.

The advantage of this is that changing preconditions have the usual safety  
net of the test, even though in release builds the checks aren't there.

> What example-based unit test cases allow you to do is present and check  
> the items specified in the contract associated with a measurable  
> outcome. In other words, specific normal outcomes in valid situations  
> and specified bad-news in unpleasant circumstances.



> Preconditions fall into two broad groups:
>
> (1) Preconditions with no associated penalty terms for violation, ie  
> undefined behaviour. These cannot be tested for in unit test cases.They   
> test the correctness of the caller rather than the called code, so there  
> is nothing to test at this level.
> (2) Preconditions associated with some form of well-defined penalty, eg  
> an exception. With the exception of a contract that leads to process  
> death, these can be tested for using unit tests.

My point is that these two cases are not always seperated - the beauty of  
UB is that you can define it if you want to your own ends.


> So, yes, unit testing is often complementary to assertions, but  
> sometimes they impinge on the same concerns.

I agree - they seem to look at testing correctness from two different  
angles.

Ric