[Exceptional C++ Style] Item 12: Exception Safety: Is it Wort hit?

[Kevlin Henney]

In message 
<20041203093046.JWYX27146.aamta01-winn.mailhost.ntl.com@[62.253.162.63]>,
  Ric Parkin <ric.parkin at ntlworld.com> writes
>
>That reminds me of a really tricky problem with unit testing under C++....
>
>...crashes.
>
>How do people's test harness cope with the code you're testing having a 
>bug that provokes an access violation/core dump?

The first thing is that the test harness should indicate progress, so on 
catastrophic failure, regardless of anything else that may happen, you 
know in which test case the failure occurred.

[As an aside, this encourages fine-grained test cases over longer ones.]

To get smarter than that you have to get platform specific to provide 
more context, eg installing a signal handler or using Windows SEH. Of 
course, the best you can get from this is more detailed context: the 
test run is over and there is no continuation.

>The partial solution I found was on windows to get the test harness to 
>set the structured exception handler so that OS exceptions were turned 
>into test failures, and everything could carry on (apart from the 
>possibility that the bug had just written over something important)

This is why I would terminate there and then. Test cases are isolated 
from one another in terms of effects, but following something that 
leaves the program state -- as opposed to just the test state -- in an 
ill-defined state means that you have less certainty about the 
reliability of the follow-on cases.

>I think boost.test does something rather clever too - runs the test 
>under a seperate thread (process?) so if that test 
>crashes/deadlocks/goes into an infinite loop then it can kill it and 
>carry on.

Running as a separate process would allow isolation, as a separate 
thread it would not, and so although kind of clever in a techie sense, 
it wouldn't seem that smart on reflection.

Kevlin
-- 
____________________________________________________________

   Kevlin Henney                   phone:  +44 117 942 2990
   mailto:kevlin at curbralan.com     mobile: +44 7801 073 508
   http://www.curbralan.com        fax:    +44 870 052 2289
   Curbralan: Consultancy + Training + Development + Review
____________________________________________________________