Test cases which require variation of the input

[Zack Weinberg]

I just investigated a bug in GCC's preprocessor which caused it to
crash on input of the form

#define S(x) #x

S(1234567890123456789012345678901234567890123456789012345678901234567890
  1234567890123456789012345678901234567890123456789012345678901234567890
  1234567890123456789012345678901234567890123456789012345678901234567890
  /* ... */
  123456789012345678901234567890123456789012345678)

S()
S()
/* ... 14 more S() here */

S(abcdefghijklmnopqrstuvwxyz)
S(abcdefghijklmnopqrstuvwxyz)

with a nasty catch: the required length of the filler text 12345...
varied with the version of the buggy compiler, the build parameters,
and the command line arguments.  This was because the bug was
triggered only by using S() with an empty argument when an internal
buffer was exactly full.

For this sort of bug, a static test case offers no assurance that the
bug has not reappeared.  All you know is that the bug has not
reappeared under the exact set of conditions described by the test
case.  It's relatively simple to instrument the preprocessor to find
out whether the problematic edge case is being hit, but you don't want
the instrumentation in the production executable.

So I'm wondering how QMTest can help with this.  What comes to mind is
a test class which takes a template and a set of parameters to vary,
and generates a whole series of test inputs; hopefully at least one of
them will hit the edge.  This is brute force.  Anyone have a better
idea?

zw