[qmtest] Handling large numbers of very similar tests

[Stefan Seefeld]

Neale Morison wrote:
> Hi all.
> We are using QMTest to run a large number (120K) of very similar tests.
> We are trying to find a way to generate the specification for these
> tests by some sort of batch or scripting, rather than creating 120K
> individual specifications. We're having trouble interpreting the manual
> and finding out whether this is possible.
> 
> Can anyone point us in the right direction, or confirm that it is not
> possible?

Handling very large numbers of tests is definitely possible, though, as you
point out, using the default test database doesn't scale well up to such
numbers.
Instead, you may consider extending QMTest by providing your own test
database implementation (see
http://www.codesourcery.com/public/qmtest/qm-snapshot/qm/test/doc/html/sec-ref-writing-database-classes.html)

I'm presently in the process of enhancing the QMTest Tutorial to better
document how to do this. One new database type I just added, and which
I will use to illustrate the process of customization and extension,
simply scans a directory for files with particular extensions (such as '.c')
and associates particular test types with them ('CompilationTest', say).

Your database implementation needs to be hooked up to whatever representation
you currently have for your tests, so you don't need one '.qmt' file per test
instance as the default database requires.

Don't hesitate to ask if you need further assistance !

Regards,
		Stefan

-- 
Stefan Seefeld
CodeSourcery
stefan at codesourcery.com
(650) 331-3385 x718