[qmtest] using an expectation database

[Mark Mitchell]

On 3/12/2011 1:08 PM, Andreas Saebjoernsen wrote:

>  1) Is the feature for having expected test-statuses on a per-target
> (e.g red hat linux, windows, etc) basis implemented?

I'm not sure.  There certainly is the ability to have a separate
expected results file (or, more generally, database) for different
platforms so that you could have an expected results file for each
platform.  But, I don't recall that we ever implemented directly support
for multiple platforms within a single expected results file.

>  2) Can you have one test results qmr file for a large project?
>         * we want to support directories running their tests run in parallel
>         * multiple qmtest instances will then write to qmr at the same time
>         * currently we have one qmr for each directory that contains tests

Yes, you can do this.

>  3) Is the feature for having an expectation SQL database implemented?

No.

>  4) What is the recommended way of updating the expectations in an
> expectation database after running a test?

It really depends on your usage model.  You probably don't want to
automatically update the expectation database after each individual test
run; that might reflect local changes of some sort.  We've generated
updated the expectation database when making a check-in to the trunk
sourcebase, though.

You can keep expectation databases around for previous releases; that's
very helpful in being able to look for regressions/progressions relative
to previous versions of the software.

Thank you,

-- 
Mark Mitchell
Mentor Graphics
mark_mitchell at mentor.com
+1 (510) 354-7337