[qmtest] The state of GUI

[Stefan Seefeld]

Vladimir Prus wrote:

>>I'm not sure I understand your concern here, as the first thing I do when
>>writing a new test is somehow expressing what result I expect. It's just
>>that QMTest until now requires you to run the test at least once to do
>>that.
> 
> 
> I'm just arguing with the statement that one can just use results of a prior 
> test run as expected result, without any manual editing. Imagine that I have 
> just one configuration and an expected results file -- that contains manually 
> changed expectation. If I add new test, rerun tests, and replace the 
> expectations file, I loose the manually set expectations already stored 
> there.

Ok, I see your point now.

>>At what granularity do you want to indicate the brokenness ? If it is not
>>per test, how else ? 
> 
> 
> test/compiler_vendor combination. Say, "this tests fails on all versions of 
> yfc", as opposed to explicitly listing 15 versions of "yfc" where the test 
> fails.

I don't think it's QMTest's job to keep track of a toolchain taxonomy.

> Yea, but if people decide that editing 15 files for 15 versions of "yfc" is 
> not too nice (even with GUI), they should be able to do something. Like using 
> custom file format.

I don't think a custom file format is a good idea, as I don't believe QMTest
itself should care about these things. However, once we have an API to manipulate
expectation files it should be easy for you to define some aliasing mechanism
that lets you make modifications that get propagated to all instances that are
in your alias list. But again, that would be on top of such a 'QMTest API'.

Regards,
		Stefan