[qmtest] The state of GUI

[Stefan Seefeld]

Vladimir Prus wrote:

> Fair enough. I'm too not entirely sure if it's best to improve the existing 
> GUI, use custom databases, or just my own GUI with PyQt. 

I find the GUI quite convenient to introspect a test database interactively,
and so I'm very interested into improving the existing one (in particular,
since it is much more portable than any GUI toolkit - based will ever be).

> I tend to like the GUI for running tests, at least when developing (nightly 
> build surely uses "run + summarize"). However, creation of tests is less 
> convenient. Say, I'd like to list a set of source files and automatically 
> compute expected results, and create a test. Whether it's better to use 
> custom database and a shell script, or standard XML database with some Python 
> script that uses QMTest API is the question, though.

Right. I'm not quite sure how important / popular this use case of modifying
a test database via the GUI really is.

> Did you look at boost results format?  See:
> 
>     http://www.meta-comm.com/engineering/boost-regression/cvs-head/developer/program_options.html

Thanks for the pointer. I have casually looked at the boost regression report,
and I definitely intend to have a close look to see what I could add to our
own reporting facility to support this.

> Essentially, there are several test results which are merged in a big table. 
> Each failure can have "notes" -- that's my proposed failure annotations. As 
> I've mentioned somewhere in the tracker, the key point is that there's one 
> expectation file for all toolsets.

Oh, I missed that. Since in QMTest expectations are simply results of prior
test executions, my initial design proposal is to provide the means to
annotate result files. (Nothing is stopping you from using the same result
file as expectation in multiple toolsets.)

Does this make sense ?

Regards,
		Stefan