[qmtest] questions to QMTest

[Stefan Seefeld]

ktemp at gmx.ch wrote:
> Dear Sirs,
> 
> I am evaluating QMTest and reached a point where I am not able to proceed. I would appreciate any help.
> 
> 
> I have installed it under Windows XP, Python 2.3.
> I will use it to start NUnit and CPP unit tests. The standard output of NUnit includes the time the test need and differs from run to run. That's why I can not create a testcase against this output. I tryed to create an extension class to parse the output but at this point I have some problems. 
> I copied the the class C:\Python23\Lib\site-packages\qm\test\classes\command.py to a new file, edit it and registered it with the command 
> QMTest.py register ... but when used this class I get an exception.
> Because I didn't know what's wrong I just copied the command.py to mycommand.py without any changes and registered it with the command:
> 
> C:\Python23\share\qm\tutorial\test\tdb2>qmtest.py register test mycommand.ExecTest
> --> Result:
> QMTest will load 'ExecTest' from '
> C:\Python23\lib\site-packages\qm\test\classes\mycommand.py'.
> 
> seems ok..
> 
> But when using this class I get the same exception as before (see below) and at this point I need help. 
> 
> 
> circle.nunit2Outcome	Cause
> ERROR 	An exception occurred. 
> Annotation	Value
> qmtest.end_time	2006-07-14T05:52:59Z 
> qmtest.exception	pywintypes.error: (123, 'CreateProcess', 'Die Syntax f\xfcr den Dateinamen, Verzeichnisnamen oder die Datentr\xe4gerbezeichnung ist falsch.')

This seems to indicate that QMTest wasn't able to run the program you passed to the test.
I assume at this point you are experimenting using the default test database as per the
tutorial, in which case your test instance will be stored in a file with the '.qmt' extension.
This file contains the parameters with which the ExecTest test class will be instantiated,
notably the 'program' argument. Does the value of this parameter name an executable file
that can be found and run by QMTest ? The error you get suggests it can not.

[...]

> General:
> 
> My scope is to create a nightly build and test sytem with QMTest. My idea is to create two extendend classes basically.
> The first is to check out the sources from the source safe reprository and build the project if the sources has changed or if binaries are out of date. I will validate against the compiler output.
> The second has to execute the unit tests. This class shall store the e-mail address of the developper and if the test failes it sends an e-mail to him.
> 
> Concerning this conecpt I have several questions to some issues.
> - Is this the right way to use QMTest or are there better solutions?

I have been using QMTest to run test suites nightly (or triggered by some other frequent events).
However, while you certainly can write some extension classes to check out source code from a
repository and compile it before running the actual tests, I don't think that QMTest is the best
tool to do that. In similar contexts I have been using dedicated tools such as buildbot
(http://buildbot.sf.net) to drive the whole process. Running QMTest then becomes one step in
that workflow (or two if you also use it to generate a test report at the end).

> - Concerning the class which compiles the project. I have found a class "compiler.py" but didn't found a way to select it in the gui. How can I use it ?

The compiler.Compiler class is used in a number of QMTest test classes, notably the
compilation_test module. You can use the compilation_test.CompilationTest to compile and
run test applets. However, compiling libraries and applications from multiple source files
is typically more involved and requires a more fine-grained control. Does the code you
want to check out and compile not have its own build system ? If so, why can't you simply
call into that (e.g. running 'make' with appropriate parameters) ?
That would also take care of the dependencies, i.e. build incrementally.

> - I can imagine that these issues already have been solved by another developper. Are there some extension classes to download ? 

All the essential extension classes are part of the QMTest package, though we are trying to
add new ones if there is demand for common functionality that isn't supported yet.

> - I searched for a public Forum for QMTest but I didn't found one. Does no forum exist ?

This mailing list is the right place to ask for help.

Regards,
		Stefan

-- 
Stefan Seefeld
CodeSourcery
stefan at codesourcery.com
(650) 331-3385 x718