[qmtest] getting started

[John Schmitt]

Apologies for being so ambiguous...

> -----Original Message-----
> From: Mark Mitchell [mailto:mark at codesourcery.com]
> Sent: Wednesday, February 19, 2003 1:45 PM
> To: John Schmitt
> Cc: qmtest at codesourcery.com
> Subject: RE: [qmtest] getting started


> [random stuff]

> I'm not sure if I understand this question, but I think the answer is
> "no".
> There's no integration between QMTest and source management systems;
how
> the tests get back into an official repository is beyond the scope of
> QMTest.

That's exactly what I was asking about.  I realise that this isn't
QMTest's responsibility, but if you have any best practice guidelines,
I'd love to hear them.

> > - automatically create bugzilla tickets for
> >   new/unexpected failures?
> 
> No -- although adding automatic support for talking to defect tracking
> systems is definitely something in which we are interested.

In the case of bugzilla, this might be a place to start:
http://quigley.durrow.com/bugzilla.html

> 
> > - when reporting a failure, how can I specify which
> >   tag should be used to check out the source when
> >   attempting to reproduce the problem
> 
> A failure in QMTest, or a failure in the tested sotware?

A failure in tested software.  I guess the test suite needs to set some
variable (the value of the tag) when the prerequisite is run (which
checks out the software about to be tested)  and then reference that
variable when the report is generated.  Advice on other approaches
welcome.

> 
> > - how is the test database propogated to the test machines?
> >   -- share them over nfs?
> 
> Correct -- or any other method you like.  The key constraint is that
the
> test database has to be accessible in the same way on all of the
machines
> being used to run tests.

My thought was that one of the first prerequisites must ensure that the
test database gets checked-out/copied/propogated to a machine that is
about to be tested.  Am I describing a chicken-and-egg thing here?

> 
> > - are there any database classes that stores tests as
> >   plain text files?
> 
> Our GCC test database handles GCC tests that are stored simply as
source
> code, which might qualify.

I browsed the source a little and I think
/home/qmtest/lib/qm/qm/test/file_database.py is what I was thinking
about.  I fear that I don't quite understand the cost/benefit of
plaintext/xml tradeoff here.  Am I missing some obvious benefit?

Thanks for the feedback.

John