[qmtest] context passed among tests?

[Mark Mitchell]

--On Monday, December 30, 2002 07:43:22 PM -0500 eichin at metacarta.com wrote:

> Annotations are output-only, right?  Or is it reasonable to have one
> test look for annotations provided by another test?

Right.

> And the context can be input by the user, or set up by resources, but
> can't actually be modified by tests, right?

Right.

Those are both important design decisions.

The idea is that tests should not be communicating amongst themselves;
that means that running a single test is no longer a well-formed idea.

(Prerequisites are not supposed to be used for ordering so much as for
optimization/diagnosis; if simple test X fails it may not even make sense
to run big test Y.  Or, the opposite -- if big test X passes, you may want
to skip small diagnostic tests Y and Z.)

There are of course games you can play (use the filesystem, modify the
classes directly, global Python variables, etc.) that you can use to
do cross-test communication, but we're not going to make it easy. :-)

> Both are sort of close; does it make more sense to
>   * stretch resource by adding resource-dependencies

This one; that's a natural extension, and one we've planned for a while.

In the abstract, resources are just an optimization -- you do some set-up
once and share it for lots of tests.  A correctly implemented resource
could be run once for each test that needed it, or just once for all of
those tests; the only difference should be how long it takes the tests to
run.  So, having one resource depend on others is no problem, that's just
allowing the resources themselves to be optimized in a similar way.

I've started on the code for this, but we need to watch out for and
diagnose circular dependencies, and that will take a little thought to
get right.

-- 
Mark Mitchell                mark at codesourcery.com
CodeSourcery, LLC            http://www.codesourcery.com