[qmtest] Wrong Exit Codes

[Kelly Lucas]

Well, here's an exmple of a test that fails on Breezy, kernel 2.6.16.1, 
and the same exact test that passes on Redhat [kernel 2.4x] and Hoary, 
kernel 2.6.13.4:

My test is a shell script, and the shell script is simply:

touch <pathname>

The pathname is a place where the user running the test does not have 
permissions. On Redhat and Hoary, I get the correct exit Code from 
QMTest, which is 1. I'm using QMTest's built in checks for exit codes. I 
do get the expected standard error in all cases, which is:

touch: cannot touch <pathname>:  Permission denied

The system where the wrong exit code is returned is running Python 
2.4.2. The other two systems where the exit code is correct, are 
running: Python 2.4.1 and Python 2.2.2.

kdl

Kelly D. Lucas
lucaskeli at fastmail.fm



Stefan Seefeld wrote:
> Kelly Lucas wrote:
>> I recently moved my test suite for another platform, which is code 
>> named Breezy, the Ubuntu Linux release.
>>
>> I've previously been running my testsuite using QMTest on RedHat and 
>> Ubuntu Hoary. However, I was amazed to see that when I ran some 
>> negative tests, such as attempting to create a file or directory 
>> where I don't have permissions. On Redhat and Ubuntu Hoary [kernel 
>> 2.4x], I would always get non zero exit codes on such tests. However, 
>> now I'm getting exit codes of 0. I don't understand what happened. 
>> Code the version of python I'm using affect this? Here's an example 
>> of a test:
>>
>> A simple script that tries to touch a file in a directory where it 
>> does not have write permissions:
>>
>> touch /home/user/testfile
>>
>> If I try the same test from a shell script, and echo the exit code, 
>> it is non zero.
>>
>> I'm kind of baffled!
>
> I have seen situations in which a non-zero exit code of a command was not
> correctly reported back, though I don't remember whether that was a bug
> in the script or a problem with a particular shell.
> However, I fear I don't have enough details to be able to make meaningful
> suggestions. Are you sure you use the same version of QMTest in both 
> cases ?
> What about the test class ? Is it any of QMTest's builtin classes ? If 
> not,
> could you post some code ? (Especially the one that evaluates the exit 
> code.)
> What does your command look like ?
>
>
> Regards,
>         Stefan