[Gluster-devel] Improve EXPECT/EXPECT_WITHIN result check in tests

[Niels de Vos]

On Mon, May 02, 2016 at 01:49:08PM +0200, Xavier Hernandez wrote:
> Hi,
> 
> I've found an spurious failure caused by an incorrect check of the expected
> value in EXPECT_WITHIN.
> 
> The problem is that the value passed to EXPECT_WITHIN (EXPECT also has the
> same problem) is considered a regular expression but most tests do not pass
> a full/valid regular expression.
> 
> For example, most tests expect a '0' result and they pass "0" as an argument
> to EXPECT/EXPECT_WITHIN. This will match with "0", that's ok, but it will
> also match with 10, 20, 102, ... and that's bad.
> 
> There are also some tests that do use a regular expression, like "^0$" to
> correctly match only "0", however current implementation of EXPECT_WITHIN
> uses the following check:
> 
>     if [[ "$a" =~ "$e" ]]; then
> 
> Where "$e" is the regular expression. However putting $e between quotation
> marks (") makes special regular expression characters to not be considered.
> This means that "^0$" will be searched literally and it won't never match.
> When the timeout expires, test_expect_footer is called, which does the same
> check but without using quotation marks. At this time the check succeeds,
> but we have waited an unnecessary amount of time.
> 
> This is not the first time that I've found something similar.
> 
> Would it be ok to change all regular expression checks to something like
> this in include.rc ?
> 
>     if [[ "$a" =~ ^$e$ ]]; then
> 
> This will allow using regular expressions in EXPECT and EXPECT_WITHIN, but
> will enforce full answer match in all cases, avoiding some possible side
> effects.
> 
> This needs some changes in many tests, but I think it's worth doing.
> 
> What do you think ?

Sounds like a good approach to me. New test-cases will fail if
developers dont use it correctly, so they'll fix it automatically. How
much work is it to fix existing tests?

Thanks,
Niels
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://www.gluster.org/pipermail/gluster-devel/attachments/20160502/0a84c308/attachment.sig>