As talked earlier today,
Here are my comments - I understand they are mostly related to the environment it is running into, but it may be interesting to look into it to improve the cases.
Regards Thierry (nick - thf )
-------- Forwarded Message -------- Subject: sig-core-t_functional suite for 7.2.1511 Date: Tue, 22 Dec 2015 11:14:29 +0100 From: Thierry Fauck@linux.vnet.ibm.com thierry@linux.vnet.ibm.com To: centos-qa@centos.org
Hello,
While checking the suite for ppc64 and ppc64le, I got the following changes and comment - let me know if you want a diff file.
- runtest.sh If script fails and you relaunch it a second time, you may get a failure because tests users are already created - I suggested to remove all test users in runtests.sh to have a clean environment.
In t_CheckExitStatus shouldn't we add the test name being run to ease tracing and script debugging like ? [ $1 -eq 0 ] && { t_Log "$0 PASS"; return $PASS; }
- Depending on your setting, you may come with LC_ALL setting as utf8 or C and 10-cmp-tests may fails - checking for byte or char would allow both settings to be run. cmp -i 15:16 ${FILE}a ${FILE}b | egrep -q "byte 1, line 1|char 1, line 1"|| ExitFail
- tests/p_mtr/mtr_test.sh fails if you are not in the centos.org env as it tries to reach the qa host. As this test is done at start of runtest.sh, in case it is not true, shouldn't you use another hosts found on your local network (an easy bypass is to use like the local DNS server or the gateway ?).
- p_shadow-utils/20-chage_tests may fails because of capital letters jan versus Jan - I would suggest to use chage -l testshadow | grep Last | grep -qi "Jan 01, 2013"
- tests/p_shim/01_shim_secureboot_signed.sh tests/p_kernel/02_kernel_secureboot_signed.sh are x86_64 dependant
- Had to increase -T delay from 2 to 8 in tests/p_squid/squid_test.sh
- tests/p_systemd/25-journalctl.sh - for fast processors, you need to had a delay for journalctl to cope with your message - I added a sleep 1 after writting to /dev/kmsg
- tests/p_which/which_basic.sh - sounds like wich is now in /bin instead of /usr/bin - /usr/bin/which which | grep -q "/bin/which" would cope with both cases
Hope that helps