On 13/04/11 10:07, David Hrbá?? wrote: > Dne 13.4.2011 8:39, Fabian Arrotin napsal(a): >> Hi, >> Thanks a lot for your submissions, i'll integrate them in the scripts >> repository. > > Fabian, > Would you mind opening the process slightly? I mean it'd be helpful > having the test list to avoid people doing the same thing. Also it would > really help to codify the return values, etc. Hi David, Sorry for the late answer but due to some illness on my side (and so being very tired when going back home from $work) i wasn't able to answer that question. Yes it's really a need to open that part of the tests so that people can contribute to those tests and also open the QA process (and also the status) to the community > >> One simple remark : we launch the tests on a minimal setup (from a >> minimal ks) so the first thing to do in the script itself is to install >> the required package, verify that it installs correctly and then test >> the feature that you want to test. > > This is something that should be written on Wiki page, so that people know: > - what are the current scripts > - expected return values > - testing environment and requirements > - etc. Yes, and so first a need to design it before documenting it. Let me clarify this a little bit : actually we (Karan and myself) designed quickly something that could be used directly during the 5.6 process (we did that during Fosdem, between discussions with people at the CentOS booth ...) We started 'thin' and so actually it's just a file with different functions declared and some bash scripts (one per VM/type) sourcing that first file and using the functions. Those 'functions' contain CentOS QA hardware informations (like on which machine to deploy a vm, which type of vm (xen, kvm, virtualbox, etc ...) disk-size, memory etc .. So we'd like first to separate maybe those informations from the tests themselves. The tests scripts would be available in a git repo and we'd use them during QA and post the results somewhere. I would like to redesign our QA subnet/vlan a little bit and reflect that in our existing QA script. (I don't think that this part interests the Community as it also involves public ip, etc ...) We'd try to have a small meeting with KB about that part and then a public discussion (an IRC meeting maybe) for people wanting to contribute their tests. Part of the discussion would be also about what to test. For example, testing if dovecot imap/pop3 is working is good, but my personal approach would be to test if centos packages for example are also 'centosified'. Let me explain that with a real example : if we have to change a logo (and that we use the same logo during the whole distro lifetime) a basic script would be to retrieve that logo, compare it with the known md5sum and so guess if the logo was replaced or not correctly. Stupid example, but believe me it's worth writing such stupid script :-) Fabian