[CentOS-devel] New QA / Testing potential

Thu May 29 00:15:15 UTC 2008
John Summerfield <debian at herakles.homelinux.org>

Karanbir Singh wrote:
> Tim Verhoeven wrote:
>> I actually promised to do this a while ago. So having time now here at
>> LinuxTag I did finaly write up what I had in mind. The proposal is
>> here "http://wiki.centos.org/QaWiki/TestingRepo". It is probably only
>> accessable for people in the QA team or people having admin rights.
>> But I think for the discussion about the proposal itself this should
>> be ok. If not let me know and I move it.
> The proposal looks good, but apart from the 3 week cutoff ( which would 
> lead to the death of the testing repo ), its about the same as what we 
> have in place right now. And we all know that the process isnt working.
> I think what we need to do is setup some form of expectations and layout 
> details on what functionality needs tested, how and where. And the 
> responsibility for laying those down should be on the person requesting 
> or pushing the packages. In the current scenario where we just ask 
> people to go test something, they dont have any clear idea as to what 
> they are looking for, and there is no way that would sync with the 
> expectations of the packager / pusher / requestor.
> Perhaps what we need is a page on the wiki that gives the name and 
> details of the package, who is responsible for it now, and how is going 
> to maintain it going further, and a list of issues / tests that need to 
> be done on those packages. People can then tick the box's indicating 
> they have tested those bits, along with some feedback if they have any. 
> Just having a list of things to look againt and a box to tick Pass / 
> Fail would be good.
>> I've also made a list in that proposal about what to do with the
>> current pacakges in the testing repo.
> I dont agree with the 'Action to take' for a large number of packages 
> there. eg. why should cft and facter be deleted ? and smolt is a part of 
> the cobbler + func + smolt admin stack, and should stay together.

If those packages are travelling together, then they should stay 
together. However, if new cft appears without its companions, and gets 
no testing, it's hard to justify its promotion.

And this is just testing, deleting it here doesn't mean it vanishes from 
the target repo.
> The biggest issue, as far as I can see it, is communitcation itself.

Always, it's communication.

I think email should _always_ go out, even if the builder doesn't intend 
it to be promoted. If there's a point to putting it into the repo, it's 
to share it, and if you want people to use it you have to tell them.

The email must contain a summary of what's changed, and a summary of 
what the package does. If I don't use it now, tell me why I might want to!

A lot of package descriptions are next to incomprehensible. Take this, 
from F9:
FAAD 2 is a LC, MAIN and LTP profile, MPEG2 and MPEG-4 AAC decoder, 
written from scratch.

This package contains libfaad.

Better, something like "FAAD 2" decodes various (generic description of 
just what MAIN, LTP etc are), including (name them) and is used for/by ....

If they're used by Amorok and I know I use that, then I know maybe I 
should be interested in this.

The email might link to a wiki, but don't expect users to use the wiki 
to decide whether it's something they should test.

I presume that these announcements would amount to quite a bit of email, 
maybe enough to seriously change the nature of this list should they 
come here.

You might consider a new list, maybe "updates-announcements" where these 
announcements go, and that responses be directed to this list. This list 
might help where/if there are different packagers on different 
platforms: Joe builds on IA32, Fred sees he needs to build on zSeries 
and Freda on Power.

Seed it with members of this list, it's little different from just 
sending mail here, and gives users of this list the ability to opt out. 
This would ensure there's a decent number of eyeballs seeing its 
contents from day one.

The list description for the new list would recommend also subscribing 
to this list, but I don't think everyone will want to.

The description of this list should be updated to recommend new 
subscribers also subscribe to updates-announcements so as to keep 
abreast of threatened changes. Again, not everyone will want to.

If there's a formal test procedure for a particular package, then this 
should be described in its wiki document. I imagine some packages - gcc, 
postgresql for example - might have established test cases to run, and 
others might be added as new bugs are reported and resolved.



-- spambait
1aaaaaaa at coco.merseine.nu  Z1aaaaaaa at coco.merseine.nu
-- Advice

You cannot reply off-list:-)