[CentOS-devel] New QA / Testing potential

Tue Jun 3 16:56:22 UTC 2008
Tim Verhoeven <tim.verhoeven.be at gmail.com>

On Wed, May 28, 2008 at 5:42 PM, Karanbir Singh <mail-lists at karan.org> wrote:
> Tim Verhoeven wrote:
> The proposal looks good, but apart from the 3 week cutoff ( which would lead
> to the death of the testing repo ), its about the same as what we have in
> place right now. And we all know that the process isnt working.
> I think what we need to do is setup some form of expectations and layout
> details on what functionality needs tested, how and where. And the
> responsibility for laying those down should be on the person requesting or
> pushing the packages. In the current scenario where we just ask people to go
> test something, they dont have any clear idea as to what they are looking
> for, and there is no way that would sync with the expectations of the
> packager / pusher / requestor.
> Perhaps what we need is a page on the wiki that gives the name and details
> of the package, who is responsible for it now, and how is going to maintain
> it going further, and a list of issues / tests that need to be done on those
> packages. People can then tick the box's indicating they have tested those
> bits, along with some feedback if they have any. Just having a list of
> things to look againt and a box to tick Pass / Fail would be good.

My feeling is we need 2 things here. A place for the packages to put
info about the packages, what they are for, version info, changelog,
... Everything that tester need to know what the package is for and
what needs to be tested. Also, this should be the place were a tester
leaves behind his feedback. Secondly a simple systems is needed to
inform testers of the existence of the package and where to find it
and also maybe some sort of remember when feedback isn't forthcoming.

I don't know what the wiki can or cannot do. But what about a special
wiki secion where builders can put the info about the packages (inside
a pre-created template) with a script that then can send a automated
mail to centos-devel to annouce a new version/build of a package so
people know that there is something to test. Maybe we can even link
the build system to create entries inside the wiki when stuff gets
pushed to the testing repo.

This should make it relatively simple for builders to document their
packages and testers are informed with a simple email with a link to
that inform them what is required to be tested and a place to leave
feedback behind.

>> I've also made a list in that proposal about what to do with the
>> current pacakges in the testing repo.
> I dont agree with the 'Action to take' for a large number of packages there.
> eg. why should cft and facter be deleted ? and smolt is a part of the
> cobbler + func + smolt admin stack, and should stay together.

I know, I just when over the list quickly. I don't know all the
software inside the testing repo. It was meant to help make a choice

> The biggest issue, as far as I can see it, is communitcation itself.

Agreed, the solution needs to be as simple as possible for both
builders and testers.

Tim Verhoeven - tim.verhoeven.be at gmail.com - 0479 / 88 11 83

Hoping the problem magically goes away by ignoring it is the
"microsoft approach to programming" and should never be allowed.
(Linus Torvalds)