[CentOS-devel] testing / qa process

--- Johnny Hughes <mailing-lists at hughesjr.com> wrote:

> On Fri, 2006-10-13 at 10:01 -0400,
> danieldk at pobox.com wrote:
> > Hi Karanbir,
> > 
> > > The CentOS Project provides packages in addition
> to the packages from
> > > RH's sources. We need to ensure that these
> packages are tested in a user
> > > environment for a while, and only released to
> the public via the
> > > mirror.centos.org network once there is some
> reasonable surety as to the
> > > functional and bugs situation with the package.
> > 
> > I have a question if I may: what should the
> packages be tested for?
> > 
> > 1. Readiness of the packaged software for
> enterprise environments.
> > 2. Quality of the packaging.
> > 
> > If both, what has most weight? If [1] is
> important, I suppose the quality
> > can partly be tested with regression tests (if the
> software includes
> > regression tests, IMO all enterprise-class
> software should). [2]
> > specifically requires people with enough knowledge
> of packaging to go over
> > it.
> > 
> 
> Daniel,
> 
> The stuff in the testing repo has already been
> initially tested (and
> packaged) by one of the CentOS Developers.  It
> doesn't get in there if
> it is not at least WorksForMe quality for someone
> who should understand
> enterprise ready.
> 
> Now I not suggesting that it is perfect, thus the
> need for testing.
> 
> What we need is people to use the products, make
> sure they work as
> expected, and tell us it does or does not work for
> them.
point is that peoples usually do not provide feedback,
you already know that,  they give feedback  if:

1- they think the developer think the products are ok
and ready (in your case the package is officialy out)
2- are too excited about the product because it solve
a great problem
3- they are officialy beta tester (or feel like that)
so they feel an obligation to give feedback

I can think another reasons ...

> I would settle for  ...  I downloaded it and it
> works for me or doesn't
> work for me because of this.  If we can get at least
> that much
> participation, we can fix it and get it back out, if
> required.

I guess a balance approach is to move the packages
from testing when:
1- some amount of peoples download the packages from
testing _and_
2- nobody complain about the packages in testing

and then be prepared for more feedback :-)
I know this is unwanted but I can't think a way to
avoid it because:
1- not all core developers needs or use the packages
so  not all can fully test the package (unlike what
happen when there is only one package or product:
linux kernel, apache or CentOS distribution as a
whole).
2- users usually do not look into the testing repo,
just developers (not only core developers) know where
to look, and again not everybody are interested or has
the time to test it.

so, finally, my adivice is: release when you, core
developer, think is ready and be prepared to
re-release after users, and not only developers, give
feedback if they do :-)

cu
roger

__________________________________________
RedHat Certified Engineer ( RHCE )
Cisco Certified Network Associate ( CCNA )

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com