On 6/1/2011 2:49 PM, Ned Slider wrote:
>
>>>> can go wrong. My opinion is that it would be best to expose the initial
>>>> release as 'test' quality and let a large number of people try it in a
>>>> large number of environments - knowing that they should treat it as a test.
>>>
>>> In which case, how would one estimate 'enough' people have used it and
>>> 'enough' people have said ok ? Or, in other words 'enough' people have
>>> not reported anything breaking for them.
>>
>> Off the top of my head I'd say a few dozen people explicitly reporting a
>> tested-good status or a few thousand downloads and a few days with no negative
>> reports.
>
> Les,
>
> From where I'm standing, what you describe is pretty much exactly what
> we currently have in place with the current QA system. The whole point
> of a CR repository is to get the updates out quickly rather than wait
> for ISO to be built and pushed to the mirror network. If you add an
> additional layer of testing that's not going to happen.
Not quite. You need to balance the risk of not making the updates
available as the default 'yum update' action (the bazillion internet
connected Centos boxes aren't all going to enable some new funky
repository just to get an update quicker) against the risk of pushing a
broken build. My opinion leans toward pushing security fixes as fast as
possible to everyone with default settings (which, I suppose, could
involve pushing a new centos-release that activates a new repo but then
it takes 2 updates to get them). But that doesn't mean it should bypass
testing entirely or that it should wait for anaconda or iso completion.
> Unless I misunderstand, KB is talking about releasing packages to CR as
> soon as they are ready (meaning as soon as they have passed all internal
> tests + QA) rather than delaying their release whilst the ISOs are
> subsequently built and everything is synced up to the mirrors. The
> packages released to CR would be the exact same set used to build the
> ISOs and eventually released to the mirrors, just you'd get access to
> them maybe a week earlier.
>
> If more testing is needed then we need to add that to the existing QA
> phase, either by adding the required tests and/or by recruiting more
> testers. Anyone can contribute to that process by writing automated test
> suites.
There are probably some new possibilities for problems with an
out-of-order delivery or some new build issues that don't show up until
the whole system is completed. It might be a good test to populate the
updates into a stock upstream system to match what you expect CentOS to
duplicate when the point release is completed - but you'd need a
licensed copy to do that.
--
Les Mikesell
lesmikesell at gmail.com