On 6/1/2011 2:49 PM, Ned Slider wrote:
can go wrong. My opinion is that it would be best to expose the initial release as 'test' quality and let a large number of people try it in a large number of environments - knowing that they should treat it as a test.
In which case, how would one estimate 'enough' people have used it and 'enough' people have said ok ? Or, in other words 'enough' people have not reported anything breaking for them.
Off the top of my head I'd say a few dozen people explicitly reporting a tested-good status or a few thousand downloads and a few days with no negative reports.
Les,
From where I'm standing, what you describe is pretty much exactly what we currently have in place with the current QA system. The whole point of a CR repository is to get the updates out quickly rather than wait for ISO to be built and pushed to the mirror network. If you add an additional layer of testing that's not going to happen.
Not quite. You need to balance the risk of not making the updates available as the default 'yum update' action (the bazillion internet connected Centos boxes aren't all going to enable some new funky repository just to get an update quicker) against the risk of pushing a broken build. My opinion leans toward pushing security fixes as fast as possible to everyone with default settings (which, I suppose, could involve pushing a new centos-release that activates a new repo but then it takes 2 updates to get them). But that doesn't mean it should bypass testing entirely or that it should wait for anaconda or iso completion.
Unless I misunderstand, KB is talking about releasing packages to CR as soon as they are ready (meaning as soon as they have passed all internal tests + QA) rather than delaying their release whilst the ISOs are subsequently built and everything is synced up to the mirrors. The packages released to CR would be the exact same set used to build the ISOs and eventually released to the mirrors, just you'd get access to them maybe a week earlier.
If more testing is needed then we need to add that to the existing QA phase, either by adding the required tests and/or by recruiting more testers. Anyone can contribute to that process by writing automated test suites.
There are probably some new possibilities for problems with an out-of-order delivery or some new build issues that don't show up until the whole system is completed. It might be a good test to populate the updates into a stock upstream system to match what you expect CentOS to duplicate when the point release is completed - but you'd need a licensed copy to do that.