[CentOS-devel] what / how could test reports be published

[Dag Wieers]

On Thu, 7 Apr 2011, Les Mikesell wrote:

> On 4/7/2011 7:39 AM, Karanbir Singh wrote:
>> On 04/07/2011 01:21 PM, Thomas Bendler wrote:
>>
>>> What do you expect, BFO adapted for CentOS in one day completely tested
>>> and ready to roll-out? Sorry but this need a bit more work and
>>> coordination (and I guess most people here have regular jobs which had
>>> in the end more priority).
>>
>> As do I, have a day job. And I'm also not looking for anything being
>> developed or implemented. At the moment, its just a case of asking
>> people for enough feedback so we can start with the design stages.
>> Implementing stuff comes a fair bit down the road.
>>
>>> I think (my personal opinion) that the first thing that should be done
>>> is a kind of an architecture overview to see what the goal of this
>>> project is. Based on this tasks need to be defined and names need to be
>>
>> Right, and therefore I am looking for people's feedback on what they
>> would expect to see in such a tool. Or are you suggesting that you are
>> not clear about why we are doing tests ?
>
> It's not clear what the test should show.  Also, I haven't seen anything
> that would help to prioritize such work.  That is, where is time
> currently being consumed, and how can the tests be designed to predict
> how to solve the problems they detect in a way that the results help
> move the project forward?

This gives also the impression that the reason releases are delayed for so 
long is because of the lack of QA, while it often takes weeks for anything 
to be submitted to QA. So even if the QA could be improved (which no doubt 
it can), it is reasonable to assume we could also make 'going to QA' 
faster by opening up the process, by feeding QA faster with new updates 
and by opening up the QA process so more people can actually help test the 
intermediate builds so we get more problems faster.

It seems to me a set of kickstart files is only going to achieve a small 
improvement to the whole process (mostly testing anaconda) and the causes 
for the delays during the 85 days CentOS 5.6 has been developed.

(The above is based on my experience with previous QA processes and what I 
have heard from people involved in the CentOS 5.6 QA process.)

It would be nice to have metrics that indicate when the first build was 
ready for QA, where things got blocked and how to unblock those in the 
future. But that requires a transparent/open process to even discuss this 
here.

-- 
-- dag wieers, dag at wieers.com, http://dag.wieers.com/
-- dagit linux solutions, info at dagit.net, http://dagit.net/

[Any errors in spelling, tact or fact are transmission errors]