Since there are quite a few people offering to help test apps /packages in CentOS - I am wondering if now is a good time to organise the CentOS-Testing repository and process a bit more. There are a *lot* of things that need looking at and pushing into the production repo's however a lack of testing feedback means they dont.
Does anyone want to take this up ? I can hammer together some ideas into a wiki page, but dont have the time to really follow the process up or to organise it on a ongoing basis.
Hello Karan, I am interested in testing. Just let me know the basic procedures/considerations and etc. I am sure i can help.
On Wed, May 28, 2008 at 2:25 PM, Karanbir Singh mail-lists@karan.org wrote:
Since there are quite a few people offering to help test apps /packages in CentOS - I am wondering if now is a good time to organise the CentOS-Testing repository and process a bit more. There are a *lot* of things that need looking at and pushing into the production repo's however a lack of testing feedback means they dont.
Does anyone want to take this up ? I can hammer together some ideas into a wiki page, but dont have the time to really follow the process up or to organise it on a ongoing basis.
-- Karanbir Singh : http://www.karan.org/ : 2522219@icq _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org http://lists.centos.org/mailman/listinfo/centos-devel
I also want to contribute something, since I am using CentOS from past 2 year.
On Wed, May 28, 2008 at 3:09 AM, KarthiKeyan karthixinbox@gmail.com wrote:
Hello Karan, I am interested in testing. Just let me know the basic procedures/considerations and etc. I am sure i can help.
On Wed, May 28, 2008 at 2:25 PM, Karanbir Singh mail-lists@karan.org wrote:
Since there are quite a few people offering to help test apps /packages in CentOS - I am wondering if now is a good time to organise the CentOS-Testing repository and process a bit more. There are a *lot* of things that need looking at and pushing into the production repo's however a lack of testing feedback means they dont.
Does anyone want to take this up ? I can hammer together some ideas into a wiki page, but dont have the time to really follow the process up or to organise it on a ongoing basis.
-- Karanbir Singh : http://www.karan.org/ : 2522219@icq _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org http://lists.centos.org/mailman/listinfo/centos-devel
-- Regards, Karthi Keyan _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org http://lists.centos.org/mailman/listinfo/centos-devel
On Wed, May 28, 2008 at 10:55 AM, Karanbir Singh mail-lists@karan.org wrote:
Since there are quite a few people offering to help test apps /packages in CentOS - I am wondering if now is a good time to organise the CentOS-Testing repository and process a bit more. There are a *lot* of things that need looking at and pushing into the production repo's however a lack of testing feedback means they dont.
Hi,
I actually promised to do this a while ago. So having time now here at LinuxTag I did finaly write up what I had in mind. The proposal is here "http://wiki.centos.org/QaWiki/TestingRepo". It is probably only accessable for people in the QA team or people having admin rights. But I think for the discussion about the proposal itself this should be ok. If not let me know and I move it.
I've also made a list in that proposal about what to do with the current pacakges in the testing repo.
Regards, Tim
Tim Verhoeven wrote:
I actually promised to do this a while ago. So having time now here at LinuxTag I did finaly write up what I had in mind. The proposal is here "http://wiki.centos.org/QaWiki/TestingRepo". It is probably only accessable for people in the QA team or people having admin rights. But I think for the discussion about the proposal itself this should be ok. If not let me know and I move it.
The proposal looks good, but apart from the 3 week cutoff ( which would lead to the death of the testing repo ), its about the same as what we have in place right now. And we all know that the process isnt working.
I think what we need to do is setup some form of expectations and layout details on what functionality needs tested, how and where. And the responsibility for laying those down should be on the person requesting or pushing the packages. In the current scenario where we just ask people to go test something, they dont have any clear idea as to what they are looking for, and there is no way that would sync with the expectations of the packager / pusher / requestor.
Perhaps what we need is a page on the wiki that gives the name and details of the package, who is responsible for it now, and how is going to maintain it going further, and a list of issues / tests that need to be done on those packages. People can then tick the box's indicating they have tested those bits, along with some feedback if they have any. Just having a list of things to look againt and a box to tick Pass / Fail would be good.
I've also made a list in that proposal about what to do with the current pacakges in the testing repo.
I dont agree with the 'Action to take' for a large number of packages there. eg. why should cft and facter be deleted ? and smolt is a part of the cobbler + func + smolt admin stack, and should stay together.
The biggest issue, as far as I can see it, is communitcation itself.
- KB
Karanbir Singh wrote:
Tim Verhoeven wrote:
I actually promised to do this a while ago. So having time now here at LinuxTag I did finaly write up what I had in mind. The proposal is here "http://wiki.centos.org/QaWiki/TestingRepo". It is probably only accessable for people in the QA team or people having admin rights. But I think for the discussion about the proposal itself this should be ok. If not let me know and I move it.
The proposal looks good, but apart from the 3 week cutoff ( which would lead to the death of the testing repo ), its about the same as what we have in place right now. And we all know that the process isnt working.
I think what we need to do is setup some form of expectations and layout details on what functionality needs tested, how and where. And the responsibility for laying those down should be on the person requesting or pushing the packages. In the current scenario where we just ask people to go test something, they dont have any clear idea as to what they are looking for, and there is no way that would sync with the expectations of the packager / pusher / requestor.
Perhaps what we need is a page on the wiki that gives the name and details of the package, who is responsible for it now, and how is going to maintain it going further, and a list of issues / tests that need to be done on those packages. People can then tick the box's indicating they have tested those bits, along with some feedback if they have any. Just having a list of things to look againt and a box to tick Pass / Fail would be good.
I've also made a list in that proposal about what to do with the current pacakges in the testing repo.
I dont agree with the 'Action to take' for a large number of packages there. eg. why should cft and facter be deleted ? and smolt is a part of the cobbler + func + smolt admin stack, and should stay together.
If those packages are travelling together, then they should stay together. However, if new cft appears without its companions, and gets no testing, it's hard to justify its promotion.
And this is just testing, deleting it here doesn't mean it vanishes from the target repo.
The biggest issue, as far as I can see it, is communitcation itself.
Always, it's communication.
I think email should _always_ go out, even if the builder doesn't intend it to be promoted. If there's a point to putting it into the repo, it's to share it, and if you want people to use it you have to tell them.
The email must contain a summary of what's changed, and a summary of what the package does. If I don't use it now, tell me why I might want to!
A lot of package descriptions are next to incomprehensible. Take this, from F9: FAAD 2 is a LC, MAIN and LTP profile, MPEG2 and MPEG-4 AAC decoder, completely written from scratch.
This package contains libfaad.
Better, something like "FAAD 2" decodes various (generic description of just what MAIN, LTP etc are), including (name them) and is used for/by ....
If they're used by Amorok and I know I use that, then I know maybe I should be interested in this.
The email might link to a wiki, but don't expect users to use the wiki to decide whether it's something they should test.
I presume that these announcements would amount to quite a bit of email, maybe enough to seriously change the nature of this list should they come here.
You might consider a new list, maybe "updates-announcements" where these announcements go, and that responses be directed to this list. This list might help where/if there are different packagers on different platforms: Joe builds on IA32, Fred sees he needs to build on zSeries and Freda on Power.
Seed it with members of this list, it's little different from just sending mail here, and gives users of this list the ability to opt out. This would ensure there's a decent number of eyeballs seeing its contents from day one.
The list description for the new list would recommend also subscribing to this list, but I don't think everyone will want to.
The description of this list should be updated to recommend new subscribers also subscribe to updates-announcements so as to keep abreast of threatened changes. Again, not everyone will want to.
If there's a formal test procedure for a particular package, then this should be described in its wiki document. I imagine some packages - gcc, postgresql for example - might have established test cases to run, and others might be added as new bugs are reported and resolved.
On Thu, May 29, 2008 at 2:15 AM, John Summerfield debian@herakles.homelinux.org wrote:
I dont agree with the 'Action to take' for a large number of packages there. eg. why should cft and facter be deleted ? and smolt is a part of the cobbler + func + smolt admin stack, and should stay together.
If those packages are travelling together, then they should stay together. However, if new cft appears without its companions, and gets no testing, it's hard to justify its promotion.
Well, that is for the builder to indicate. If he/she does not inform the rest about that we don't know that. So that is back to the communication part :-)
And this is just testing, deleting it here doesn't mean it vanishes from the target repo.
Correct.
The biggest issue, as far as I can see it, is communitcation itself.
Always, it's communication.
I think email should _always_ go out, even if the builder doesn't intend it to be promoted. If there's a point to putting it into the repo, it's to share it, and if you want people to use it you have to tell them.
Not completely true, some packages but through the built system are only meant for the builder (to test stuff, a new package not ready for general consumption, ...). It is the builder that decide when a package is ready for promotion to CentOSPlus or Extras and only then testing is needed IMHO.
The email must contain a summary of what's changed, and a summary of what the package does. If I don't use it now, tell me why I might want to!
Yep, true.
A lot of package descriptions are next to incomprehensible. Take this, from
...snip...
The email might link to a wiki, but don't expect users to use the wiki to decide whether it's something they should test.
I presume that these announcements would amount to quite a bit of email, maybe enough to seriously change the nature of this list should they come here.
You might consider a new list, maybe "updates-announcements" where these announcements go, and that responses be directed to this list. This list might help where/if there are different packagers on different platforms: Joe builds on IA32, Fred sees he needs to build on zSeries and Freda on Power.
Seed it with members of this list, it's little different from just sending mail here, and gives users of this list the ability to opt out. This would ensure there's a decent number of eyeballs seeing its contents from day one.
The list description for the new list would recommend also subscribing to this list, but I don't think everyone will want to.
The description of this list should be updated to recommend new subscribers also subscribe to updates-announcements so as to keep abreast of threatened changes. Again, not everyone will want to.
Looking at the amount of packages built, I do not think it will generate a huge amount of additional traffic, but still something to look out for. If the amount of traffic gets to high we should indeed consider a seperate list.
If there's a formal test procedure for a particular package, then this should be described in its wiki document. I imagine some packages - gcc, postgresql for example - might have established test cases to run, and others might be added as new bugs are reported and resolved.
Yes, we need test cases for the most common packages.
Regards, Tim
Karanbir Singh wrote:
I actually promised to do this a while ago. So having time now here at LinuxTag I did finaly write up what I had in mind. The proposal is here "http://wiki.centos.org/QaWiki/TestingRepo". It is probably only accessable for people in the QA team or people having admin rights. But I think for the discussion about the proposal itself this should be ok. If not let me know and I move it.
The proposal looks good, but apart from the 3 week cutoff ( which would lead to the death of the testing repo ), its about the same as what we have in place right now. And we all know that the process isnt working.
I tried to implement some of the stuff I wrote in my previous email - implement on the wiki that is, and its too much work getting all the stuff together. And its 'too much' enough, to make it something that most people wont bother with, it raises the barrier too far.
Perhaps a php or python+django or Rails hosted app that can manage this for us would be good. Anyone fancy a shot ?
I can work with Tim and get some sort of a specification up...
On Wed, May 28, 2008 at 5:42 PM, Karanbir Singh mail-lists@karan.org wrote:
Tim Verhoeven wrote:
...snip...
The proposal looks good, but apart from the 3 week cutoff ( which would lead to the death of the testing repo ), its about the same as what we have in place right now. And we all know that the process isnt working.
I think what we need to do is setup some form of expectations and layout details on what functionality needs tested, how and where. And the responsibility for laying those down should be on the person requesting or pushing the packages. In the current scenario where we just ask people to go test something, they dont have any clear idea as to what they are looking for, and there is no way that would sync with the expectations of the packager / pusher / requestor.
Perhaps what we need is a page on the wiki that gives the name and details of the package, who is responsible for it now, and how is going to maintain it going further, and a list of issues / tests that need to be done on those packages. People can then tick the box's indicating they have tested those bits, along with some feedback if they have any. Just having a list of things to look againt and a box to tick Pass / Fail would be good.
My feeling is we need 2 things here. A place for the packages to put info about the packages, what they are for, version info, changelog, ... Everything that tester need to know what the package is for and what needs to be tested. Also, this should be the place were a tester leaves behind his feedback. Secondly a simple systems is needed to inform testers of the existence of the package and where to find it and also maybe some sort of remember when feedback isn't forthcoming.
I don't know what the wiki can or cannot do. But what about a special wiki secion where builders can put the info about the packages (inside a pre-created template) with a script that then can send a automated mail to centos-devel to annouce a new version/build of a package so people know that there is something to test. Maybe we can even link the build system to create entries inside the wiki when stuff gets pushed to the testing repo.
This should make it relatively simple for builders to document their packages and testers are informed with a simple email with a link to that inform them what is required to be tested and a place to leave feedback behind.
I've also made a list in that proposal about what to do with the current pacakges in the testing repo.
I dont agree with the 'Action to take' for a large number of packages there. eg. why should cft and facter be deleted ? and smolt is a part of the cobbler + func + smolt admin stack, and should stay together.
I know, I just when over the list quickly. I don't know all the software inside the testing repo. It was meant to help make a choice :-)
The biggest issue, as far as I can see it, is communitcation itself.
Agreed, the solution needs to be as simple as possible for both builders and testers.
-
John Summerfield -
Karanbir Singh -
KarthiKeyan -
prabhat kumar -
Tim Verhoeven