Hi all,
Please, first of all, I don't want to start a flame about this subject. I only wnat to know CentOS's developers point of view about this particular.
Due to the delays that occurs in the release of new CentOS's versions and patches, and because CentOS developers involved in these tasks may not have sufficient time to devote to it (and also make if freely and spending their time), and I think that has been much talk about the whys for users and developers:
- Why not a distribution fusion between Scientific Linux and CentOS?
- Why not join efforts in the development of a single Enterprise Linux solid distribution based on RHEL?
I repeat: I only want to know your point of view.
Thanks.
Scientific Linux uses upstream source to create their own repo, without desire to be 100% compatible.
CentOS project is dedicated to provide (as close as possible) 100% compatibility. It's not just a rebuild of upstream sources, goal is tu *duplicate* RHEL.
It's that simple. And this was answer many times in this and other mailing lists, forum threads....
Ljubomir
carlopmart wrote:
Hi all,
Please, first of all, I don't want to start a flame about this subject. I only wnat to know CentOS's developers point of view about this particular.
Due to the delays that occurs in the release of new CentOS's versions and patches, and because CentOS developers involved in these tasks may not have sufficient time to devote to it (and also make if freely and spending their time), and I think that has been much talk about the whys for users and developers:
Why not a distribution fusion between Scientific Linux and CentOS?
Why not join efforts in the development of a single Enterprise Linux
solid distribution based on RHEL?
I repeat: I only want to know your point of view.
Thanks.
On 03/23/2011 10:27 AM, Ljubomir Ljubojevic wrote:
Scientific Linux uses upstream source to create their own repo, without desire to be 100% compatible.
CentOS project is dedicated to provide (as close as possible) 100% compatibility. It's not just a rebuild of upstream sources, goal is tu *duplicate* RHEL.
It's that simple. And this was answer many times in this and other mailing lists, forum threads....
Ljubomir
I know that SL includes some custom components like OpenAFS in their distribution, but base system is the same as CentOS. Then, I repeat, why not?
On Wed, Mar 23, 2011 at 10:46:21AM +0100, carlopmart wrote:
I know that SL includes some custom components like OpenAFS in their distribution, but base system is the same as CentOS. Then, I repeat, why not?
Because unless something has changed SL does not profess to be binary compatible with upstream; and for _many_ users that isn't only a strong desire, but a show-stopping requirement.
John
On 03/23/2011 10:52 AM, John R. Dennison wrote:
On Wed, Mar 23, 2011 at 10:46:21AM +0100, carlopmart wrote:
I know that SL includes some custom components like OpenAFS in their distribution, but base system is the same as CentOS. Then, I repeat, why not?
Because unless something has changed SL does not profess to be binary compatible with upstream
Are you sure?? What does it means "binary compatible" for you?? To me it means that the software "foo" works perfectly on both distros. And all the software I've tried, commercial and GNU, it works in both distributions with the same mistakes and equally effective and performance.
On 03/23/2011 12:00 PM, carlopmart wrote:
On 03/23/2011 10:52 AM, John R. Dennison wrote:
On Wed, Mar 23, 2011 at 10:46:21AM +0100, carlopmart wrote:
I know that SL includes some custom components like OpenAFS in their distribution, but base system is the same as CentOS. Then, I repeat, why not?
Because unless something has changed SL does not profess to be binary compatible with upstream
Are you sure?? What does it means "binary compatible" for you??
binary compatible means that ALL dependencies are identical. each and every one.
To me it means that the software "foo" works perfectly on both distros.
if you examine certain packages you will notice that there are linking differences between what SL ships and what RH ships.
And all the software I've tried, commercial and GNU, it works in both distributions with the same mistakes and equally effective and performance.
You'll be surprised how subtle differences can influence programs behaviour. My colleagues investigated for half a week a <2 second difference for the execution time of a commercial application ( we are beta testing ) when using a specific test scenario.
On 03/23/2011 12:20 PM, Rainer Traut wrote:
Am 23.03.2011 11:09, schrieb Manuel Wolfshant:
if you examine certain packages you will notice that there are linking differences between what SL ships and what RH ships.
Can you give an example of such package/binary?
I can but I am not sure that I may. So I won't.
On 03/23/2011 11:42 AM, Manuel Wolfshant wrote:
On 03/23/2011 12:20 PM, Rainer Traut wrote:
Am 23.03.2011 11:09, schrieb Manuel Wolfshant:
if you examine certain packages you will notice that there are linking differences between what SL ships and what RH ships.
Can you give an example of such package/binary?
I can but I am not sure that I may. So I won't.
Then, why you say that without evidence? -- CL Martinez carlopmart {at} gmail {d0t} com
On Wed, Mar 23, 2011 at 03:45:16AM -0700, carlopmart wrote:
On 03/23/2011 11:42 AM, Manuel Wolfshant wrote:
On 03/23/2011 12:20 PM, Rainer Traut wrote:
Am 23.03.2011 11:09, schrieb Manuel Wolfshant:
if you examine certain packages you will notice that there are linking differences between what SL ships and what RH ships.
Can you give an example of such package/binary?
I can but I am not sure that I may. So I won't.
Then, why you say that without evidence?
My two cents are that if you have those stringent of requirements you'd be using RHEL, anyways.
I'd be interested to hear an expansion on this a bit more. Claims or no claims, SL is rebuilt from RHEL sources just as CentOS is. Are there specific examples of ABI breakage introduced?
It would be great to see these two fantastic projects collaborate (and I know they do to some degree already).
Ray
On 03/23/2011 11:53 AM, Ray Van Dolson wrote:
My two cents are that if you have those stringent of requirements you'd be using RHEL, anyways.
I'd be interested to hear an expansion on this a bit more. Claims or no claims, SL is rebuilt from RHEL sources just as CentOS is. Are there specific examples of ABI breakage introduced?
It would be great to see these two fantastic projects collaborate (and I know they do to some degree already).
I think what really gets lost here is that just because a package is rebuilt from the same package sources, it doesn't mean that it is exactly 100% binary compatible. SL has stated that they maintain compatibility but certainly do not guarantee it. By and large, most packages probably are ABI identical and some may be slightly off but you may never even notice it. However, if you ever did encounter cases where there differences, that may cause major headaches for you.
The only way I could see the two projects 'merging' would be if the CentOS goals remained and established the base OS and SL became the addons and enhancements repo. Similar to the CentOS plus repo, ABI compatibility may have to be broken by the SL additions but that would be an end user choice.
I kind of thing that as others have mentioned already, keeping them separate allows for more choice for the end user base. Some people could care less about RHEL compatibility. Some may not care a helluva lot but would prefer to have it lest they be bitten in the backside at some point, and others demand it.
For those that demand it, sure purchasing RHEL would be a logical option, but some people really do not need the support agreement. I (certainly like many others on this list) have been running Linux boxes for more years than I can count and I have never had to contact a vendor for support. When I have had issues, I've been able to contact the upstream package provider directly and have managed to get fantastic support far beyond you would ever get from a commercial offering of any sort.
SL has extra packages (it means they wasn't included into RHEL) that was included in it.
... we have added several packages to Scientific Linux that are not found anywhere in the upstream release, including IceWM, OpenAFS, Revisor, Live USB Creator, YUM auto-update, external YUM repositories... (c) from http://distrowatch.com/?newsid=06549
SL and CentOS like two people that go different ways, but have the common base.
Kind regards, Alexander Ustinov
2011/3/23 Устинов Александр Александрович rusfearuth@gmail.com:
SL has extra packages (it means they wasn't included into RHEL) that was included in it. ... we have added several packages to Scientific Linux that are not found anywhere in the upstream release, including IceWM, OpenAFS, Revisor, Live USB Creator, YUM auto-update, external YUM repositories... (c) from http://distrowatch.com/?newsid=06549 SL and CentOS like two people that go different ways, but have the common base. Kind regards, Alexander Ustinov
Can those packages go in a separate channel, much as "centosplus" is a separate channel? I don't have the spare environments to really spin that distribution up right now for the side by side comparison.
Am 24.03.2011 03:07, schrieb Устинов Александр Александрович:
SL has extra packages (it means they wasn't included into RHEL) that was included in it.
... we have added several packages to Scientific Linux that are not found anywhere in the upstream release, including IceWM, OpenAFS, Revisor, Live USB Creator, YUM auto-update, external YUM repositories... (c) from http://distrowatch.com/?newsid=06549
SL and CentOS like two people that go different ways, but have the common base.
Kind regards, Alexander Ustinov
The funny thing is, the point you're bringing up is quite the reverse.
In SL it reads: "> ... we have added several packages to Scientific Linux that are not found anywhere in the upstream release, including IceWM, OpenAFS, "
In Centos it reads: "CentOS Extras - This repository contains items that provide additional functionality to CentOS ... horde framework and packages, freenx, apt, XFCE, and yumex"
What is right though, in SL these extra packages exist in the default repo, not in an extra repo - but it is enabled by default in Centos.
info from here: http://wiki.centos.org/AdditionalResources/Repositories
Rainer
2011/3/24 Rainer Traut tr.ml@gmx.de
Am 24.03.2011 03:07, schrieb Устинов Александр Александрович:
SL has extra packages (it means they wasn't included into RHEL) that was included in it.
... we have added several packages to Scientific Linux that are not found anywhere in the upstream release, including IceWM, OpenAFS, Revisor, Live USB Creator, YUM auto-update, external YUM repositories... (c) from http://distrowatch.com/?newsid=06549
SL and CentOS like two people that go different ways, but have the common base.
Kind regards, Alexander Ustinov
The funny thing is, the point you're bringing up is quite the reverse.
In SL it reads: "> ... we have added several packages to Scientific Linux that are not found anywhere in the upstream release, including IceWM, OpenAFS, "
In Centos it reads: "CentOS Extras - This repository contains items that provide additional functionality to CentOS ... horde framework and packages, freenx, apt, XFCE, and yumex"
What is right though, in SL these extra packages exist in the default repo, not in an extra repo - but it is enabled by default in Centos.
info from here: http://wiki.centos.org/AdditionalResources/Repositories
Rainer
CentOS-devel mailing list CentOS-devel@centos.org http://lists.centos.org/mailman/listinfo/centos-devel
Just have a read next message =) I wrote there about an another idea ;)
Kind regards, Alexander Ustinov
Am 23.03.2011 11:42, schrieb Manuel Wolfshant:
On 03/23/2011 12:20 PM, Rainer Traut wrote:
Am 23.03.2011 11:09, schrieb Manuel Wolfshant:
if you examine certain packages you will notice that there are linking differences between what SL ships and what RH ships.
Can you give an example of such package/binary?
I can but I am not sure that I may. So I won't.
So this means you are making assertions you cannot prove - for whatever reason. I cannot take you serious then.
Am 23.03.2011 11:42, schrieb Manuel Wolfshant:
On 03/23/2011 12:20 PM, Rainer Traut wrote:
Am 23.03.2011 11:09, schrieb Manuel Wolfshant:
if you examine certain packages you will notice that there are linking differences between what SL ships and what RH ships.
Can you give an example of such package/binary?
I can but I am not sure that I may. So I won't.
So this means you are making assertions you cannot prove - for whatever reason. I cannot take you serious then.
Go to the centos list archives and look things up yourself, this has been discussed there recently.
On 03/24/2011 10:31 AM, Rainer Traut wrote:
Am 23.03.2011 11:42, schrieb Manuel Wolfshant:
On 03/23/2011 12:20 PM, Rainer Traut wrote:
Am 23.03.2011 11:09, schrieb Manuel Wolfshant:
if you examine certain packages you will notice that there are linking differences between what SL ships and what RH ships.
Can you give an example of such package/binary?
I can but I am not sure that I may. So I won't.
So this means you are making assertions you cannot prove - for whatever reason. I cannot take you serious then.
Thank you for the vote of confidence, I appreciate it. Especially coming after the answer that you demand was already given. http://lists.centos.org/pipermail/centos/2011-March/108389.html
"It does not exist because I cannot see it" was never a valid reasoning. Not even in the era when people believed the Earth is flat because there was no evidence of roundness.
On Wed, 23 Mar 2011, Manuel Wolfshant wrote:
On 03/23/2011 12:20 PM, Rainer Traut wrote:
Am 23.03.2011 11:09, schrieb Manuel Wolfshant:
if you examine certain packages you will notice that there are linking differences between what SL ships and what RH ships.
Can you give an example of such package/binary?
I can but I am not sure that I may. So I won't.
Manuel,
I don't see any reason why you may not share incompatibilities found in Scientific Linux ? Can you shed a light on why that is ?
Other team members already disclosed two of those, so is there a reason why not all of them are shared with the Scientific Linux team ?
On 03/24/2011 11:07 AM, Dag Wieers wrote:
if you examine certain packages you will notice that there are linking differences between what SL ships and what RH ships.
Can you give an example of such package/binary?
I can but I am not sure that I may. So I won't.
Manuel,
I don't see any reason why you may not share incompatibilities found in Scientific Linux ? Can you shed a light on why that is ?
Other team members already disclosed two of those, so is there a reason why not all of them are shared with the Scientific Linux team ?
There was no deep secret, just a matter of timming. It was not my answer to give, I wanted to be sure that Johny & Karanbir take the needed fixing steps first. And as you might have already found out, Johny has already provided the answer and RH has already acknowledged the bug in their bugzilla. It's as simple as that.
On Thu, Mar 24, 2011 at 2:12 AM, Manuel Wolfshant wolfy@nobugconsulting.ro wrote:
On 03/24/2011 11:07 AM, Dag Wieers wrote:
if you examine certain packages you will notice that there are linking differences between what SL ships and what RH ships.
Can you give an example of such package/binary?
I can but I am not sure that I may. So I won't.
Manuel,
I don't see any reason why you may not share incompatibilities found in Scientific Linux ? Can you shed a light on why that is ?
Other team members already disclosed two of those, so is there a reason why not all of them are shared with the Scientific Linux team ?
There was no deep secret, just a matter of timming. It was not my answer to give, I wanted to be sure that Johny & Karanbir take the needed fixing steps first.
I just wanted to make a quick note that you/we are referring to the *Alpha* version of SL 5.6. At that stage of the game, chances are that they have not gone through all the testing. There are good reasons why it is Alpha. I'm not saying the "incompatibility" issue will definitely disappear in their Beta but this is just an observation most people seem to be overlooking.
Akemi
On 3/24/11 7:33 AM, Akemi Yagi wrote:
On Thu, Mar 24, 2011 at 2:12 AM, Manuel Wolfshant
Other team members already disclosed two of those, so is there a reason why not all of them are shared with the Scientific Linux team ?
There was no deep secret, just a matter of timming. It was not my answer to give, I wanted to be sure that Johny& Karanbir take the needed fixing steps first.
I just wanted to make a quick note that you/we are referring to the *Alpha* version of SL 5.6. At that stage of the game, chances are that they have not gone through all the testing. There are good reasons why it is Alpha. I'm not saying the "incompatibility" issue will definitely disappear in their Beta but this is just an observation most people seem to be overlooking.
Also note that you can't just 'yum update' from those SL alpha versions to the final release, so even if CentOS did ship alpha/beta versions it wouldn't make life that much easier. On the other hand it would be nice if yum knew enough to do that - or to understand different repositories and be able to always update from the same source or be told when to switch and reinstall.
Also note that you can't just 'yum update' from those SL alpha versions to the final release, so even if CentOS did ship alpha/beta versions it wouldn't make life that much easier. On the other hand it would be nice if yum knew enough to do that - or to understand different repositories and be able to always update from the same source or be told when to switch and reinstall.
the new yum ( in fedora ) knows that, basically it prefers to install from the same repo from where you already have stuff.
On Thu, 24 Mar 2011, Manuel Wolfshant wrote:
Also note that you can't just 'yum update' from those SL alpha versions to the final release, so even if CentOS did ship alpha/beta versions it wouldn't make life that much easier. On the other hand it would be nice if yum knew enough to do that - or to understand different repositories and be able to always update from the same source or be told when to switch and reinstall.
the new yum ( in fedora ) knows that, basically it prefers to install from the same repo from where you already have stuff.
Great ! This helps with cross-repository use in Yum.
On 3/24/2011 8:42 AM, Dag Wieers wrote:
On Thu, 24 Mar 2011, Manuel Wolfshant wrote:
Also note that you can't just 'yum update' from those SL alpha versions to the final release, so even if CentOS did ship alpha/beta versions it wouldn't make life that much easier. On the other hand it would be nice if yum knew enough to do that - or to understand different repositories and be able to always update from the same source or be told when to switch and reinstall.
the new yum ( in fedora ) knows that, basically it prefers to install from the same repo from where you already have stuff.
Great ! This helps with cross-repository use in Yum.
I never quite understood why that wasn't designed in from the start. It always had the assumption that all repositories were coordinated (or forced the end user to know more than was usually possible to know about them), when from the beginning the base repositories involved had policies that guaranteed that uncoordinated 3rd party repositories would always be a necessity. The new rpmforge layout helps somewhat, but I've pretty much had to keep all 3rd party repos disabled and explicitly enable them with the command line switch just to install/update single packages.
And while I'm ranting about yum, there has to be a way to pick a repository mirror that doesn't load your caching proxy with a copy of every file from every mirror. Since at least Centos 5 the mirrors are insanely fast so it really doesn't matter any more, but it still doesn't make much sense to make the mirrors do the extra work.
On Thursday, March 24, 2011 10:40:37 am Les Mikesell wrote:
On 3/24/2011 8:42 AM, Dag Wieers wrote:
On Thu, 24 Mar 2011, Manuel Wolfshant wrote:
the new yum ( in fedora ) knows that, basically it prefers to install from the same repo from where you already have stuff.
Great ! This helps with cross-repository use in Yum.
I never quite understood why that wasn't designed in from the start.
Because the yellowdog updater-modified developers didn't have that lofty of a goal in mind from the start?
Hindsight is 20/20; perhaps people thought at the time that cooperation in a Debianic way could happen between the various 3rd party repos and make repo mixing problems a thing of the past? We know now that that didn't happen, almost entirely due to personality/political things. Not casting any blame at all; but the people who saw the 'unified Debianic repository' dream die, in real-time (you were around then, Les, being just as much of a gadfly then as now) know what happened. That's about the time I dropped all my fedora list subscriptions; I've just since F13 resubscribed to the fedora-users list.
And there was a bigger issue to address, that of dependency solving. I'm glad the yellowdog updater-modified developers chose to solve that problem first, and not worry about overdesigning for a problem that had not yet reared its ugly head (at the time). Oh, and let's not forget the absolutely wonderful presto and delta rpm work.
Or we could all still be on apt-rpm and still be having multilib and related issues. And while synaptic is still far and away the best package management GUI out there bar none, the apt-rpm backend has (or at least had) significant issues at that time.
On Thu, 24 Mar 2011, Lamar Owen wrote:
Or we could all still be on apt-rpm and still be having multilib and related issues. And while synaptic is still far and away the best package management GUI out there bar none, the apt-rpm backend has (or at least had) significant issues at that time.
Hey ! There was nothing wrong with apt-rpm, I've been using it until November last year on CentOS-5 before I switched to RHEL6. The multi-lib problems were exagerated and fixed, the biggest problem was the lack of development after the main developer moved to Conectiva, and the second maintainer to Red Hat.
But I do prefer python over C++ anytime, though speedwise it was unbeatable.
On Mar 24, 2011, at 11:42 AM, Dag Wieers wrote:
On Thu, 24 Mar 2011, Lamar Owen wrote:
Or we could all still be on apt-rpm and still be having multilib and related issues. And while synaptic is still far and away the best package management GUI out there bar none, the apt-rpm backend has (or at least had) significant issues at that time.
Hey ! There was nothing wrong with apt-rpm, I've been using it until November last year on CentOS-5 before I switched to RHEL6. The multi-lib problems were exagerated and fixed, the biggest problem was the lack of development after the main developer moved to Conectiva, and the second maintainer to Red Hat.
There's *lots* that's wrong with apt-rpm, don't start me.
Meanwhile -- now that there's an interested/active maintainer at Caixa Magica -- there's some hope of improving apt-rpm.
But I do prefer python over C++ anytime, though speedwise it was unbeatable.
FICL! OCAML! Java! Perl! Pyhon (but which, sigh)! ...
choice of implementation language increasingly fragments deployable solutions.
73 de Jeff
On Thursday, March 24, 2011 11:42:30 am Dag Wieers wrote:
On Thu, 24 Mar 2011, Lamar Owen wrote:
Or we could all still be on apt-rpm and still be having multilib and related issues. And while synaptic is still far and away the best package management GUI out there bar none, the apt-rpm backend has (or at least had) significant issues at that time.
Hey !
HiHi! :-) I figured you'd reply, Dag.... in a good way, of course.
There was nothing wrong with apt-rpm, I've been using it until November last year on CentOS-5 before I switched to RHEL6. The multi-lib problems were exagerated and fixed,
As the fix occurred after I switched off of the apt-rpm setup (I had apt-rpm primarily to get the excellent DAG repo running on one box and Axel's repo running on another) to a fully yum setup (with no really useful GUI; that was the bummer), at least until F8, after which I did the Grand two-year Kubuntu Experiment (from 7.04 up through 9.04; I never had that much trouble with Fedora's KDE!) and then back to F11.... And while I am less than thrilled with any of the PackageKit GUI's, it is what it is. One can always dream that a real 'synaptic for RPM' could happen.....
the biggest problem was the lack of development after the main developer moved to Conectiva, and the second maintainer to Red Hat.
Those are the issues to which I referred....
But I do prefer python over C++ anytime, though speedwise it was unbeatable.
At the time I last used it, synaptic could have all the updated headers downloaded, I could have my new selections made, and the update could be halfway done before the yumex of that time had finished updating the yum cache. Yumex is still not a speed demon, that's for sure. This is one place the Ubuntu and Debian updaters excel: they are wicked fast. But no presto; and on slower lines and for bigger updates presto is a major win.
But the apt-rpm code is not the easiest code in the world, and you're very right: my preference to the maintainability of python over C++ means that I'll just have a less useful GUI for real package managment. At least until either I get motivated enough to do it, or someone else gets motivated enough to give packagekit that powerful synaptic goodness.
On Thu, 24 Mar 2011, Lamar Owen wrote:
On Thursday, March 24, 2011 11:42:30 am Dag Wieers wrote:
On Thu, 24 Mar 2011, Lamar Owen wrote:
Or we could all still be on apt-rpm and still be having multilib and related issues. And while synaptic is still far and away the best package management GUI out there bar none, the apt-rpm backend has (or at least had) significant issues at that time.
Hey !
HiHi! :-) I figured you'd reply, Dag.... in a good way, of course.
Good, because it was not meant to be very serious. We have all moved on by now ! :-)
On Thu, 2011-03-24 at 11:56 -0400, Lamar Owen wrote:
But I do prefer python over C++ anytime, though speedwise it was unbeatable.
At the time I last used it, synaptic could have all the updated headers downloaded, I could have my new selections made, and the update could be halfway done before the yumex of that time had finished updating the yum cache. Yumex is still not a speed demon, that's for sure. This is one place the Ubuntu and Debian updaters excel: they are wicked fast. But no presto; and on slower lines and for bigger updates presto is a major win.
Have you looked at yumex in f14 or rawhide? Or from Tim's builds here: http://repos.fedorapeople.org/repos/timlau/yumex/
I ask b/c he's made a lot of great changes to yumex and it is faster.
worth looking at.
-sv
On Thursday, March 24, 2011 12:06:31 pm seth vidal wrote:
Have you looked at yumex in f14 or rawhide? Or from Tim's builds here: http://repos.fedorapeople.org/repos/timlau/yumex/
I ask b/c he's made a lot of great changes to yumex and it is faster.
worth looking at.
I will do that; while yumex is no synaptic, feature-wise, it beats kpackagekit hands down, IMO. Thanks for the pointers, and thanks for yum.
And I have noticed the speed improvements in yum over the years; I still have a CentOS 3 box running in production, and comparing yum of that day (2.0.8) with today's yum is....well, no comparison, really.
On Mar 24, 2011, at 11:35 AM, Lamar Owen wrote:
On Thursday, March 24, 2011 10:40:37 am Les Mikesell wrote:
On 3/24/2011 8:42 AM, Dag Wieers wrote:
On Thu, 24 Mar 2011, Manuel Wolfshant wrote:
the new yum ( in fedora ) knows that, basically it prefers to install from the same repo from where you already have stuff.
Great ! This helps with cross-repository use in Yum.
I never quite understood why that wasn't designed in from the start.
Because the yellowdog updater-modified developers didn't have that lofty of a goal in mind from the start?
Actually because (historically) cross-repositories Just Don't Work without some artifact like "repository priority" which is too coarse grained fto work generally (Dag's priority scheme works well, just not everyone does as carefukl work as Dag et al do).
Just restating what you said more specifically ...
73 de Jeff
On 3/24/2011 10:44 AM, Jeff Johnson wrote:
On Thu, 24 Mar 2011, Manuel Wolfshant wrote:
the new yum ( in fedora ) knows that, basically it prefers to install from the same repo from where you already have stuff.
Great ! This helps with cross-repository use in Yum.
I never quite understood why that wasn't designed in from the start.
Because the yellowdog updater-modified developers didn't have that lofty of a goal in mind from the start?
I think it was the opposite - that they had the lofty goal of having all repositories coordinated even though that is clearly impossible unless you can dictate a jailed iphone-like world.
Actually because (historically) cross-repositories Just Don't Work without some artifact like "repository priority" which is too coarse grained fto work generally (Dag's priority scheme works well, just not everyone does as carefukl work as Dag et al do).
Just restating what you said more specifically ...
Sure, it can't work if you generalize it to an infinite number of repositories that each can have any version of any library they want, but in practice what you want to happen is to pull only the necessary components from a 3rd party repo, and thereafter only update those from the repo where they originated. This mostly works and is what people usually try to accomplish in ways that end up being worse than having yum track it automatically.
On Thu, Mar 24, 2011 at 9:50 AM, Les Mikesell lesmikesell@gmail.com wrote:
On 3/24/2011 10:44 AM, Jeff Johnson wrote:
I never quite understood why that wasn't designed in from the start.
Because the yellowdog updater-modified developers didn't have that lofty of a goal in mind from the start?
I think it was the opposite - that they had the lofty goal of having all repositories coordinated even though that is clearly impossible unless you can dictate a jailed iphone-like world.
There weren't really "external" repositories when yum came out. I won't speak for Seth, but in my opinion, the goal of yum was "help end RPM dep hell". And thankfully it did (combined with lots of packaging improvements in rh/fedora, etc.).
-Jeff
On Thu, 24 Mar 2011, Jeff Sheltren wrote:
On Thu, Mar 24, 2011 at 9:50 AM, Les Mikesell lesmikesell@gmail.com wrote:
On 3/24/2011 10:44 AM, Jeff Johnson wrote:
I never quite understood why that wasn't designed in from the start.
Because the yellowdog updater-modified developers didn't have that lofty of a goal in mind from the start?
I think it was the opposite - that they had the lofty goal of having all repositories coordinated even though that is clearly impossible unless you can dictate a jailed iphone-like world.
There weren't really "external" repositories when yum came out. I won't speak for Seth, but in my opinion, the goal of yum was "help end RPM dep hell". And thankfully it did (combined with lots of packaging improvements in rh/fedora, etc.).
Well, you had plenty, RPMforge, FreshRPMS, ATrpms, NewRPMS, PlanetCCMA, and many many more. That was before something like Fedora Extras existed.
On Thursday, March 24, 2011 01:01:11 pm Dag Wieers wrote:
Well, you had plenty, RPMforge, FreshRPMS, ATrpms, NewRPMS, PlanetCCMA, and many many more. That was before something like Fedora Extras existed.
It was even before Fedora the distribution existed; you of course had the fedora.us repository.....
But I distinctly remember using a mix of the above with Red Hat Linux 9, and prior, using apt-rpm (and by hand before apt-rpm existed), before Seth tackled the problem yum solved really well.
On Thursday, March 24, 2011 01:48:17 pm Lamar Owen wrote:
It was even before Fedora the distribution existed; you of course had the fedora.us repository.....
After sending this out I realized I should have said 'there of course was the fedora.us repository' because that was Warren's thing, not yours. Dastardly colloquial usage of the second person personal pronoun as a generic address to a set of people but easily misinterpreted as singular when plural was meant.... where's thou when you need him?
On Thu, 2011-03-24 at 13:48 -0400, Lamar Owen wrote:
On Thursday, March 24, 2011 01:01:11 pm Dag Wieers wrote:
Well, you had plenty, RPMforge, FreshRPMS, ATrpms, NewRPMS, PlanetCCMA, and many many more. That was before something like Fedora Extras existed.
It was even before Fedora the distribution existed; you of course had the fedora.us repository.....
But I distinctly remember using a mix of the above with Red Hat Linux 9, and prior, using apt-rpm (and by hand before apt-rpm existed), before Seth tackled the problem yum solved really well.
The first release of yum was used for rhl 7.1/.2/.3
Just to get things into proper perspective.
-sv
On 3/24/2011 1:20 PM, seth vidal wrote:
Well, you had plenty, RPMforge, FreshRPMS, ATrpms, NewRPMS, PlanetCCMA, and many many more. That was before something like Fedora Extras existed.
It was even before Fedora the distribution existed; you of course had the fedora.us repository.....
But I distinctly remember using a mix of the above with Red Hat Linux 9, and prior, using apt-rpm (and by hand before apt-rpm existed), before Seth tackled the problem yum solved really well.
The first release of yum was used for rhl 7.1/.2/.3
Just to get things into proper perspective.
Which at least freshrpms supported.
On Thursday, March 24, 2011 02:20:42 pm seth vidal wrote:
The first release of yum was used for rhl 7.1/.2/.3 Just to get things into proper perspective.
Perhaps my 9 should have been a 6..... :-)
I do remember having to depsolve by hand, and using a third party RPM repository to do it, for a project way back then....
Thanks for the perspective.
On 3/24/2011 11:54 AM, Jeff Sheltren wrote:
On Thu, Mar 24, 2011 at 9:50 AM, Les Mikeselllesmikesell@gmail.com wrote:
On 3/24/2011 10:44 AM, Jeff Johnson wrote:
I never quite understood why that wasn't designed in from the start.
Because the yellowdog updater-modified developers didn't have that lofty of a goal in mind from the start?
I think it was the opposite - that they had the lofty goal of having all repositories coordinated even though that is clearly impossible unless you can dictate a jailed iphone-like world.
There weren't really "external" repositories when yum came out. I won't speak for Seth, but in my opinion, the goal of yum was "help end RPM dep hell". And thankfully it did (combined with lots of packaging improvements in rh/fedora, etc.).
Really? I thought freshrpms was around (and wonderful) back when you had to use some apt derivative to use it.
On Mar 24, 2011, at 1:02 PM, Les Mikesell wrote:
On 3/24/2011 11:54 AM, Jeff Sheltren wrote:
On Thu, Mar 24, 2011 at 9:50 AM, Les Mikeselllesmikesell@gmail.com wrote:
On 3/24/2011 10:44 AM, Jeff Johnson wrote:
I never quite understood why that wasn't designed in from the start.
Because the yellowdog updater-modified developers didn't have that lofty of a goal in mind from the start?
I think it was the opposite - that they had the lofty goal of having all repositories coordinated even though that is clearly impossible unless you can dictate a jailed iphone-like world.
There weren't really "external" repositories when yum came out. I won't speak for Seth, but in my opinion, the goal of yum was "help end RPM dep hell". And thankfully it did (combined with lots of packaging improvements in rh/fedora, etc.).
Really? I thought freshrpms was around (and wonderful) back when you had to use some apt derivative to use it.
Yum was deployed around the same time as "repositories" were invented. Dag et al are perhaps the oldest (and imho most useful) "repositories" around.
The flaw with "repositories" is that its pretty easy to devise a split at the 1st level of "add-on". The next level of "add-on" to "add-on" is usually a disaster because of duplication and lack of coordination.
One can assign a preference mechanism like "repository priority" which looks attractive on paper: One can then tell (when faced with otherwise identical choices in a depsolver) which package to install by "priority".
But priority in the "real world" of multiple repositories is too arbitrary and not based on intrinsic details like dependency closure and difficult to administer reliably: Who *exactly* chooses the priority? What criteria are the basis?
And freshrpms was wonderful and Matthias is truly a gentleman, his involvement is missed. But everyone moves on to other interests than Linux eventually.
73 de Jeff
On Thursday, March 24, 2011 02:40:59 pm Jeff Johnson wrote:
But priority in the "real world" of multiple repositories is too arbitrary and not based on intrinsic details like dependency closure and difficult to administer reliably: Who *exactly* chooses the priority? What criteria are the basis?
Well, Jeff, it partially goes all the way back to a subject related to a discussion you and I had years ago, in Red Hat 6.1/6.2 timeframes, of why PostgreSQL database data upgrades couldn't be done in %pre, %install, %post, or %postun scriptlets. We had been talking about the anaconda chroot and some of the deficiencies of that chroot and how the environment wasn't stable enough to do the ambitious things I wanted to do, and you made a statement along the lines that all RPM knows is packages; it doesn't know squat about distributions, just packages.
Substitute repositories for distributions, and you have today's situation. RPM itself doesn't care where a package gets its dependencies from, just that they're satisfied.
I'm going to have to look up that e-mail, because I know I kept it....and I'm fairly certain it was you, but it might have been someone else, might have even been Nottingham.
And freshrpms was wonderful and Matthias is truly a gentleman, his involvement is missed.
Yes, it is.
But everyone moves on to other interests than Linux eventually.
Indeed. If it weren't my 'daily driver' OS I might not still be around these parts.
On Mar 24, 2011, at 4:13 PM, Lamar Owen wrote:
On Thursday, March 24, 2011 02:40:59 pm Jeff Johnson wrote:
But priority in the "real world" of multiple repositories is too arbitrary and not based on intrinsic details like dependency closure and difficult to administer reliably: Who *exactly* chooses the priority? What criteria are the basis?
Well, Jeff, it partially goes all the way back to a subject related to a discussion you and I had years ago, in Red Hat 6.1/6.2 timeframes, of why PostgreSQL database data upgrades couldn't be done in %pre, %install, %post, or %postun scriptlets. We had been talking about the anaconda chroot and some of the deficiencies of that chroot and how the environment wasn't stable enough to do the ambitious things I wanted to do, and you made a statement along the lines that all RPM knows is packages; it doesn't know squat about distributions, just packages.
Well my comments way back when were based on real world experience attempting to automate postgresql database format conversions. That was actually attempted in RPM scriptlets (RHL5.2? I fergit) and turned out to be rather a disaster with ENOSPC failure conditions.
(aside) These days sqlite3 is embedded @rpm5.org, so its quite possible to do a full dump/reload operation now that "disk space is cheap".
But the salient point is that RPM is a pretty stoopid program, implements mechanism but not policy, and largely does whatever its told to do.
That's merely a restatement of "... doesn't know squat about distributions ..."
Substitute repositories for distributions, and you have today's situation. RPM itself doesn't care where a package gets its dependencies from, just that they're satisfied.
Depends on POV. The operant POV is rpm == dpkg which is no more accurate than rpm == apt RPM is a weird mixture between apt (and depsolvers) and dpkg (and archiver operations).
(aside) RPM @rpm5.org is tracking not only the *.rpm file digest, but also the origin/stat(2) of the original *.rpm file, and ghas many other features that aren't "doesn't care".
But you are likely quoting a statement from me that If two packages have the same Provides:, either will do. That's merely a statement of logic: the Requires: assertion will have an identical truth table no matter which package is chosen.
But "preference" and installer "policy" isn't well implemented anywhere that I see. Sure there's "repositories" and "applications" and "distros", but there isn't any implementation of "preference".
Go ask for Suggests: to be implemented in RPM and listen carefully to the random replies. Everyone wants "preference", but there's no meaningful (afaics) implementation around. It *is* a non-trivial problem to solve.
I'm going to have to look up that e-mail, because I know I kept it....and I'm fairly certain it was you, but it might have been someone else, might have even been Nottingham.
"Patience grasshoppers" still cracks me up (but I have a warped sense of humor)
And freshrpms was wonderful and Matthias is truly a gentleman, his involvement is missed.
Yes, it is.
But everyone moves on to other interests than Linux eventually.
Indeed. If it weren't my 'daily driver' OS I might not still be around these parts.
And if it wasn't for an RPM fork (which pissed me off: I have no intent of "losing") I'd likely have moved on to something else too.
73 de Jeff
CentOS-devel mailing list CentOS-devel@centos.org http://lists.centos.org/mailman/listinfo/centos-devel
On Thursday, March 24, 2011 12:50:47 pm Les Mikesell wrote:
I think it was the opposite - that they had the lofty goal of having all repositories coordinated even though that is clearly impossible unless you can dictate a jailed iphone-like world.
I used my own adjective 'Debianic' to indicate that Debian has for the most part already done this with no 'jail' required. It can be done, and it can be done with a fully open structure, if all the players will just do it. Of course, instead of third party repos you get whole different distributions, like Ubuntu, as a result, but that's preferrable to multiple overlapping and in many areas mutually exclusive repositories. The situation right now is not as bad as I have seen it, and that is good.
Repository unification is the better goal; making the various repos work together when they aren't coordinated is much more difficult, and is the very definition of the word 'kludge.' It didn't happen; and won't happen: but it would really have been nice had it happened.
There are going to be situations where your mix isn't going to work at all, ever, because the packages you want need conflicting versions of various components; there is no technical solution for this other than not mixing and putting the two mutually exclusive 'things' on two different boxes, real or virtual.
On 3/24/2011 12:45 PM, Lamar Owen wrote:
I think it was the opposite - that they had the lofty goal of having all repositories coordinated even though that is clearly impossible unless you can dictate a jailed iphone-like world.
I used my own adjective 'Debianic' to indicate that Debian has for the most part already done this with no 'jail' required. It can be done, and it can be done with a fully open structure, if all the players will just do it. Of course, instead of third party repos you get whole different distributions, like Ubuntu, as a result, but that's preferrable to multiple overlapping and in many areas mutually exclusive repositories.
But it can't be done starting with Red Hat's policy, which was/is the real world starting point.
Repository unification is the better goal; making the various repos work together when they aren't coordinated is much more difficult, and is the very definition of the word 'kludge.' It didn't happen; and won't happen: but it would really have been nice had it happened.
An even better goal would be an LSB standard that actually lets 3rd party binaries work. Period, no qualifications except maybe minimum kernel and glibc versions. Let me know when any distro cares.
There are going to be situations where your mix isn't going to work at all, ever, because the packages you want need conflicting versions of various components; there is no technical solution for this other than not mixing and putting the two mutually exclusive 'things' on two different boxes, real or virtual.
Yes, and this tends to become a big problem as enterprise distributions age well beyond the 'best used by' date of a lot of components and more than one 3rd party wants to update a base library. But it could be improved if you could keep those from jumping around as the different repos bump numbers at random times.
On Thu, 2011-03-24 at 15:15 +0200, Manuel Wolfshant wrote:
Also note that you can't just 'yum update' from those SL alpha versions to the final release, so even if CentOS did ship alpha/beta versions it wouldn't make life that much easier. On the other hand it would be nice if yum knew enough to do that - or to understand different repositories and be able to always update from the same source or be told when to switch and reinstall.
the new yum ( in fedora ) knows that, basically it prefers to install from the same repo from where you already have stuff.
If it's the feature I think you are talking about, it is also in RHEL-5/CentOS-5. But it only alters install (actually just changes the behaviour of compare_providers¹). AFAIK that doesn't solve the entire set of problems that Les/Dag want solved.
Adding more logic based on "repoid of the installed package" is somewhat problematic, due to anaconda using different repoids and the fact there are a number of "repos" in the wild which are just URLs (so clients have semi-random repoids).
The known "plans" atm. are more of the "make the priorities plugin suck less" variety, than to add vendor/buildtime/repoid/etc. directly into update calculations.
On 3/24/2011 11:57 AM, James Antill wrote:
The known "plans" atm. are more of the "make the priorities plugin suck less" variety, than to add vendor/buildtime/repoid/etc. directly into update calculations.
Yes, but given more than 2 choices, priorities are likely transient things over an enterprise release's lifetime and even if your favorite 2nd choice changes you aren't going to want its likely incompatible versions to clobber the ones you've already installed. I think you'd see that now if you flip between epel/rpmforge's viewvc. The code may even be the same but the config files are different enough to break things.
The other thing I've always thought should be included in yum is an option to ignore packages in a repository newer than a specified timestamp. That would let you update one machine, test everything, then update some other set of machines to the tested state even if new packages have been introduced to the repositories between the times in question. That might not cover every possible change, but it would be better than nothing or having to build snapshot copies of entire repositories to avoid new items.
On Thu, 2011-03-24 at 12:57 -0500, Les Mikesell wrote:
The other thing I've always thought should be included in yum is an option to ignore packages in a repository newer than a specified timestamp. That would let you update one machine, test everything, then update some other set of machines to the tested state even if new packages have been introduced to the repositories between the times in question. That might not cover every possible change, but it would be better than nothing or having to build snapshot copies of entire repositories to avoid new items.
We've explained why this isn't possible before, but to repeat: There is no "available from the repo." timestamp, and generating one is non-trivial to do correctly.
However, something that should be in the 6.1 yum is that you can do:
# -- Goto test machine yum update # Do all your testing to make sure this is good. # Get the "saved transaction" from the above: yum -q history addon-info last saved_tx > saved_tx.yumtx
# -- Goto N other machines with a copy of the above yum load-ts saved_tx.yumtx
On Thu, 24 Mar 2011, Les Mikesell wrote:
Also note that you can't just 'yum update' from those SL alpha versions to the final release, so even if CentOS did ship alpha/beta versions it wouldn't make life that much easier. On the other hand it would be nice if yum knew enough to do that - or to understand different repositories and be able to always update from the same source or be told when to switch and reinstall.
Yes, I have contemplated writing two Yum plugins for such cases:
- to only consider to update packages from the same repository/source or in case dependency resolution requires a different course of action, to ask the user (no more exclude/include nonsense)
- to always update to same/newer updates based on build-time, this would be absolutely useful for CentOS QA testing where packages are being rebuild with the exact same version/release and in those cases you want to get the latest build.
If anyone is interested, don't wait for me to implement it as free time is pretty scarce around here :-/
On 03/24/2011 01:32 PM, Dag Wieers wrote:
- to always update to same/newer updates based on build-time, this would be absolutely useful for CentOS QA testing where packages are being rebuild with the exact same version/release and in those cases you want to get the latest build.
yum reinstall; does this quite well - also we dont want anything like that for centos QA. We dont test how pkgs change in place, the aim is to test how user side stuff changes through upgrades.
- KB
On Thursday, March 24, 2011 04:13:25 pm Karanbir Singh wrote:
yum reinstall; does this quite well - also we dont want anything like that for centos QA. We dont test how pkgs change in place, the aim is to test how user side stuff changes through upgrades.
If I may ask a CentOS QA process question, then, is what is the QA on making sure that the package that gets pushed to -testing and then to -updates is the latest build of a particular EVR tuple? Just curious, since buildtime seems a reasonable thing to use for that sort of thing. Or, for that matter, that the QA feedback is for the correct build of a particular EVR for a package?
Like I said, just curious, because I did run into that issue, which is why I always bumped release on RPMs I pushed out even for the most trivial rebuild reason; I got burned by it once; but if the question is inappropriate, my apologies.
On 03/24/2011 08:23 PM, Lamar Owen wrote:
If I may ask a CentOS QA process question, then, is what is the QA on making sure that the package that gets pushed to -testing and then to -updates is the latest build of a particular EVR tuple? Just curious, since buildtime seems a reasonable thing to use for that sort of thing. Or, for that matter, that the QA feedback is for the correct build of a particular EVR for a package?
There are three things here that are important :
1) the QA team guys are very wired up to this process - and are very aware of the fact that packages will change with no EVR change; they are also aware of the fact that the testing that were doing is from <whats out there now> => <what we want to put out there > ( as opposed to <what we tested yesterday> => < what we want to test today> )
2) We only ever sign packages once they are accepted; and only signed packages are pushed publicly
3) Every package build ( so srpm -> binary rpm conversion ) runs with a uniq build-tag; the buildtag has no relation to package name-E:VR.arch or anything like that. Its a purely generated ID. So its still possible to go back and get stats, logs etc for every build that we did. irrespective of what we did ( eg: there are lots of things that were exclusive arch: s390 :D )
None of that probably answers your question. So here is how the 'move to repo happens'. By hand :) or the system is smart enough to replace all binary rpms with newer builds if the srpm metadata is the same. So you can ever only have one <name>.<arch> in the os/repo as output from one srpm; and their buildid needs to match ( which will handle corner cases like a subpackage going away etc ); for updates/<arch>/repo there can only be one name-E:VR.arch from one source rpm; and since the test is only run when a new set of binaries are output, the latest will always win[1]
- KB
[1]: this is not always true or the desired result, and needs manual intervention.
On Thursday, March 24, 2011 04:44:43 pm Karanbir Singh wrote:
- the QA team guys are very wired up to this process
[snip]
- We only ever sign packages once they are accepted; and only signed
packages are pushed publicly
[snip]
None of that probably answers your question.
No, these two items do answer what I really wanted to know very nicely, at least to my satisfaction, thanks much for taking the time to answer. In a nutshell, the QA people only ever work with unsigned packages, you keep them tagged with unique tag ID's, and when it's signed it's Golden.
While this thread has taken a few sweeping off-topic turns (not a small number of which I'm sure I'm at least partially responsible for), I've learned yet a little more about these processes, and I hope others have as well.
On Thursday, March 24, 2011 08:51:30 am Les Mikesell wrote:
Also note that you can't just 'yum update' from those SL alpha versions to the final release, so even if CentOS did ship alpha/beta versions it wouldn't make life that much easier.
The problem with updates from a rolling alpha/beta to the GA release is that it is possible that the package contents change but the epoch:version-release tuple doesn't; especially given a rebuild of the upstream source package (which cannot have its EVR changed and maintain strict compatibility) due to things like the libtalloc versioning situation previously mentioned.
I haven't checked to see if it's implemented in EL6, but this sounds like a situation tailor-made for yum distro-sync. I'm not sure, however, how distro-sync acts when packages are actually different but their EVR stays the same.
On Thu, 2011-03-24 at 09:55 -0400, Lamar Owen wrote:
On Thursday, March 24, 2011 08:51:30 am Les Mikesell wrote:
Also note that you can't just 'yum update' from those SL alpha versions to the final release, so even if CentOS did ship alpha/beta versions it wouldn't make life that much easier.
The problem with updates from a rolling alpha/beta to the GA release is that it is possible that the package contents change but the epoch:version-release tuple doesn't; especially given a rebuild of the upstream source package (which cannot have its EVR changed and maintain strict compatibility) due to things like the libtalloc versioning situation previously mentioned.
This is kind of a unique situation, in that it's pretty well accepted that if you are rebuilding for any reason you need to change at least the release in some way. I kind of understand why you don't want to, but I'm not sure it's worth the problem of having M versions of a particular NEVRA. Does this happen anytime after initial distro. creation?
I haven't checked to see if it's implemented in EL6, but this sounds like a situation tailor-made for yum distro-sync. I'm not sure, however, how distro-sync acts when packages are actually different but their EVR stays the same.
As above, we currently mostly only look at NEVRA for identifying packages in any part of yum. For rpmdbv's we also look at the checksum, so I had thought about doing something so you could more easily "reinstall" anything with changed checksums:
import yum yb = yum.YumBase() for ipkg in sorted(yb.rpmdb): for apkg in yb.pkgSack.searchPkgTuple(ipkg.pkgtup): if ('checksum_type' in ipkg.yumdb_info and 'checksum_data' in ipkg.yumdb_info): if ipkg.yumdb_info.checksum_data != apkg.pkgId: print "Diff chksum:", ipkg, ipkg.ui_from_repo, apkg.repoid print " :", ipkg, ipkg.yumdb_info.checksum_type, ipkg.yumdb_info.checksum_data print " :", apkg, apkg.checksum_type, apkg.pkgId else: print "No chksum:", ipkg, ipkg.ui_from_repo, apkg.repoid
...that does have some minor downsides though. For instance changing the repo. from "sha" to sha256 would fake the above into reinstalling everything (but then again, doing that would break rpmdbv compatibility, and so would be very bad). It also currently changes if the signing key changes, but again you'd probably want that. Probably the most significant downside is that you can't sort based on that, you can only tell that X is different from Y not which is preferred but then I'd _really_ want to discourage people publishing multiple NEVRAs that are identical ... and one assumes that when you are doing your builds you rarely (if ever?) want to go backwards.
In theory we could also use "buildtime", but I'm very wary of making that into a minor release id as it would encourage the multiple NEVRAs problem and it is much less linear in other distributions (although they'd tend to respect NEVRA :). Also worth noting is that the one NEVRA dup. with different checksums I can see on F13 is libjingle in updates and fedora-chromium ... and those have the same buildtime (I assume spot just resigned it).
On Thursday, March 24, 2011 11:58:03 am James Antill wrote:
This is kind of a unique situation, in that it's pretty well accepted that if you are rebuilding for any reason you need to change at least the release in some way.
The problem, at least for CentOS, becomes the stated policy of no changes to the upstream source RPM unless absolutely necessary. Changing the release is a change to the upstream source RPM, and doesn't qualify as absolutely necessary.
I kind of understand why you don't want to, but I'm not sure it's worth the problem of having M versions of a particular NEVRA. Does this happen anytime after initial distro. creation?
It would happen any time a source RPM would have to be rebuilt due to binary linkage differences; the initial build(s) that had the wrong linkage version(s) and the final, binary compatible confirmed build would have the same EVR but different contents and different dependencies. As long as a 'cannot change the source RPM' policy is to be followed this will be true. This is true specifically for the problem RPM with the wrong libtalloc dependency that has been mentioned, for update 6 of C5 (I really, really, really strongly dislike the use of a minor version to reflect the upstream's quarterly large updates).
As above, we currently mostly only look at NEVRA for identifying packages in any part of yum. For rpmdbv's we also look at the checksum, so I had thought about doing something so you could more easily "reinstall" anything with changed checksums:
[snip code segment]
Probably the most significant downside is that you can't sort based on that, you can only tell that X is different from Y not which is preferred but then I'd _really_ want to discourage people publishing multiple NEVRAs that are identical ... and one assumes that when you are doing your builds you rarely (if ever?) want to go backwards.
Methinks that is the strongest argument against a CentOS alpha or beta being published, since by definition there are going to be exactly that: two or more packages with identical NEVRA tuples with different contents and different checksums. I say by definition since it is one of the stated goals of CentOS to not change the upstream source package in any way unless a trademark issue or in the case of kernel signing where it's necessary. If you can't change the source RPM you can't change NEVRA. Unless you want to tag every single RPM in the distribution with a distribution tag.....or a repository tag....
Please read Connie Sieh's first paragraph, and Troy's immediately previous paragraph, in a December 17, 2010 posting to scientific-linux-devel, found in their archives at: http://listserv.fnal.gov/scripts/wa.exe?A2=ind1012&L=scientific-linux-de...
And while I could easily quote that here, I think reading the whole e-mail, including the quoted lines from Troy, will be beneficial.
In theory we could also use "buildtime", but I'm very wary of making that into a minor release id as it would encourage the multiple NEVRAs problem and it is much less linear in other distributions (although they'd tend to respect NEVRA :).
Well, other distributions have control of their own source RPM's NEVRA; rebuilding from source RPM by default reliquishes control of NEVRA to upstream. So if you don't get the build right on the first go, you are going to have more than one package with the same NEVRA, and either buildtime or checksum will show the difference; buildtime is more desireable, I would think. If the distribution is being built out of an SCM or full buildsystem like koji, and the upstream is not already built into a NEVRA-frozen source RPM, then you don't have this problem. This is only a problem when you're rebuilding from somebody else's source RPMs.
Or when you're maintaining an upstream RPM set and find out that you didn't update before churning out that last build..... you get a quandary of having to bump release because you forgot to issue yum update before building. With mock, of course, that problem goes away, but I maintained RPMs before mock (or mach) existed, and I tried to always bump release for every single build, regardless. Of course, then you have to put in a changelog line and explain the rebuild.... and many times it was just tempting to not bump release and just throw the rebuild up there, slipstreaming it into the ftp site....
Also worth noting is that the one NEVRA dup. with different checksums I can see on F13 is libjingle in updates and fedora-chromium ... and those have the same buildtime (I assume spot just resigned it).
Could be.
On Thu, 2011-03-24 at 13:19 -0400, Lamar Owen wrote:
On Thursday, March 24, 2011 11:58:03 am James Antill wrote:
This is kind of a unique situation, in that it's pretty well accepted that if you are rebuilding for any reason you need to change at least the release in some way.
The problem, at least for CentOS, becomes the stated policy of no changes to the upstream source RPM unless absolutely necessary. Changing the release is a change to the upstream source RPM, and doesn't qualify as absolutely necessary.
Right. Personally, given that you aren't shipping the exact binaries that upstream ship, I would say that all CentOS builds should start with a .1 added to the end of release, and then increment that for any rebuilds you need to do. I'd also have the packages provide the upstream NEVR, as well. To me that would still satisfy the "don't unnecessarily alter specfiles" mission and all of the "we changed FOO.bin but can't reflect that in FOO.src" problems go away.
Of course this technically is changing the .spec ... and there _could_ be problems with 3rd party rpms requires/conflicts/obsoletes, but it seems like a much cleaner solution. Obviously changing this before CentOS-7 would be complete insanity, but I think it's worth considering for then.
I kind of understand why you don't want to, but I'm not sure it's worth the problem of having M versions of a particular NEVRA. Does this happen anytime after initial distro. creation?
It would happen any time a source RPM would have to be rebuilt due to binary linkage differences; the initial build(s) that had the wrong linkage version(s) and the final, binary compatible confirmed build would have the same EVR but different contents and different dependencies. As long as a 'cannot change the source RPM' policy is to be followed this will be true. This is true specifically for the problem RPM with the wrong libtalloc dependency that has been mentioned, for update 6 of C5 (I really, really, really strongly dislike the use of a minor version to reflect the upstream's quarterly large updates).
Right, sorry, I worded my question badly. What I meant to ask is:
I understand that you probably rebuild the same NEVRAs "a lot" during the initial distro. creation, as you are guessing about what the build environment of each SRPM was. I also understand that this happens "sometimes" for updates, Eg. libtalloc or util-linux, but ... how often is this a problem for updates? Is it 1-2 packages a year, 1-2 a month, more?
As above, we currently mostly only look at NEVRA for identifying packages in any part of yum. For rpmdbv's we also look at the checksum, so I had thought about doing something so you could more easily "reinstall" anything with changed checksums:
[snip code segment]
Probably the most significant downside is that you can't sort based on that, you can only tell that X is different from Y not which is preferred but then I'd _really_ want to discourage people publishing multiple NEVRAs that are identical ... and one assumes that when you are doing your builds you rarely (if ever?) want to go backwards.
Methinks that is the strongest argument against a CentOS alpha or beta being published, since by definition there are going to be exactly that: two or more packages with identical NEVRA tuples with different contents and different checksums.
Personally I doubt a beta would be useful anyway. As I don't think people want a CentOS beta released quickly ... they "just" want a CentOS release released quickly, and if you release a beta (esp. with the same NEVRAs as release) they'll either ignore it or treat it as a release.
Unless you want to tag every single RPM in the distribution with a distribution tag.....or a repository tag....
Please read Connie Sieh's first paragraph, and Troy's immediately previous paragraph, in a December 17, 2010 posting to scientific-linux-devel, found in their archives at: http://listserv.fnal.gov/scripts/wa.exe?A2=ind1012&L=scientific-linux-de...
Yeh, this is the above "we change FOO.bin, but don't want to change FOO.src" problem. I understand, but I'm still tempted to say "don't do that then".
On Thursday, March 24, 2011 01:52:24 pm James Antill wrote:
Yeh, this is the above "we change FOO.bin, but don't want to change FOO.src" problem. I understand, but I'm still tempted to say "don't do that then".
Distilling the whole mesage down to this, as this is the core issue.
I personally used the decimal release scheme when doing the PostgreSQL RPMs. While it's been a long time ago, I think I used a 0.x release while developing and honing the packaging, and rebuilds of a release where nothing was actually changed in the spec (other than the release) got a decimal, but, like I say, it has been a long time.
But the more critical question is 'would upstream ever use a decimal release number that might conflict with mine?'
On Thu, 2011-03-24 at 14:39 -0400, Lamar Owen wrote:
On Thursday, March 24, 2011 01:52:24 pm James Antill wrote:
Yeh, this is the above "we change FOO.bin, but don't want to change FOO.src" problem. I understand, but I'm still tempted to say "don't do that then".
Distilling the whole mesage down to this, as this is the core issue.
I personally used the decimal release scheme when doing the PostgreSQL RPMs. While it's been a long time ago, I think I used a 0.x release while developing and honing the packaging, and rebuilds of a release where nothing was actually changed in the spec (other than the release) got a decimal, but, like I say, it has been a long time.
But the more critical question is 'would upstream ever use a decimal release number that might conflict with mine?'
I do not speak for upstream. I would guess that (if they care at all) upstream would be happy to have you use different NEVRAs, and certainly wouldn't go out of their way to choose NEVRAs that are the same.
Certainly it seems like you should be pretty safe if you did something like:
Upstream: foo-1.2.4-8.el7 CentOS: foo-1.2.4-8.el7_0.0c1 CentOS: foo-1.2.4-8.el7_0.0c2
Upstream: foo-1.2.4-8.el7_0.1 CentOS: foo-1.2.4-8.el7_0.1_0.0c1 CentOS: foo-1.2.4-8.el7_0.1_0.0c2
...and that should provide the same level of "upgrade compatibility" you have now (every CentOS build would upgrade to the next upstream build).
On 03/24/2011 08:14 PM, James Antill wrote:
Upstream: foo-1.2.4-8.el7 CentOS: foo-1.2.4-8.el7_0.0c1 CentOS: foo-1.2.4-8.el7_0.0c2
This would make a lot of people, very angry :)
Specially the crazy vendors who use pkg names and payload content matching to workout what platform they are running on and conditionals that result from it
we do change tag's for packages we need to change, or force a user side bump on;
- KB
On 03/24/2011 05:52 PM, James Antill wrote:
Right. Personally, given that you aren't shipping the exact binaries that upstream ship, I would say that all CentOS builds should start with a .1 added to the end of release, and then increment that for any rebuilds you need to do. I'd also have the packages provide the upstream NEVR, as well.
We do that already, but it depends on what state its in and why there was a bump and if the package has made it to user machines. We never rebuild or push a package that will need to be considered with buildtime etc.
- KB
On 03/24/2011 01:52 PM, James Antill wrote:
Right. Personally, given that you aren't shipping the exact binaries that upstream ship, I would say that all CentOS builds should start with a .1 added to the end of release, and then increment that for any rebuilds you need to do. I'd also have the packages provide the upstream NEVR, as well. To me that would still satisfy the "don't unnecessarily alter specfiles" mission and all of the "we changed FOO.bin but can't reflect that in FOO.src" problems go away.
This policy would be murder for end users that have to maintain compliance with various requirements such as SOX404, PCI, etc.
If you really wanted to modify the release number, while likely maintaining ABI compatibility, you would in many ways be a new distro based on upstream sources.
As much as it adds a bunch of fun for the CentOS devs rebuilding the same package differently to get it to match upstreams binary while maintaining the same NVR, it's just the nature of the beast.
On 03/24/2011 03:58 PM, James Antill wrote:
I kind of understand why you don't want to, but I'm not sure it's worth the problem of having M versions of a particular NEVRA. Does this happen anytime after initial distro. creation?
no, not in CentOS - once a package is public, every change will bring a EVR bump.
- KB
carlopmart wrote:
On 03/23/2011 10:27 AM, Ljubomir Ljubojevic wrote:
Scientific Linux uses upstream source to create their own repo, without desire to be 100% compatible.
CentOS project is dedicated to provide (as close as possible) 100% compatibility. It's not just a rebuild of upstream sources, goal is tu *duplicate* RHEL.
It's that simple. And this was answer many times in this and other mailing lists, forum threads....
Ljubomir
I know that SL includes some custom components like OpenAFS in their distribution, but base system is the same as CentOS. Then, I repeat, why not?
Then, I repeat, because SL *does not care* to build 100% *binary* compatible packages, fo r CentOS it's a must.
Look at it this way. Upstream is a Coca-Cola Co. SL is Pepsi. They use publicly available formulas from upstream in order to create product that is as good as upstreams, but is not *the 100% same* since their production formulas are not ***100%/absolutely*** the same.
In this analogy, CentOS is the industrial espionage guy who constantly steals new formulas from upstream in order to create **exact/100%** replicas of upstream product. For better analogy, lets say that acquiring that formula is illegal.
So, how do you propose that Pepsi and counterfeit work together.
P.S. "counterfeit" is extremely wrong to say for CentOS, but analogy demands clear distinction of persons involved, so I beg all to forgive me for this.
Ljubomir
On 03/23/2011 11:00 AM, Ljubomir Ljubojevic wrote:
carlopmart wrote:
On 03/23/2011 10:27 AM, Ljubomir Ljubojevic wrote:
Scientific Linux uses upstream source to create their own repo, without desire to be 100% compatible.
CentOS project is dedicated to provide (as close as possible) 100% compatibility. It's not just a rebuild of upstream sources, goal is tu *duplicate* RHEL.
It's that simple. And this was answer many times in this and other mailing lists, forum threads....
Ljubomir
I know that SL includes some custom components like OpenAFS in their distribution, but base system is the same as CentOS. Then, I repeat, why not?
Then, I repeat, because SL *does not care* to build 100% *binary* compatible packages, fo r CentOS it's a must.
Sorry, but that is not what is said from the SL website.:
from "http://www.scientificlinux.org":
"The base SL distribution is basically Enterprise Linux, recompiled from source.
Our main goal for the base distribution is to have everything compatible with Enterprise, with only a few minor additions or changes. An example of of items that were added are Alpine, and OpenAFS.
Our secondary goal is to allow easy customization for a site, without disturbing the Scientific Linux base. The various labs are able to add their own modifications to their own site areas. By the magic of scripts, and the anaconda installer, each site is to be able to create their own distributions with minimal effort. Or, if a users wishes, they can simply install the base SL release."
And from SL FAQ:
"Q. What is Scientific Linux?
A. Scientific Linux is in essence, a commercial enterprise linux distribution, recompiled. What we have done is taken the source code from Enterprise (in srpm form) and recompiled them. The resulting binaries (now in rpm form) are then ours to do with as we desire as long as we follow the license from that original source code, which we are doing. We then bundle all these binaries into a linux distribution that is as close to the commercial enterprise distribution as we can get it. The goal is to ensure that if a program runs and is certified on the commercial enterprise linux distribution, then it will run on the corresponding Scientific release."
Where is the difference here against CentOS??
Look at it this way. Upstream is a Coca-Cola Co. SL is Pepsi. They use publicly available formulas from upstream in order to create product that is as good as upstreams, but is not *the 100% same* since their production formulas are not ***100%/absolutely*** the same.
It is not a valid example
Compare those 2 sentances:
"The base SL distribution is basically Enterprise Linux, recompiled from source.
-snip-
"Q. What is Scientific Linux?
A. Scientific Linux is in essence, a commercial enterprise linux distribution, recompiled.
and compare it with this statement:
"CentOS conforms fully with the upstream vendors redistribution policy and aims to be 100% binary compatible. (CentOS mainly changes packages to remove upstream vendor branding and artwork)."
For me, "basically" and "in essence" is not as same as "aims to be 100% binary compatible"
If you can not see and understand, then I, or anyone else, can not help you to understand the difference.
Ljubomir
On 03/23/2011 11:17 AM, Ljubomir Ljubojevic wrote:
Compare those 2 sentances:
"The base SL distribution is basically Enterprise Linux, recompiled from source.
-snip-
"Q. What is Scientific Linux?
A. Scientific Linux is in essence, a commercial enterprise linux distribution, recompiled.
Please don't cut the paragraph where you're interested. Add:
Our main goal for the base distribution is to have everything compatible with Enterprise, with only a few minor additions or changes. An example of of items that were added are Alpine, and OpenAFS.
and compare it with this statement:
"CentOS conforms fully with the upstream vendors redistribution policy and aims to be 100% binary compatible. (CentOS mainly changes packages to remove upstream vendor branding and artwork)."
Both statements says the same with different words ... SL says "compatible" too, like CentOS ...
On Wed, Mar 23, 2011 at 11:43:53AM +0100, carlopmart wrote:
Both statements says the same with different words ... SL says "compatible" too, like CentOS ...
Honestly... This is a development list. If you need to have the concept of binary compatibility explained to you then I fear you are in the wrong place.
John
On 03/23/2011 11:48 AM, John R. Dennison wrote:
On Wed, Mar 23, 2011 at 11:43:53AM +0100, carlopmart wrote:
Both statements says the same with different words ... SL says "compatible" too, like CentOS ...
Honestly... This is a development list. If you need to have the concept of binary compatibility explained to you then I fear you are in the wrong place.
Honestly ... I know the meaning of the concept of "binary compatible". I don't understand is where you see the difference between CentOS and SL about this. Where is the difference?
2011/3/23 carlopmart carlopmart@gmail.com:
On 03/23/2011 11:48 AM, John R. Dennison wrote:
On Wed, Mar 23, 2011 at 11:43:53AM +0100, carlopmart wrote:
Both statements says the same with different words ... SL says "compatible" too, like CentOS ...
Honestly... This is a development list. If you need to have the concept of binary compatibility explained to you then I fear you are in the wrong place.
Honestly ... I know the meaning of the concept of "binary compatible". I don't understand is where you see the difference between CentOS and SL about this. Where is the difference?
carlopmart, please go to the -users list with this, one of the CentOS devs actually posted a relevant example of the SL/CentOS differences earlier today: http://lists.centos.org/pipermail/centos/2011-March/108389.html
Best regards Kenni
On 03/23/2011 12:11 PM, Kenni Lund wrote:
2011/3/23 carlopmartcarlopmart@gmail.com:
On 03/23/2011 11:48 AM, John R. Dennison wrote:
On Wed, Mar 23, 2011 at 11:43:53AM +0100, carlopmart wrote:
Both statements says the same with different words ... SL says "compatible" too, like CentOS ...
Honestly... This is a development list. If you need to have the concept of binary compatibility explained to you then I fear you are in the wrong place.Honestly ... I know the meaning of the concept of "binary compatible". I don't understand is where you see the difference between CentOS and SL about this. Where is the difference?
carlopmart, please go to the -users list with this, one of the CentOS devs actually posted a relevant example of the SL/CentOS differences earlier today: http://lists.centos.org/pipermail/centos/2011-March/108389.html
Best regards Kenni
I don't doubt about Johnny Hughes says in his email, but there is an important point is not to taken into account: SL5.6 is not released and Johnny makes his comparision between SL and CentOS 5.6.
ok, then the principal two reasons to don't fusion CentOS and SL are:
a) SL is not "binary compatible"
b) SL binaries are linked in different manner than TUV does.
?? am I right??
On 03/23/2011 06:24 AM, carlopmart wrote:
On 03/23/2011 12:11 PM, Kenni Lund wrote:
2011/3/23 carlopmartcarlopmart@gmail.com:
On 03/23/2011 11:48 AM, John R. Dennison wrote:
On Wed, Mar 23, 2011 at 11:43:53AM +0100, carlopmart wrote:
Both statements says the same with different words ... SL says "compatible" too, like CentOS ...
Honestly... This is a development list. If you need to have the concept of binary compatibility explained to you then I fear you are in the wrong place.Honestly ... I know the meaning of the concept of "binary compatible". I don't understand is where you see the difference between CentOS and SL about this. Where is the difference?
carlopmart, please go to the -users list with this, one of the CentOS devs actually posted a relevant example of the SL/CentOS differences earlier today: http://lists.centos.org/pipermail/centos/2011-March/108389.html
Best regards Kenni
I don't doubt about Johnny Hughes says in his email, but there is an important point is not to taken into account: SL5.6 is not released and Johnny makes his comparision between SL and CentOS 5.6.
ok, then the principal two reasons to don't fusion CentOS and SL are:
a) SL is not "binary compatible"
b) SL binaries are linked in different manner than TUV does.
?? am I right??
I never said that SL does anything wrong. And yes, they have not "released" 5.6 yet, so it might be fixed then.
If people want CentOS, they should use CentOS. If they want Scientific Linux, they should use Scientific Linux.
I was only pointing out that one can find fault with any distribution.
I do not know what exact process is for SL ... I do know what it is for CentOS.
For the record, neither CentOS nor SL is 100% compatible with the upstream product. Ned Slider pointed out that some of our Kernel modules are build against some different than upstream because they built on a version of the kernel that they did not release. We do not have access to that kernel source code, so we built against a released kernel.
The real issue is that the upstream product is not necessarily self hosting and they sometimes build against stuff that is not released. That is just how it is. Then again, they never said their goal was to produce a self hosting distribution. They also do not need to provide every iteration of their software that resides on the build system, and they do provide more information than any other enterprise vendor.
There is no reason to slam SL to build up CentOS (or vice versa) ... both CentOS and SL are quality free enterprise distributions. Each can be used for enterprise level server or workstation uses.
I can say that if I was not using CentOS then I would be using SL.
on 3/23/2011 3:00 AM Ljubomir Ljubojevic spake the following:
carlopmart wrote:
On 03/23/2011 10:27 AM, Ljubomir Ljubojevic wrote:
Scientific Linux uses upstream source to create their own repo, without desire to be 100% compatible.
CentOS project is dedicated to provide (as close as possible) 100% compatibility. It's not just a rebuild of upstream sources, goal is tu *duplicate* RHEL.
It's that simple. And this was answer many times in this and other mailing lists, forum threads....
Ljubomir
I know that SL includes some custom components like OpenAFS in their distribution, but base system is the same as CentOS. Then, I repeat, why not?
Then, I repeat, because SL *does not care* to build 100% *binary* compatible packages, fo r CentOS it's a must.
Look at it this way. Upstream is a Coca-Cola Co. SL is Pepsi. They use publicly available formulas from upstream in order to create product that is as good as upstreams, but is not *the 100% same* since their production formulas are not ***100%/absolutely*** the same.
In this analogy, CentOS is the industrial espionage guy who constantly steals new formulas from upstream in order to create **exact/100%** replicas of upstream product. For better analogy, lets say that acquiring that formula is illegal.
So, how do you propose that Pepsi and counterfeit work together.
P.S. "counterfeit" is extremely wrong to say for CentOS, but analogy demands clear distinction of persons involved, so I beg all to forgive me for this.
Ljubomir
It would be more like, CentOS uses freely available list of ingredients, but without the secret recipe. So they work and work until the end product is as close as humanly possible. SL takes those ingredients and when they get "close enough" they say "done!"
I say if anyone thinks SL is better... Don't let the virtual door hit you in the nethers on your way out... If you want CentOS... Just be patient. Getting 4.9 and 5.6 working first was a good and valid decision, as a new deployment of a new and untested distro is usually lower on the priority list anyway. The unpaid and overworked staff is working as fast as they can...
On 03/23/2011 09:13 AM, carlopmart wrote:
Hi all,
Please, first of all, I don't want to start a flame about this subject. I only wnat to know CentOS's developers point of view about this particular.
we looked at this in the past and :
- SL and CentOS have different goals, and neither of us are keen on adapting the other's - its good to have two projects, its keep user options open
thats about it.
- KB
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Karanbir Singh said the following on 23/03/11 13:07:
- SL and CentOS have different goals, and neither of us are keen on
adapting the other's
- its good to have two projects, its keep user options open
It could be added here
http://www.centos.org/modules/smartfaq/category.php?categoryid=2
to avoid further useless thread
Ciao, luigi
- -- / +--[Luigi Rosa]-- \
You know you've landed gear-up when it takes full power to taxi.
On 03/23/2011 01:07 PM, Karanbir Singh wrote:
On 03/23/2011 09:13 AM, carlopmart wrote:
Hi all,
Please, first of all, I don't want to start a flame about thissubject. I only wnat to know CentOS's developers point of view about this particular.
we looked at this in the past and :
- SL and CentOS have different goals, and neither of us are keen on
adapting the other's
Ok, that's a good reason.
Thanks Karanbir.
On 03/23/2011 07:07 AM, Karanbir Singh wrote:
On 03/23/2011 09:13 AM, carlopmart wrote:
Hi all,
Please, first of all, I don't want to start a flame about this subject. I only wnat to know CentOS's developers point of view about this particular.
we looked at this in the past and :
- SL and CentOS have different goals, and neither of us are keen on
adapting the other's
- its good to have two projects, its keep user options open
thats about it.
- KB
I would also like to make sure that no one on this list thinks that we have anything but the utmost respect for the Scientific Linux project. The devels over there are very nice and we have worked with them in the past and I am sure will continue to do so in the future.
Connie and Troy are very knowledgeable and they produce a quality product. If I was not using CentOS, I would certainly be using Scientific Linux.
-
Akemi Yagi -
Alexander Ustinov -
carlopmart -
Dag Wieers -
David Hollis -
James Antill -
Jeff Johnson -
Jeff Sheltren -
John R. Dennison -
Johnny Hughes -
Karanbir Singh -
Kenni Lund -
Lamar Owen -
Les Mikesell -
Ljubomir Ljubojevic -
Luigi Rosa -
Manuel Wolfshant -
Nico Kadel-Garcia -
Rainer Traut -
Ray Van Dolson -
Scott Silva -
seth vidal -
Simon Matter -
Устинов Александр Александро вич