[CentOS-devel] Repository structures for SIG and variants in the future

[Ljubomir Ljubojevic]

On 01/17/2014 06:06 PM, Jim Perrin wrote:
> On 01/17/2014 10:06 AM, Ljubomir Ljubojevic wrote:
>> On 01/17/2014 02:35 PM, Jim Perrin wrote:
>>> On 01/17/2014 07:08 AM, Manuel Wolfshant wrote:
>>>
>>>> make it 5 or 10, please. one might want to supersede base packages with
>>>> ones from private/local repos. hplip for instance was such a candidate
>>>> for a long time, my printers were not supported by the stock version of
>>>> hplip from EL5 and I had to compile a newer one from fedora. and at the
>>>> time I was not yet relying on include/excludepkgs
>>>>
>>>
>>>
>>> This touches exactly on the point for why I don't want to include
>>> priorities by default. To make it useful for repos that provide newer
>>> software than what exists in base/updates (php, httpd, libvirt,
>>> whatever), you're automatically working against priorities. It doesn't
>>> matter if it's for a local repository or for a SIG.
>>>
>>>
>>
>> So how do you intend to solve this without priorities? Especially if
>> there is a need for some packages to be compiled with different flags?
>
> Ideally by eliminating package duplication for the SIG/repos who work
> with us, but that's an ivory tower goal that's likely unattainable.
>
> It will likely be a combination of working with SIGs to coordinate and
> cooperate, changing to SCL builds where possible and appropriate, and
> documentation to better educate the users.
>
> SCL builds provide a nice name-spaced way to have multiple versions of
> the same packages installed. For example, python27, python33, multiple
> php and httpd versions, etc. All of which are mostly impractical with
> the older rpm packaging style (see php and httpd for example). This is
> by no means a silver bullet, but it does solve a fair chunk of the issue.
>
>
>> One example is for those using Virtualmin. Virtualmin provides http,
>> postfix and few other packages compiled little differently (different
>> flags) then RH does, in their own repository. So if both repositories
>> are of same priority, when RH publishes newer version it is
>> automatically an update to modified version. And that can brake things
>> until Virtualmin releases their own version. If we "hide" RH versions
>> with higher priority to Virtualmin packages then nothing can be broken.
>
>
> These could easily be scl enabled packages. Keep in mind we have a
> scenario very much like this with the Clear and Neth groups, who seem to
> be very interested in working with us.
>
>> Same rule will apply to any package any Variant will have to compile
>> with different flags, including CentOSPlus kernels. If someone uses
>> CentOSPlus kernel because regular RH kernel does not have drivers for
>> their hardware, and happens that CentOSPlus kernel is not released the
>> same time as regular kernel (even 1-5h can make a difference), problem
>> may arise where update to regular kernel fails and we have a unhappy
>> user. With newbies around (I hope we attract in greater numbers), that
>> can be troublesome.
>
> I'm not sure this is the best example to address your point, but I do
> understand your meaning. This problem is not unique to CentOS, and I
> don't consider it to be a core distribution or board problem. The SIGs
> and Variants are up to their respective communities to maintain and keep
> current. If they falter, we'll certainly try to help but we're not going
> to take over the maintenance of a failing SIG.

Being proactive and being prepared are two different things. Taking over 
SiG maintenance would be proactive approach. Enabling Variants and 3rd 
party repositories to have safety net would be being accommodating 
while Variants and 3rd party repositories (Va3pr? :) ) would be prepared.

>
>> -----------------------------------
>> But, as I said, for now I will be happy with "Priorities=" and
>> "Enabled=" for every repo's config. It will make many things easier,
>> like creating script to change those numbers per users request (you do
>> not have to host it). Adding them to every new config (after
>> centos-release is updated) is not desired.
>
>
> Enabled, we can likely accommodate, and I'll discuss this with the
> others. Priorities, I'm not sold on.
>
>
>
>> ----------------------------------
>>
>> With new opportunities creating CentOS Variants, I already thought of
>> few things I would ask of yum project to incorporate, to further provide
>> out-of-the-box experience for Variants users.
>
> You seem to be quite passionate for the laptop/desktop users. Why not
> participate or create a SIG with that target audience? You could tailor
> this exactly as you wish with the packages (assuming legally
> distributable) you wish.

I built Desktop quasi-Variant of CentOS back in 2008 I think. I created 
my own repository and mirrored all the mayor repositories. I have them 
mirrored even now. For example, I repacked static Skype 2.x for CentOS 5 
to rpm, I was the only one to have it in a repository. I also recompiled 
32-bit kvm L. Farkas built and used it for several years. And I rebuilt 
any Fedora package I needed, somewhere 50-100 rpm's overall. My focus 
was solely on desktop usage. I use exclusively CentOS for Desktops and 
Laptops for those 6 years. And I battle with 3rd party repositories 
including my own all those 6 years. I tried numerous solutions and 
variations, but every one failed due to CentOS restrictions and 
compilations. In those 6 years, only way to provide out-of-the box 
experience, but not changing centos-release/CentOS-Base.repo involved 
guiding newbies to login as root and start editing CentOS-Base.repo 
after they install my plnet-release package. Tiresome and discouraging job.

In January 2012 I created "DentOS" brand with full set of .repo files 
for all popular 3rd party repos and a script that will be able to switch 
.repo files among different sets (in subdirectories), just like I 
described in one of earlier mails. I would move CentOS-Base.repo to 
backup subdirectory and replace it with my own .repo files with 
incorporated "Priority=" and "Enabled=", with priorities set in such a 
way so Tier3 repositories including Firefox, Thunderbird, etc. packages 
have greater priority then base, updates, etc. That way there are no 
surprises on the Desktop systems I maintain for other people.

If you ask why they do not have same priority, reason is for example 
gstreamer version. gstreamer packages in CentOS have one version, and 
all gstreamer-* packages from 3rd party repositories (with legal issues) 
have to keep up with main gstreamer version, or they will not work. So 
"yum update" will fail when gstreamer package changes version since 
plugins must be compiled for exactly that version. And if newbie can not 
update it's system without problems, GUY IN SUPPORT has a problem. 
Responce from CentOS will be/was take it with 3rd party repo, user will 
get irritated and will talk about it.

 From 2008 to 2010 I was on LinuxQuestions.org forums, only in 
RHEL/CentOS sections providing help to users (DrLove73).
At the same time I was on every Linux forum in my language supporting 
and promoting CentOS, even as a Desktop solution, explaining 3rd party 
repositories.
Roughly from 2010 to 2012 I was on CentOS forums providing help (DrLove73).
In 2012 took on official CentOS Facebook group as main/regular admin. 
Before I started in CentOS group, I was one of the admins in biggest 
Linux group on Facebook, promoting CentOS and arguing pro/against with 
pro-Debian members there. Today CentOS group passed 6100 members, vast 
majority are newbies, some barely know what Linux is.

So I have WAST experience with newbies, their expectations and what they 
are looking for. And it is NOT current CentOS with problematic setting 
up 3rd party repositories and solving conflicts. If you ask people on 
CentOS forums (and other where someone supports CentOS), you will find 
out that newbies biggest problems revolve around 3rd party repositories 
providing "forbidden" packages.

And so we came to the main problem regarding Desktop SiG.

If CentOS project want's bigger adoption amongst newbies, it has to have 
Desktop Variant that "just work", providing easy add-on model for 3rd 
party repositories carrying "forbidden" packages. Otherwise newbies will 
use Mint/Ubuntu/Debian for Desktop and Mint/Ubuntu/Debian for server. 
They will not bother with learning both Debian and CentOS way. And keep 
in mind that Windows XP is expiring, Windows 7 is off the shelf's, and 
people hate Windows 8. So many newbies are already shopping for 
Desktop/Laptop Linux distro, and many will follow. Once first ones 
decide to take a Debian route, they will want Debian on servers also.

So if this crucial problem is not solved having in mind easy add-on 
solution without manual interventions, there is not reason what so ever 
in even creating Desktop SiG. CentOS will remain server distro for 
geeks, just like 90% of free-Linux-distro users think today.

If I were smart enough to chose Debian 9 years ago for my server in 
stead of ClarkConnect, I could have made easy living installing and 
maintaining Ubuntu/Mint systems in my area.

>
>> With all of my suggestions so far, I think something like "Skip=" for
>> every repo would be useful for those who use their internal mirrors.
>> If you set "Enabled=0" for some repo, using "--enablerepo=*" will enable
>> it. Another use is to have installed but disabled one or more 3rd party
>> repos, so you can quickly locate and install some needed package. No
>> need for search around internet, double checking.
>
> There is a skip_if_unavailable option, but I'm not sure that's what
> you're getting at. I believe fedora was also working on some sort of
> package library that would allow you to search directly. Perhaps this is
> something we could see about appropriating from them for our own uses.
>
>
>> Working around current "restrictions" in CentOS concerning 3rd party
>> repository manipulation is possible, but I think CentOS-Core should be
>> as close to Variants as possible in terms of repository syntax and use,
>> so we do not have to build (in-house or 3rd party) additional layer over
>> centos-release package or replace it all together (for Variants).
>>
>
> The formats and guidelines we're establishing will apply as equally as
> possible to all groups. However, I would much prefer overlay release
> files, similar to how there's an epel-release, elrepo-release,
> gluster-release, etc. This way one can 'at-a-glance' see what's in play
> with a simple 'yum list installed *-release'

That is my sentiment exactly. That is why I would like changes on the 
level of CentOS-Base.repo file, so it can be compatible with .repo files 
3rd party and Variants .repo files will carry, so CentOS-Base.repo file 
is changed as little as possible.

>
>
>> I will go over to yum mailing list and post my ideas there and see what
>> they say, and maybe you can talk about this with them on OfficeHours?
>>
>
> Certainly.
>
>


-- 
Ljubomir Ljubojevic
(Love is in the Air)
PL Computers
Serbia, Europe

StarOS, Mikrotik and CentOS/RHEL/Linux consultant