How about using pxe and cobbler instead.

Sent from my iPhone

On Dec 29, 2011, at 1:06 AM, 夜神　岩男 supergiantpotato@yahoo.co.jp wrote:

On 12/29/2011 02:09 PM, lowlux wrote:

...

I am also interested in doing this...

On Wed, Dec 28, 2011 at 11:21 PM, Larry Brigman <larry.brigman@gmail.com mailto:larry.brigman@gmail.com> wrote:

I see the small ISO images but I would like to build something custom but not a live image.

What tools do you use to create these images? Is there some documentation on building a custom iso spin? I would also like to be able to drive the tools.

I was just about to ask this exact same question. I use livecd-creator and some ks files to create livecds, and that's nice, but I have a need now to create a more complete network-boot installer which includes a lot of changes/different packages.

And I completely don't remember how to do it anymore. There is a utility for this, and its not livecd-creator (or the gui revisor thing, either) grrrr...

Anyone? _______________________________________________ CentOS-devel mailing list CentOS-devel@centos.org http://lists.centos.org/mailman/listinfo/centos-devel

On 12/29/2011 08:04 AM, Patrick Lists wrote:

On 29-12-11 12:32, Scott Rineer wrote:

...

How about using pxe and cobbler instead.

I don't think that a PXE/Cobbler setup is capable of creating an ISO image. At least I have never heard of nor read about this possibility.

Not to create an ISO, to allow for user specific kickstart installs.

There is no reason to create a dozen different ISO types with this or that set of packages. Use PXE and a separate kickstart file to create as many variant installs as you want from the main tree.

On 12/29/2011 11:20 PM, Jeff Sheltren wrote:

2011/12/28 夜神　岩男 <supergiantpotato@yahoo.co.jp mailto:supergiantpotato@yahoo.co.jp>

On 12/29/2011 02:09 PM, lowlux wrote:
 > I am also interested in doing this...
 >
 > On Wed, Dec 28, 2011 at 11:21 PM, Larry Brigman
<larry.brigman@gmail.com <mailto:larry.brigman@gmail.com>
 > <mailto:larry.brigman@gmail.com
<mailto:larry.brigman@gmail.com>>> wrote:
 >
 > I see the small ISO images but I would like
 > to build something custom but not a live image.
 >
 > What tools do you use to create these images?
 > Is there some documentation on building a custom iso spin?
 > I would also like to be able to drive the tools.

I was just about to ask this exact same question. I use livecd-creator
and some ks files to create livecds, and that's nice, but I have a need
now to create a more complete network-boot installer which includes a
lot of changes/different packages.

And I completely don't remember how to do it anymore. There is a utility
for this, and its not livecd-creator (or the gui revisor thing, either)
grrrr...

I agree with Scott that it sounds like pxe booting (you could also use the netboot iso/usb disk if you prefer) and kickstart sounds like it'll meet your needs. If you really want to create a custom installer, that's a bit more difficult, but you're probably thinking of the Fedora tools like Revisor.

http://fedoraproject.org/wiki/Spins_Custom

My intent was specifically to get away from livecds, which is all that Revisor can do. Its really buggy anyway (putting it lightly).

As a followup, the ISO creation suite for installers that underlies everything is part of anaconda-runtime. /usr/lib/anaconda-runtime/buildinstall is what I was looking for.

Getting at these tools was the whole point of my post. I'm not the OP here, but I think he was after the same thing.

buildinstall is a script that changes a directory with a dependency-complete package list into a bootable file system, ready to be iso9660'd -- but as an install media (what we want), *not* as a livecd (not what we want).

Cobbler stands on top of buildinstall, yumdownloader and a few other things, but doesn't seem to be geared towards installations as much as diskless PXE boot for live systems -- though ks can of course be used for mass installs, but has its own problems when given a lot of non-standard media types to install to and dual- and triple-boot systems. Unfortunately this is the problem I have right now.

Anyway, after having run buildinstall I get a noisy build -- noisier than I remember it being. The main problems have to do with thinks like awk not being linked correctly, or selinux policy files not being found, or especially install-info docs not being located. Tracking these down is a pain, because apparently the spec files indeed do *not* contain complete dependency information... which sucks, but is workable.

On 12/30/2011 01:02 AM, Larry Brigman wrote:

That sounds good. I have built ISO's with upstream packages that weren't available even from EPEL.

This is part of what we're doing in my situation. There are a lot of custom ERP/office automation, CAD, etc things that no distros include just yet, *and* the customer uses the systems for showroom stuff so rebranding (at least surface stuff that's readily visible) is an issue as well. There is also the matter of our Fluendo codecs and media things that can't go into a community distro because they have to get paid for somewhere along the way, and the applications that are being ported to PostgreSQL 9.1 from evil-empire type software...

Some of this goes beyond what ks can do alone, and having clean isos that the users can carry for demos and things go a long way to reducing confusion and manpower needs (sales != devepment types...).

I take it the tools are in anaconda-runtime or their dependency set?

 > Is there some documentation on building a custom iso spin?
 > I would also like to be able to drive the tools.

What does that mean 'drive the tools' ?

I have used buildinstall from anaconda-runtime package but trying my same recipe that I used from Centos 5 doesn't work. So I wonder, if I was giving the correct input to the tool or if the project has a set of scripts that provide that input in some manner that would better document the tools?

This command works so long as the ../Packages/ directory actually has all the packages necessary for the system to work correctly. This boots straight to the installer, no livecd stuff, and gives you the chance to install whatever is in Packages.

A command very similar to this is what gets fed automatically to /usr/lib/anaconda-runtime/buildinstall by other tools.

/usr/lib/anaconda-runtime/buildinstall --version 6 --brand $COMPANY --product $PRODUCTNAME --release "Tester" --output /home/iwao/remaster/ /home/iwao/remaster/Packages/

One quirk is you still have to do this as root. Looking for a way around that. Using revisor you have to disable selinux as well (eh?). So there are some weird things that I feel were never worked out properly because the drive to "virtualize everything!" got in the way of writing for proper system segregation at the user level. So this root/disable SELinux thing is likely a development oversight that was never straightened out, but may not be all that hard to fix.

I'm curious what the CentOS build infrastructure looks like. Fedora's is quite robust/complex but works pretty well considering the things they put it through from time to time.

On 12/30/2011 04:22 AM, Johnny Hughes wrote:

Well, when KB says we will add stuff, obviously he means we will add items that are GPL or other fully recognized OSS licenses that all us to obtain sources and properly redistribute the packages.

If it is things like the upstream supplementary disk items where one would need some kind of special agreement to distribute, then we can't do that.

Certainly! I what I was describing was the build situation where I need a simple way to build installation ISOs. That's just one situation, and the specific packages being assembled is quite a different problem from the issue of finding a simple ISO creation system that can be automated with profiles.

Maybe have different profiles inherit from a master ks or something. Not sure. Anyway...

Cobbler actually looks a bit promising, but I've yet to explore it more fully. It loooks like it can be simplified a lot, too -- there are some pretty weird error message conditions in the code that don't play nicely with SELinux and would probably confuse folks. I'm also unsure if it supports building an installer from scratch by just pointing it at a list of packages (which would need to include anaconda, of course). If it could do it from nothing but a list of src.rpms that would be great. I'll probably discover whether it can tonight, and if not what needs to be added. A koji+cobbler that is easy to setup and doesn't require root to run (other than initial setup) or turning selinux off would be ideal.

Some managers have problem with things like the installer displaying branding information that isn't *their* local brand. Some customers also get tickled by seeing their own logos and names flash by on the screen. Technically pointless? Yes. I sort of hate all that. *but* it does sell. Actually, if the Linux community understood why pixels and chicken lipstick does sell so well we would be in a very different situation today than we are now.

On 12/30/2011 01:45 PM, Karanbir Singh wrote:

my talk on application hosting on CentOS from Fosdem 2008 went into some details about this, there should be a copy of the talk online.

I appreciate the reference, I'll give it a look.

happy to answer specific questions... our build services are fairly robust as well ( remember over 100k builds were done leading to 6.0 ); the plague setup in place for c4/c5 has been fairly resilient as well.

Specific question. You mentioned plague. Are you using Koji now for build management or something else?

On 12/30/2011 10:28 PM, Johnny Hughes wrote:

On 12/30/2011 04:55 AM, 夜神　岩男 wrote:

...

On 12/30/2011 01:45 PM, Karanbir Singh wrote:

...

my talk on application hosting on CentOS from Fosdem 2008 went into some details about this, there should be a copy of the talk online.

I appreciate the reference, I'll give it a look.

...

happy to answer specific questions... our build services are fairly robust as well ( remember over 100k builds were done leading to 6.0 ); the plague setup in place for c4/c5 has been fairly resilient as well.

Specific question. You mentioned plague. Are you using Koji now for build management or something else?

Something else ... no koji at all here.

...which would be?

Or was that supposed to be mysterious and intriguing?

On 12/30/2011 04:25 PM, Johnny Hughes wrote:

We have a custom build system. It uses a beanstalkd workqueue to submit packages to mock and can scale up or down as required and submit

Just to add a few more bits : the queue system is meant to extend in both directions ( import and tracking of sources on one end - and testing + release management at the other end ). At the moment there is a bunch of band-aid and Johnny and me typing up stuff to bridge those bits, but I hope to have the entire setup in place end to end by end of March 2012. Apart from the cli, some of the other interesting interfaces that are semi functional at this point include an irc-bot, a rest api and a publicly available http callback server that allows arbitrary subscription.

There is, by design, no web interface. The plan is to focus on the mechanics and make the entire system work, with enough hooks that allows anyone / everyone to do the client side work. Eg. being able to write a yum plugin that would harvest update metadata from the buildsystem and be able to report on 'upcoming updates' ( a PoC for this is functional now.. )

The entire system is called 'reimzul'

As soon as I have got the basic functional stuff in place, and some docs around it I'll have it up in a git repo where people can clone and join the fun. And yes, this is very custom done for the CentOS ecosystem and address's the problems and hurdles that we run into with the localised process. Koji is a great system, just like OBS is - but they solve different problems and do so in a way that makes both of them really bad solutions for the distro-rebuild and tracking process's.

In other interesting artifacts ( and mostly from free win's ) include being able to test external third party repo's for pre-updates-release breakage's.... Another feature is the constant compose ( the next-release-tree is constantly composed on every update build, so were pretty much ready to go for the next release every night.. a feature that came in handy for the 6.2 release.)

again, just to be clear - this system works for c6 now, and will for c7 etc ( unless we come up with something else ). We are very much committed to using plague for c4/c5 for the time being. The pain associated with migrating and then proving the system for c5 would be more than is justifiable at this point.

Bit of a brain dump, but there you have it.

On 2011-12-30 Fri 20:47 +0000，Karanbir Singh wrote：

On 12/30/2011 04:25 PM, Johnny Hughes wrote:

...

We have a custom build system. It uses a beanstalkd workqueue to submit packages to mock and can scale up or down as required and submit

Just to add a few more bits : the queue system is meant to extend in both directions ( import and tracking of sources on one end - and testing + release management at the other end ). At the moment there is a bunch of band-aid and Johnny and me typing up stuff to bridge those bits, but I hope to have the entire setup in place end to end by end of March 2012. Apart from the cli, some of the other interesting interfaces that are semi functional at this point include an irc-bot, a rest api and a publicly available http callback server that allows arbitrary subscription.

There is, by design, no web interface. The plan is to focus on the mechanics and make the entire system work, with enough hooks that allows anyone / everyone to do the client side work. Eg. being able to write a yum plugin that would harvest update metadata from the buildsystem and be able to report on 'upcoming updates' ( a PoC for this is functional now.. )

The entire system is called 'reimzul'

Great works.

As soon as I have got the basic functional stuff in place, and some docs around it I'll have it up in a git repo where people can clone and join the fun.

How can I take part in? I'm glad to contribute more works.

And yes, this is very custom done for the CentOS ecosystem and address's the problems and hurdles that we run into with the localised process. Koji is a great system, just like OBS is - but they solve different problems and do so in a way that makes both of them really bad solutions for the distro-rebuild and tracking process's.

In other interesting artifacts ( and mostly from free win's ) include being able to test external third party repo's for pre-updates-release breakage's.... Another feature is the constant compose ( the next-release-tree is constantly composed on every update build, so were pretty much ready to go for the next release every night.. a feature that came in handy for the 6.2 release.)

again, just to be clear - this system works for c6 now, and will for c7 etc ( unless we come up with something else ). We are very much committed to using plague for c4/c5 for the time being. The pain associated with migrating and then proving the system for c5 would be more than is justifiable at this point.

Bit of a brain dump, but there you have it.

On Wed, Jan 4, 2012 at 10:29 AM, Karanbir Singh mail-lists@karan.org wrote:

the original plan was also to get a public facing buildsystem

Who are you, and what have you done with the CentOS team? :-)

Hi Guys,

On 12/30/2011 02:55 PM, Lamar Owen wrote:

I'd like to make an install USB with all the CentOS packages, sorted into repositories, and with updates integrated. Also, I'd like to put some select third party repos, and be able to use them at install time (anaconda is capable of using any repo at install time, of course). With 16 and 32GB USB bootable USB sticks somewhat affordable now, I can see great utility in being able to carry a key around to do installs to non-network attached PC's (of which we have a few) as well as being able to yum update off of that media (as documented in a form in the CentOS-Media.repo file).

And, like the OP, I'd like to drive the tools, meaning build it myself so that I can include packages that CentOS can't distribute (in particular, some site-developed packages that for various reasons we can't or won't distribute). The SL project documents a good deal of this process in their 'sites' documentation, incidentally.

the actual tools are all just anaconda/scripts - included in the rpms's - BUT, you dont need to rebuild the installer to achieve any of that. All you need to do is decide on the layout ( ideally, dont change the layout already in place so you dont lose the ability to use/abuse/incorpotate existing tools ), add your content into place and setup a new installclasses/<lamar.py> which sets up whatever install options you want pre-selected ( or just do that via kickstart ).

anaconda in c6 will work with arbitary media:// style repos as well as the usual http:// ftp:// etc ones.