<div dir="ltr"><div class="gmail_extra"><div class="gmail_quote">Hi,</div><div class="gmail_quote"><br></div><div class="gmail_quote">On Tue, Apr 8, 2014 at 11:03 PM, Juerg Haefliger <span dir="ltr"><<a href="mailto:juergh@gmail.com" target="_blank" class="cremed">juergh@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div class=""><div>On Tue, Apr 8, 2014 at 2:24 PM, Nux! <<a href="mailto:nux@li.nux.ro" target="_blank" class="cremed">nux@li.nux.ro</a>> wrote:<br>
><br>> Hello,<br>><br>> While the Cloud SIG is still being established, let's get to actual<br>
> work and think of a set of features for a CentOS cloud template.<br>> I am referring here to VMs, not containers (e.g. docker).<br>><br>> This is how I see it so far, please feel free to come with<br>> suggestions/comments/questions.<br>
><br>> A - Single partition for simplicity (and lack of good arguments against<br>> it)<br><br></div></div>I was wondering about LVM. It makes reconfiguration much easier (like adding swap). But growroot doesn't support LVM.</div>
</div></blockquote><div><br></div><div>Single partition is good for both simplicity and LVM support. LVM also adds a performance overhead which is fine for some customer use cases but not others.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div><div class=""><div>> - dracut-modules-growroot included so the template partition will<br>> expand to match target, cloud-init in charge of resize2fs<br><br></div></div><div>Only required for kernel < 3.8. Later kernels can do online partition resizing (handled by cloud-init post initrd).</div>
</div></div></blockquote><div><br></div><div>Unless we plan to ignore CentOS 6, we need to handle kernels before 3.8 as well as CentOS 7's 3.10. </div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div><div class=""><div>> B - To swap or not to swap?<br><br></div></div><div>Some service providers charge for disk IOs and nobody wants to pay for swap activity, so I vote against swap.<br></div></div>
</div></blockquote><div><br></div><div>I also don't see a need for swap in a cloud image. </div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div><div class=""><div>> C - "tuned-adm profile virtual-host" which translates to:<br>
> - kern.sched_min_granularity_ns 10ms<br>> - kernel.sched_wakeup_granularity_ns 15ms<br>> - vm.dirty_ratio 40%<br>> - vm.swappiness 30<br>> - IO scheduler "deadline"<br>
> - fs barriers off<br>
> - CPU governor "performance"<br>> - disk readahead 4x<br><br></div></div><div>Where do these come from? What's the rational?<br></div></div></div></blockquote><div><br></div><div>This might be a good place to link to GCE's recommendations for image settings, assembled from several different teams inside Google, with a bent toward maximal security but also discussing other areas:</div>
<div><a href="https://developers.google.com/compute/docs/building-image">https://developers.google.com/compute/docs/building-image</a><br></div><div><br></div><div>Some of them are more important than others, and clearly distributions will make the decisions that are right for them. Examples might be compiling virtio-scsi/virtio-net support as kernel modules for generality even though kernel modules would be disabled in the most single-vendor locked-down security-minded kernel.</div>
<div><br></div><div>Very few of these beyond the bare hardware support are strictly mandatory, but e.g. disabling password-based SSH, disabling root SSH login, and having root's password field locked are good cloud defaults except anywhere a specific vendor's environment needs otherwise.</div>
<div><br></div><div>We should also consider installing yum-cron by default; that adds a lot of automatic security protection for hands-off cloud users, but some behaviors or software versions occasionally change between 6.x and 6.{x+1}. Interesting tradeoff, and one that many users of a configuration management system handle through that software. GCE's image currently does preinstall yum-cron, though of course the CentOS community will eventually own the image and have the final say.</div>
<div><br></div><div>More recommendations might surface over time through things like performance testing or advice from our hypervisor or kernel hackers.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div><div class=""><div>> D - tso and gso off on the network interfaces <a href="http://s.nux.ro/gsotso" target="_blank" class="cremed">http://s.nux.ro/gsotso</a><br>
<br></div></div><div>These seem to be settings on the host, not the guest.<br></div></div></div></blockquote><div><br></div><div>No opinion here, though if this is a guest-side setting, I can ask around within Google to give a well-informed GCE perspective.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div></div><div class="">
<div>
> E - network interface remapping (75-persistent-net-generator.rules, BZ<br>> 912801)<br><br></div></div><div>Not authorized to access that bug.</div></div></div></blockquote><div><br></div><div>Same.</div><div> </div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div class=""><div>> F - Selinux on. Do we relabel for uniqueness? Seen small VMs run out of<br>
> memory while relabelling..<br></div><br></div></div>Ack.<br></div></blockquote><div><br></div><div>I don't think GCE's current image does anything specific here beyond leaving SELinux on and ensuring some of our environment-specific hacks get properly labeled. No opinion on what's optimal, but we do offer small VMs as well as normal-sized ones, so handling both use cases is good.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div><div class="">> G - PERSISTENT_DHCLIENT="1" (BZ 1011013)<br>
<br></div>Ack.</div></div></div></blockquote><div><br></div><div>Seems reasonable based on the RHBA linked from the BZ - we haven't noticed a problem without this but it could be useful.</div><div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<div dir="ltr"><div><div><div class="">> H - Bundle all the paravirt drivers in the ramdisk<br>> (virtio/xen/vmware/hyperv) so the same image can boot everywhere?<br><br></div></div><div>Seems reasonable. What's the impact on the initrd size?<br>
</div></div></div></blockquote><div><br></div><div>Seems good to me too. The ones GCE cares about are virtio-scsi, virtio-net, and virtio-pci/virtio-blk, but no objection to the others in the initrd if the result is reasonably sized.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div class=""><div>> I - Per "stack" requirements (e.g. cloudstack relies a lot on root user<br>
> and password logins, openstack tends not to, SSH key only logins etc<br>> etc)<br><br></div></div><div>Can we have a single image that fits all the different requirements?</div></div></div></blockquote><div><br></div>
<div>We are unlikely to have that in the end, but we can certainly start with one base and customize the output slightly for each environment.</div><div><br></div><div>Examples: GCE currently has two (Apache2-licensed Python) daemons running in our instances: one handles SSH keys via our metadata server in a way that's tied in to Google accounts and Google Cloud project access control lists, the other one facilitates some of our advanced networking features. We also ship gcutil and gsutil, two (Apache2-licensed Python) command-line utilities which are useful for interacting with the environment. The container format varies across environments too.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div class=""><div>> That's about all that crosses my mind for now.<br>
<br></div></div><div>K - No firwall. Handled by the service provider.<br></div></div></div></blockquote><div><br></div><div>Mostly the same in GCE too. To avoid breaking configs which expect the firewall on by default, we're currently going with a default-open iptables firewall (at least for TCP/UDP - I'd have to check for ICMP). If CentOS prefers to disable it entirely, no strong objection from me.</div>
<div><br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div></div><div>L - Timezone is set to UTC, Hostname is set to 'centos', lang is en_US.UTF-8, keyboard is us (or whatever you guys think makes sense).<br>
</div></div></div></blockquote><div><br></div><div>Agreed, although in the GCE case the hostname is set dynamically via DHCP based on the instance name given to the API. We also set the NTP server to metadata.google.internal, served by the host the VM is running on. While this is baked into our images via kickstart, the DHCP server also recently started providing this via NTP option.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div></div><div>M - NOZEROCONF=yes<br>
</div></div></div></blockquote><div><br></div><div>No opinion from me here. The same RHBA as before makes this seem wise to enable, although I haven't noticed a problem without it (our metadata server is at 169.254.169.254).</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div dir="ltr"><div><div></div><div>N - Along with the image, we'll also provide md5/sha1/sha256 checksums, gpg signed files and a manifest (list of installed packages and their versions).</div>
</div></div></blockquote><div><br></div><div>Sounds reasonable.</div><div> </div><div>- Jimmy</div></div></div></div>