I'm staring at the free CentOS images on AWS, and seeing that whoever set those up elected to use a partition for /dev/xvda1 rather than taking advantage of Amazon's tendency to use "/dev/xvda", "/dev/xvdb", etc. for each disk and use those directly as a file system.
The result is that if you elect to allocate a larger base disk image, for example allocating 50 Gig to allow local home directories or space for "mock" or for bulky logs, and don't spend the time to select and allocate new disk images, it's awkward to simply expand the "/" partition. And with only 8 Gig allocated in the latest CentOS 6 images that I see in AWS, it's possible to get pretty pressed for space pretty quickly.
Now, AWS published guidelines on manipulating partition size, and expanding a matching filesystem, but they're very clear to "unmount the parition before you touch it!!!" That's a bit difficult to unmount with a "/" partition, and they understandably don't have the kind of "boot from CD and work from the console" setup I'd normally use for that kind of work.
So: why did the creators of that CentOS AMI elect to use such a small / partition? And how dangerous is it, with the system essentially idle, to use "parted" to expand the "/dev/xvda1" parition and then use "resize2fs" to expand the "/" file system while the system is alive?
Note that, because I'm a complete weasel, I know at least one way around this: add a second disk, copy the OS to *that*, set grub to boot from the second disk, reboot from that, paritition the first disk as desired, copy the OS back, reset grub to boot from the first disk, and pray. I've had good success with the approach in the past, and have rebuilt rougly 15,000 Linux systems this way. But the work predates CentOS, and I dont't want to go through that again.
So, has anyone resized "/" successfully and gracefully on AWS CentOS instances?
Nico Kadel-Garcia
This is not really a problem at all. when you launch your image for the first time, you can specify a larger / volume size and cloud-init-tools will take care of the rest. This is well documented in the AWS userguides.
-- Kelly Prescott
On Wed, 29 Apr 2015, Nico Kadel-Garcia wrote:
I'm staring at the free CentOS images on AWS, and seeing that whoever set those up elected to use a partition for /dev/xvda1 rather than taking advantage of Amazon's tendency to use "/dev/xvda", "/dev/xvdb", etc. for each disk and use those directly as a file system.
The result is that if you elect to allocate a larger base disk image, for example allocating 50 Gig to allow local home directories or space for "mock" or for bulky logs, and don't spend the time to select and allocate new disk images, it's awkward to simply expand the "/" partition. And with only 8 Gig allocated in the latest CentOS 6 images that I see in AWS, it's possible to get pretty pressed for space pretty quickly.
Now, AWS published guidelines on manipulating partition size, and expanding a matching filesystem, but they're very clear to "unmount the parition before you touch it!!!" That's a bit difficult to unmount with a "/" partition, and they understandably don't have the kind of "boot from CD and work from the console" setup I'd normally use for that kind of work.
So: why did the creators of that CentOS AMI elect to use such a small / partition? And how dangerous is it, with the system essentially idle, to use "parted" to expand the "/dev/xvda1" parition and then use "resize2fs" to expand the "/" file system while the system is alive?
Note that, because I'm a complete weasel, I know at least one way around this: add a second disk, copy the OS to *that*, set grub to boot from the second disk, reboot from that, paritition the first disk as desired, copy the OS back, reset grub to boot from the first disk, and pray. I've had good success with the approach in the past, and have rebuilt rougly 15,000 Linux systems this way. But the work predates CentOS, and I dont't want to go through that again.
So, has anyone resized "/" successfully and gracefully on AWS CentOS instances?
Nico Kadel-Garcia
CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
On Wed, Apr 29, 2015 at 11:24 PM, Kelly Prescott kprescott@coolip.net wrote:
This is not really a problem at all. when you launch your image for the first time, you can specify a larger / volume size and cloud-init-tools will take care of the rest. This is well documented in the AWS userguides.
-- Kelly Prescott
I just had that discussion with an experienced AWS user, who hadn't noticed that the documentation at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/storage_expand_partition.... says the following, very clearly:
Unmount the partition if it is mounted. Run the umount command with the value of MOUNTPOINT from the lsblk command. In this example, the MOUNTPOINT value for the partition is /mnt.
Notice the "unmount the parition part". This is not feasible with a "/" partition unless you've booted from separate media and have console access or remote SSH access with that separate media. I'm experienced enough to actually configure a second disk and ensure booting from *that* to re-arrange the primary disk's filesystems, but I'm looking for a better way.
I'm also staring at the "cloud-init" docs you mentioned at http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/AmazonLinuxAMIBasics.html.... I see nothing about resizing the base OS image size, especially the "/" partition. Do you see something relevant?
Nico Kadel-Garcia
to follow-up, I will give an example. Here is the listing for the official centos AMI:
IMAGE ami-96a818fe aws-marketplace/CentOS 7 x86_64 (2014_09_29) EBS HVM-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-d2a117ba.2 aws-marketplace available public [marketplace: aw0evgkw8e5c1q413zgy5pjce] x86_64 machineebs hvm xen BLOCKDEVICEMAPPING EBS /dev/sda1 snap-591037fd 8 false standard Not Encrypted as you can see the block device mapping is by default set to BLOCKDEVICEMAPPING EBS /dev/sda1 snap-591037fd 8 false standard Not Encrypted
it is a standard volume, not encrypted, and 8 GB my modification consists in adding this to my run command for my ami launch: -b /dev/sda1=snap-591037fd:20:false:gp2
I set the drive the same, the snapshot the same, and I give it 20GB instead of 8, I also use the gp2 type instead of the standard as well as telling it not to delete the volume when the instance terminates.
Hope this helps. kp
On Wed, 29 Apr 2015, Nico Kadel-Garcia wrote:
I'm staring at the free CentOS images on AWS, and seeing that whoever set those up elected to use a partition for /dev/xvda1 rather than taking advantage of Amazon's tendency to use "/dev/xvda", "/dev/xvdb", etc. for each disk and use those directly as a file system.
The result is that if you elect to allocate a larger base disk image, for example allocating 50 Gig to allow local home directories or space for "mock" or for bulky logs, and don't spend the time to select and allocate new disk images, it's awkward to simply expand the "/" partition. And with only 8 Gig allocated in the latest CentOS 6 images that I see in AWS, it's possible to get pretty pressed for space pretty quickly.
Now, AWS published guidelines on manipulating partition size, and expanding a matching filesystem, but they're very clear to "unmount the parition before you touch it!!!" That's a bit difficult to unmount with a "/" partition, and they understandably don't have the kind of "boot from CD and work from the console" setup I'd normally use for that kind of work.
So: why did the creators of that CentOS AMI elect to use such a small / partition? And how dangerous is it, with the system essentially idle, to use "parted" to expand the "/dev/xvda1" parition and then use "resize2fs" to expand the "/" file system while the system is alive?
Note that, because I'm a complete weasel, I know at least one way around this: add a second disk, copy the OS to *that*, set grub to boot from the second disk, reboot from that, paritition the first disk as desired, copy the OS back, reset grub to boot from the first disk, and pray. I've had good success with the approach in the past, and have rebuilt rougly 15,000 Linux systems this way. But the work predates CentOS, and I dont't want to go through that again.
So, has anyone resized "/" successfully and gracefully on AWS CentOS instances?
Nico Kadel-Garcia
CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
On Wed, Apr 29, 2015 at 11:33 PM, Kelly Prescott kprescott@coolip.net wrote:
to follow-up, I will give an example. Here is the listing for the official centos AMI:
IMAGE ami-96a818fe aws-marketplace/CentOS 7 x86_64 (2014_09_29) EBS HVM-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-d2a117ba.2 aws-marketplace available public [marketplace: aw0evgkw8e5c1q413zgy5pjce] x86_64 machineebs hvm xen BLOCKDEVICEMAPPING EBS /dev/sda1 snap-591037fd 8 false standard Not Encrypted as you can see the block device mapping is by default set to BLOCKDEVICEMAPPING EBS /dev/sda1 snap-591037fd 8 false standard Not Encrypted
it is a standard volume, not encrypted, and 8 GB my modification consists in adding this to my run command for my ami launch: -b /dev/sda1=snap-591037fd:20:false:gp2
I set the drive the same, the snapshot the same, and I give it 20GB instead of 8, I also use the gp2 type instead of the standard as well as telling it not to delete the volume when the instance terminates.
Hope this helps. kp
Perhaps so, and I appreciate the pointer. I can try to work with that to integrate command line based deployment and get this option.
So you're working from the command line tools in the EPEL 'cloud-init' package, not the AWS GUI? Because when I tried expanding the size of the base disk image in the GUI, I wound up with an an 8 Gig default /dev/xvda1 on a 20 Gig /dev/xvda. That's why I was looking at "how do I resize this thing safel?"
Unfortunately, it doesn't help a lot with what I already have built, but could be useful going forward.
I think the command-line is far more flexable then the GUI interface. I use ec2-api-tools, but the python boto stuff works virtually the same.
On Thu, 30 Apr 2015, Nico Kadel-Garcia wrote:
On Wed, Apr 29, 2015 at 11:33 PM, Kelly Prescott kprescott@coolip.net wrote:
to follow-up, I will give an example. Here is the listing for the official centos AMI:
IMAGE ami-96a818fe aws-marketplace/CentOS 7 x86_64 (2014_09_29) EBS HVM-b7ee8a69-ee97-4a49-9e68-afaee216db2e-ami-d2a117ba.2 aws-marketplace available public [marketplace: aw0evgkw8e5c1q413zgy5pjce] x86_64 machineebs hvm xen BLOCKDEVICEMAPPING EBS /dev/sda1 snap-591037fd 8 false standard Not Encrypted as you can see the block device mapping is by default set to BLOCKDEVICEMAPPING EBS /dev/sda1 snap-591037fd 8 false standard Not Encrypted
it is a standard volume, not encrypted, and 8 GB my modification consists in adding this to my run command for my ami launch: -b /dev/sda1=snap-591037fd:20:false:gp2
I set the drive the same, the snapshot the same, and I give it 20GB instead of 8, I also use the gp2 type instead of the standard as well as telling it not to delete the volume when the instance terminates.
Hope this helps. kp
Perhaps so, and I appreciate the pointer. I can try to work with that to integrate command line based deployment and get this option.
So you're working from the command line tools in the EPEL 'cloud-init' package, not the AWS GUI? Because when I tried expanding the size of the base disk image in the GUI, I wound up with an an 8 Gig default /dev/xvda1 on a 20 Gig /dev/xvda. That's why I was looking at "how do I resize this thing safel?"
Unfortunately, it doesn't help a lot with what I already have built, but could be useful going forward. _______________________________________________ CentOS-virt mailing list CentOS-virt@centos.org http://lists.centos.org/mailman/listinfo/centos-virt
So you're working from the command line tools in the EPEL 'cloud-init' package, not the AWS GUI? Because when I tried expanding the size of the base disk image in the GUI, I wound up with an an 8 Gig default /dev/xvda1 on a 20 Gig /dev/xvda. That's why I was looking at "how do I resize this thing safel?"
No experience with amazon here, but I routinely resize filesystems online without issues.
Repartition xvda so that xvda1 is the size you want (make sure you use the same start sector, just change the end of the partition). Run partprobe and confirm that fdisk -l /dev/xvda1 shows the new size. (You may need to reboot). After that just run resize2fs /dev/xvda1 (works online).
- Nathan
-
Kelly Prescott -
Nathan March -
Nico Kadel-Garcia