Hi,
I´m running a CentOS server in a VPS. Backups of the VPS take quite much space if I don´t claim unused space.
Currently I´m using dd if=/dev/zero of=/mytempfile and remove that file to claim that unused space. Any automatic way of doing a similar thing in CentOS? I have googled for it but I have only found Debian commands.
Thanks in advance!
Miguel
I realize this is wandering off-topic but, if you have found Debian commands, you're doing better than me. What are they? Also, are you allowing dd to totally fill the partition (what I have found on the web as a recommendation)? If so, is the OS surviving acceptably?
----- Original Message ----- From: "Miguel González" miguel_3_gonzalez@yahoo.es To: "centos" centos@centos.org Sent: Saturday, July 29, 2017 5:11:33 AM Subject: [CentOS] claiming unsused space back
Hi,
I´m running a CentOS server in a VPS. Backups of the VPS take quite much space if I don´t claim unused space.
Currently I´m using dd if=/dev/zero of=/mytempfile and remove that file to claim that unused space. Any automatic way of doing a similar thing in CentOS? I have googled for it but I have only found Debian commands.
Thanks in advance!
Miguel _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On Mon, Jul 31, 2017 at 08:28:49AM -0500, Leroy Tennison wrote:
I realize this is wandering off-topic but, if you have found Debian commands, you're doing better than me. What are they? Also, are you allowing dd to totally fill the partition (what I have found on the web as a recommendation)? If so, is the OS surviving acceptably?
----- Original Message ----- From: "Miguel González" miguel_3_gonzalez@yahoo.es To: "centos" centos@centos.org Sent: Saturday, July 29, 2017 5:11:33 AM Subject: [CentOS] claiming unsused space back
Hi,
I´m running a CentOS server in a VPS. Backups of the VPS take quite much space if I don´t claim unused space.
Currently I´m using dd if=/dev/zero of=/mytempfile and remove that file to claim that unused space. Any automatic way of doing a similar thing in CentOS? I have googled for it but I have only found Debian commands.
Thanks in advance!
I may be blind, but I don't seehow that technique can "reclaim" any space. all it does is fill up all the space not allocated to other files by creating one large file that occupies all otherwise unused disk space.
presumably you'll delete that file once it is created, but you won't have any more free disk space than you had before. the only difference will be that that unused space will then be filled with zeroes.
what are you actually wanting to do here?
On 07/31/2017 08:50 AM, Fred Smith wrote:
On Mon, Jul 31, 2017 at 08:28:49AM -0500, Leroy Tennison wrote:
I realize this is wandering off-topic but, if you have found Debian commands, you're doing better than me. What are they? Also, are you allowing dd to totally fill the partition (what I have found on the web as a recommendation)? If so, is the OS surviving acceptably?
----- Original Message ----- From: "Miguel González" miguel_3_gonzalez@yahoo.es To: "centos" centos@centos.org Sent: Saturday, July 29, 2017 5:11:33 AM Subject: [CentOS] claiming unsused space back
Hi,
I´m running a CentOS server in a VPS. Backups of the VPS take quite much space if I don´t claim unused space.
Currently I´m using dd if=/dev/zero of=/mytempfile and remove that file to claim that unused space. Any automatic way of doing a similar thing in CentOS? I have googled for it but I have only found Debian commands.
Thanks in advance!
I may be blind, but I don't seehow that technique can "reclaim" any space. all it does is fill up all the space not allocated to other files by creating one large file that occupies all otherwise unused disk space.
presumably you'll delete that file once it is created, but you won't have any more free disk space than you had before. the only difference will be that that unused space will then be filled with zeroes.
what are you actually wanting to do here?
I agree with fred .. this would OVERWRITE empty space with zeros and remove it .. that MIGHT be something you would do for security because it would replace what was on there before. But it would not free up any space and should not impact your backup at all (if you are backing up files).
You're right, there's a procedure following it, once the space is zeroed qemu-img will recognize it as such and will eliminate it when 'convert' is used. Apparently Fedora qemu has some better capabilities to shrink partitions but they haven't made it to "long term support" distributions yet. For now, what has to be done to shrink qcow[2] partitions (raw works) is (regardless of client OS, for Windows defragment is first used followed by resizing the partitions in Disk Management then finally Sysinternals' sdelete to zero disk space - I have used this process and it works but with surprises): defragment (even Linux, look into e2defrag, shake, a defrag script or e4defrag - can be found on the web, haven't used them, YMMV), zero disk space, resize the partition, then use qemu-img to convert to raw (or even qcow - it works). However, to permanently resize you must convert to raw, shrink and re-convert to qcow2 if you want those capabilities.
----- Original Message ----- From: "Fred Smith" fredex@fcshome.stoneham.ma.us To: "centos" centos@centos.org Sent: Monday, July 31, 2017 8:50:57 AM Subject: Re: [CentOS] claiming unsused space back
On Mon, Jul 31, 2017 at 08:28:49AM -0500, Leroy Tennison wrote:
I realize this is wandering off-topic but, if you have found Debian commands, you're doing better than me. What are they? Also, are you allowing dd to totally fill the partition (what I have found on the web as a recommendation)? If so, is the OS surviving acceptably?
----- Original Message ----- From: "Miguel González" miguel_3_gonzalez@yahoo.es To: "centos" centos@centos.org Sent: Saturday, July 29, 2017 5:11:33 AM Subject: [CentOS] claiming unsused space back
Hi,
I´m running a CentOS server in a VPS. Backups of the VPS take quite much space if I don´t claim unused space.
Currently I´m using dd if=/dev/zero of=/mytempfile and remove that file to claim that unused space. Any automatic way of doing a similar thing in CentOS? I have googled for it but I have only found Debian commands.
Thanks in advance!
I may be blind, but I don't seehow that technique can "reclaim" any space. all it does is fill up all the space not allocated to other files by creating one large file that occupies all otherwise unused disk space.
presumably you'll delete that file once it is created, but you won't have any more free disk space than you had before. the only difference will be that that unused space will then be filled with zeroes.
what are you actually wanting to do here?
If you are using XFS - there is mount option "discard|nodiscard"
From XFS man page:
discard|nodiscard Enable/disable the issuing of commands to let the block device reclaim space freed by the filesystem. This is useful for SSD devices, thinly provisioned LUNs and virtual machine images, but may have a performance impact.
Note: It is currently recommended that you use the fstrim application to discard unused blocks rather than the discard mount option because the performance impact of this option is quite severe. For this reason, nodiscard is the default.
Vladimir
-----Original Message----- From: CentOS [mailto:centos-bounces@centos.org] On Behalf Of Leroy Tennison Sent: Monday, July 31, 2017 4:42 PM To: centos centos@centos.org Subject: Re: [CentOS] claiming unsused space back
You're right, there's a procedure following it, once the space is zeroed qemu-img will recognize it as such and will eliminate it when 'convert' is used. Apparently Fedora qemu has some better capabilities to shrink partitions but they haven't made it to "long term support" distributions yet. For now, what has to be done to shrink qcow[2] partitions (raw works) is (regardless of client OS, for Windows defragment is first used followed by resizing the partitions in Disk Management then finally Sysinternals' sdelete to zero disk space - I have used this process and it works but with surprises): defragment (even Linux, look into e2defrag, shake, a defrag script or e4defrag - can be found on the web, haven't used them, YMMV), zero disk space, resize the partition, then use qemu-img to convert to raw (or even qcow - it works). However, to permanently resize you must convert to raw, shrink and re-convert to qcow2 if you want those capabilities.
----- Original Message ----- From: "Fred Smith" fredex@fcshome.stoneham.ma.us To: "centos" centos@centos.org Sent: Monday, July 31, 2017 8:50:57 AM Subject: Re: [CentOS] claiming unsused space back
On Mon, Jul 31, 2017 at 08:28:49AM -0500, Leroy Tennison wrote:
I realize this is wandering off-topic but, if you have found Debian commands, you're doing better than me. What are they? Also, are you allowing dd to totally fill the partition (what I have found on the web as a recommendation)? If so, is the OS surviving acceptably?
----- Original Message ----- From: "Miguel González" miguel_3_gonzalez@yahoo.es To: "centos" centos@centos.org Sent: Saturday, July 29, 2017 5:11:33 AM Subject: [CentOS] claiming unsused space back
Hi,
I´m running a CentOS server in a VPS. Backups of the VPS take quite much space if I don´t claim unused space.
Currently I´m using dd if=/dev/zero of=/mytempfile and remove that file to claim that unused space. Any automatic way of doing a similar thing in CentOS? I have googled for it but I have only found Debian commands.
Thanks in advance!
I may be blind, but I don't seehow that technique can "reclaim" any space. all it does is fill up all the space not allocated to other files by creating one large file that occupies all otherwise unused disk space.
presumably you'll delete that file once it is created, but you won't have any more free disk space than you had before. the only difference will be that that unused space will then be filled with zeroes.
what are you actually wanting to do here?
Interesting, thanks, my situation was (obviously) using NTFS. I should add clarification that, although a qcow[2] to qcow[2] convert will reclaim the zeroed space, it does nothing to change the virtual size (qemu-img info ...) so the image can grow back to that size. Currently (on long term support distributions) you need to convert to raw, use qemu-resize to reduce the physical file size then convert back to qcow2 to get an adjusted virtual size.
----- Original Message ----- From: "Ruttkay Vladimir" vladimir.ruttkay@telekom.sk To: "centos" centos@centos.org Sent: Monday, July 31, 2017 9:54:27 AM Subject: Re: [CentOS] claiming unsused space back
If you are using XFS - there is mount option "discard|nodiscard"
From XFS man page:
discard|nodiscard Enable/disable the issuing of commands to let the block device reclaim space freed by the filesystem. This is useful for SSD devices, thinly provisioned LUNs and virtual machine images, but may have a performance impact.
Note: It is currently recommended that you use the fstrim application to discard unused blocks rather than the discard mount option because the performance impact of this option is quite severe. For this reason, nodiscard is the default.
Vladimir
-----Original Message----- From: CentOS [mailto:centos-bounces@centos.org] On Behalf Of Leroy Tennison Sent: Monday, July 31, 2017 4:42 PM To: centos centos@centos.org Subject: Re: [CentOS] claiming unsused space back
You're right, there's a procedure following it, once the space is zeroed qemu-img will recognize it as such and will eliminate it when 'convert' is used. Apparently Fedora qemu has some better capabilities to shrink partitions but they haven't made it to "long term support" distributions yet. For now, what has to be done to shrink qcow[2] partitions (raw works) is (regardless of client OS, for Windows defragment is first used followed by resizing the partitions in Disk Management then finally Sysinternals' sdelete to zero disk space - I have used this process and it works but with surprises): defragment (even Linux, look into e2defrag, shake, a defrag script or e4defrag - can be found on the web, haven't used them, YMMV), zero disk space, resize the partition, then use qemu-img to convert to raw (or even qcow - it works). However, to permanently resize you must convert to raw, shrink and re-convert to qcow2 if you want those capabilities.
----- Original Message ----- From: "Fred Smith" fredex@fcshome.stoneham.ma.us To: "centos" centos@centos.org Sent: Monday, July 31, 2017 8:50:57 AM Subject: Re: [CentOS] claiming unsused space back
On Mon, Jul 31, 2017 at 08:28:49AM -0500, Leroy Tennison wrote:
I realize this is wandering off-topic but, if you have found Debian commands, you're doing better than me. What are they? Also, are you allowing dd to totally fill the partition (what I have found on the web as a recommendation)? If so, is the OS surviving acceptably?
----- Original Message ----- From: "Miguel González" miguel_3_gonzalez@yahoo.es To: "centos" centos@centos.org Sent: Saturday, July 29, 2017 5:11:33 AM Subject: [CentOS] claiming unsused space back
Hi,
I´m running a CentOS server in a VPS. Backups of the VPS take quite much space if I don´t claim unused space.
Currently I´m using dd if=/dev/zero of=/mytempfile and remove that file to claim that unused space. Any automatic way of doing a similar thing in CentOS? I have googled for it but I have only found Debian commands.
Thanks in advance!
I may be blind, but I don't seehow that technique can "reclaim" any space. all it does is fill up all the space not allocated to other files by creating one large file that occupies all otherwise unused disk space.
presumably you'll delete that file once it is created, but you won't have any more free disk space than you had before. the only difference will be that that unused space will then be filled with zeroes.
what are you actually wanting to do here?
On Jul 31, 2017, at 7:50 AM, Fred Smith fredex@fcshome.stoneham.ma.us wrote:
On Mon, Jul 31, 2017 at 08:28:49AM -0500, Leroy Tennison wrote:
dd to totally fill the partition
I may be blind, but I don't seehow that technique can "reclaim" any space.
In addition to the OP’s qemu case, zeroing the free space can also be valuable when building binary OS images from physical media.
The first time you do it with a fresh drive, the disk image contains only what you put onto the drive. Then later when you update that drive for the next release, you cause files to be overwritten, thus leaving outdated copies of file system blocks laying around which you don’t want to be dd’d into the resulting disk image.
Zeroing the free space not only prevents inclusion of these discarded FS blocks, they compress better, too.
Once upon a time, Warren Young warren@etr-usa.com said:
Zeroing the free space not only prevents inclusion of these discarded FS blocks, they compress better, too.
Check out the "virt-sparsify" command - it does all of this for you.
As has already been mentioned, some commands (or command options) are only supported on later releases, the man pages don't say this. Does anyone know of a source of information listing the command, option and version it is implemented in? That alone would be a great help.
----- Original Message ----- From: "Chris Adams" linux@cmadams.net To: "centos" centos@centos.org Sent: Monday, July 31, 2017 11:45:20 AM Subject: Re: [CentOS] claiming unsused space back
Once upon a time, Warren Young warren@etr-usa.com said:
Zeroing the free space not only prevents inclusion of these discarded FS blocks, they compress better, too.
Check out the "virt-sparsify" command - it does all of this for you.
On 07/31/2017 05:27 PM, Leroy Tennison wrote:
As has already been mentioned, some commands (or command options) are only supported on later releases, the man pages don't say this. Does anyone know of a source of information listing the command, option and version it is implemented in? That alone would be a great help.
----- Original Message ----- From: "Chris Adams" linux@cmadams.net To: "centos" centos@centos.org Sent: Monday, July 31, 2017 11:45:20 AM Subject: Re: [CentOS] claiming unsused space back
Once upon a time, Warren Young warren@etr-usa.com said:
Zeroing the free space not only prevents inclusion of these discarded FS blocks, they compress better, too.
Check out the "virt-sparsify" command - it does all of this for you.
Yes .. just run man on the machine in question. That has the commands for the man for the version of software installed on that specific machine.
I should have been more specific (and maybe ask "Are you seeing something different?") Admittedly, Ubuntu 16.04 LTS, but the qemu-img man page says for resize (What does the CentOS7 man page say?):
resize filename [+ | -]size Change the disk image as if it had been created with size.
Before using this command ... (warning about doing guest resizing first)
After using ... (somewhat different message about guest resizing)
No mention that shrinking only works with raw, not qcow2. Similar issue with virsh blockresize. I probably should have been more clear that the issue isn't commands or just command options, but significant limitations in scope for some of those options.
----- Original Message ----- From: "Johnny Hughes" johnny@centos.org To: "centos" centos@centos.org Sent: Tuesday, August 1, 2017 6:31:14 AM Subject: Re: [CentOS] claiming unsused space back
On 07/31/2017 05:27 PM, Leroy Tennison wrote:
As has already been mentioned, some commands (or command options) are only supported on later releases, the man pages don't say this. Does anyone know of a source of information listing the command, option and version it is implemented in? That alone would be a great help.
----- Original Message ----- From: "Chris Adams" linux@cmadams.net To: "centos" centos@centos.org Sent: Monday, July 31, 2017 11:45:20 AM Subject: Re: [CentOS] claiming unsused space back
Once upon a time, Warren Young warren@etr-usa.com said:
Zeroing the free space not only prevents inclusion of these discarded FS blocks, they compress better, too.
Check out the "virt-sparsify" command - it does all of this for you.
Yes .. just run man on the machine in question. That has the commands for the man for the version of software installed on that specific machine.
_______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
On Jul 31, 2017, at 4:27 PM, Leroy Tennison leroy@datavoiceint.com wrote:
some commands (or command options) are only supported on later releases, the man pages don't say this.
You only run into that problem when trying to use man pages from one system but then run commands on a very different system. The man pages actually installed on the system you’re running the command on lists only those options that are supported by that version of the command.
Does anyone know of a source of information listing the command, option and version it is implemented in?
The closest thing I’m aware of is the man page collection at unix.com:
http://www.unix.com/man-page/linux/1/ls/
They don’t have man pages for absolutely every version of Linux — that would require hundreds of sets! — and it only includes commands in the base system, not those for add-on packages.
In this particular case, I suspect the problem is that you haven’t got the libguestfs-tools-c package installed, which is what owns the virt-sparsify command. And I found that out with one Google search and one “yum search” command.
With that package installed, now you can say “man virt-sparsify” to find out what the CentOS 7 version of that command understands.
On Jul 31, 2017, at 7:28 AM, Leroy Tennison leroy@datavoiceint.com wrote:
if you have found Debian commands, you're doing better than me. What are they?
I’m not aware of a generic Debian-to-CentOS command translation dictionary. You’ll have to be specific about the commands you’re not finding.
Almost all of the commands you can run on a Debian system apply to a CentOS system, and vice versa. The main differences today are in the packaging system. Prior to the systemd takeover of the world, there were also significant command differences in the init system.
(See, haters, there’s actual value in systemd!)
Anyway, your `dd` command will run just fine on CentOS. The only thing I’d look at changing is that it will only affect the root filesystem, which may not include all of the other filesystems you want to affect. Post your mount table (i.e. output of the `mount` command) if you want advice here.
Hi,
Why wandering off-topic? This is about CentOS on VPSes. I can´t remember which Debian or Ubuntu commands I found since I didn´t use them.
I´m allowing dd to totally fill the partition and the OS is surviving perfectly (It´s a web server and works fine as long as I don´t keep it "full" too long).
Regarding other comments that I have read:
I´m using qcow2, I don´t need to shrink the HD size since sometimes it grows temporarily (backups).
Regards,
Miguel
On 07/31/17 3:28 PM, Leroy Tennison wrote:
I realize this is wandering off-topic but, if you have found Debian commands, you're doing better than me. What are they? Also, are you allowing dd to totally fill the partition (what I have found on the web as a recommendation)? If so, is the OS surviving acceptably?
----- Original Message ----- From: "Miguel González" miguel_3_gonzalez@yahoo.es To: "centos" centos@centos.org Sent: Saturday, July 29, 2017 5:11:33 AM Subject: [CentOS] claiming unsused space back
Hi,
I´m running a CentOS server in a VPS. Backups of the VPS take quite much space if I don´t claim unused space.
Currently I´m using dd if=/dev/zero of=/mytempfile and remove that file to claim that unused space. Any automatic way of doing a similar thing in CentOS? I have googled for it but I have only found Debian commands.
Thanks in advance!
Miguel _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos _______________________________________________ CentOS mailing list CentOS@centos.org https://lists.centos.org/mailman/listinfo/centos
-
Chris Adams -
Fred Smith -
Johnny Hughes -
Leroy Tennison -
Miguel González -
Ruttkay Vladimir -
Warren Young