Hello,
I was asked if it is possible to zero unused space in ext3 partition?
Users write to the server via Samba and are far from computer geeks so teaching them to use some safedelete utility is quite impossible.
Is there some way or utility to wipe out all the data from unused space?
Thanks,
Mindaugas
On Thu, 2006-09-07 at 10:25 +0300, Mindaugas wrote:
Hello,
I was asked if it is possible to zero unused space in ext3 partition?
Users write to the server via Samba and are far from computer geeks so teaching them to use some safedelete utility is quite impossible.
Is there some way or utility to wipe out all the data from unused space?
Thanks,
Mindaugas
wow...long time...look read up on like running multi 0S on same box ...like say centos and windows. In order to do that you have to play w/your hard drive space and spec it out... Now I don't remember the command or program you use but I think it's a good start for ya
jr
On 9/7/06, Mindaugas mind@bi.lt wrote:
Hello,
I was asked if it is possible to zero unused space in ext3 partition?
Users write to the server via Samba and are far from computer geeks so teaching them to use some safedelete utility is quite impossible.
A safedelete may not work via samba anyway. It doesn't work very well on Windows Native CIFS shares from what my forensics friends have told me.
Is there some way or utility to wipe out all the data from unused space?
For the simplest level of 'wiping' you can try the following:
make sure your samba share is its own physical partition seperate from say /var, etc.
have a regular job that runs at off hours and does the following:
dd if=/dev/zero of=/my/local/filesystem/zero_file
when dd dies from lack of space remove /my/local/filesystem/zero_file
It wont be DOD regs but it will give some level of cleanliness [where level is better than nothing, but not much.] Your big problem will be that this can/will cause problems with clients that are writing regular data to that disk.
Thanks,
Mindaugas
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
"Mindaugas" mind@bi.lt wrote:
I was asked if it is possible to zero unused space in ext3 partition?
I have a couple of patches that add a zerofree mount option to ext2 and ext3 filesystems. The ext2 version is much better tested and more complete: it zeros all file data blocks, directory blocks and extended attributes (though not inode data). The ext3 patch only handles file data, not metadata.
I've been meaning to submit these to LKML, but since you ask let's give them an airing here first.
Since this is being copied to the CentOS mailing list I should point out that I also have versions of the patches that apply cleanly to the RHEL 4 kernel. I don't have them to hand at the moment but if there's any interest I can provide them later.
Some background information and other tools are on my website:
http://intgat.tigress.co.uk/rmy/uml/index.html
Ron
Add a zerofree mount option to the ext2 filesystem. This causes freed blocks to be filled with zeros.
ext2_zero_blocks has an additional argument to specify whether or not zeroing is required: there's no point in zeroing blocks that have just come from the free list.
Some rerrangement of code in xattr.c is required to ensure that ext2_zero_blocks is never called with a locked buffer.
Signed-off-by: Ron Yorston rmy@tigress.co.uk
---
--- linux-2.6.17/Documentation/filesystems/ext2.txt.zerofree2 2006-06-18 02:49:35.000000000 +0100 +++ linux-2.6.17/Documentation/filesystems/ext2.txt 2006-08-25 20:08:06.000000000 +0100 @@ -58,6 +58,8 @@ nobh Do not attach buffer_heads to fi
xip Use execute in place (no caching) if possible
+zerofree Zero data blocks when they are freed. + grpquota,noquota,quota,usrquota Quota options are silently ignored by ext2.
--- linux-2.6.17/fs/ext2/balloc.c.zerofree2 2006-06-18 02:49:35.000000000 +0100 +++ linux-2.6.17/fs/ext2/balloc.c 2006-08-25 20:08:26.000000000 +0100 @@ -174,9 +174,28 @@ static void group_release_blocks(struct } }
+static void ext2_zero_blocks(struct super_block *sb, unsigned long block, + unsigned long count) +{ + unsigned long i; + struct buffer_head * bh; + + for (i = 0; i < count; i++) { + bh = sb_getblk(sb, block+i); + if (!bh) + continue; + + lock_buffer(bh); + memset(bh->b_data, 0, bh->b_size); + mark_buffer_dirty(bh); + unlock_buffer(bh); + brelse(bh); + } +} + /* Free given blocks, update quota and i_blocks field */ void ext2_free_blocks (struct inode * inode, unsigned long block, - unsigned long count) + unsigned long count, int zero) { struct buffer_head *bitmap_bh = NULL; struct buffer_head * bh2; @@ -201,6 +220,9 @@ void ext2_free_blocks (struct inode * in
ext2_debug ("freeing block(s) %lu-%lu\n", block, block + count - 1);
+ if (test_opt(sb, ZEROFREE) && zero) + ext2_zero_blocks(sb, block, count); + do_more: overflow = 0; block_group = (block - le32_to_cpu(es->s_first_data_block)) / --- linux-2.6.17/fs/ext2/super.c.zerofree2 2006-06-18 02:49:35.000000000 +0100 +++ linux-2.6.17/fs/ext2/super.c 2006-08-25 20:08:06.000000000 +0100 @@ -287,7 +287,7 @@ enum { Opt_err_ro, Opt_nouid32, Opt_nocheck, Opt_debug, Opt_oldalloc, Opt_orlov, Opt_nobh, Opt_user_xattr, Opt_nouser_xattr, Opt_acl, Opt_noacl, Opt_xip, Opt_ignore, Opt_err, Opt_quota, - Opt_usrquota, Opt_grpquota + Opt_usrquota, Opt_grpquota, Opt_zerofree };
static match_table_t tokens = { @@ -310,6 +310,7 @@ static match_table_t tokens = { {Opt_oldalloc, "oldalloc"}, {Opt_orlov, "orlov"}, {Opt_nobh, "nobh"}, + {Opt_zerofree, "zerofree"}, {Opt_user_xattr, "user_xattr"}, {Opt_nouser_xattr, "nouser_xattr"}, {Opt_acl, "acl"}, @@ -393,6 +394,9 @@ static int parse_options (char * options case Opt_nobh: set_opt (sbi->s_mount_opt, NOBH); break; + case Opt_zerofree: + set_opt (sbi->s_mount_opt, ZEROFREE); + break; #ifdef CONFIG_EXT2_FS_XATTR case Opt_user_xattr: set_opt (sbi->s_mount_opt, XATTR_USER); --- linux-2.6.17/fs/ext2/xattr.c.zerofree2 2006-06-18 02:49:35.000000000 +0100 +++ linux-2.6.17/fs/ext2/xattr.c 2006-08-25 20:08:06.000000000 +0100 @@ -676,7 +676,7 @@ ext2_xattr_set2(struct inode *inode, str
new_bh = sb_getblk(sb, block); if (!new_bh) { - ext2_free_blocks(inode, block, 1); + ext2_free_blocks(inode, block, 1, 0); error = -EIO; goto cleanup; } @@ -715,25 +715,26 @@ ext2_xattr_set2(struct inode *inode, str
error = 0; if (old_bh && old_bh != new_bh) { + unsigned long block = old_bh->b_blocknr; struct mb_cache_entry *ce;
/* * If there was an old block and we are no longer using it, * release the old block. */ - ce = mb_cache_entry_get(ext2_xattr_cache, old_bh->b_bdev, - old_bh->b_blocknr); + ce = mb_cache_entry_get(ext2_xattr_cache, old_bh->b_bdev, block); lock_buffer(old_bh); if (HDR(old_bh)->h_refcount == cpu_to_le32(1)) { /* Free the old block. */ if (ce) mb_cache_entry_free(ce); ea_bdebug(old_bh, "freeing"); - ext2_free_blocks(inode, old_bh->b_blocknr, 1); + unlock_buffer(old_bh); /* We let our caller release old_bh, so we * need to duplicate the buffer before. */ get_bh(old_bh); bforget(old_bh); + ext2_free_blocks(inode, block, 1, 1); } else { /* Decrement the refcount only. */ HDR(old_bh)->h_refcount = cpu_to_le32( @@ -744,8 +745,8 @@ ext2_xattr_set2(struct inode *inode, str mark_buffer_dirty(old_bh); ea_bdebug(old_bh, "refcount now=%d", le32_to_cpu(HDR(old_bh)->h_refcount)); + unlock_buffer(old_bh); } - unlock_buffer(old_bh); }
cleanup: @@ -789,10 +790,10 @@ ext2_xattr_delete_inode(struct inode *in if (HDR(bh)->h_refcount == cpu_to_le32(1)) { if (ce) mb_cache_entry_free(ce); - ext2_free_blocks(inode, EXT2_I(inode)->i_file_acl, 1); + unlock_buffer(bh); get_bh(bh); bforget(bh); - unlock_buffer(bh); + ext2_free_blocks(inode, EXT2_I(inode)->i_file_acl, 1, 1); } else { HDR(bh)->h_refcount = cpu_to_le32( le32_to_cpu(HDR(bh)->h_refcount) - 1); --- linux-2.6.17/fs/ext2/inode.c.zerofree2 2006-06-18 02:49:35.000000000 +0100 +++ linux-2.6.17/fs/ext2/inode.c 2006-08-25 20:08:06.000000000 +0100 @@ -100,7 +100,7 @@ void ext2_discard_prealloc (struct inode ei->i_prealloc_count = 0; ei->i_prealloc_block = 0; write_unlock(&ei->i_meta_lock); - ext2_free_blocks (inode, block, total); + ext2_free_blocks (inode, block, total, 0); return; } else write_unlock(&ei->i_meta_lock); @@ -467,7 +467,7 @@ static int ext2_alloc_branch(struct inod for (i = 1; i < n; i++) bforget(branch[i].bh); for (i = 0; i < n; i++) - ext2_free_blocks(inode, le32_to_cpu(branch[i].key), 1); + ext2_free_blocks(inode, le32_to_cpu(branch[i].key), 1, 0); return err; }
@@ -527,7 +527,7 @@ changed: for (i = 1; i < num; i++) bforget(where[i].bh); for (i = 0; i < num; i++) - ext2_free_blocks(inode, le32_to_cpu(where[i].key), 1); + ext2_free_blocks(inode, le32_to_cpu(where[i].key), 1, 1); return -EAGAIN; }
@@ -837,7 +837,7 @@ static inline void ext2_free_data(struct count++; else { mark_inode_dirty(inode); - ext2_free_blocks (inode, block_to_free, count); + ext2_free_blocks (inode, block_to_free, count, 1); free_this: block_to_free = nr; count = 1; @@ -846,7 +846,7 @@ static inline void ext2_free_data(struct } if (count > 0) { mark_inode_dirty(inode); - ext2_free_blocks (inode, block_to_free, count); + ext2_free_blocks (inode, block_to_free, count, 1); } }
@@ -889,7 +889,7 @@ static void ext2_free_branches(struct in (__le32*)bh->b_data + addr_per_block, depth); bforget(bh); - ext2_free_blocks(inode, nr, 1); + ext2_free_blocks(inode, nr, 1, 1); mark_inode_dirty(inode); } } else --- linux-2.6.17/fs/ext2/ext2.h.zerofree2 2006-06-18 02:49:35.000000000 +0100 +++ linux-2.6.17/fs/ext2/ext2.h 2006-08-25 20:08:06.000000000 +0100 @@ -94,7 +94,7 @@ extern unsigned long ext2_bg_num_gdb(str extern int ext2_new_block (struct inode *, unsigned long, __u32 *, __u32 *, int *); extern void ext2_free_blocks (struct inode *, unsigned long, - unsigned long); + unsigned long, int); extern unsigned long ext2_count_free_blocks (struct super_block *); extern unsigned long ext2_count_dirs (struct super_block *); extern void ext2_check_blocks_bitmap (struct super_block *); --- linux-2.6.17/include/linux/ext2_fs.h.zerofree2 2006-06-18 02:49:35.000000000 +0100 +++ linux-2.6.17/include/linux/ext2_fs.h 2006-08-25 20:08:06.000000000 +0100 @@ -310,6 +310,7 @@ struct ext2_inode { #define EXT2_MOUNT_MINIX_DF 0x000080 /* Mimics the Minix statfs */ #define EXT2_MOUNT_NOBH 0x000100 /* No buffer_heads */ #define EXT2_MOUNT_NO_UID32 0x000200 /* Disable 32-bit UIDs */ +#define EXT2_MOUNT_ZEROFREE 0x000400 /* Zero freed blocks */ #define EXT2_MOUNT_XATTR_USER 0x004000 /* Extended user attributes */ #define EXT2_MOUNT_POSIX_ACL 0x008000 /* POSIX Access Control Lists */ #define EXT2_MOUNT_XIP 0x010000 /* Execute in place */
Add a zerofree mount option to the ext3 filesystem. This causes freed blocks to be filled with zeros.
Zeroing is only applied to data blocks, not metadata. This means that directory blocks and extended attributes are not zeroed.
Signed-off-by; Ron Yorston rmy@tigress.co.uk
---
--- linux-2.6.17/Documentation/filesystems/ext3.txt.zerofree3 2006-06-18 02:49:35.000000000 +0100 +++ linux-2.6.17/Documentation/filesystems/ext3.txt 2006-08-26 19:07:34.000000000 +0100 @@ -113,6 +113,8 @@ noquota grpquota usrquota
+zerofree Zero data blocks when they are freed. +
Specification ============= --- linux-2.6.17/fs/ext3/super.c.zerofree3 2006-08-26 19:06:57.000000000 +0100 +++ linux-2.6.17/fs/ext3/super.c 2006-08-26 19:07:34.000000000 +0100 @@ -675,7 +675,7 @@ enum { Opt_usrjquota, Opt_grpjquota, Opt_offusrjquota, Opt_offgrpjquota, Opt_jqfmt_vfsold, Opt_jqfmt_vfsv0, Opt_quota, Opt_noquota, Opt_ignore, Opt_barrier, Opt_err, Opt_resize, Opt_usrquota, - Opt_grpquota + Opt_grpquota, Opt_zerofree };
static match_table_t tokens = { @@ -720,6 +720,7 @@ static match_table_t tokens = { {Opt_jqfmt_vfsold, "jqfmt=vfsold"}, {Opt_jqfmt_vfsv0, "jqfmt=vfsv0"}, {Opt_grpquota, "grpquota"}, + {Opt_zerofree, "zerofree"}, {Opt_noquota, "noquota"}, {Opt_quota, "quota"}, {Opt_usrquota, "usrquota"}, @@ -1052,6 +1053,9 @@ clear_qf_name: case Opt_nobh: set_opt(sbi->s_mount_opt, NOBH); break; + case Opt_zerofree: + set_opt(sbi->s_mount_opt, ZEROFREE); + break; default: printk (KERN_ERR "EXT3-fs: Unrecognized mount option "%s" " --- linux-2.6.17/fs/ext3/balloc.c.zerofree3 2006-06-18 02:49:35.000000000 +0100 +++ linux-2.6.17/fs/ext3/balloc.c 2006-08-26 19:08:14.000000000 +0100 @@ -491,9 +491,28 @@ error_return: return; }
+static void ext3_zero_blocks(struct super_block *sb, unsigned long block, + unsigned long count) +{ + unsigned long i; + struct buffer_head *bh; + + for (i = 0; i < count; i++) { + bh = sb_getblk(sb, block+i); + if (!bh) + continue; + + lock_buffer(bh) ; + memset(bh->b_data, 0, bh->b_size); + mark_buffer_dirty(bh); + unlock_buffer(bh) ; + brelse(bh); + } +} + /* Free given blocks, update quota and i_blocks field */ void ext3_free_blocks(handle_t *handle, struct inode *inode, - unsigned long block, unsigned long count) + unsigned long block, unsigned long count, int zero) { struct super_block * sb; int dquot_freed_blocks; @@ -503,6 +522,8 @@ void ext3_free_blocks(handle_t *handle, printk ("ext3_free_blocks: nonexistent device"); return; } + if (test_opt(sb, ZEROFREE) && zero && !ext3_should_journal_data(inode)) + ext3_zero_blocks(sb, block, count); ext3_free_blocks_sb(handle, sb, block, count, &dquot_freed_blocks); if (dquot_freed_blocks) DQUOT_FREE_BLOCK(inode, dquot_freed_blocks); --- linux-2.6.17/fs/ext3/inode.c.zerofree3 2006-08-26 19:06:57.000000000 +0100 +++ linux-2.6.17/fs/ext3/inode.c 2006-08-26 19:07:34.000000000 +0100 @@ -562,7 +562,7 @@ static int ext3_alloc_blocks(handle_t *h return ret; failed_out: for (i = 0; i <index; i++) - ext3_free_blocks(handle, inode, new_blocks[i], 1); + ext3_free_blocks(handle, inode, new_blocks[i], 1, 0); return ret; }
@@ -661,9 +661,9 @@ failed: ext3_journal_forget(handle, branch[i].bh); } for (i = 0; i <indirect_blks; i++) - ext3_free_blocks(handle, inode, new_blocks[i], 1); + ext3_free_blocks(handle, inode, new_blocks[i], 1, 0);
- ext3_free_blocks(handle, inode, new_blocks[i], num); + ext3_free_blocks(handle, inode, new_blocks[i], num, 0);
return err; } @@ -760,9 +760,9 @@ err_out: for (i = 1; i <= num; i++) { BUFFER_TRACE(where[i].bh, "call journal_forget"); ext3_journal_forget(handle, where[i].bh); - ext3_free_blocks(handle,inode,le32_to_cpu(where[i-1].key),1); + ext3_free_blocks(handle,inode,le32_to_cpu(where[i-1].key),1,0); } - ext3_free_blocks(handle, inode, le32_to_cpu(where[num].key), blks); + ext3_free_blocks(handle, inode, le32_to_cpu(where[num].key), blks, 0);
return err; } @@ -1996,7 +1996,7 @@ static void ext3_clear_blocks(handle_t * } }
- ext3_free_blocks(handle, inode, block_to_free, count); + ext3_free_blocks(handle, inode, block_to_free, count, 1); }
/** @@ -2169,7 +2169,7 @@ static void ext3_free_branches(handle_t ext3_journal_test_restart(handle, inode); }
- ext3_free_blocks(handle, inode, nr, 1); + ext3_free_blocks(handle, inode, nr, 1, 0);
if (parent_bh) { /* --- linux-2.6.17/fs/ext3/xattr.c.zerofree3 2006-06-18 02:49:35.000000000 +0100 +++ linux-2.6.17/fs/ext3/xattr.c 2006-08-26 19:07:34.000000000 +0100 @@ -484,7 +484,7 @@ ext3_xattr_release_block(handle_t *handl ea_bdebug(bh, "refcount now=0; freeing"); if (ce) mb_cache_entry_free(ce); - ext3_free_blocks(handle, inode, bh->b_blocknr, 1); + ext3_free_blocks(handle, inode, bh->b_blocknr, 1, 0); get_bh(bh); ext3_forget(handle, 1, inode, bh, bh->b_blocknr); } else { @@ -804,7 +804,7 @@ inserted: new_bh = sb_getblk(sb, block); if (!new_bh) { getblk_failed: - ext3_free_blocks(handle, inode, block, 1); + ext3_free_blocks(handle, inode, block, 1, 0); error = -EIO; goto cleanup; } --- linux-2.6.17/include/linux/ext3_fs.h.zerofree3 2006-08-26 19:06:57.000000000 +0100 +++ linux-2.6.17/include/linux/ext3_fs.h 2006-08-26 19:07:34.000000000 +0100 @@ -376,6 +376,7 @@ struct ext3_inode { #define EXT3_MOUNT_QUOTA 0x80000 /* Some quota option set */ #define EXT3_MOUNT_USRQUOTA 0x100000 /* "old" user quota */ #define EXT3_MOUNT_GRPQUOTA 0x200000 /* "old" group quota */ +#define EXT3_MOUNT_ZEROFREE 0x400000 /* Zero freed blocks */
/* Compatibility, for having both ext2_fs.h and ext3_fs.h included at once */ #ifndef _LINUX_EXT2_FS_H @@ -745,7 +746,7 @@ extern int ext3_new_block (handle_t *, s extern int ext3_new_blocks (handle_t *, struct inode *, unsigned long, unsigned long *, int *); extern void ext3_free_blocks (handle_t *, struct inode *, unsigned long, - unsigned long); + unsigned long, int); extern void ext3_free_blocks_sb (handle_t *, struct super_block *, unsigned long, unsigned long, int *); extern unsigned long ext3_count_free_blocks (struct super_block *);
Ron Yorston wrote:
"Mindaugas" mind@bi.lt wrote:
I was asked if it is possible to zero unused space in ext3 partition?
I have a couple of patches that add a zerofree mount option to ext2 and ext3 filesystems. The ext2 version is much better tested and more complete: it zeros all file data blocks, directory blocks and extended attributes (though not inode data). The ext3 patch only handles file data, not metadata.
I've been meaning to submit these to LKML, but since you ask let's give them an airing here first.
Ron,
thanks for these patch's - I dont think we can have them included in any official centos-repository hosted kernel, but its good to know that people, should they need this, can get to them here.
- K
-
Karanbir Singh -
Mindaugas -
rado -
Ron Yorston -
Stephen John Smoogen