Hello all,
I am looking to build a larger array (6TB) using CentOS 4.4 to archive data to. We want to have the Windows server mount this array as a local drive so we were looking at iSCSI to do it. I have played with it in the past and gotten it to work in this combo, but I have a question about access to the data on the local (Centos) machine.
If I understand correctly, when I mount the device on windows, I need to format the array in a format that Windows (2003 server) can understand. Also, iSCSI only allows you to mount the array on one server at a time. Once I do that and write files to this array, is there anyway to access those files from the local machine (Centos)? I may want to do things like rsync to another location, copy files to another removable SATA disk, or just plain delete something.
Is it a choice of the OS so both windows and linux can read it? Little help with which one then. I know NTFS is still somewhat in its infancy.
Any thoughts would be much appreciated!
Andrew
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Andrew Cotter Sent: Friday, February 23, 2007 7:32 AM To: centos@centos.org Subject: [CentOS] iSCSI, windows, & local linux access
Hello all,
I am looking to build a larger array (6TB) using CentOS 4.4 to archive data to. We want to have the Windows server mount this array as a local drive so we were looking at iSCSI to do it. I have played with it in the past and gotten it to work in this combo, but I have a question about access to the data on the local (Centos) machine.
If I understand correctly, when I mount the device on windows, I need to format the array in a format that Windows (2003 server) can understand.
Yes, it will be an NTFS partition, but with the "plus" kernel you can use the read-only ntfs module to read the data on the Linux side, just take an LVM snapshot, use the loopback with the sector offset of the partition in the snapshot lv to mount it locally. Remove the snapshot when done.
1) Find the offset
# fdisk -l -u /dev/VG/lv-snap
Disk /dev/VG/lv-snap: XXX MB, XXX bytes 255 heads, 63 sectors/track, XXXX cylinders, total 0 sectors Units = sectors of 1 * 512 = 512 bytes
Device Boot Start End Blocks Id System /dev/VG/lv-snap1 *63 XXXXXX XXXXXXX 7 HPFS/NTFS
63 sectors = 63*512 = 32256 bytes
2) Mount into the offset using the loopback option
# mount -t ntfs -o loop,offset=32256 /dev/VG/lv-snap1 /mnt
Also, iSCSI only allows you to mount the array on oneserver at a time. Once I do that and write files to this array, is there anyway to access those files from the local machine (Centos)? I may want to do things like rsync to another location, copy files to another removable SATA disk, or just plain delete something.
The method above is read-only access, which is what I highly highly advise. If you want to delete something, delete it on the Windows server.
Hint: Look at the latest Sharepoint Services 3.0 for fine-grained file management and look at DFSR for file level replication of Sharepoint managed files.
Multiple Windows hosts can log into the iSCSI volume at the same time, but only 1 can mount the NTFS partition at a time, and you would use Microsoft Clustering to do that.
You can't rsync NTFS partitions, but you could use drbd and scheduled replication to block-level replicate it. Use drbd without heartbeat, have it sync up asynchronously using Prot A, once it is sync'd have it disconnect and run standalone with secondary in wait-for-connect, using 'cron' bring up the connection again, when it is sync'd bring it down again.
Is it a choice of the OS so both windows and linux can read it? Little help with which one then. I know NTFS is still somewhat in its infancy.
CentOS is a good platform, I use CentOS here to do exactly what you are talking about. I have Exchange, SQL and Virtual Server running off iSCSI.
Any thoughts would be much appreciated!
Andrew
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
Yes, it will be an NTFS partition, but with the "plus" kernel you can use the read-only ntfs module to read the data on the Linux side, just take an LVM snapshot, use the loopback with the sector offset of the partition in the snapshot lv to mount it locally. Remove the snapshot when done.
Does the snapshot take utilize the same amount of space? If I have 3TB of files, do a snapshot, do I then have 6TB of data on disk?
A little more background....
Why we are doing this is we are (re)building our array to store our disk-disk backups using BackupExec. The older versions did no mind using a smb share but since we have updated to 10d it will not connect and Symantec/Veritas does not support this. That is why we thought iSCSI would do the trick.
You can't rsync NTFS partitions, but you could use drbd and scheduled replication to block-level replicate it. Use drbd without heartbeat, have it sync up asynchronously using Prot A, once it is sync'd have it disconnect and run standalone with secondary in wait-for-connect, using 'cron' bring up the connection again, when it is sync'd bring it down again.
Where rysnc came in for us was we then take some of those backups and spool them off over a VPN to a remote site. In addition, we would take a SATA drive, mount it, copy some data, pull it, then take it offsite like people traditionally do with tape. I have read a bit about drdb and that might work. We would have to be selective somehow since we have a smaller array at the other end meant to only hold 2wks of backups.
We used to do this with a 1TB array and smb but our data has grown fairly rapidly.
Would something like NFS possibly suit us better?
Thanks for the suggestions!
Andrew
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of Andrew Cotter Sent: Friday, February 23, 2007 10:59 AM To: CentOS mailing list Subject: RE: [CentOS] iSCSI, windows, & local linux access
Yes, it will be an NTFS partition, but with the "plus"
kernel you can
use the read-only ntfs module to read the data on the Linux
side, just
take an LVM snapshot, use the loopback with the sector offset of the partition in the snapshot lv to mount it locally. Remove
the snapshot
when done.
Does the snapshot take utilize the same amount of space? If I have 3TB of files, do a snapshot, do I then have 6TB of data on disk?
No, the snapshot just stores the changes from the original volume for the duration of the snapshot from the point-in-time of when the snapshot is taken.
A little more background....
Why we are doing this is we are (re)building our array to store our disk-disk backups using BackupExec. The older versions did no mind using a smb share but since we have updated to 10d it will not connect and Symantec/Veritas does not support this. That is why we thought iSCSI would do the trick.
Hmmm, backups use large block io, I would definitely look at using the latest SVN of iscsitarget and using the block-io module, not just because I helped develop it, but because it performs large block io a lot faster than the file-io module.
You can't rsync NTFS partitions, but you could use drbd and
scheduled
replication to block-level replicate it. Use drbd without heartbeat, have it sync up asynchronously using Prot A, once it is
sync'd have it
disconnect and run standalone with secondary in
wait-for-connect, using
'cron' bring up the connection again, when it is sync'd
bring it down
again.
Where rysnc came in for us was we then take some of those backups and spool them off over a VPN to a remote site. In addition, we would take a SATA drive, mount it, copy some data, pull it, then take it offsite like people traditionally do with tape. I have read a bit about drdb and that might work. We would have to be selective somehow since we have a smaller array at the other end meant to only hold 2wks of backups.
If you take your 6TB volume and split it into LVs for each job type, you can selectively replicate LVs between storage units. Don't allocate all your 6TB right away, just allocate what you need for each job type now, then you can always grow a LV as the job grows. Simple take the iSCSI volume offline, lvresize it, bring it back online, use Windows volume resize (in 2003 I believe) and then you have a larger volume.
If LV contiguous space is a concern, which it should with such a large volume, start by breaking it into large regions, say 4 by creating 4 large LVs, then resize them down to what you need which will leave contiguous gaps in between for growth or other needs, once they are resized down then export them via iSCSI and create your Windows partition. Playing around with this concept will allow you to properly utilize the storage surface. The original IBM LVM had the concept of drive region allocation where you could have a LV allocate extends from the beginning (fastest), the middle (avg) and the end (slowest) of the vg, by using these regions (3) and some trickery you can accomplish the same type of affect.
We used to do this with a 1TB array and smb but our data has grown fairly rapidly.
It always does.
Would something like NFS possibly suit us better?
Not with Backup Exec being your application. No, iSCSI will definitely work a lot better for this application than NFS or CIFS.
Thanks for the suggestions!
Andrew
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
Andrew Cotter wrote:
Hello all,
I am looking to build a larger array (6TB) using CentOS 4.4 to archive data to. We want to have the Windows server mount this array as a local drive so we were looking at iSCSI to do it. I have played with it in the past and gotten it to work in this combo, but I have a question about access to the data on the local (Centos) machine.
If I understand correctly, when I mount the device on windows, I need to format the array in a format that Windows (2003 server) can understand. Also, iSCSI only allows you to mount the array on one server at a time. Once I do that and write files to this array, is there anyway to access those files from the local machine (Centos)? I may want to do things like rsync to another location, copy files to another removable SATA disk, or just plain delete something.
Is it a choice of the OS so both windows and linux can read it? Little help with which one then. I know NTFS is still somewhat in its infancy.
if its mounted as a block device, nothing else should touch it, read OR write. block devices are NOT sharable, with the exception of specialized filesystems like GFS or IBrix Fusion Even a read only access would have issues with metadata consistency, if windows is updating a directory or MFT or whatever, linux could see stale/mixed data and would just throw up
the only 'safe' way to do it would be for the windows machine to share the logical file system, and have the linux system access it as a smbmount via samba. or use replication running on the windows server (rsync from mingw, etc)
Windows 2003 server has its own snapshotting capabilities, btw.
-----Original Message----- From: centos-bounces@centos.org [mailto:centos-bounces@centos.org] On Behalf Of John R Pierce Sent: Friday, February 23, 2007 11:41 AM To: CentOS mailing list Subject: Re: [CentOS] iSCSI, windows, & local linux access
Andrew Cotter wrote:
Hello all,
I am looking to build a larger array (6TB) using CentOS 4.4
to archive
data to. We want to have the Windows server mount this array as a local drive so we were looking at iSCSI to do it. I have
played with
it in the past and gotten it to work in this combo, but I have a question about access to the data on the local (Centos) machine.
If I understand correctly, when I mount the device on
windows, I need
to format the array in a format that Windows (2003 server) can understand. Also, iSCSI only allows you to mount the
array on one
server at a time. Once I do that and write files to this array, is there anyway to access those files from the local machine
(Centos)? I
may want to do things like rsync to another location, copy files to another removable SATA disk, or just plain delete something.
Is it a choice of the OS so both windows and linux can read it? Little help with which one then. I know NTFS is still
somewhat in its
infancy.
if its mounted as a block device, nothing else should touch it, read OR write. block devices are NOT sharable, with the exception of specialized filesystems like GFS or IBrix Fusion Even a read only access would have issues with metadata consistency, if windows is updating a directory or MFT or whatever, linux could see stale/mixed data and would just throw up
Technically yes, if the volume has long periods of no activity though there should be no problem in taking a snapshot and mounting that. Understand that if the volume is being actively used by a backup process the data contained therein will not be consistent, but it will not harm the production volume, just the snapshot will not be considered prime.
the only 'safe' way to do it would be for the windows machine to share the logical file system, and have the linux system access it as a smbmount via samba. or use replication running on the windows server (rsync from mingw, etc)
Yes sharing it and mounting it back via CIFS is more reliable for a highly active volume, but if you plan to then copy-out data over the network it will slow things down to a crawl.
Windows 2003 server has its own snapshotting capabilities, btw.
Which IMHO are crap. If it's backup jobs that are accessed during a run, but otherwise sit there unused, then you can snapshot them off LVM fairly safely.
-Ross
______________________________________________________________________ This e-mail, and any attachments thereto, is intended only for use by the addressee(s) named herein and may contain legally privileged and/or confidential information. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this e-mail, and any attachments thereto, is strictly prohibited. If you have received this e-mail in error, please immediately notify the sender and permanently delete the original and any copy or printout thereof.
-
Andrew Cotter -
John R Pierce -
Ross S. W. Walker