Rodrigo Barbosa wrote:
On Fri, Jun 09, 2006 at 03:56:36PM -0400, Bowie Bailey wrote:
I have a backup spooling onto a DLT drive. Is there any tool that will let me monitor the data throughput on the device?
Throughput on a tape drive is very variable, depending on how you are writing/reading the data to it.
So for some meaningful numbers, you would need your backup solution to provide them.
I'm just looking for raw bps flowing to the device. I don't really care about compression and such at the moment, I just want a rough idea of what the drive is doing.
Bowie Bailey wrote:
I just want a rough idea of what the drive is doing.
Do a google search for 'dbuffer' and I think you will find something useful. I think it's on freshmeat, too. It's a filter. And it shows you the throughput it's getting over stdout. It's great for buffering data coming from a relatively slow pipe and going to a tape drive, as well.
-Steve
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, Jun 09, 2006 at 04:15:38PM -0400, Bowie Bailey wrote:
Rodrigo Barbosa wrote:
On Fri, Jun 09, 2006 at 03:56:36PM -0400, Bowie Bailey wrote:
I have a backup spooling onto a DLT drive. Is there any tool that will let me monitor the data throughput on the device?
Throughput on a tape drive is very variable, depending on how you are writing/reading the data to it.
So for some meaningful numbers, you would need your backup solution to provide them.
I'm just looking for raw bps flowing to the device. I don't really care about compression and such at the moment, I just want a rough idea of what the drive is doing.
There is no such thing. You "raw bps" will vary greatly depending on your blocksize and other factors.
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
On Friday 09 June 2006 16:35, Rodrigo Barbosa wrote:
On Fri, Jun 09, 2006 at 04:15:38PM -0400, Bowie Bailey wrote:
Rodrigo Barbosa wrote:
On Fri, Jun 09, 2006 at 03:56:36PM -0400, Bowie Bailey wrote:
I have a backup spooling onto a DLT drive. Is there any tool that will let me monitor the data throughput on the device?
Throughput on a tape drive is very variable, depending on how you are writing/reading the data to it.
So for some meaningful numbers, you would need your backup solution to provide them.
I'm just looking for raw bps flowing to the device. I don't really care about compression and such at the moment, I just want a rough idea of what the drive is doing.
There is no such thing. You "raw bps" will vary greatly depending on your blocksize and other factors.
I know this won't help you much but just assume that tapes are very slow ...
On Fri, 9 Jun 2006 at 4:40pm, Charles Lacroix wrote
I know this won't help you much but just assume that tapes are very slow ...
This is no longer a valid assumption with tapes. LTO3 (yes, I know that's not what the OP is using) gets 80MB/s natively. I wouldn't call that slow.
Rodrigo Barbosa wrote:
There is no such thing. You "raw bps" will vary greatly depending on your blocksize and other factors.
It's not 1996 anymore. The -C option to cpio and the block size for tar don't make the difference that they did back then.
I don't see any point in perpetuating the myth that with Windows it "just works" but with Linux you have to be a Guru to make things work.
Using a big enough buffer size to keep things from being toally inefficient it not rocket science. The kernel does that for you. And if it didn't, the best thing to do would be to steer clear of it.
-Steve
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, Jun 09, 2006 at 03:49:51PM -0500, Steve wrote:
Rodrigo Barbosa wrote:
There is no such thing. You "raw bps" will vary greatly depending on your blocksize and other factors.
It's not 1996 anymore. The -C option to cpio and the block size for tar don't make the difference that they did back then.
I don't see any point in perpetuating the myth that with Windows it "just works" but with Linux you have to be a Guru to make things work.
Using a big enough buffer size to keep things from being toally inefficient it not rocket science. The kernel does that for you. And if it didn't, the best thing to do would be to steer clear of it.
Maybe you should take your time to read the manpage for "mt". Also, check the "blocksize" configuration for your tape drive.
I'm not talking about cpio -C or dump -b or tar -b. I'm talking about "mt setblk" and "blocksize" for stinit.
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
Maybe you should take your time to read the manpage for "mt". Also, check the "blocksize" configuration for your tape drive.
I'm not talking about cpio -C or dump -b or tar -b. I'm talking about "mt setblk" and "blocksize" for stinit.
I'm quite familiar with mt's man page. I use 'setblk 0' for my clients' backups... but for compatibility reasons, not for performance ones. (Most OSes know about variable block size, BTW. Even SCO Open Server.) I see no reason to turn Bowie's simple question into a "this is really complicated" answer when it's not.
-Steve
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, Jun 09, 2006 at 04:28:38PM -0500, Steve wrote:
Maybe you should take your time to read the manpage for "mt". Also, check the "blocksize" configuration for your tape drive.
I'm not talking about cpio -C or dump -b or tar -b. I'm talking about "mt setblk" and "blocksize" for stinit.
I'm quite familiar with mt's man page. I use 'setblk 0' for my clients' backups... but for compatibility reasons, not for performance ones. (Most OSes know about variable block size, BTW. Even SCO Open Server.) I see no reason to turn Bowie's simple question into a "this is really complicated" answer when it's not.
Yes, this IS a complicated question. Not because it is "technicaly complicatd" but because "many factors affect it".
If you use "setblk 0", you are using "Variable Blocksize". Which just makes it more complicated to calculate the throughput, since there is no way for you to know what block size will be under use in a given moment.
But I'll give you the benefit of the doubt. Since you say it is not a complicated issue, please give us the throughput he is getting for this DLT Unit.
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
Rodrigo Barbosa wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, Jun 09, 2006 at 04:28:38PM -0500, Steve wrote:
Maybe you should take your time to read the manpage for "mt". Also, check the "blocksize" configuration for your tape drive.
I'm not talking about cpio -C or dump -b or tar -b. I'm talking about "mt setblk" and "blocksize" for stinit.
I'm quite familiar with mt's man page. I use 'setblk 0' for my clients' backups... but for compatibility reasons, not for performance ones. (Most OSes know about variable block size, BTW. Even SCO Open Server.) I see no reason to turn Bowie's simple question into a "this is really complicated" answer when it's not.
Yes, this IS a complicated question. Not because it is "technicaly complicatd" but because "many factors affect it".
If you use "setblk 0", you are using "Variable Blocksize". Which just makes it more complicated to calculate the throughput, since there is no way for you to know what block size will be under use in a given moment.
But I'll give you the benefit of the doubt. Since you say it is not a complicated issue, please give us the throughput he is getting for this DLT Unit.
Depends on the unit, of course. If the OS/Software is not delivering something reasonably close to that in the real world, then you have a problem. That's not complicated.
What Bowie asked was not what the rate was, but how he could measure what he is actually getting.
But I would be *very* interested to know how you could calculate the total throughput of a random drive if you *did* know the fixed block size.
-Steve
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, Jun 09, 2006 at 05:05:16PM -0500, Steve wrote:
But I would be *very* interested to know how you could calculate the total throughput of a random drive if you *did* know the fixed block size.
Knowing the blocksize used, I would just tell him to use "mt setblk" to the same block size, then use "dump" to dump a random ammount of that to the tape. Dump will provide you with the throughtput rate you want.
If you don't know the blocksize, or if you are using variable (0), you really can't measure it outside the specific application you are using to write the data to the tape.
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
Rodrigo Barbosa wrote:
But I would be *very* interested to know how you could calculate the total throughput of a random drive if you *did* know the fixed block size.
Knowing the blocksize used, I would just tell him to use "mt setblk" to the same block size, then use "dump" to dump a random ammount of that to the tape. Dump will provide you with the throughtput rate you want.
If you don't know the blocksize, or if you are using variable (0), you really can't measure it outside the specific application you are using to write the data to the tape.
Seems like it would be easier to just run it through something like buffer, mbuffer, or dbuffer and have one of those report the value second by second, in kilobytes per second.
-Steve
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, Jun 09, 2006 at 05:38:30PM -0500, Steve wrote:
But I would be *very* interested to know how you could calculate the total throughput of a random drive if you *did* know the fixed block size.
Knowing the blocksize used, I would just tell him to use "mt setblk" to the same block size, then use "dump" to dump a random ammount of that to the tape. Dump will provide you with the throughtput rate you want.
If you don't know the blocksize, or if you are using variable (0), you really can't measure it outside the specific application you are using to write the data to the tape.
Seems like it would be easier to just run it through something like buffer, mbuffer, or dbuffer and have one of those report the value second by second, in kilobytes per second.
Should work too, but is it possible to do with the application he is using ?
Actually, what IS the application he is using ?
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)
Rodrigo Barbosa wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, Jun 09, 2006 at 05:38:30PM -0500, Steve wrote:
But I would be *very* interested to know how you could calculate the total throughput of a random drive if you *did* know the fixed block size.
Knowing the blocksize used, I would just tell him to use "mt setblk" to the same block size, then use "dump" to dump a random ammount of that to the tape. Dump will provide you with the throughtput rate you want.
If you don't know the blocksize, or if you are using variable (0), you really can't measure it outside the specific application you are using to write the data to the tape.
Seems like it would be easier to just run it through something like buffer, mbuffer, or dbuffer and have one of those report the value second by second, in kilobytes per second.
Should work too, but is it possible to do with the application he is using ?
Actually, what IS the application he is using ?
Good question.
Let's assume that he is using "BackMeUpNow Proprietary Editition(tm)" newly ported from Win32 and running under Winelib.
Another strategy, then, might be to do something like:
# Clear the cache dd if=/dev/hda of=/dev/null bs=1M count=2048
# Run the backup utility ./BackMeUpNowProForLinux
And then monitor it with:
vmstat 60
What gets written to tape has to be read from disk first, right?
-Steve
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On Fri, Jun 09, 2006 at 06:29:41PM -0500, Steve wrote:
Actually, what IS the application he is using ?
Good question.
Let's assume that he is using "BackMeUpNow Proprietary Editition(tm)" newly ported from Win32 and running under Winelib.
Another strategy, then, might be to do something like:
# Clear the cache dd if=/dev/hda of=/dev/null bs=1M count=2048
# Run the backup utility ./BackMeUpNowProForLinux
And then monitor it with:
vmstat 60
What gets written to tape has to be read from disk first, right?
I'm not sure that would be enough. To get anything with at least 90% accuracy, he would have to run it for a good time. And considering other processes might mess up with the disk, it can give him problems.
Unless he is running at runlevel 1, when those occurences will be much more rare.
[]s
PS: Aren't we getting too much offtopic here ? If you want, we can move this discussion to the linux-practices list, or even simply offlist, then come back here with any conclusions (tentative or not).
- -- Rodrigo Barbosa "Quid quid Latine dictum sit, altum viditur" "Be excellent to each other ..." - Bill & Ted (Wyld Stallyns)