I want to set up a new CentOS install using version 7 and would like to experiment with various RAID levels. Anyone care to point out a tutorial?
TIA
Dave
On 9/7/2014 7:19 PM, Dave Stevens wrote:
I want to set up a new CentOS install using version 7 and would like to experiment with various RAID levels. Anyone care to point out a tutorial?
how many drives do you have for this experiment? whats the target usecase for the file systems on the raid(s)? whats the level of data resiliance required by said use case?
Raid only protects against one specific sort of failure, where an entire disk drive fails. It doesn't protect against data corruption, or system failure, or software failure or any other catastrophes.
On Sun, September 7, 2014 9:30 pm, John R Pierce wrote:
On 9/7/2014 7:19 PM, Dave Stevens wrote:
I want to set up a new CentOS install using version 7 and would like to experiment with various RAID levels. Anyone care to point out a tutorial?
how many drives do you have for this experiment? whats the target usecase for the file systems on the raid(s)? whats the level of data resiliance required by said use case?
Raid only protects against one specific sort of failure, where an entire disk drive fails. It doesn't protect against data corruption, or system failure,
Even more: system failure or power loss is more likely to destroy all data on software RAID than on a single drive when there is a lot of IO present (to the best of my understanding, loss of cache software RAID is using is more catastrophic compared to journaled filesystem under same circumstances - somebody may correct me). So, there may be worth thinking about hardware RAID.
Just my 2c.
Valeri
or software failure or any other catastrophes.
-- john r pierce 37N 122W somewhere on the middle of the left coast
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++
On 2014-09-08, Valeri Galtsev galtsev@kicp.uchicago.edu wrote:
Even more: system failure or power loss is more likely to destroy all data on software RAID than on a single drive when there is a lot of IO present (to the best of my understanding, loss of cache software RAID is using is more catastrophic compared to journaled filesystem under same circumstances - somebody may correct me). So, there may be worth thinking about hardware RAID.
I think an essential feature of any md RAID that's not a RAID1 is a UPS and a mechanism for a clean shutdown in case of extended power failure. (An md RAID1 might be okay in this instance, but I wouldn't want to risk it.) But this is true for any RAID, which is why many controllers come with a BBU (and if you don't have a BBU on your hardware RAID controller, then you absolutely need the UPS setup I described).
OTOH, the OP wasn't clear on what he was doing; perhaps he is just playing around, and doesn't care about data preservation at this time. If you're just testing performance then data integrity in the face of a power failure is less of a concern.
--keith
--keith
On 07/09/14 11:01 PM, Keith Keller wrote:
On 2014-09-08, Valeri Galtsev galtsev@kicp.uchicago.edu wrote:
Even more: system failure or power loss is more likely to destroy all data on software RAID than on a single drive when there is a lot of IO present (to the best of my understanding, loss of cache software RAID is using is more catastrophic compared to journaled filesystem under same circumstances - somebody may correct me). So, there may be worth thinking about hardware RAID.
I think an essential feature of any md RAID that's not a RAID1 is a UPS and a mechanism for a clean shutdown in case of extended power failure. (An md RAID1 might be okay in this instance, but I wouldn't want to risk it.) But this is true for any RAID, which is why many controllers come with a BBU (and if you don't have a BBU on your hardware RAID controller, then you absolutely need the UPS setup I described).
OTOH, the OP wasn't clear on what he was doing; perhaps he is just playing around, and doesn't care about data preservation at this time. If you're just testing performance then data integrity in the face of a power failure is less of a concern.
--keith
A UPS is certainly better than nothing, but I would not consider it safe enough. A BBU/FBU will protect you if the node loses power, right up to the failure of the PSU(s). I've seen shorted cable harnesses taking out servers with redundant power supplies, popped breakers in PDUs/UPSes, knocked out power cords, etc. So a UPS is not a silver-bullet to safe write-back caching in software arrays. Good, yes, but not perfect.
Quoting Digimer lists@alteeve.ca:
On 07/09/14 11:01 PM, Keith Keller wrote:
On 2014-09-08, Valeri Galtsev galtsev@kicp.uchicago.edu wrote:
Even more: system failure or power loss is more likely to destroy all data on software RAID than on a single drive when there is a lot of IO present (to the best of my understanding, loss of cache software RAID is using is more catastrophic compared to journaled filesystem under same circumstances - somebody may correct me). So, there may be worth thinking about hardware RAID.
I think an essential feature of any md RAID that's not a RAID1 is a UPS and a mechanism for a clean shutdown in case of extended power failure. (An md RAID1 might be okay in this instance, but I wouldn't want to risk it.) But this is true for any RAID, which is why many controllers come with a BBU (and if you don't have a BBU on your hardware RAID controller, then you absolutely need the UPS setup I described).
OTOH, the OP wasn't clear on what he was doing; perhaps he is just playing around, and doesn't care about data preservation at this time. If you're just testing performance then data integrity in the face of a power failure is less of a concern.
--keith
A UPS is certainly better than nothing, but I would not consider it safe enough. A BBU/FBU will protect you if the node loses power, right up to the failure of the PSU(s). I've seen shorted cable harnesses taking out servers with redundant power supplies, popped breakers in PDUs/UPSes, knocked out power cords, etc. So a UPS is not a silver-bullet to safe write-back caching in software arrays. Good, yes, but not perfect.
This is a pretty interesting discussion but has not revealed an on-line tutorial. Anyone?
Dave
-- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Mon, Sep 8, 2014 at 2:58 PM, Dave Stevens geek@uniserve.com wrote:
This is a pretty interesting discussion but has not revealed an on-line tutorial. Anyone?
Dave
Use mdadm commands. ~]$ man mdadm
I don't follow any one "tutorial" or "guide" in particular, I consult the mdadm manpage in most cases.
Though, the following URLs look to have suitable information (at a quick glance): https://raid.wiki.kernel.org/index.php/RAID_setup http://www.ducea.com/2009/03/08/mdadm-cheat-sheet/ http://www.mysolutions.it/tutorial-mdadm-software-raid-ubuntu-debian-systems...
On 07/09/14 10:43 PM, Valeri Galtsev wrote:
On Sun, September 7, 2014 9:30 pm, John R Pierce wrote:
On 9/7/2014 7:19 PM, Dave Stevens wrote:
I want to set up a new CentOS install using version 7 and would like to experiment with various RAID levels. Anyone care to point out a tutorial?
how many drives do you have for this experiment? whats the target usecase for the file systems on the raid(s)? whats the level of data resiliance required by said use case?
Raid only protects against one specific sort of failure, where an entire disk drive fails. It doesn't protect against data corruption, or system failure,
Even more: system failure or power loss is more likely to destroy all data on software RAID than on a single drive when there is a lot of IO present (to the best of my understanding, loss of cache software RAID is using is more catastrophic compared to journaled filesystem under same circumstances - somebody may correct me). So, there may be worth thinking about hardware RAID.
Just my 2c.
Valeri
Valeri makes an excellent point, which I would like to elaborate on;
Hardware RAID *with* flash-backed/battery-backed cache. I find it endlessly fascinating how many machines out there have hardware RAID with BBU/FBU. When using write-back caching without a battery leaves you in no better position.
Note that if you do get hardware RAID with BBU/FBU, be sure the cache policy is set to "Write-Back with BBU" (or your controllers wording). The idea here is that, if the battery/caps fail/drain, the controller switches to write-through (no) caching.
I'm not so familiar with software RAID, but I would be surprised if there isn't a way to force write-through caching. If this is possible, then Valeri's concern can be addressed (at the cost of performance).
digimer
On 9/7/2014 8:09 PM, Digimer wrote:
I'm not so familiar with software RAID, but I would be surprised if there isn't a way to force write-through caching. If this is possible, then Valeri's concern can be addressed (at the cost of performance).
software raid on enterprise grade JBOD *is* write-through caching. the OS will only cache writes til an fsync/fdatasync/etc and then it will flush them to the md device, which will immediately flush them to the physical media. where it goes sideways is when you use cheap consumer grade desktop drives, those often lie about write complete to improve windows performance... but these would be a problem with or without mdraid, indeed, they would be a problem with hardware raid, too.
this is why I really like ZFS (on solaris and bsd, at least), because it timestamps and checksums every block it writes to disk... a conventional raid1, if the two copies don't match, you don't know which one is the 'right' one. the ZFS scrub process will check these timestamps and crc's, and correct the 'wrong' block.
I did a fair bit of informal(*) benchmarking of some storage systems at work before they were deployed. using a hardware raid card such as a LSI Megaraid 9260 with 2GB BBU cache, (or HP P410i or similar) is most certainly faster at transactional database style random read/write testing than using a simple SAS2 JBOD controller. But using mdraid with the Megaraid configured just as a bunch of disks, gave the same results if writeback caching was enabled in the controller. At different times, using different-but-similar SAS2 raid cards, I benchmarked 10-20 disk raids in various levels like 10, 5, 6, 50, and 60, built with 7200RPM SAS2 'nearline server' drives, 7200rpm SATA desktop drives, and 15000rpm SAS2 enterprise server drives. For an OLTP style database server under high concurrency and high transaction/second rates, raid10 with lots of 15k disks is definitely the way to go. for bulk file storage that's write-once and read-mostly, raid 5, 6, 60 perform adequately.
(*) my methodology was ad-hoc rather than rigorous, I primarily observed trends, so I can't publish any hard data to back these conclusions.. My tests including postgresql with pgbench, and various bonnie++ and iozone tests. most of these tests were on Xeon X5600 class servers with 8-12 cores, and 24-48GB ram.
Raid only protects against one specific sort of failure, where an entire disk drive fails. It doesn't protect against data corruption, or system failure, or software failure or any other catastrophes.
+1
Try ZFS
On 09/08/2014 10:00 PM, Andrew Holway wrote:
+1
Try ZFS
How stable is it on linux?
Eliezer
On 09/09/14 05:36 PM, Eliezer Croitoru wrote:
On 09/08/2014 10:00 PM, Andrew Holway wrote:
+1
Try ZFS
How stable is it on linux?
Eliezer
The problem with ZFS on linux is license related more than technical.
On 09/10/2014 02:33 AM, Digimer wrote:
The problem with ZFS on linux is license related more than technical.
It exists for ubutnu so I can use it from a ppa for testing. I would like to understand more about this license issue. If you can sound me with more about it will help me understand the issue.
Thanks, Eliezer
On 10/09/14 05:35 PM, Eliezer Croitoru wrote:
On 09/10/2014 02:33 AM, Digimer wrote:
The problem with ZFS on linux is license related more than technical.
It exists for ubutnu so I can use it from a ppa for testing. I would like to understand more about this license issue. If you can sound me with more about it will help me understand the issue.
Thanks, Eliezer
http://zfsonlinux.org/faq.html#WhatAboutTheLicensingIssue
https://en.wikipedia.org/wiki/ZFS#Linux
Quoting Digimer lists@alteeve.ca:
On 10/09/14 05:35 PM, Eliezer Croitoru wrote:
On 09/10/2014 02:33 AM, Digimer wrote:
The problem with ZFS on linux is license related more than technical.
It exists for ubutnu so I can use it from a ppa for testing. I would like to understand more about this license issue. If you can sound me with more about it will help me understand the issue.
Thanks, Eliezer
http://zfsonlinux.org/faq.html#WhatAboutTheLicensingIssue
https://en.wikipedia.org/wiki/ZFS#Linux
-- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Dear All,
This list reminds me of the wizards in the Terry Pratchett novels - confronted with a need to take action they would by far rather discuss all possibilities, however remote, than address the need in a simple way, after all they're WIZARDS.
The discussion has been pretty interesting, I've figured out what I wanted to know through other means.
Thanks for the perspectives.
Dave
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 11.09.2014 00:45, Dave Stevens wrote:
Dear All,
This list reminds me of the wizards in the Terry Pratchett novels
- confronted with a need to take action they would by far rather
discuss all possibilities, however remote, than address the need in a simple way, after all they're WIZARDS.
The discussion has been pretty interesting, I've figured out what I wanted to know through other means.
Thanks for the perspectives.
Dave
I am under the same sad (and a little funny) impression.
But for people who may have the same problem I guess here is a good answer, obvious somehow, but hey, nobody gave it until now:
Just follow upstream documentation:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/htm...
HTH
Sven
On Thu, Sep 11, 2014 at 01:00:02AM +0200, Sven Kieske wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 11.09.2014 00:45, Dave Stevens wrote:
Dear All,
This list reminds me of the wizards in the Terry Pratchett novels
- confronted with a need to take action they would by far rather
discuss all possibilities, however remote, than address the need in a simple way, after all they're WIZARDS.
The discussion has been pretty interesting, I've figured out what I wanted to know through other means.
Thanks for the perspectives.
Dave
I am under the same sad (and a little funny) impression.
But for people who may have the same problem I guess here is a good answer, obvious somehow, but hey, nobody gave it until now:
Just follow upstream documentation:
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/htm...
Page one. Define RAID. Page 2, who should use it. Configuring Raid (after a few more wasted pages). Read the man page. It seems too sparse for a beginner and, with such sparseness, not much use to an experienced admin.
I can't see that being very useful to someone with little or no RAID experience. (And actually, it's so sparse that the experienced won't need the little bit of suggestion it gives.)
In contrast, the CentOS wiki article, if running CentOS 5 or 6, gives an easy to follow guide, complete with commands one might actually type. (At least some of the instructions don't seem to work with CentOS 7 though)
On 11.09.2014 01:27, Scott Robbins wrote:
In contrast, the CentOS wiki article, if running CentOS 5 or 6, gives an easy to follow guide, complete with commands one might actually type. (At least some of the instructions don't seem to work with CentOS 7 though)
Did you read the topic of this thread? This is about version 7 ;)
furthermore, which wiki article are you referring to? the search yields many results like: http://wiki.centos.org/HowTos/CentOS5ConvertToRAID?highlight=%28raid%29
which might be not what you want, depending on what you want. I'm no fan of copy and paste tutorials if they are not used just for very specific use cases. and just using a centos 5 tutorial on centos 7 seems not to be the best way to start things.
On Thu, Sep 11, 2014 at 09:48:36PM +0200, Sven Kieske wrote:
On 11.09.2014 01:27, Scott Robbins wrote:
In contrast, the CentOS wiki article, if running CentOS 5 or 6, gives an easy to follow guide, complete with commands one might actually type. (At least some of the instructions don't seem to work with CentOS 7 though)
Did you read the topic of this thread? This is about version 7 ;)
Sorry, I wasn't clear, and I apologize. My point is that the tutorial, which at least judging from my experience, won't work in 7, is detailed and helpful for both novice and the more experienced. In contrast, the upstream seems as if they basically paid someone to dress up "read the man page," in 10 pages.
furthermore, which wiki article are you referring to? the search yields many results like: http://wiki.centos.org/HowTos/CentOS5ConvertToRAID?highlight=%28raid%29
Again, apologies. Some mental shorthand on my part, as I was recently using that article in a work situation. I meant this one.
http://wiki.centos.org/HowTos/Install_On_Partitionable_RAID1
Which I suppose you can call it cut and paste. Now, I did, for my own knowledge, see if I could get that to work on CentOS-7, but I couldn't.
which might be not what you want, depending on what you want. I'm no fan of copy and paste tutorials if they are not used just for very specific use cases. and just using a centos 5 tutorial on centos 7 seems not to be the best way to start things.
I don't know how much knowledge the OP has or doesn't have. I don't think it's unreasonable to expect instructions to include examples and commands, but we're now very much outside the scope of this thread. :)
TL;DR I wasn't clear. My point, in one sentence is that I don't consider the RH documentation very good, and a tutorial for CentOS 7, written in the style of the tutorial to which I link, would be far more helpful.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Thanks for the clarification and the link!
No need to apologize, I tend to forget thinks myself from time to time.
kind regards
Sven
On Fri, Sep 12, 2014 at 01:03:17AM +0200, Sven Kieske wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
Thanks for the clarification and the link!
No need to apologize, I tend to forget thinks myself from time to time.
Thanks for the understanding. :) Mental shorthand is a bad habit of mine.
On 10/09/14 06:45 PM, Dave Stevens wrote:
Quoting Digimer lists@alteeve.ca:
On 10/09/14 05:35 PM, Eliezer Croitoru wrote:
On 09/10/2014 02:33 AM, Digimer wrote:
The problem with ZFS on linux is license related more than technical.
It exists for ubutnu so I can use it from a ppa for testing. I would like to understand more about this license issue. If you can sound me with more about it will help me understand the issue.
Thanks, Eliezer
http://zfsonlinux.org/faq.html#WhatAboutTheLicensingIssue
https://en.wikipedia.org/wiki/ZFS#Linux
-- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Dear All,
This list reminds me of the wizards in the Terry Pratchett novels - confronted with a need to take action they would by far rather discuss all possibilities, however remote, than address the need in a simple way, after all they're WIZARDS.
The discussion has been pretty interesting, I've figured out what I wanted to know through other means.
Thanks for the perspectives.
Dave
Hi Dave,
I can understand your feeling, but I need to say that "storage", as a topic, is a very big one. People form entire careers around the topic. So when discussing storage without a specific context, conversations like this are inevitable.
All the points that have been made in this thread are valid and important. So I suppose the better thing would be, if you were still looking for answers, would be to ask the question with a particular use-case in mind. That would allow people to stay more focused in their answers.
Cheers!
digimer
Quoting Digimer lists@alteeve.ca:
On 10/09/14 06:45 PM, Dave Stevens wrote:
Quoting Digimer lists@alteeve.ca:
On 10/09/14 05:35 PM, Eliezer Croitoru wrote:
On 09/10/2014 02:33 AM, Digimer wrote:
The problem with ZFS on linux is license related more than technical.
It exists for ubutnu so I can use it from a ppa for testing. I would like to understand more about this license issue. If you can sound me with more about it will help me understand the issue.
Thanks, Eliezer
http://zfsonlinux.org/faq.html#WhatAboutTheLicensingIssue
https://en.wikipedia.org/wiki/ZFS#Linux
-- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Dear All,
This list reminds me of the wizards in the Terry Pratchett novels - confronted with a need to take action they would by far rather discuss all possibilities, however remote, than address the need in a simple way, after all they're WIZARDS.
The discussion has been pretty interesting, I've figured out what I wanted to know through other means.
Thanks for the perspectives.
Dave
Hi Dave,
I can understand your feeling, but I need to say that "storage", as a topic, is a very big one. People form entire careers around the topic. So when discussing storage without a specific context, conversations like this are inevitable.
All the points that have been made in this thread are valid and important. So I suppose the better thing would be, if you were still looking for answers, would be to ask the question with a particular use-case in mind. That would allow people to stay more focused in their answers.
Cheers!
digimer
Quoting Digimer lists@alteeve.ca:
On 10/09/14 06:45 PM, Dave Stevens wrote:
Quoting Digimer lists@alteeve.ca:
On 10/09/14 05:35 PM, Eliezer Croitoru wrote:
On 09/10/2014 02:33 AM, Digimer wrote:
The problem with ZFS on linux is license related more than technical.
It exists for ubutnu so I can use it from a ppa for testing. I would like to understand more about this license issue. If you can sound me with more about it will help me understand the issue.
Thanks, Eliezer
http://zfsonlinux.org/faq.html#WhatAboutTheLicensingIssue
https://en.wikipedia.org/wiki/ZFS#Linux
-- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education? _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
Dear All,
This list reminds me of the wizards in the Terry Pratchett novels - confronted with a need to take action they would by far rather discuss all possibilities, however remote, than address the need in a simple way, after all they're WIZARDS.
The discussion has been pretty interesting, I've figured out what I wanted to know through other means.
Thanks for the perspectives.
Dave
Hi Dave,
I can understand your feeling, but I need to say that "storage", as a topic, is a very big one. People form entire careers around the topic. So when discussing storage without a specific context, conversations like this are inevitable.
All the points that have been made in this thread are valid and important. So I suppose the better thing would be, if you were still looking for answers, would be to ask the question with a particular use-case in mind. That would allow people to stay more focused in their answers.
Cheers!
digimer
Ok, maybe one more. I manage a server with CentOS 5.10 that has a raid 10 array with a hot spare. It was well set up by someone else and has worked very well. It also has a Xen kernel and several VMs, also all working well.
The age of the OS and accumulated cruft in the application side, together with the absence of the person who did the original setup, have me thinking about a new clean install - first on a transitional box for continuity, then a new config on the current hardware with the same basic design but more up-to-date; we are now a long way from version 5.3.
I had seen reference to a much improved raid installation procedure in version 7. I can now confirm that the process is indeed much simpler and so I have been able to get on with performance testing in various scenarios, which was what I wanted to do. My experience has been that of you want a tutorial on any topic the internet is flooded with the. But I didn't see one for this topic, so I thought a routine query on this list would get me a starting point.
That didn't exactly happen. But there's lots of food for thought in the answers I got and I'll be able to take at least some of that forward. And if I'm motivated enough may I'LL make the tutorial!
Dave
-- Digimer Papers and Projects: https://alteeve.ca/w/ What if the cure for cancer is trapped in the mind of a person without access to education?
On 09/11/2014 01:27 AM, Digimer wrote:
On 10/09/14 05:35 PM, Eliezer Croitoru wrote:
On 09/10/2014 02:33 AM, Digimer wrote:
The problem with ZFS on linux is license related more than technical.
It exists for ubutnu so I can use it from a ppa for testing. I would like to understand more about this license issue. If you can sound me with more about it will help me understand the issue.
Thanks, Eliezer
Thanks!
On 2014-09-09, Eliezer Croitoru eliezer@ngtech.co.il wrote:
How stable is it on linux?
My own experience is very limited, but I have heard from a handful of reliable sources who are very happy with ZFS. As Digimer noted, much of the challenge is in the licensing, not a technical issue.
ZFS has been packaged for RHEL/CentOS:
http://zfsonlinux.org/epel.html
So it should be fairly straightforward to get things working. (If you've installed your own kernel you may need the generic RPM packages instead.)
--keith
On 09/08/2014 09:00 PM, Andrew Holway wrote:
Try ZFS http://zfsonlinux.org/
Maybe you can tune ZFS further, but I tried it in userspace (with FUSE) and reading was a almost 5 times slower than MDADM.
On 2014-09-15, Chris chris2014@postbox.xyz wrote:
On 09/08/2014 09:00 PM, Andrew Holway wrote:
Try ZFS http://zfsonlinux.org/
Maybe you can tune ZFS further, but I tried it in userspace (with FUSE) and reading was a almost 5 times slower than MDADM.
Just running ZFS in the kernel is going to be a lot faster than running it under FUSE. You should try it in kernel space before making any final decisions.
--keith
Maybe you can tune ZFS further, but I tried it in userspace (with FUSE) and reading was a almost 5 times slower than MDADM.
ZFS on Linux is backed by the US government as ZFS will be used as the primary filesystem to back the parallel distributed filesystem 'Lustre'. Lustre is used in the majority of the worlds supercomputers as the primary filesystem (When you gotta have 2TB/s of IO over 100PB then Lustre is pretty much the only viable opensource option)
To set expectation. Actually, the most recent release (0.63) of ZFS on Linux is not that quick. In certain circumstances is actually quite bad. This is almost by design as the developers of ZFS have been focussing on stability and have pretty much ignored enhancing performance. This will change in the near future.
On 9/15/2014 01:37, Andrew Holway wrote:
To set expectation. Actually, the most recent release (0.63) of ZFS on Linux is not that quick.
Compared to what? To ZFS on Solaris, ZFS on FreeBSD, or ext4 on Linux?
Any comparison between ZFS and non-ZFS probably overlooks things like fully-checksummed data (not just metadata) and redundant copies. ZFS will always be slower than filesystems without these features. TANSTAAFL.
Any comparison between ZFS and non-ZFS probably overlooks things like fully-checksummed data (not just metadata) and redundant copies. ZFS will always be slower than filesystems without these features. TANSTAAFL.
Not really true. It hugely depends on your workload. For example, if you have a 20TB filesystem with 128GB of "ARC" (adaptive replacement cache) then ZFS will be many many times faster then ext4 assuming that the "hot data" is under 128GB as all reads will come from memory or a dedicated cache SSD. If however you are streaming the whole 20TB from the filesystem then the cache makes no difference and you just see the performance of the disks. The checksumming for example does not typically add a performance penalty as this calculation is done in parallel to normal disk operations.
The atomistic write mechanism of ZFS can be hundreds of times faster than EXT in cases where your IO is random SYNC assuming you have a dedicated ZIL (ZFS intent log) which keeps the journal. Read up on ZFS transaction groups for more details.
On 9/15/2014 14:48, Andrew Holway wrote:
Any comparison between ZFS and non-ZFS probably overlooks things like fully-checksummed data (not just metadata) and redundant copies. ZFS will always be slower than filesystems without these features. TANSTAAFL.
Not really true. It hugely depends on your workload. For example, if you have a 20TB filesystem with 128GB of "ARC" (adaptive replacement cache) then ZFS will be many many times faster then ext4 assuming that the "hot data" is under 128GB as all reads will come from memory or a dedicated cache SSD. If however you are streaming the whole 20TB from the filesystem then the cache makes no difference and you just see the performance of the disks. The checksumming for example does not typically add a performance penalty as this calculation is done in parallel to normal disk operations.
The SSD and second CPU core are not free.
ZFS does more than pretty much every other mainstream filesystem, and hence needs more resources when you use its additional features.
My only point is that there are people who will compare a ZFS pool with all the features turned on to a bare-bones ext4 setup, then complain that ZFS is "slow."
The atomistic write mechanism of ZFS can be hundreds of times faster than EXT in cases where your IO is random SYNC assuming you have a dedicated ZIL (ZFS intent log) which keeps the journal. Read up on ZFS transaction groups for more details.
The "ZFS is slow" crowd generally doesn't care about reliable fsyncs. Hence why most PC OSes lie through their teeth when you tell them to sync a block to disk:
On Mon, September 15, 2014 4:49 pm, Andrew Holway wrote:
The SSD and second CPU core are not free.
Where I come from, the streets are paved with SLC
Is it Salt Lake City that you are from? (that is so reach with Second Level Cache... That is what you actually meant I figure)
Am I the only one who is tempted to say: people, could you kindly start deciphering your abbreviations. I know, I know, computed science uses _that_ abbreviation for years. But we definitely have people without computer science degree (not me, I do have one ;-) on the list who will be happier to learn terms and abbreviations if it is mentioned what stands for what.
Valeri
++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++
On Mon, 15 Sep 2014, Valeri Galtsev wrote:
On Mon, September 15, 2014 4:49 pm, Andrew Holway wrote:
The SSD and second CPU core are not free.
Where I come from, the streets are paved with SLC
Is it Salt Lake City that you are from? (that is so reach with Second Level Cache... That is what you actually meant I figure)
Am I the only one who is tempted to say: people, could you kindly start deciphering your abbreviations. I know, I know, computed science uses _that_ abbreviation for years. But we definitely have people without computer science degree (not me, I do have one ;-) on the list who will be happier to learn terms and abbreviations if it is mentioned what stands for what.
In defense of the OP ("original poster"), he mentioned "SLC" in
1. a throw-away line meant as a joke,
2. the clear context of SSD ("solid-state drive(s)"), where the distinction between MLC ("multi-level cell") and SLC ("single-level cell") technology has been widely discussed.
In this case, I'd give more weight to point number 1: I'd estimate that 493 out of 500 readers of this list would have immediately grasped he was making a joke. Even if they didn't get the SLC reference, they understood that jokes can be safely ignored without missing any important bits of the discussion.
In the case of point #2, I'd say that abuse is definitely in the eye of the beholder. Who is the arbiter of when an acronym is obvious or obscure? I mean that as a serious question. Who gets to say that USB is an acceptable acronym while SLC is not? If you know, how does one become that person?
On 9/15/2014 16:54, Paul Heinlein wrote:
On Mon, 15 Sep 2014, Valeri Galtsev wrote:
- a throw-away line meant as a joke,
I didn't take it as a joke so much as a comment that where he works, high-end hardware is available for the asking. SLC is the most expensive sort of SSD; if it's so readily available that he can joke about using them for paving tiles, he's pooh-poohing my observation that adding an SSD to a ZFS pool to accelerate it isn't free. Where he works, it effectively *is* free.
Where *I* work, a freed-up second-hand 40 GB first-generation SSD is a welcome boon.
Who gets to say that USB is an acceptable acronym while SLC is not? If you know, how does one become that person?
Prescriptivist lexicographers are popinjays that try to tell you what the legal words are.
The descriptivists are the real lexicographers, because they merely try to figure out which words are actually being used in a widespread fashion and try to document the meaning(s) of those words. They understand how language actually works.
SLC is perfectly cromulent to a descriptivist. It's in wide use in a large community, and a large fraction of that community agrees on its meaning.
Keep in mind that there are only about 17,000 three-letter acronyms. Most will have conflicts.
Yes, most. XQZ is yet unused according to http://www.acronymfinder.com/
On Tue, September 16, 2014 12:03 pm, Warren Young wrote:
On 9/15/2014 16:54, Paul Heinlein wrote:
On Mon, 15 Sep 2014, Valeri Galtsev wrote:
- a throw-away line meant as a joke,
I didn't take it as a joke so much as a comment that where he works, high-end hardware is available for the asking. SLC is the most expensive sort of SSD; if it's so readily available that he can joke about using them for paving tiles, he's pooh-poohing my observation that adding an SSD to a ZFS pool to accelerate it isn't free. Where he works, it effectively *is* free.
Where *I* work, a freed-up second-hand 40 GB first-generation SSD is a welcome boon.
Who gets to say that USB is an acceptable acronym while SLC is not? If you know, how does one become that person?
Prescriptivist lexicographers are popinjays that try to tell you what the legal words are.
The descriptivists are the real lexicographers, because they merely try to figure out which words are actually being used in a widespread fashion and try to document the meaning(s) of those words. They understand how language actually works.
SLC is perfectly cromulent to a descriptivist. It's in wide use in a large community, and a large fraction of that community agrees on its meaning.
I would comment here even though you answered someone's else argument - put much better than I can do. (after all I'm not linguist in any language, not even in my native one ;-)
My take on it is it is your decision to be understood by me or not. If I'm anxious to decipher what is not obvious I will look it up. I often do (lying of course: I do it only occasionally). And I can easily find out that SLC stands for Salt Lake City (and quite a few other things). But, of course, if I didn't spare enough effort I may not deserve to understand discussions above some technical (or abbreviational ;-) level.
As they will say in volleyball: the ball is on other side.
Valeri
Keep in mind that there are only about 17,000 three-letter acronyms. Most will have conflicts.
Yes, most. XQZ is yet unused according to http://www.acronymfinder.com/ _______________________________________________ CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++
Warren Young wrote:
On 9/15/2014 16:54, Paul Heinlein wrote:
On Mon, 15 Sep 2014, Valeri Galtsev wrote:
- a throw-away line meant as a joke,
I didn't take it as a joke so much as a comment that where he works, high-end hardware is available for the asking. SLC is the most expensive sort of SSD; if it's so readily available that he can joke about using them for paving tiles, he's pooh-poohing my observation that adding an SSD to a ZFS pool to accelerate it isn't free. Where he works, it effectively *is* free.
Where *I* work, a freed-up second-hand 40 GB first-generation SSD is a welcome boon.
We don't use 'em. We get more annoyed at OEMs who really, *really* want you to buy their new systems with the k3wl 2.5" drives, that are amazingly expensive, and you're lucky to find 'em at 1TB, when you really want to slap in three 4TB WD Reds....
Who gets to say that USB is an acceptable acronym while SLC is not? If
you know, how does one become that person?
Prescriptivist lexicographers are popinjays that try to tell you what the legal words are.
The descriptivists are the real lexicographers, because they merely try to figure out which words are actually being used in a widespread fashion and try to document the meaning(s) of those words. They understand how language actually works.
SLC is perfectly cromulent to a descriptivist. It's in wide use in a large community, and a large fraction of that community agrees on its meaning.
Keep in mind that there are only about 17,000 three-letter acronyms. Most will have conflicts.
Yes, most. XQZ is yet unused according to http://www.acronymfinder.com/
http://en.wikipedia.org/wiki/SLC
Given the upcoming elections, I like Scottish Law Commission, though where I work, Single-Layer Cell might be more appropos.
mark, who works for a TLC (Three Letter Company)
I would strongly suggest anyone interested in ZFS on Linux join the zfs-discuss list. http://zfsonlinux.org/lists.html There is a fairly good signal to noise ratio.
On 16 September 2014 20:17, Andrew Holway andrew.holway@gmail.com wrote:
Referendum.
I sit, and type, corrected.
:)
On Tue, September 16, 2014 12:47 pm, m.roth@5-cent.us wrote:
Warren Young wrote:
On 9/15/2014 16:54, Paul Heinlein wrote:
On Mon, 15 Sep 2014, Valeri Galtsev wrote:
- a throw-away line meant as a joke,
I didn't take it as a joke so much as a comment that where he works, high-end hardware is available for the asking. SLC is the most expensive sort of SSD; if it's so readily available that he can joke about using them for paving tiles, he's pooh-poohing my observation that adding an SSD to a ZFS pool to accelerate it isn't free. Where he works, it effectively *is* free.
Where *I* work, a freed-up second-hand 40 GB first-generation SSD is a welcome boon.
We don't use 'em. We get more annoyed at OEMs who really, *really* want you to buy their new systems with the k3wl 2.5" drives
What is 'k3wl 2.5' drives"? Just fascinated by what you are saying, yet fail to find what it is. Even switched from my usual "DuckDuckGo" searching engine to google, yet no dice... I feel like a cave man in the middle of today's New York City...
Valeri
, that are amazingly
expensive, and you're lucky to find 'em at 1TB, when you really want to slap in three 4TB WD Reds....
Who gets to say that USB is an acceptable acronym while SLC is not? If
you know, how does one become that person?
Prescriptivist lexicographers are popinjays that try to tell you what the legal words are.
The descriptivists are the real lexicographers, because they merely try to figure out which words are actually being used in a widespread fashion and try to document the meaning(s) of those words. They understand how language actually works.
SLC is perfectly cromulent to a descriptivist. It's in wide use in a large community, and a large fraction of that community agrees on its meaning.
Keep in mind that there are only about 17,000 three-letter acronyms. Most will have conflicts.
Yes, most. XQZ is yet unused according to http://www.acronymfinder.com/
http://en.wikipedia.org/wiki/SLC
Given the upcoming elections, I like Scottish Law Commission, though where I work, Single-Layer Cell might be more appropos.
mark, who works for a TLC (Three Letter Company)
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics University of Chicago Phone: 773-702-4247 ++++++++++++++++++++++++++++++++++++++++
Valeri Galtsev wrote:
On Tue, September 16, 2014 12:47 pm, m.roth@5-cent.us wrote:
Warren Young wrote:
On 9/15/2014 16:54, Paul Heinlein wrote:
On Mon, 15 Sep 2014, Valeri Galtsev wrote:
<snip>
that adding an SSD to a ZFS pool to accelerate it isn't free. Where he works, it effectively *is* free.
Where *I* work, a freed-up second-hand 40 GB first-generation SSD is a welcome boon.
We don't use 'em. We get more annoyed at OEMs who really, *really* want you to buy their new systems with the k3wl 2.5" drives
What is 'k3wl 2.5' drives"? Just fascinated by what you are saying, yet fail to find what it is. Even switched from my usual "DuckDuckGo" searching engine to google, yet no dice... I feel like a cave man in the middle of today's New York City...
I see, you're out of the mainstream - you've missed reading about people like "script kiddies", and their "l33t sp38k" (substitute letters like e for 3).
mark "no, they're not a pretty picture"
On Tue, 2014-09-16 at 13:47 -0400, m.roth@5-cent.us wrote:
Given the upcoming elections, I like Scottish Law Commission,
I don't think they will get independence from the clowns in London, England, this time, but I do wish them every possible success !
On Mon, Sep 15, 2014 at 4:37 AM, Andrew Holway andrew.holway@gmail.com wrote:
ZFS on Linux is backed by the US government as ZFS will be used as the primary filesystem to back the parallel distributed filesystem 'Lustre'.
wow, the US government!!. *sarcasm implied*
FC
On Mon, Sep 15, 2014 at 1:16 PM, Fernando Cassia fcassia@gmail.com wrote:
On Mon, Sep 15, 2014 at 4:37 AM, Andrew Holway andrew.holway@gmail.com wrote:
ZFS on Linux is backed by the US government as ZFS will be used as the primary filesystem to back the parallel distributed filesystem 'Lustre'.
wow, the US government!!. *sarcasm implied*
Ummm, like you've walked on the moon....
Les Mikesell wrote:
On Mon, Sep 15, 2014 at 1:16 PM, Fernando Cassia fcassia@gmail.com wrote:
On Mon, Sep 15, 2014 at 4:37 AM, Andrew Holway andrew.holway@gmail.com wrote:
ZFS on Linux is backed by the US government as ZFS will be used as the primary filesystem to back the parallel distributed filesystem 'Lustre'.
wow, the US government!!. *sarcasm implied*
Ummm, like you've walked on the moon....
We'll just ignore the trivial detail that this conversation is taking place over the Internet, invented for DARPA (US gov't)....
mark
On Mon, Sep 15, 2014 at 3:20 PM, Les Mikesell lesmikesell@gmail.com wrote:
Ummm, like you've walked on the moon....
LOL. I will begin saying that "the US government backs JavaFX" then, just because NASA uses it in some projects. https://weblogs.java.net/blog/seanmiphillips/archive/2013/11/20/visualizing-...
There´s a difference between a govt organization vs "the US government" as a whole.
FC
On 09/07/2014 09:19 PM, Dave Stevens wrote:
I want to set up a new CentOS install using version 7 and would like to experiment with various RAID levels. Anyone care to point out a tutorial?
That's a very broad question, so the responses thus far shouldn't be surprising. I suggest reading:
http://en.wikipedia.org/wiki/RAID
Figure out approximately what you want to do, and come back with any questions.
Sound good?
-
Always Learning -
Andrew Holway -
Chris -
Dave Stevens -
Digimer -
Eliezer Croitoru -
Fernando Cassia -
Ian Pilcher -
John R Pierce -
Keith Keller -
Les Mikesell -
m.roth＠5-cent.us -
Paul Heinlein -
Scott Robbins -
SilverTip257 -
Sven Kieske -
Valeri Galtsev -
Warren Young