So I'm curious how SAS JBOD arrays and linux MDraid as implemented in CentOS6, and SES (SCSI/SAS Enclosure Services) backplane controllers 'get along' and how much configuration is needed to get the warning lights to work properly.
scenario: whitebox server with a SAS backplane or two, daisy chained on a SAS HBA (like an LSI Logic 2008), and disks organized as several raid5/6 using linux mdraid. Does this system have to be configured to ensure that the activity and failure lights match the drive bays, or does the SAS/SES spec all sort that out 'plug and play' ?
does mdraid properly set those warning lights automatically on a failed raid element?
On Mon, Aug 15, 2011 at 11:19:22AM -0700, John R Pierce wrote:
So I'm curious how SAS JBOD arrays and linux MDraid as implemented in CentOS6, and SES (SCSI/SAS Enclosure Services) backplane controllers 'get along' and how much configuration is needed to get the warning lights to work properly.
scenario: whitebox server with a SAS backplane or two, daisy chained on a SAS HBA (like an LSI Logic 2008), and disks organized as several raid5/6 using linux mdraid. Does this system have to be configured to ensure that the activity and failure lights match the drive bays, or does the SAS/SES spec all sort that out 'plug and play' ?
does mdraid properly set those warning lights automatically on a failed raid element?
Afaik the LEDs won't work out-of-the-box.
There's no daemon currently in Linux responsible for monitoring disk status and lighting up SES LEDs accordingly..
Feel free to write such a daemon! :) (or even better, prove me wrong)
-- Pasi
Afaik the LEDs won't work out-of-the-box.
Maybe, I presume its vendor specific but if they appear to linux in sys, maybe a solution like the first link suggests?
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=560432 http://lkml.org/lkml/2008/2/3/193
On 08/15/11 3:31 PM, Pasi Kärkkäinen wrote:
Afaik the LEDs won't work out-of-the-box.
There's no daemon currently in Linux responsible for monitoring disk status and lighting up SES LEDs accordingly..
yikes. thats pretty amateur. I thought Linux was better than that.
I've found some man pages for sg_ses, which appears to be a util for querying and commanding the SES, including turning the lights on and off.
On Tue, 16 Aug 2011, John R Pierce wrote:
To: centos@centos.org From: John R Pierce pierce@hogranch.com Subject: Re: [CentOS] SAS storage arrays, C6, and SES lights
On 08/15/11 3:31 PM, Pasi Kärkkäinen wrote:
Afaik the LEDs won't work out-of-the-box.
There's no daemon currently in Linux responsible for monitoring disk status and lighting up SES LEDs accordingly..
yikes. thats pretty amateur. I thought Linux was better than that.
I've found some man pages for sg_ses, which appears to be a util for querying and commanding the SES, including turning the lights on and off.
Not sure if you can write a plugin for GKrelM, that is used to monitor disk I/O and temperatures on my C5.6 box?
Installed Packages Name : gkrellm Arch : i386 Version : 2.3.0 Release : 4.el5.1 Size : 1.8 M Repo : installed Summary : Multiple stacked system monitors in one process URL : http://www.gkrellm.net/ License : GPLv3+ Description: GKrellM charts CPU, load, Disk, and all active net interfaces : automatically. An on/off button and online timer for the PPP : interface is provided, as well as monitors for memory and swap : usage, file system, internet connections, APM laptop battery, : mbox style mailboxes, and temperature sensors on supported : systems. Also included is an uptime monitor, a hostname label, : and a clock/calendar. Additional features are: : : * Autoscaling grid lines with configurable grid line : resolution. : * LED indicators for the net interfaces. : * A gui popup for configuration of chart sizes and : resolutions.
Available Packages Name : gkrellm-daemon Arch : i386 Version : 2.2.7 Release : 0.el5.rf Size : 50 k Repo : rpmforge Summary : The GNU Krell Monitor Daemon URL : http://www.gkrellm.net/ License : GPL Description: This contains only the gkrellm daemon, which you can install on : its own on machines you intend to monitor with gkrellm from a : different location.
Name : gkrellm-devel Arch : i386 Version : 2.2.7 Release : 0.el5.rf Size : 18 k Repo : rpmforge Summary : Include headers from the GNU Krell Monitor URL : http://www.gkrellm.net/ License : GPL Description: Install this package if you intend to compile plugins to use : with the GKrellM monitor.
Kind Regards,
Keith Roberts
----------------------------------------------------------------- Websites: http://www.karsites.net http://www.php-debuggers.net http://www.raised-from-the-dead.org.uk
All email addresses are challenge-response protected with TMDA [http://tmda.net] -----------------------------------------------------------------
On Tue, Aug 16, 2011 at 12:19:17AM -0700, John R Pierce wrote:
On 08/15/11 3:31 PM, Pasi Kärkkäinen wrote:
Afaik the LEDs won't work out-of-the-box.
There's no daemon currently in Linux responsible for monitoring disk status and lighting up SES LEDs accordingly..
yikes. thats pretty amateur. I thought Linux was better than that.
I've found some man pages for sg_ses, which appears to be a util for querying and commanding the SES, including turning the lights on and off.
Yep.. you could (would) use sg_ses as a part of the solution.
.. but there's more than that. If you get IO errors from some disk, you need to figure out on which backplane slot that disk is, and only then you can light up the LED..
ie. you need a daemon/tool that monitors status of disks, and keeps the Linux disk <-> ses slot mapping up-to-date.
-- Pasi
On 08/16/11 12:59 AM, Pasi Kärkkäinen wrote:
ie. you need a daemon/tool that monitors status of disks, and keeps the Linux disk<-> ses slot mapping up-to-date.
i'm amazed this doesn't exist. isn't this a really common problem with storage arrays?
On Tue, Aug 16, 2011 at 4:03 PM, John R Pierce pierce@hogranch.com wrote:
On 08/16/11 12:59 AM, Pasi Kärkkäinen wrote:
ie. you need a daemon/tool that monitors status of disks, and keeps the Linux disk<-> ses slot mapping up-to-date.
i'm amazed this doesn't exist. isn't this a really common problem with storage arrays?
-- john r pierce N 37, W 122 santa cruz ca mid-left coast
yup.
I inquired about it a while ago (something about switching LED's on) and didn't get very far either.
OpenIndiana has all that builtin...
/me ducks. ----- Original Message ----- From: "John R Pierce" pierce@hogranch.com To: centos@centos.org Sent: Tuesday, August 16, 2011 10:03 PM Subject: Re: [CentOS] SAS storage arrays, C6, and SES lights
On 08/16/11 12:59 AM, Pasi Kärkkäinen wrote:
ie. you need a daemon/tool that monitors status of disks, and keeps the Linux disk<-> ses slot mapping up-to-date.
i'm amazed this doesn't exist. isn't this a really common problem with storage arrays?
On Wed, Aug 17, 2011 at 8:26 AM, Christopher Chan christopher.chan@bradbury.edu.hk wrote:
OpenIndiana has all that builtin...
/me ducks. ----- Original Message -----
Yes, and so does many commercial NAS appliances that run on Linux - but do you think they'll tell us how they got it working?
On Wed, Aug 17, 2011 at 02:26:51PM +0800, Christopher Chan wrote:
OpenIndiana has all that builtin...
Well in a way. Afaik out-of-the-box opensolaris/openindiana FMA (Fault Management Agent) only works with some certain Sun/Oracle chassises.
It requires plugins for other models.
-- Pasi
/me ducks. ----- Original Message ----- From: "John R Pierce" pierce@hogranch.com To: centos@centos.org Sent: Tuesday, August 16, 2011 10:03 PM Subject: Re: [CentOS] SAS storage arrays, C6, and SES lights
On 08/16/11 12:59 AM, Pasi Kärkkäinen wrote:
ie. you need a daemon/tool that monitors status of disks, and keeps the Linux disk<-> ses slot mapping up-to-date.
i'm amazed this doesn't exist. isn't this a really common problem with storage arrays?
-- john r pierce N 37, W 122 santa cruz ca mid-left coast
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
CentOS mailing list CentOS@centos.org http://lists.centos.org/mailman/listinfo/centos
On Tue, Aug 16, 2011 at 07:03:01AM -0700, John R Pierce wrote:
On 08/16/11 12:59 AM, Pasi Kärkkäinen wrote:
ie. you need a daemon/tool that monitors status of disks, and keeps the Linux disk<-> ses slot mapping up-to-date.
i'm amazed this doesn't exist. isn't this a really common problem with storage arrays?
It is. I guess that's one of the reasons why many people still use hardware RAID adapters.. (firmware of the hardware RAID adapter takes case of the SES/LED management.)
Someone should start an opensource project to build this kind of management daemon for software RAID on Linux. It's not very difficult.. someone just has to do it :)
The only solution that I'm aware right now is Linux dm-raid.. I think it can handle some LEDs on some chassises. But that's implemented in kernel-driver..
-- Pasi
-
Christopher Chan -
John R Pierce -
Joseph L. Casale -
Keith Roberts -
Pasi Kärkkäinen -
Rudi Ahlers