Discuss

discuss@lists.centos.org

34758 discussions

Cluster Failover Troubleshooting (luci and ricci)
by Ryan Bunce 01 Jul '11

01 Jul '11

Hello all. I posted this in the forum and was told to instead post it to the mailing list. My apologies for the redundancy if you have already seen and been irritated by my blatherings. Thanks. _________________________ I am working on a CentOS clustered LAMP stack and running into problems. I have searched extensively and have come up empty. Here's my setup: Two node cluster identical hardware. IBM x226 with RSAII adapters for fencing. Configured for Active/Passive failover - no load balancing. No shared storage - manual rsync of data (shared SSH keys, rsync over SSH, cron job). Single shared IP address I used luci and ricci to configure the cluster. It's a bit confusing that there's an 'apache' script but you have to use the custom init script. I'm past that though. The failover function is working when it's kicked off manually from the luci web interface. I can tell it to transfer the services (IP, httpd, msqld) to the secondary server and it works fine. I run into problems when I attempt to simulate a failure (a pulled network cord for instance). The primary system recognizes the failure, shuts down it's services, attempts to inform the secondary server to take over and then it never does. Here is a log excerpt from a cable pull test: Jun 16 15:33:27 flex kernel: tg3: eth0: Link is down. Jun 16 15:33:34 flex clurgmgrd: [2970]: <warning> Link for eth0: Not detected Jun 16 15:33:34 flex clurgmgrd: [2970]: <warning> No link on eth0... Jun 16 15:33:34 flex clurgmgrd[2970]: <notice> status on ip "10.6.2.25" returned 1 (generic error) Jun 16 15:33:34 flex clurgmgrd[2970]: <notice> Stopping service service:web Jun 16 15:33:35 flex proftpd[6321]: 10.6.2.47 - ProFTPD killed (signal 15) Jun 16 15:33:35 flex proftpd[6321]: 10.6.2.47 - ProFTPD 1.3.3c standalone mode SHUTDOWN Jun 16 15:33:39 flex avahi-daemon[2850]: Withdrawing address record for 10.6.2.25 on eth0. Jun 16 15:33:49 flex clurgmgrd[2970]: <notice> Service service:web is recovering Jun 16 15:33:49 flex clurgmgrd[2970]: <notice> Recovering failed service service:web Jun 16 15:33:49 flex clurgmgrd: [2970]: <warning> Link for eth0: Not detected Jun 16 15:33:49 flex clurgmgrd[2970]: <notice> start on ip "10.6.2.25" returned 1 (generic error) Jun 16 15:33:49 flex clurgmgrd[2970]: <warning> #68: Failed to start service:web; return value: 1 Jun 16 15:33:49 flex clurgmgrd[2970]: <notice> Stopping service service:web Jun 16 15:33:49 flex clurgmgrd: [2970]: <err> script:mysqld: stop of /etc/rc.d/init.d/mysqld failed (returned 1) Jun 16 15:33:49 flex clurgmgrd[2970]: <notice> stop on script "mysqld" returned 1 (generic error) Jun 16 15:33:49 flex clurgmgrd[2970]: <crit> #12: RG service:web failed to stop; intervention required Jun 16 15:33:49 flex clurgmgrd[2970]: <notice> Service service:web is failed Jun 16 15:33:49 flex clurgmgrd[2970]: <crit> #13: Service service:web failed to stop cleanly Jun 16 15:36:43 flex kernel: tg3: eth0: Link is up at 100 Mbps, full duplex. Jun 16 15:36:43 flex kernel: tg3: eth0: Flow control is off for TX and off for RX. Jun 16 16:04:52 flex luci[2904]: Unable to retrieve batch 306226694 status from web2:11111: Unable to disable failed service web before starting it:clusvcadm failed to stop web: Jun 16 16:05:28 flex clurgmgrd[2970]: <notice> Starting disabled service service:web Jun 16 16:05:31 flex avahi-daemon[2850]: Registering new address record for 10.6.2.25 on eth0. Jun 16 16:05:31 flex luci[2904]: Unable to retrieve batch 1997354692 status from web2:11111: module scheduled for execution Jun 16 16:05:33 flex proftpd[1926]: 10.6.2.47 - ProFTPD 1.3.3c (maint) (built Thu Nov 18 2010 03:38:57 CET) standalone mode STARTUP Jun 16 16:05:33 flex clurgmgrd[2970]: <notice> Service service:web started I have followed the HowTos for setting up the cluster (with the exception of the shared storage) as closely as possible. Here's what I've already troubleshot: No IPTables running No SELinux running Hosts file resolves all IP address/host names properly. I must say that I am less familiar with how all of the cluster components work together. All of the Linux clusters I have built thus far have been heartbeat+mon style clusters. I'm looking to find out if there is an additional debug layer that I can put in place to get some more detailed information about what is transacting (or not) between the two cluster members. Many thanks.

3 2

Anyway to ensure SSH availability?
by Emmanuel Noobadmin 01 Jul '11

01 Jul '11

I was having problems with the same server locking up to the point I can't even get in via SSH. I've already used HTB/TC to reserve bandwidth for my SSH port but the problem now isn't an attack on the bandwidth. So I'm trying to figure out if there's a way to ensure that SSH is given cpu and i/o priority. However, so far reading seems to imply that it's probably not going to help if the issue is i/o related and/or it would require escalating SSH to such levels (above paging/filesystem processes) that makes it a really bad idea. Since I'm not the only person who face problems trying to remotely access a locked up server, surely somebody must had come up with a solution that didn't involve somebody/something hitting the power button?

21 59

OT: 1.1.1.1 is not private anymore?
by Fajar Priyanto 01 Jul '11

01 Jul '11

Hi all, Our network is suspected to be infected by malware by the detector in upline network. Turns out that some of our developers use 1.1.1.1 as a "pinging testing". Google comes to my knowledge that 1.1.1.1 is not a private IP anymore? Since when? Also Google says 1.1.1.1 is well-known to be used by botnet command and control host?? I've blocked it in the local gateway. Just curious......

3 2

Re: [CentOS] Power-outage
by Brent L. Bates 01 Jul '11

01 Jul '11

apcupsd is only suppose to work with APC's UPS's and the apcupsd developer will not deal with complaints about UPS's from other manufacturers. If your non-APC UPS works with apcupsd, then count yourself lucky. If you want to use another manufacturer's UPS, check out NUT (http://www.networkupstools.org/) to see if it will work with your UPS. One last thing, APC has changed their communications protocol (it's called microlink now) on new UPS's, so apcupsd doesn't work nearly as well as it used to. -- Brent L. Bates (UNIX Sys. Admin.) M.S. 912 Phone:(757) 865-1400, x204 NASA Langley Research Center FAX:(757) 865-8177 Hampton, Virginia 23681-0001 Email: B.L.BATES(a)larc.nasa.gov http://www.vigyan.com/~blbates/

1 0

yum update -> [Errno 4] Socket Error: timed out
by ken 30 Jun '11

30 Jun '11

Trying to update a second CentOS box, I'm getting this error repeatedly: [Errno 4] Socket Error: timed out I'm getting this on every mirror and have gone through the list of mirrors more than a dozen times. Oddly, the RPMs I'm trying to upgrade I upgraded just yesterday without a problem on another machine on the same LAN with no problems whatsoever. I can ping mirrors fine. There were a spate of these errors back in 2006. The fix for many was to add this line to yum.conf: timeout=300 So I did that on the machine where yum is having the problem, but the same errors are returned. Anyone else seeing this? Anyone know what the problem is?

7 18

raid1: solve smart error
by Helmut Drodofsky 30 Jun '11

30 Jun '11

In a server with raid1 a first smart error was reported. Smart attributes: SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000f 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0003 185 183 021 Pre-fail Always - 7750 4 Start_Stop_Count 0x0032 100 100 000 Old_age Always - 20 5 Reallocated_Sector_Ct 0x0033 200 200 140 Pre-fail Always - 0 7 Seek_Error_Rate 0x000e 200 200 000 Old_age Always - 0 9 Power_On_Hours 0x0032 075 075 000 Old_age Always - 18868 10 Spin_Retry_Count 0x0012 100 253 000 Old_age Always - 0 11 Calibration_Retry_Count 0x0012 100 253 000 Old_age Always - 0 12 Power_Cycle_Count 0x0032 100 100 000 Old_age Always - 18 192 Power-Off_Retract_Count 0x0032 200 200 000 Old_age Always - 10 193 Load_Cycle_Count 0x0032 200 200 000 Old_age Always - 28 194 Temperature_Celsius 0x0022 121 115 000 Old_age Always - 31 196 Reallocated_Event_Count 0x0032 200 200 000 Old_age Always - 0 197 Current_Pending_Sector 0x0012 200 200 000 Old_age Always - 1 198 Offline_Uncorrectable 0x0010 200 200 000 Old_age Offline - 0 199 UDMA_CRC_Error_Count 0x003e 200 200 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 200 000 Old_age Offline - 0 I assume, raw read error rate is ok. So I would use the same disk for resyncing the raid. Other recommendations? Helmut

1 0

centos 5.6 and intel MHD4500 graphics card
by Hüvely Balázs 30 Jun '11

30 Jun '11

Hi! I've a toshiba satellite notebook, and I tried to install centos 5.6. The installer starts, but when it turns to graphical mode, the backlight goes off. I see the windows, buttons, but it's very dark, maybe the backlight switching off.. How can I get the installer to work correctly? The machine details: core i3 330m (The graphics controller maybe integrated to the cpu...) it's a MHD4500 (GMA4500) Thanks: Balazs

5 7

preventing symlinks?
by Smithies, Russell 30 Jun '11

30 Jun '11

This may be more of a general linux question but is there a simple way of preventing users from creating symlinks to or from certain directories? I have a /scratch dir that's a single 27TB volume and I don't want users linking their home dirs as there's a chance it will screw up our external backups. Is this a job for SELinux? Any ideas? Thanx, Russell ======================================================================= Attention: The information contained in this message and/or attachments from AgResearch Limited is intended only for the persons or entities to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipients is prohibited by AgResearch Limited. If you have received this message in error, please notify the sender immediately. =======================================================================

3 2

unsubscribes
by Dharma Kusumah 30 Jun '11

30 Jun '11

1 0

Redirecting PHP error messages into a log file
by Alexander Farber 29 Jun '11

29 Jun '11

Hello, I'm using CentOS release 5.5 x86_64 with the stock php-5.1.6-27.el5 and would like to redirect PHP messages into /var/log/httpd/php_log First I tried adding that file name to /etc/php.ini: error_log = "/var/log/httpd/php_log" and restarted httpd, but the file didn't appear. I've touched it and changed owner to apache.apache, but that didn't help. Then I've changed that line to error_log = syslog and now I get the messages in /var/log/messages: Jul 29 21:29:24 hXXX httpd: PHP Notice: Undefined index: REMOTE_ADDR in /var/www/html/odnoklassniki/index.php on line 44 Jul 29 21:29:45 hXXX httpd: PHP Parse error: syntax error, unexpected '[' in /var/www/html/odnoklassniki/index.php on line 44 I still would like to move those message to a separate file though, so I've added the following line to /etc/syslog.conf and reloaded syslogd service: httpd.* /var/log/httpd/php_log but the file is still empty. Any hints please? Thank you Alex -- http://preferans.de

6 6

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss