Discuss July 2014

discuss@lists.centos.org

211 participants
227 discussions

corruption of in-memory data detected (xfs)
by Alexandru Cardaniuc 07 Jul '14

07 Jul '14

Hi All, I am having an issue with an XFS filesystem shutting down under high load with very many small files. Basically, I have around 3.5 - 4 million files on this filesystem. New files are being written to the FS all the time, until I get to 9-11 mln small files (35k on average). at some point I get the following in dmesg: [2870477.695512] Filesystem "sda5": XFS internal error xfs_trans_cancel at line 1138 of file fs/xfs/xfs_trans.c. Caller 0xffffffff8826bb7d [2870477.695558] [2870477.695559] Call Trace: [2870477.695611] [<ffffffff88262c28>] :xfs:xfs_trans_cancel+0x5b/0xfe [2870477.695643] [<ffffffff8826bb7d>] :xfs:xfs_mkdir+0x57c/0x5d7 [2870477.695673] [<ffffffff8822f3f8>] :xfs:xfs_attr_get+0xbf/0xd2 [2870477.695707] [<ffffffff88273326>] :xfs:xfs_vn_mknod+0x1e1/0x3bb [2870477.695726] [<ffffffff80264929>] _spin_lock_irqsave+0x9/0x14 [2870477.695736] [<ffffffff802230e6>] __up_read+0x19/0x7f [2870477.695764] [<ffffffff8824f8f4>] :xfs:xfs_iunlock+0x57/0x79 [2870477.695776] [<ffffffff80264929>] _spin_lock_irqsave+0x9/0x14 [2870477.695784] [<ffffffff802230e6>] __up_read+0x19/0x7f [2870477.695791] [<ffffffff80209f4c>] __d_lookup+0xb0/0xff [2870477.695803] [<ffffffff8020cd4a>] _atomic_dec_and_lock+0x39/0x57 [2870477.695814] [<ffffffff8022d6db>] mntput_no_expire+0x19/0x89 [2870477.695829] [<ffffffff80264929>] _spin_lock_irqsave+0x9/0x14 [2870477.695837] [<ffffffff802230e6>] __up_read+0x19/0x7f [2870477.695861] [<ffffffff8824f8f4>] :xfs:xfs_iunlock+0x57/0x79 [2870477.695887] [<ffffffff882680af>] :xfs:xfs_access+0x3d/0x46 [2870477.695899] [<ffffffff80264929>] _spin_lock_irqsave+0x9/0x14 [2870477.695923] [<ffffffff802df4a3>] vfs_mkdir+0xe3/0x152 [2870477.695933] [<ffffffff802dfa79>] sys_mkdirat+0xa3/0xe4 [2870477.695953] [<ffffffff80260295>] tracesys+0x47/0xb6 [2870477.695963] [<ffffffff802602f9>] tracesys+0xab/0xb6 [2870477.695977] [2870477.695985] xfs_force_shutdown(sda5,0x8) called from line 1139 of file fs/xfs/xfs_trans.c. Return address = 0xffffffff88262c46 [2870477.696452] Filesystem "sda5": Corruption of in-memory data detected. Shutting down filesystem: sda5 [2870477.696464] Please umount the filesystem, and rectify the problem(s) # ls -l /store ls: /store: Input/output error ?--------- 0 root root 0 Jan 1 1970 /store Filesystems is ~1T in size # df -hT /store Filesystem Type Size Used Avail Use% Mounted on /dev/sda5 xfs 910G 142G 769G 16% /store Using CentOS 5.9 with kernel 2.6.18-348.el5xen The filesystem is in a virtual machine (Xen) and on top of LVM. Filesystem was created using mkfs.xfs defaults with xfsprogs-2.9.4-1.el5.centos (that's the one that comes with CentOS 5.x by default.) These are the defaults with which the filesystem was created: # xfs_info /store meta-data=/dev/sda5 isize=256 agcount=32, agsize=7454720 blks = sectsz=512 attr=0 data = bsize=4096 blocks=238551040, imaxpct=25 = sunit=0 swidth=0 blks, unwritten=1 naming =version 2 bsize=4096 log =internal bsize=4096 blocks=32768, version=1 = sectsz=512 sunit=0 blks, lazy-count=0 realtime =none extsz=4096 blocks=0, rtextents=0 The problem is reproducible and I don't think it's hardware related. The problem was reproduced on multiple servers of the same type. So, I doubt it's a memory issue or something like that. Is that a known issue? If it is then what's the fix? I went through the kernel updates for CentOS 5.10 (newer kernel), but didn't see any xfs related fixes since CentOS 5.9 Any help will be greatly appreciated... -- "If we really understand the problem, the answer will come out of it, because the answer is not separate from the problem." - Krishnamurti

8 18

cannot hibernate, not enough swap
by Michael Hennebry 07 Jul '14

07 Jul '14

The last couple time I tried to hibernate, I got an error message: > PM: Not enough free swap I find that hard to believe: [hennebry@localhost doc]$ free -m total used free shared buffers cached Mem: 3833 1124 2709 0 51 286 -/+ buffers/cache: 786 3047 Swap: 11453 356 11097 [hennebry@localhost doc]$ 1124+786+356=2266< 11097 How do I figure out what is going on and fix it? -- Michael hennebry(a)web.cs.ndsu.NoDak.edu "SCSI is NOT magic. There are *fundamental technical reasons* why it is necessary to sacrifice a young goat to your SCSI chain now and then." -- John Woods

3 3

CentOS-6.5 LiveDVD problem
by James B. Byrne 07 Jul '14

07 Jul '14

I downloaded the Live DVD for CentOS-6.5 and burned a DVD-R using a CentOS-6.5 desktop. To test the validity of the new DVD I then rebooted the same desktop using the same Live DVD. No matter which Live DVD boot option I choose I see this after the splash screen completes the Circle (sounds like a good title for a novel). <--- Kernel panic - Not syncing - Attempting to kill init! Pid: 1, comm: init Not tainted 2.6.32-431.el6.x86_64 # 1 call trace [<ffffffff815271fa>] ? Panic+0xa7/0x16f [<ffffffff81077622>] ? do_exit+0x862/0x870 [<ffffffff8118a865>] ? fput+0x25/0x30 [<ffffffff81077688>] ? do_group_exit+0x58/0xd0 [<ffffffff81077717>] ? sys_exit_group+0x17/0x020 [<ffffffff81006072>] ? system_call_fastpath+0x16/0x16 dkm_kms_helper panic occurred, switching back to text console ---> The very last line appears for each boot choice saving only the text mode boot. In that case the line is absent but the rest of the output remains unchanged. An extract of the system specifications rendered by lshw: <--- description: Desktop Computer width: 64 bits capabilities: smbios-2.4 dmi-2.4 vsyscall64 vsyscall32 configuration: boot=normal chassis=desktop uuid=B4CDC46C-7E7A-11DE-87DF-0011D8A408ED *-core description: Motherboard product: DG43GT vendor: Intel Corporation physical id: 0 version: AAE62768-300 serial: BTGT931003W4 slot: To be filled by O.E.M. *-firmware description: BIOS vendor: Intel Corp. physical id: 0 version: GTG4310H.86A.0019.2009.0625.1334 (06/25/2009) size: 64KiB capacity: 960KiB capabilities: pci upgrade shadowing escd cdboot bootselect socketedrom edd int13floppy1200 int13floppy720 int13floppy2880 int5printscreen int9keyboard int14serial int17printer int10video acpi usb ls120boot zipboot biosbootspecification *-cpu description: CPU product: Intel(R) Core(TM)2 Quad CPU Q8400 @ 2.66GHz vendor: Intel Corp. physical id: 4 bus info: cpu@0 version: Intel(R) Core(TM)2 Quad CPU Q8400 @ 2.66GHz serial: To Be Filled By O.E.M. slot: PROCESSOR size: 2666MHz capacity: 4GHz width: 64 bits clock: 333MHz capabilities: fpu fpu_exception wp vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx x86-64 constant_tsc arch_perfmon pebs bts rep_good aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 xsave lahf_lm tpr_shadow vnmi flexpriority *-cache:0 description: L1 cache physical id: 5 slot: L1-Cache size: 32KiB capacity: 32KiB capabilities: internal write-back data *-cache:1 description: L2 cache physical id: 6 slot: L2-Cache size: 2MiB capacity: 2MiB capabilities: internal varies unified *-memory description: System Memory physical id: 2b slot: System board or motherboard size: 16GiB *-bank:0 description: DIMM DDR2 Synchronous 800 MHz (1.2 ns) product: PartNum0 vendor: Manufacturer0 physical id: 0 serial: SerNum0 slot: CHANNEL_A size: 4GiB width: 64 bits clock: 800MHz (1.2ns) *-bank:1 ---> Help??? -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3

4 3

Re: [CentOS] CentOS-6.5 LiveDVD problem
by James B. Byrne 07 Jul '14

07 Jul '14

On Fri, July 4, 2014 17:15, Ljubomir Ljubojevic wrote: > On 07/04/2014 07:43 PM, James B. Byrne wrote: >> I downloaded the Live DVD for CentOS-6.5 and burned a DVD-R using a >> CentOS-6.5 >> desktop. To test the validity of the new DVD I then rebooted the same >> desktop >> using the same Live DVD. No matter which Live DVD boot option I choose I >> see >> this after the splash screen completes the Circle (sounds like a good title >> for a novel). > > Validity of ISO file is tested with md5sum program. Have you done that? > Yes. # cat md5sum65.txt 83221db52687c7b857e65bfe60787838 CentOS-6.5-x86_64-bin-DVD1.iso 91018b86ca338360bc1212f06ea1719f CentOS-6.5-x86_64-bin-DVD2.iso 8c37390fa5d932d03feb05fba13fe92e CentOS-6.5-x86_64-LiveCD.iso 7b571e13ce6c3f61dc176bd34f8d60bb CentOS-6.5-x86_64-LiveDVD.iso 0d9dc37b5dd4befa1c440d2174e88a87 CentOS-6.5-x86_64-minimal.iso 939fd1d87c11ffe7795324438b85adfb CentOS-6.5-x86_64-netinstall.iso # md5sum -c md5sum65.txt CentOS-6.5-x86_64-bin-DVD1.iso: OK CentOS-6.5-x86_64-bin-DVD2.iso: OK CentOS-6.5-x86_64-LiveCD.iso: OK CentOS-6.5-x86_64-LiveDVD.iso: OK CentOS-6.5-x86_64-minimal.iso: OK CentOS-6.5-x86_64-netinstall.iso: OK -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3

1 0

Re: [CentOS] block level changes at the file system level?
by Ljubomir Ljubojevic 07 Jul '14

07 Jul '14

On 07/07/2014 02:56 PM, Reindl Harald wrote: > > > Am 07.07.2014 14:53, schrieb Ljubomir Ljubojevic: >> Also, check needs to be made if xz supports multitrheading like pk7zip > > enter "xz --help" would have answered this > > --threads=NUM > use at most NUM threads; the default is 1; > set to 0 to use the number of processor cores > > Usage: xz [OPTION]... [FILE]... > Compress or decompress FILEs in the .xz format. > > -z, --compress force compression > -d, --decompress, --uncompress > force decompression > -t, --test test compressed file integrity > -l, --list list information about .xz files > -k, --keep keep (don't delete) input files > -f, --force force overwrite of output file and (de)compress links > -c, --stdout, --to-stdout > write to standard output and don't delete input files > -0 ... -9 compression preset; default is 6; take compressor *and* > decompressor memory usage into account before using 7-9! > -e, --extreme try to improve compression ratio by using more CPU time; > does not affect decompressor memory requirements > -T, --threads=NUM use at most NUM threads; the default is 1; set to 0 > to use the number of processor cores > -q, --quiet suppress warnings; specify twice to suppress errors too > -v, --verbose be verbose; specify twice for even more verbose > -h, --help display this short help and exit > -H, --long-help display the long help (lists also the advanced options) > -V, --version display the version number and exit > Ok, I spent several minutes on this. xz --long-help and man xz are much more informative. https://en.wikipedia.org/wiki/Xz: "Design xz compresses single files as input, and does not bundle multiple files into a single archive. It is therefore common to compress a file that is itself an archive, such as those created by the tar or cpio Unix programs.[2]" "xz is a lossless data compression program and file format which incorporates the LZMA compression algorithm. xz can be thought of as a stripped down version of the 7-Zip program. xz has its own file format rather than the .7z format used by 7-Zip (which lacks support for Unix-like file system metadata[2])." https://en.wikipedia.org/wiki/7-Zip: "By default, 7-Zip creates 7z format archives with a .7z file extension. Each archive can contain multiple directories and files. As a container format, security or size reduction are achieved using a stacked combination of filters. These can consist of pre-processors, compression algorithms, and encryption filters." So either tar.xz or 7z container, and looks like LZM2 is only option, with several switches for xz for block reduction. And, no, --help does not actually help with assessing if xz is viable replacement for some particular purpose, testing is needed. P.S. You should REALLY fix your mail client, to sent replies to centos(a)centos.org so that conversations is seen by others (it looks like you reply 99% to the person instead back to the list). -- Ljubomir Ljubojevic (Love is in the Air) PL Computers Serbia, Europe StarOS, Mikrotik and CentOS/RHEL/Linux consultant

1 0

Mon morn
by m.roth＠5-cent.us 07 Jul '14

07 Jul '14

And, no mourn. Took three-quarters of an hour, which someone inconvenienced Beecher, but the server I came in so early to reboot did it's thing (think DOS or Windows chkdisk, except with a *lot* more disk), and I got the memory that Dell shipped me last Thurs into the other server, and everything's up swimmingly. Oh, and I'd gotten a email the other day - for some reason, they thought the card I'd reserved the room at the con with expired before our stay, so I updated their info, and got a thanks from Rob, who I've been dealing with. I replied to that before we left this morning, and mentioned it had been a week, and we hadn't heard anything about Bath, etc... and I got a nice email from him, saying he'd walk down the hall and prod them. Now, let me reattach my IV of tea/caffeine....

1 1

Re: [CentOS] block level changes at the file system level?
by Martín Cigorraga 05 Jul '14

05 Jul '14

> > Perhaps there is a file system that supports compression and would do a > good job with the snapshots transparently. Maybe even ZFS or btrfs do? > Hi, I agree with Lee. Btrfs actually does sport built-in compression as a mount argument/flag and the delta snapshots works beautifully well but I wouldn't dare to use it with any kernel previous to 3.13; in fact best practice would be to use it with kernel 3.14.x or later, preferably 3.15.x and later. B.R.

1 0

semi-OT: perl question
by m.roth＠5-cent.us 03 Jul '14

03 Jul '14

We've had a ton of selinux errors recently, and the other day, one of the managers here, after I started inquiring what their perl CGI was doing, found a way to reproduce the issue at will. It seems that before I upgraded their servers from CentOS 5.x, they were trapping interrupts such as a browser breaking the connection before getting a response, and under CentOS 6, it's looking like the traps aren't working. Has anyone seen anything like that? mark

1 0

SCL SRPMS repos URL?
by James B. Byrne 03 Jul '14

03 Jul '14

What is the correct URL for the SRMS for packages found in: http://mirror.centos.org/centos-6/6/SCL/ ? -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3

1 0

Miredo server for Centos 6
by Robert Moskowitz 03 Jul '14

03 Jul '14

On delving deeper into Miredo support, it seems that Miredo Server is a separate program from the Miredo client/relay. And that there is no Miredo Server available for Centos 6. Not in EPEL 6, or repoforge. So far the maintainer of Miredo for Fedora/EPEL has not reponded to a query on its status for EPEL 6. I have to ask on repoforge about miredo-server. Anyone know of anywhere else to look? I suppose I can bring up a Fedora 20 box, as this is 'just' a testbed. But to move the testbed out of my network into the corp testbed, I need it for Centos. thanks

3 10

Jump to page:

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss July 2014