Discuss

discuss@lists.centos.org

34753 discussions

s3 as mysql directory
by Tim Dunphy 01 Oct '12

01 Oct '12

Hello list, I am soliciting opinion here as opposed technical help with an idea I have. I've setup a bacula backup system on an AWS volume. Bacula stores a LOT of information in it's mysql database (in my setup, you can also use postgres or sqlite if you chose). Since I've started doing this I notice that the mysql data directory has swelled to over 700GB! That's quite a lot and its' easting up valuable disk space. So I had an idea. What about uses the fuse based s3fs to mount an S3 bucket on the local filesystem and use that as your mysql data dir? In other words mount your s3 bucket on /var/lib/mysql I used this article to setup the s3fs file system http://benjisimon.blogspot.com/2011/01/setting-up-s3-backup-solution-on-cen… And everything went as planned. So my question to you dear listers is if I do start using a locally mounted s3 bucket as my mysqld data dir, will performance of the database be acceptable? If so, why? If not are there any other reasons why it would NOT be a good idea to do this? The steps I have in mind are basically this: 1) mysqldump --all-databases > alldb.sql 2) stop mysql 3) rm -rf /var/lib/mysql/* 4) mount the s3 bucket on /var/lib/mysql 5) start mysql 6) restore the alldb.sql dump Thanks for your opinions on this! Tim 4) -- GPG me!! gpg --keyserver pool.sks-keyservers.net --recv-keys F186197B

7 7

Deduplication data for CentOS?
by Rainer Traut 01 Oct '12

01 Oct '12

Hi list, is there any working solution for deduplication of data for centos? We are trying to find a solution for our backup server which runs a bash script invoking xdelta(3). But having this functionality in fs is much more friendly... We have looked into lessfs, sdfs and ddar. Are these filesystems ready to use (on centos)? ddar is sthg different, I know. Thx Rainer

14 28

load balancer recommendation
by Boris Epstein 01 Oct '12

01 Oct '12

Hello all, If I were looking for a load balancer to run on a Linux - specifically, CentOS - machine - what would you recommend? Thanks. Boris.

6 5

High memory needs
by Jérémie Dubois-Lacoste 01 Oct '12

01 Oct '12

Dear All, We recently reinstalled our computing cluster. We were using CentOS 5.3 (32 bits). It is now CentOS 6.3 (64 bits), installed from the CentOS 6.2 x64 CD, then upgraded to 6.3. We have some issues with the memory needs of our running jobs. They require much more than before, it may be due to the switch from 32 to 64 bits, but to me this cannot explain the whole difference. Here are our investigations. We used the following simple benchmark: 1. Run a python script and check the memory that it requires (field "VIRT" of the "top" command). This script is: ---- import time time.sleep(30) print("done") ---- 2. Similarly, run and check the memory of a simple bash script: ---- #!/bin/bash sleep 30 echo "done" ---- 3. Open a R session and check the memory used I asked 10 of our users to run these three things on their personal PCs. They are running different distributions (mainly ubuntu, slackware), half of them use a 32 bits system, the other half a 64 one. Here is a summary of the results: Bash script: Avg Min Max 32 bits 5400 4192 9024 64 bits 12900 10000 16528 Python script: Avg Min Max 32 bits 8500 5004 11132 64 bits 32800 30000 36336 R: Avg Min Max 32 bits 26900 21000 33452 64 bits 100200 93008 97496 (as a side remark, the difference between 32 and 64 is surprisingly big to me...). Then we ran the same things on our CentOS cluster, getting surprisingly high results. I installed a machine from scratch with the CentOS CD (6.2 x64) to be sure another component of the cluster was not playing a role. On this freshly installed machine I get the following results: SH: 103MB PYTHON: 114MB R: 200MB So, compared to the highest of our users (among the 64 bits ones), we have a ratio of ~7, ~3, ~2, respectively. It is very problematic for us because many jobs now cannot run properly, because they lack memory on most of our computing nodes. So we really cannot stand the situation... Do you see any reason for this? Do you have suggestions? Sincerely, Jérémie

6 11

Doubled up RAM to 32 GB - now how to speed up a LAPP server?
by Alexander Farber 01 Oct '12

01 Oct '12

Dear CentOS users, I run a small Facebook game at a CentOS 6.3 machine with PostgreSQL 8.4.3 + few PHP scripts + 1 Perl daemon and even though the server worked ok, I've suggested my users to double up the RAM to 32 GB and they have collected money for that. Now my problem is that I don't know, which knob to turn and how to really use the additional memory. Below is my top output at the peak time (evenings) - as you see, 27 GB of RAM aren't used: # top - 18:47:55 up 23:12, 2 users, load average: 2.17, 2.31, 2.56 Tasks: 246 total, 2 running, 244 sleeping, 0 stopped, 0 zombie Cpu(s): 12.1%us, 0.2%sy, 0.0%ni, 87.7%id, 0.0%wa, 0.0%hi, 0.0%si, 0.0%st Mem: 32790380k total, 5296664k used, 27493716k free, 197132k buffers Swap: 2096056k total, 0k used, 2096056k free, 3815840k cached PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 12363 postgres 20 0 4376m 717m 710m S 40.6 2.2 2:03.48 postmaster 3842 nobody 20 0 118m 23m 3920 S 12.0 0.1 46:24.68 pref.pl 9178 postgres 20 0 4375m 518m 514m S 11.3 1.6 53:50.63 postmaster 12380 postgres 20 0 4377m 668m 660m S 11.3 2.1 2:33.26 postmaster 12243 postgres 20 0 4377m 668m 662m S 9.3 2.1 2:20.49 postmaster 12438 postgres 20 0 4374m 502m 498m S 6.3 1.6 1:03.34 postmaster 12249 postgres 20 0 4384m 852m 839m S 3.0 2.7 3:59.11 postmaster 12241 postgres 20 0 4378m 632m 625m S 1.7 2.0 2:48.62 postmaster 12156 apache 20 0 366m 27m 17m S 1.0 0.1 0:05.12 httpd 36 root 20 0 0 0 0 S 0.3 0.0 0:01.32 events/1 100 root 39 19 0 0 0 S 0.3 0.0 0:06.04 khugepaged 9217 postgres 20 0 21976 1036 516 S 0.3 0.0 1:01.07 pgbouncer 12010 apache 20 0 376m 37m 17m S 0.3 0.1 0:07.58 httpd 12280 apache 20 0 370m 30m 16m S 0.3 0.1 0:03.17 httpd 12362 apache 20 0 365m 15m 6816 R 0.3 0.0 0:01.90 httpd 12457 apache 20 0 360m 9.8m 3456 S 0.3 0.0 0:00.14 httpd 1 root 20 0 19352 1584 1284 S 0.0 0.0 0:01.03 init 2 root 20 0 0 0 0 S 0.0 0.0 0:00.00 kthreadd # vmstat 10 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu----- r b swpd free buff cache si so bi bo in cs us sy id wa st 6 2 0 27323416 196988 3852536 0 0 3 165 15 9 14 0 84 2 0 1 0 0 27422148 197012 3814860 0 0 0 5284 4784 2104 46 1 51 2 0 1 0 0 27454748 197020 3826656 0 0 4 1734 2021 1200 16 0 83 1 0 0 2 0 27514008 197028 3813464 0 0 0 702 1475 1208 8 0 90 2 0 0 0 0 27465612 197040 3813968 0 0 0 1435 1764 1725 10 0 85 5 0 1 0 0 27459260 197060 3814248 0 0 0 2032 2667 1304 22 0 76 1 0 1 0 0 27440076 197064 3827064 0 0 0 1604 3146 2109 27 0 72 1 0 1 0 0 27466796 197068 3814868 0 0 2 1241 2014 1637 13 0 83 3 0 4 0 0 27380104 197072 3848256 0 0 0 1064 2375 894 20 0 79 1 0 1 0 0 27488168 197096 3815296 0 0 0 2075 2697 2220 23 0 75 1 0 1 0 0 27462168 197116 3821380 0 0 0 871 1750 943 13 0 86 1 0 4 0 0 27432100 197128 3822320 0 0 0 3980 4767 2340 46 1 53 1 0 0 0 0 27493716 197132 3815844 0 0 0 1871 3209 2078 27 0 72 1 0 3 0 0 27424284 197132 3827036 0 0 0 1452 2551 1487 18 0 78 3 0 3 0 0 27435428 197160 3824116 0 0 0 2066 3430 2082 29 0 70 1 0 2 0 0 27452004 197172 3817440 0 0 0 1356 2722 1895 23 0 76 1 0 2 0 0 27436668 197176 3826648 0 0 0 1633 3629 2162 30 0 69 1 0 1 0 0 27439924 197204 3823124 0 0 0 1502 1786 1293 14 0 86 0 0 0 0 0 27466696 197212 3816780 0 0 0 1200 1701 1164 13 0 86 0 0 3 0 0 27432204 197212 3818344 0 0 0 2587 2098 2154 16 0 83 1 0 2 0 0 27421088 197224 3827224 0 0 0 1229 2635 1421 21 0 75 3 0 3 0 0 27319136 197232 3832088 0 0 13 2965 4220 1951 40 0 59 1 0 procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu----- What I've already done: /var/lib/pgsql/data/postgresql.conf (local connections only) max_connections = 10 shared_buffers = 4096MB work_mem = 16M /etc/php.ini memory_limit = 300M [PostgresSQL] pgsql.allow_persistent = Off # because I use pgbouncer /etc/pgbouncer.ini (local connections only too) max_client_conn = 600 default_pool_size = 80 /etc/httpd/conf/httpd.conf (didn't modify yet): <IfModule prefork.c> StartServers 10 MinSpareServers 8 MaxSpareServers 30 ServerLimit 512 MaxClients 512 MaxRequestsPerChild 4000 </IfModule> Any suggestions please? I was actually hoping that Linux would use the additional memory for caching disks, but this doesn't seem to happen? Thank you Alex P.S. Below my dmesg output - # dmesg Initializing cgroup subsys cpuset Initializing cgroup subsys cpu Linux version 2.6.32-279.9.1.el6.x86_64 (mockbuild(a)c6b9.bsys.dev.centos.org) (gcc version 4.4.6 20120305 (Red Hat 4.4.6-4) (GCC) ) #1 SMP Tue Sep 25 21:43:11 UTC 2012 Command line: ro root=/dev/md2 crashkernel=auto SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=de KERNEL supported cpus: Intel GenuineIntel AMD AuthenticAMD Centaur CentaurHauls BIOS-provided physical RAM map: BIOS-e820: 0000000000000000 - 000000000009d800 (usable) BIOS-e820: 000000000009d800 - 00000000000a0000 (reserved) BIOS-e820: 00000000000e0000 - 0000000000100000 (reserved) BIOS-e820: 0000000000100000 - 0000000020000000 (usable) BIOS-e820: 0000000020000000 - 0000000020200000 (reserved) BIOS-e820: 0000000020200000 - 0000000040000000 (usable) BIOS-e820: 0000000040000000 - 0000000040200000 (reserved) BIOS-e820: 0000000040200000 - 00000000bac15000 (usable) BIOS-e820: 00000000bac15000 - 00000000bac71000 (ACPI NVS) BIOS-e820: 00000000bac71000 - 00000000bada4000 (reserved) BIOS-e820: 00000000bada4000 - 00000000badb5000 (ACPI NVS) BIOS-e820: 00000000badb5000 - 00000000badcc000 (reserved) BIOS-e820: 00000000badcc000 - 00000000badce000 (usable) BIOS-e820: 00000000badce000 - 00000000badd6000 (reserved) BIOS-e820: 00000000badd6000 - 00000000bade0000 (ACPI NVS) BIOS-e820: 00000000bade0000 - 00000000bae3a000 (reserved) BIOS-e820: 00000000bae3a000 - 00000000bae7d000 (ACPI NVS) BIOS-e820: 00000000bae7d000 - 00000000bb000000 (usable) BIOS-e820: 00000000bb800000 - 00000000bfa00000 (reserved) BIOS-e820: 00000000fed1c000 - 00000000fed20000 (reserved) BIOS-e820: 00000000ff000000 - 0000000100000000 (reserved) BIOS-e820: 0000000100000000 - 000000083fe00000 (usable) DMI 2.6 present. SMBIOS version 2.6 @ 0xF0450 DMI: System manufacturer System Product Name/P8H67-M PRO, BIOS 1004 06/08/2011 AMI BIOS detected: BIOS may corrupt low RAM, working around it. e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved) e820 update range: 0000000000000000 - 0000000000001000 (usable) ==> (reserved) e820 remove range: 00000000000a0000 - 0000000000100000 (usable) last_pfn = 0x83fe00 max_arch_pfn = 0x400000000 MTRR default type: uncachable MTRR fixed ranges enabled: 00000-9FFFF write-back A0000-BFFFF uncachable C0000-CFFFF write-protect D0000-E7FFF uncachable E8000-FFFFF write-protect MTRR variable ranges enabled: 0 base 000000000 mask 800000000 write-back 1 base 800000000 mask FC0000000 write-back 2 base 0BB800000 mask FFF800000 uncachable 3 base 0BC000000 mask FFC000000 uncachable 4 base 0C0000000 mask FC0000000 uncachable 5 base 83FE00000 mask FFFE00000 uncachable 6 disabled 7 disabled 8 disabled 9 disabled x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106 original variable MTRRs reg 0, base: 0GB, range: 32GB, type WB reg 1, base: 32GB, range: 1GB, type WB reg 2, base: 3000MB, range: 8MB, type UC reg 3, base: 3008MB, range: 64MB, type UC reg 4, base: 3GB, range: 1GB, type UC reg 5, base: 33790MB, range: 2MB, type UC total RAM covered: 32694M Found optimal setting for mtrr clean up gran_size: 64K chunk_size: 128M num_reg: 9 lose cover RAM: 0G New variable MTRRs reg 0, base: 0GB, range: 2GB, type WB reg 1, base: 2GB, range: 1GB, type WB reg 2, base: 3000MB, range: 8MB, type UC reg 3, base: 3008MB, range: 64MB, type UC reg 4, base: 4GB, range: 4GB, type WB reg 5, base: 8GB, range: 8GB, type WB reg 6, base: 16GB, range: 16GB, type WB reg 7, base: 32GB, range: 1GB, type WB reg 8, base: 33790MB, range: 2MB, type UC e820 update range: 00000000bb800000 - 0000000100000000 (usable) ==> (reserved) last_pfn = 0xbb000 max_arch_pfn = 0x400000000 initial memory mapped : 0 - 20000000 init_memory_mapping: 0000000000000000-00000000bb000000 0000000000 - 00bb000000 page 2M kernel direct mapping tables up to bb000000 @ 10000-14000 init_memory_mapping: 0000000100000000-000000083fe00000 0100000000 - 083fe00000 page 2M kernel direct mapping tables up to 83fe00000 @ 12000-34000 RAMDISK: 1f2bd000 - 1ffef50f ACPI: RSDP 00000000000f0420 00024 (v02 ALASKA) ACPI: XSDT 00000000bac63068 0004C (v01 ALASKA A M I 01072009 AMI 00010013) ACPI: FACP 00000000bac6cfa0 000F4 (v04 ALASKA A M I 01072009 AMI 00010013) ACPI: DSDT 00000000bac63140 09E5F (v02 ALASKA A M I 00000000 INTL 20051117) ACPI: FACS 00000000badd7f80 00040 ACPI: APIC 00000000bac6d098 00092 (v03 ALASKA A M I 01072009 AMI 00010013) ACPI: SSDT 00000000bac6d130 001D6 (v01 AMICPU PROC 00000001 MSFT 03000001) ACPI: MCFG 00000000bac6d308 0003C (v01 ALASKA A M I 01072009 MSFT 00000097) ACPI: HPET 00000000bac6d348 00038 (v01 ALASKA A M I 01072009 AMI. 00000004) ACPI: Local APIC address 0xfee00000 No NUMA configuration found Faking a node at 0000000000000000-000000083fe00000 Bootmem setup node 0 0000000000000000-000000083fe00000 NODE_DATA [000000000002f000 - 0000000000062fff] bootmap [0000000000100000 - 0000000000207fbf] pages 108 (8 early reservations) ==> bootmem [0000000000 - 083fe00000] #0 [0000000000 - 0000001000] BIOS data page ==> [0000000000 - 0000001000] #1 [0000006000 - 0000008000] TRAMPOLINE ==> [0000006000 - 0000008000] #2 [0001000000 - 0002012024] TEXT DATA BSS ==> [0001000000 - 0002012024] #3 [001f2bd000 - 001ffef50f] RAMDISK ==> [001f2bd000 - 001ffef50f] #4 [000009d800 - 0000100000] BIOS reserved ==> [000009d800 - 0000100000] #5 [0002013000 - 000201334e] BRK ==> [0002013000 - 000201334e] #6 [0000010000 - 0000012000] PGTABLE ==> [0000010000 - 0000012000] #7 [0000012000 - 000002f000] PGTABLE ==> [0000012000 - 000002f000] found SMP MP-table at [ffff8800000fcd90] fcd90 Reserving 131MB of memory at 48MB for crashkernel (System RAM: 33790MB) [ffffea0000000000-ffffea00147fffff] PMD -> [ffff88002c600000-ffff88003fffffff] on node 0 [ffffea0014800000-ffffea001cdfffff] PMD -> [ffff880040200000-ffff8800487fffff] on node 0 Zone PFN ranges: DMA 0x00000010 -> 0x00001000 DMA32 0x00001000 -> 0x00100000 Normal 0x00100000 -> 0x0083fe00 Movable zone start PFN for each node early_node_map[7] active PFN ranges 0: 0x00000010 -> 0x0000009d 0: 0x00000100 -> 0x00020000 0: 0x00020200 -> 0x00040000 0: 0x00040200 -> 0x000bac15 0: 0x000badcc -> 0x000badce 0: 0x000bae7d -> 0x000bb000 0: 0x00100000 -> 0x0083fe00 On node 0 totalpages: 8365863 DMA zone: 56 pages used for memmap DMA zone: 132 pages reserved DMA zone: 3793 pages, LIFO batch:0 DMA32 zone: 14280 pages used for memmap DMA32 zone: 745938 pages, LIFO batch:31 Normal zone: 103929 pages used for memmap Normal zone: 7497735 pages, LIFO batch:31 ACPI: PM-Timer IO Port: 0x408 ACPI: Local APIC address 0xfee00000 ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled) ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled) ACPI: LAPIC (acpi_id[0x03] lapic_id[0x04] enabled) ACPI: LAPIC (acpi_id[0x04] lapic_id[0x06] enabled) ACPI: LAPIC (acpi_id[0x05] lapic_id[0x01] enabled) ACPI: LAPIC (acpi_id[0x06] lapic_id[0x03] enabled) ACPI: LAPIC (acpi_id[0x07] lapic_id[0x05] enabled) ACPI: LAPIC (acpi_id[0x08] lapic_id[0x07] enabled) ACPI: LAPIC_NMI (acpi_id[0xff] high edge lint[0x1]) ACPI: IOAPIC (id[0x00] address[0xfec00000] gsi_base[0]) IOAPIC[0]: apic_id 0, version 32, address 0xfec00000, GSI 0-23 ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl) ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level) ACPI: IRQ0 used by override. ACPI: IRQ2 used by override. ACPI: IRQ9 used by override. Using ACPI (MADT) for SMP configuration information ACPI: HPET id: 0x8086a701 base: 0xfed00000 SMP: Allowing 8 CPUs, 0 hotplug CPUs nr_irqs_gsi: 24 PM: Registered nosave memory: 000000000009d000 - 000000000009e000 PM: Registered nosave memory: 000000000009e000 - 00000000000a0000 PM: Registered nosave memory: 00000000000a0000 - 00000000000e0000 PM: Registered nosave memory: 00000000000e0000 - 0000000000100000 PM: Registered nosave memory: 0000000020000000 - 0000000020200000 PM: Registered nosave memory: 0000000040000000 - 0000000040200000 PM: Registered nosave memory: 00000000bac15000 - 00000000bac71000 PM: Registered nosave memory: 00000000bac71000 - 00000000bada4000 PM: Registered nosave memory: 00000000bada4000 - 00000000badb5000 PM: Registered nosave memory: 00000000badb5000 - 00000000badcc000 PM: Registered nosave memory: 00000000badce000 - 00000000badd6000 PM: Registered nosave memory: 00000000badd6000 - 00000000bade0000 PM: Registered nosave memory: 00000000bade0000 - 00000000bae3a000 PM: Registered nosave memory: 00000000bae3a000 - 00000000bae7d000 PM: Registered nosave memory: 00000000bb000000 - 00000000bb800000 PM: Registered nosave memory: 00000000bb800000 - 00000000bfa00000 PM: Registered nosave memory: 00000000bfa00000 - 00000000fed1c000 PM: Registered nosave memory: 00000000fed1c000 - 00000000fed20000 PM: Registered nosave memory: 00000000fed20000 - 00000000ff000000 PM: Registered nosave memory: 00000000ff000000 - 0000000100000000 Allocating PCI resources starting at bfa00000 (gap: bfa00000:3f31c000) Booting paravirtualized kernel on bare hardware NR_CPUS:4096 nr_cpumask_bits:8 nr_cpu_ids:8 nr_node_ids:1 PERCPU: Embedded 31 pages/cpu @ffff88002c200000 s94424 r8192 d24360 u262144 pcpu-alloc: s94424 r8192 d24360 u262144 alloc=1*2097152 pcpu-alloc: [0] 0 1 2 3 4 5 6 7 Built 1 zonelists in Zone order, mobility grouping on. Total pages: 8247466 Policy zone: Normal Kernel command line: ro root=/dev/md2 crashkernel=131M@0M SYSFONT=latarcyrheb-sun16 LANG=en_US.UTF-8 KEYTABLE=de PID hash table entries: 4096 (order: 3, 32768 bytes) xsave/xrstor: enabled xstate_bv 0x7, cntxt size 0x340 Checking aperture... No AGP bridge found PCI-DMA: Using software bounce buffering for IO (SWIOTLB) Placing 64MB software IO TLB between ffff880024000000 - ffff880028000000 software IO TLB at phys 0x24000000 - 0x28000000 Memory: 32772900k/34600960k available (5153k kernel code, 1137508k absent, 690552k reserved, 7165k data, 1260k init) Hierarchical RCU implementation. NR_IRQS:33024 nr_irqs:472 Extended CMOS year: 2000 Console: colour VGA+ 80x25 console [tty0] enabled allocated 268435456 bytes of page_cgroup please try 'cgroup_disable=memory' option if you don't want memory cgroups hpet clockevent registered Fast TSC calibration using PIT Detected 3411.656 MHz processor. Calibrating delay loop (skipped), value calculated using timer frequency.. 6823.31 BogoMIPS (lpj=3411656) pid_max: default: 32768 minimum: 301 Security Framework initialized SELinux: Initializing. SELinux: Starting in permissive mode Dentry cache hash table entries: 4194304 (order: 13, 33554432 bytes) Inode-cache hash table entries: 2097152 (order: 12, 16777216 bytes) Mount-cache hash table entries: 256 Initializing cgroup subsys ns Initializing cgroup subsys cpuacct Initializing cgroup subsys memory Initializing cgroup subsys devices Initializing cgroup subsys freezer Initializing cgroup subsys net_cls Initializing cgroup subsys blkio Initializing cgroup subsys perf_event Initializing cgroup subsys net_prio CPU: Physical Processor ID: 0 CPU: Processor Core ID: 0 mce: CPU supports 9 MCE banks CPU0: Thermal monitoring enabled (TM1) using mwait in idle threads. ACPI: Core revision 20090903 ftrace: converting mcount calls to 0f 1f 44 00 00 ftrace: allocating 21019 entries in 83 pages x2apic not enabled, IRQ remapping init failed Setting APIC routing to flat ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1 CPU0: Intel(R) Core(TM) i7-2600 CPU @ 3.40GHz stepping 07 Performance Events: PEBS fmt1+, SandyBridge events, Intel PMU driver. ... version: 3 ... bit width: 48 ... generic registers: 4 ... value mask: 0000ffffffffffff ... max period: 000000007fffffff ... fixed-purpose events: 3 ... event mask: 000000070000000f NMI watchdog enabled, takes one hw-pmu counter. Booting Node 0, Processors #1 #2 #3 #4 #5 #6 #7 Ok. Brought up 8 CPUs Total of 8 processors activated (54586.49 BogoMIPS). sizeof(vma)=200 bytes sizeof(page)=56 bytes sizeof(inode)=592 bytes sizeof(dentry)=192 bytes sizeof(ext3inode)=800 bytes sizeof(buffer_head)=104 bytes sizeof(skbuff)=232 bytes sizeof(task_struct)=2648 bytes devtmpfs: initialized PM: Registering ACPI NVS region at bac15000 (376832 bytes) PM: Registering ACPI NVS region at bada4000 (69632 bytes) PM: Registering ACPI NVS region at badd6000 (40960 bytes) PM: Registering ACPI NVS region at bae3a000 (274432 bytes) regulator: core version 0.5 NET: Registered protocol family 16 ACPI: bus type pci registered PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 63 PCI: Not using MMCONFIG. PCI: Using configuration type 1 for base access bio: create slab <bio-0> at 0 ACPI: EC: Look up EC in DSDT ACPI: Executed 1 blocks of module-level executable AML code ACPI Error (psargs-0359): [RAMB] Namespace lookup failure, AE_NOT_FOUND ACPI Exception: AE_NOT_FOUND, Could not execute arguments for [RAMW] (Region) (20090903/nsinit-347) ACPI: Interpreter enabled ACPI: (supports S0 S1 S3 S4 S5) ACPI: Using IOAPIC for interrupt routing PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 63 PCI: MCFG area at e0000000 reserved in ACPI motherboard resources PCI: Using MMCONFIG at e0000000 - e3ffffff ACPI: BIOS _OSI(Linux) query ignored ACPI: No dock devices found. HEST: Table not found. PCI: Using host bridge windows from ACPI; if necessary, use "pci=nocrs" and report a bug ACPI: PCI Root Bridge [PCI0] (domain 0000 [bus 00-ff]) pci_root PNP0A08:00: host bridge window [io 0x0000-0x0cf7] pci_root PNP0A08:00: host bridge window [io 0x0d00-0xffff] pci_root PNP0A08:00: host bridge window [mem 0x000a0000-0x000bffff] pci_root PNP0A08:00: host bridge window [mem 0x000c8000-0x000dffff] pci_root PNP0A08:00: host bridge window [mem 0xbfa00000-0xffffffff] pci_root PNP0A08:00: ignoring host bridge window [mem 0x000c8000-0x000dffff] (conflicts with Video ROM [mem 0x000c0000-0x000cd7ff]) pci 0000:00:01.0: PME# supported from D0 D3hot D3cold pci 0000:00:01.0: PME# disabled pci 0000:00:02.0: reg 10: [mem 0xfe000000-0xfe3fffff 64bit] pci 0000:00:02.0: reg 18: [mem 0xc0000000-0xcfffffff 64bit pref] pci 0000:00:02.0: reg 20: [io 0xf000-0xf03f] pci 0000:00:16.0: reg 10: [mem 0xfe504000-0xfe50400f 64bit] pci 0000:00:16.0: PME# supported from D0 D3hot D3cold pci 0000:00:16.0: PME# disabled pci 0000:00:1a.0: reg 10: [mem 0xfe503000-0xfe5033ff] pci 0000:00:1a.0: PME# supported from D0 D3hot D3cold pci 0000:00:1a.0: PME# disabled pci 0000:00:1c.0: PME# supported from D0 D3hot D3cold pci 0000:00:1c.0: PME# disabled pci 0000:00:1c.5: PME# supported from D0 D3hot D3cold pci 0000:00:1c.5: PME# disabled pci 0000:00:1c.6: PME# supported from D0 D3hot D3cold pci 0000:00:1c.6: PME# disabled pci 0000:00:1c.7: PME# supported from D0 D3hot D3cold pci 0000:00:1c.7: PME# disabled pci 0000:00:1d.0: reg 10: [mem 0xfe502000-0xfe5023ff] pci 0000:00:1d.0: PME# supported from D0 D3hot D3cold pci 0000:00:1d.0: PME# disabled pci 0000:00:1f.2: reg 10: [io 0xf0b0-0xf0b7] pci 0000:00:1f.2: reg 14: [io 0xf0a0-0xf0a3] pci 0000:00:1f.2: reg 18: [io 0xf090-0xf097] pci 0000:00:1f.2: reg 1c: [io 0xf080-0xf083] pci 0000:00:1f.2: reg 20: [io 0xf060-0xf07f] pci 0000:00:1f.2: reg 24: [mem 0xfe501000-0xfe5017ff] pci 0000:00:1f.2: PME# supported from D3hot pci 0000:00:1f.2: PME# disabled pci 0000:00:1f.3: reg 10: [mem 0xfe500000-0xfe5000ff 64bit] pci 0000:00:1f.3: reg 20: [io 0xf040-0xf05f] pci 0000:00:01.0: PCI bridge to [bus 01-01] pci 0000:00:01.0: bridge window [io 0xf000-0x0000] (disabled) pci 0000:00:01.0: bridge window [mem 0xfff00000-0x000fffff] (disabled) pci 0000:00:01.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled) pci 0000:00:1c.0: PCI bridge to [bus 02-02] pci 0000:00:1c.0: bridge window [io 0xf000-0x0000] (disabled) pci 0000:00:1c.0: bridge window [mem 0xfff00000-0x000fffff] (disabled) pci 0000:00:1c.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled) pci 0000:03:00.0: reg 10: [mem 0xfe400000-0xfe407fff 64bit] pci 0000:03:00.0: PME# supported from D3hot D3cold pci 0000:03:00.0: PME# disabled pci 0000:00:1c.5: PCI bridge to [bus 03-03] pci 0000:00:1c.5: bridge window [io 0xf000-0x0000] (disabled) pci 0000:00:1c.5: bridge window [mem 0xfe400000-0xfe4fffff] pci 0000:00:1c.5: bridge window [mem 0xfff00000-0x000fffff pref] (disabled) pci 0000:04:00.0: reg 10: [io 0xe000-0xe0ff] pci 0000:04:00.0: reg 18: [mem 0xd0004000-0xd0004fff 64bit pref] pci 0000:04:00.0: reg 20: [mem 0xd0000000-0xd0003fff 64bit pref] pci 0000:04:00.0: supports D1 D2 pci 0000:04:00.0: PME# supported from D0 D1 D2 D3hot D3cold pci 0000:04:00.0: PME# disabled pci 0000:00:1c.6: PCI bridge to [bus 04-04] pci 0000:00:1c.6: bridge window [io 0xe000-0xefff] pci 0000:00:1c.6: bridge window [mem 0xfff00000-0x000fffff] (disabled) pci 0000:00:1c.6: bridge window [mem 0xd0000000-0xd00fffff 64bit pref] pci 0000:00:1c.7: PCI bridge to [bus 05-06] (subtractive decode) pci 0000:00:1c.7: bridge window [io 0xf000-0x0000] (disabled) pci 0000:00:1c.7: bridge window [mem 0xfff00000-0x000fffff] (disabled) pci 0000:00:1c.7: bridge window [mem 0xfff00000-0x000fffff pref] (disabled) pci 0000:00:1c.7: bridge window [io 0x0000-0x0cf7] (subtractive decode) pci 0000:00:1c.7: bridge window [io 0x0d00-0xffff] (subtractive decode) pci 0000:00:1c.7: bridge window [mem 0x000a0000-0x000bffff] (subtractive decode) pci 0000:00:1c.7: bridge window [mem 0xbfa00000-0xffffffff] (subtractive decode) pci 0000:05:00.0: PCI bridge to [bus 06-06] (subtractive decode) pci 0000:05:00.0: bridge window [io 0xfff000-0x0000] (disabled) pci 0000:05:00.0: bridge window [mem 0xfff00000-0x000fffff] (disabled) pci 0000:05:00.0: bridge window [mem 0xfff00000-0x000fffff pref] (disabled) pci 0000:05:00.0: bridge window [??? 0x00000000 flags 0x0] (subtractive decode) pci 0000:05:00.0: bridge window [??? 0x00000000 flags 0x0] (subtractive decode) pci 0000:05:00.0: bridge window [??? 0x00000000 flags 0x0] (subtractive decode) pci 0000:05:00.0: bridge window [??? 0x00000000 flags 0x0] (subtractive decode) pci 0000:05:00.0: bridge window [io 0x0000-0x0cf7] (subtractive decode) pci 0000:05:00.0: bridge window [io 0x0d00-0xffff] (subtractive decode) pci 0000:05:00.0: bridge window [mem 0x000a0000-0x000bffff] (subtractive decode) pci 0000:05:00.0: bridge window [mem 0xbfa00000-0xffffffff] (subtractive decode) ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX0._PRT] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX5._PRT] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX6._PRT] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PEX7.BR19._PRT] pci0000:00: Requesting ACPI _OSC control (0x1d) pci0000:00: ACPI _OSC control (0x1c) granted ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12 14 15) ACPI: PCI Interrupt Link [LNKB] (IRQs *3 4 5 6 7 10 11 12 14 15) ACPI: PCI Interrupt Link [LNKC] (IRQs 3 *4 5 6 10 11 12 14 15) ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 10 *11 12 14 15) ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 *10 11 12 14 15) ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 *10 11 12 14 15) ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 7 10 11 12 14 15) *0 ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 *5 6 7 10 11 12 14 15) vgaarb: device added: PCI:0000:00:02.0,decodes=io+mem,owns=io+mem,locks=none vgaarb: loaded vgaarb: bridge control possible 0000:00:02.0 SCSI subsystem initialized libata version 3.00 loaded. usbcore: registered new interface driver usbfs usbcore: registered new interface driver hub usbcore: registered new device driver usb PCI: Using ACPI for IRQ routing PCI: old code would have set cacheline size to 32 bytes, but clflush_size = 64 PCI: pci_cache_line_size set to 64 bytes NetLabel: Initializing NetLabel: domain hash size = 128 NetLabel: protocols = UNLABELED CIPSOv4 NetLabel: unlabeled traffic allowed by default alloc irq_desc for 24 on node 0 alloc kstat_irqs on node 0 alloc irq_desc for 25 on node 0 alloc kstat_irqs on node 0 alloc irq_desc for 26 on node 0 alloc kstat_irqs on node 0 alloc irq_desc for 27 on node 0 alloc kstat_irqs on node 0 alloc irq_desc for 28 on node 0 alloc kstat_irqs on node 0 HPET: 8 timers in total, 5 timers will be used for per-cpu timer hpet0: at MMIO 0xfed00000, IRQs 2, 8, 24, 25, 26, 27, 28, 0 hpet0: 8 comparators, 64-bit 14.318180 MHz counter hpet: hpet2 irq 24 for MSI hpet: hpet3 irq 25 for MSI hpet: hpet4 irq 26 for MSI hpet: hpet5 irq 27 for MSI hpet: hpet6 irq 28 for MSI Switching to clocksource hpet pnp: PnP ACPI init ACPI: bus type pnp registered pnp 00:00: [io 0x0cf8-0x0cff] pnp 00:00: Plug and Play ACPI device, IDs PNP0a08 PNP0a03 (active) pnp 00:01: [mem 0xfed10000-0xfed19fff] pnp 00:01: [mem 0xe0000000-0xe3ffffff] pnp 00:01: [mem 0xfed90000-0xfed93fff] pnp 00:01: [mem 0xfed20000-0xfed3ffff] pnp 00:01: [mem 0xfee00000-0xfee0ffff] pnp 00:01: Plug and Play ACPI device, IDs PNP0c01 (active) pnp 00:02: [io 0x0000-0xffffffffffffffff disabled] pnp 00:02: [io 0x0000-0xffffffffffffffff disabled] pnp 00:02: [io 0x0290-0x029f] pnp 00:02: [io 0x0000-0xffffffffffffffff disabled] pnp 00:02: Plug and Play ACPI device, IDs PNP0c02 (active) pnp 00:03: [dma 4] pnp 00:03: [io 0x0000-0x000f] pnp 00:03: [io 0x0081-0x0083] pnp 00:03: [io 0x0087] pnp 00:03: [io 0x0089-0x008b] pnp 00:03: [io 0x008f] pnp 00:03: [io 0x00c0-0x00df] pnp 00:03: Plug and Play ACPI device, IDs PNP0200 (active) pnp 00:04: [io 0x0070-0x0071] pnp 00:04: [irq 8] pnp 00:04: Plug and Play ACPI device, IDs PNP0b00 (active) pnp 00:05: [io 0x0061] pnp 00:05: Plug and Play ACPI device, IDs PNP0800 (active) pnp 00:06: [io 0x0010-0x001f] pnp 00:06: [io 0x0022-0x003f] pnp 00:06: [io 0x0044-0x005f] pnp 00:06: [io 0x0063] pnp 00:06: [io 0x0065] pnp 00:06: [io 0x0067-0x006f] pnp 00:06: [io 0x0072-0x007f] pnp 00:06: [io 0x0080] pnp 00:06: [io 0x0084-0x0086] pnp 00:06: [io 0x0088] pnp 00:06: [io 0x008c-0x008e] pnp 00:06: [io 0x0090-0x009f] pnp 00:06: [io 0x00a2-0x00bf] pnp 00:06: [io 0x00e0-0x00ef] pnp 00:06: [io 0x04d0-0x04d1] pnp 00:06: Plug and Play ACPI device, IDs PNP0c02 (active) pnp 00:07: [io 0x00f0-0x00ff] pnp 00:07: [irq 13] pnp 00:07: Plug and Play ACPI device, IDs PNP0c04 (active) pnp 00:08: [io 0x0060] pnp 00:08: [io 0x0064] pnp 00:08: [irq 1] pnp 00:08: Plug and Play ACPI device, IDs PNP0303 PNP030b (active) pnp 00:09: [io 0x0400-0x0453] pnp 00:09: [io 0x0458-0x047f] pnp 00:09: [io 0x0000-0xffffffffffffffff disabled] pnp 00:09: [io 0x0500-0x057f] pnp 00:09: [mem 0xfed1c000-0xfed1ffff] pnp 00:09: [mem 0xfec00000-0xfecfffff] pnp 00:09: [mem 0xfed08000-0xfed08fff] pnp 00:09: [mem 0xff000000-0xffffffff] pnp 00:09: Plug and Play ACPI device, IDs PNP0c01 (active) pnp 00:0a: [io 0x0454-0x0457] pnp 00:0a: Plug and Play ACPI device, IDs INT3f0d PNP0c02 (active) pnp 00:0b: [mem 0xfed00000-0xfed003ff] pnp 00:0b: Plug and Play ACPI device, IDs PNP0103 (active) pnp: PnP ACPI: found 12 devices ACPI: ACPI bus type pnp unregistered system 00:01: [mem 0xfed10000-0xfed19fff] has been reserved system 00:01: [mem 0xe0000000-0xe3ffffff] has been reserved system 00:01: [mem 0xfed90000-0xfed93fff] has been reserved system 00:01: [mem 0xfed20000-0xfed3ffff] has been reserved system 00:01: [mem 0xfee00000-0xfee0ffff] has been reserved system 00:02: [io 0x0290-0x029f] has been reserved system 00:06: [io 0x04d0-0x04d1] has been reserved system 00:09: [io 0x0400-0x0453] has been reserved system 00:09: [io 0x0458-0x047f] has been reserved system 00:09: [io 0x0500-0x057f] has been reserved system 00:09: [mem 0xfed1c000-0xfed1ffff] has been reserved system 00:09: [mem 0xfec00000-0xfecfffff] could not be reserved system 00:09: [mem 0xfed08000-0xfed08fff] has been reserved system 00:09: [mem 0xff000000-0xffffffff] has been reserved system 00:0a: [io 0x0454-0x0457] has been reserved PCI: max bus depth: 2 pci_try_num: 3 pci 0000:00:01.0: PCI bridge to [bus 01-01] pci 0000:00:01.0: PCI bridge to [bus 01-01] pci 0000:00:01.0: bridge window [io disabled] pci 0000:00:01.0: bridge window [mem disabled] pci 0000:00:01.0: bridge window [mem pref disabled] pci 0000:00:1c.0: PCI bridge to [bus 02-02] pci 0000:00:1c.0: PCI bridge to [bus 02-02] pci 0000:00:1c.0: bridge window [io disabled] pci 0000:00:1c.0: bridge window [mem disabled] pci 0000:00:1c.0: bridge window [mem pref disabled] pci 0000:00:1c.5: PCI bridge to [bus 03-03] pci 0000:00:1c.5: PCI bridge to [bus 03-03] pci 0000:00:1c.5: bridge window [io disabled] pci 0000:00:1c.5: bridge window [mem 0xfe400000-0xfe4fffff] pci 0000:00:1c.5: bridge window [mem pref disabled] pci 0000:00:1c.6: PCI bridge to [bus 04-04] pci 0000:00:1c.6: PCI bridge to [bus 04-04] pci 0000:00:1c.6: bridge window [io 0xe000-0xefff] pci 0000:00:1c.6: bridge window [mem disabled] pci 0000:00:1c.6: bridge window [mem 0xd0000000-0xd00fffff 64bit pref] pci 0000:05:00.0: PCI bridge to [bus 06-06] pci 0000:05:00.0: PCI bridge to [bus 06-06] pci 0000:05:00.0: bridge window [io disabled] pci 0000:05:00.0: bridge window [mem disabled] pci 0000:05:00.0: bridge window [mem pref disabled] pci 0000:00:1c.7: PCI bridge to [bus 05-06] pci 0000:00:1c.7: PCI bridge to [bus 05-06] pci 0000:00:1c.7: bridge window [io disabled] pci 0000:00:1c.7: bridge window [mem disabled] pci 0000:00:1c.7: bridge window [mem pref disabled] alloc irq_desc for 16 on node -1 alloc kstat_irqs on node -1 pci 0000:00:01.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 pci 0000:00:01.0: setting latency timer to 64 alloc irq_desc for 17 on node -1 alloc kstat_irqs on node -1 pci 0000:00:1c.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17 pci 0000:00:1c.0: setting latency timer to 64 pci 0000:00:1c.5: PCI INT B -> GSI 16 (level, low) -> IRQ 16 pci 0000:00:1c.5: setting latency timer to 64 alloc irq_desc for 18 on node -1 alloc kstat_irqs on node -1 pci 0000:00:1c.6: PCI INT C -> GSI 18 (level, low) -> IRQ 18 pci 0000:00:1c.6: setting latency timer to 64 alloc irq_desc for 19 on node -1 alloc kstat_irqs on node -1 pci 0000:00:1c.7: PCI INT D -> GSI 19 (level, low) -> IRQ 19 pci 0000:00:1c.7: setting latency timer to 64 pci 0000:05:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17 pci 0000:05:00.0: setting latency timer to 64 pci_bus 0000:00: resource 4 [io 0x0000-0x0cf7] pci_bus 0000:00: resource 5 [io 0x0d00-0xffff] pci_bus 0000:00: resource 6 [mem 0x000a0000-0x000bffff] pci_bus 0000:00: resource 7 [mem 0xbfa00000-0xffffffff] pci_bus 0000:03: resource 1 [mem 0xfe400000-0xfe4fffff] pci_bus 0000:04: resource 0 [io 0xe000-0xefff] pci_bus 0000:04: resource 2 [mem 0xd0000000-0xd00fffff 64bit pref] pci_bus 0000:05: resource 4 [io 0x0000-0x0cf7] pci_bus 0000:05: resource 5 [io 0x0d00-0xffff] pci_bus 0000:05: resource 6 [mem 0x000a0000-0x000bffff] pci_bus 0000:05: resource 7 [mem 0xbfa00000-0xffffffff] pci_bus 0000:06: resource 8 [io 0x0000-0x0cf7] pci_bus 0000:06: resource 9 [io 0x0d00-0xffff] pci_bus 0000:06: resource 10 [mem 0x000a0000-0x000bffff] pci_bus 0000:06: resource 11 [mem 0xbfa00000-0xffffffff] NET: Registered protocol family 2 IP route cache hash table entries: 524288 (order: 10, 4194304 bytes) TCP established hash table entries: 524288 (order: 11, 8388608 bytes) TCP bind hash table entries: 65536 (order: 8, 1048576 bytes) TCP: Hash tables configured (established 524288 bind 65536) TCP reno registered NET: Registered protocol family 1 pci 0000:00:02.0: Boot video device alloc irq_desc for 23 on node -1 alloc kstat_irqs on node -1 pci 0000:00:1a.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23 pci 0000:00:1a.0: PCI INT A disabled pci 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23 pci 0000:00:1d.0: PCI INT A disabled pci 0000:03:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17 pci 0000:03:00.0: PCI INT A disabled Trying to unpack rootfs image as initramfs... Freeing initrd memory: 13513k freed audit: initializing netlink socket (disabled) type=2000 audit(1348853735.531:1): initialized HugeTLB registered 2 MB page size, pre-allocated 0 pages VFS: Disk quotas dquot_6.5.2 Dquot-cache hash table entries: 512 (order 0, 4096 bytes) msgmni has been set to 32768 SELinux: Registering netfilter hooks alg: No test for stdrng (krng) ksign: Installing public key data Loading keyring - Added public key CBB175BA83D55050 - User ID: CentOS (Kernel Module GPG key) Block layer SCSI generic (bsg) driver version 0.4 loaded (major 252) io scheduler noop registered io scheduler anticipatory registered io scheduler deadline registered io scheduler cfq registered (default) pcieport 0000:00:01.0: setting latency timer to 64 alloc irq_desc for 29 on node -1 alloc kstat_irqs on node -1 pcieport 0000:00:01.0: irq 29 for MSI/MSI-X pcieport 0000:00:1c.0: setting latency timer to 64 alloc irq_desc for 30 on node -1 alloc kstat_irqs on node -1 pcieport 0000:00:1c.0: irq 30 for MSI/MSI-X pcieport 0000:00:1c.5: setting latency timer to 64 alloc irq_desc for 31 on node -1 alloc kstat_irqs on node -1 pcieport 0000:00:1c.5: irq 31 for MSI/MSI-X pcieport 0000:00:1c.6: setting latency timer to 64 alloc irq_desc for 32 on node -1 alloc kstat_irqs on node -1 pcieport 0000:00:1c.6: irq 32 for MSI/MSI-X pci_hotplug: PCI Hot Plug PCI Core version: 0.5 pciehp: PCI Express Hot Plug Controller Driver version: 0.4 acpiphp: ACPI Hot Plug PCI Controller Driver version: 0.5 intel_idle: MWAIT substates: 0x1120 intel_idle: v0.4 model 0x2A intel_idle: lapic_timer_reliable_states 0xffffffff input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0C0C:00/input/input0 ACPI: Power Button [PWRB] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input1 ACPI: Power Button [PWRF] ACPI: acpi_idle yielding to intel_idleACPI: SSDT 00000000baddf898 006F4 (v01 AMI IST 00000001 MSFT 03000001) ACPI: SSDT 00000000badd6d98 000E4 (v01 AMI CST 00000001 MSFT 03000001) ERST: Table is not found! GHES: HEST is not enabled! Non-volatile memory driver v1.3 Linux agpgart interface v0.103 agpgart-intel 0000:00:00.0: Intel Sandybridge Chipset agpgart-intel 0000:00:00.0: detected gtt size: 2097152K total, 262144K mappable agpgart-intel 0000:00:00.0: detected 65536K stolen memory agpgart-intel 0000:00:00.0: AGP aperture is 256M @ 0xc0000000 crash memory driver: version 1.1 Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled brd: module loaded loop: module loaded input: Macintosh mouse button emulation as /devices/virtual/input/input2 Fixed MDIO Bus: probed ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver ehci_hcd 0000:00:1a.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23 ehci_hcd 0000:00:1a.0: setting latency timer to 64 ehci_hcd 0000:00:1a.0: EHCI Host Controller ehci_hcd 0000:00:1a.0: new USB bus registered, assigned bus number 1 ehci_hcd 0000:00:1a.0: debug port 2 ehci_hcd 0000:00:1a.0: cache line size of 64 is not supported ehci_hcd 0000:00:1a.0: irq 23, io mem 0xfe503000 ehci_hcd 0000:00:1a.0: USB 2.0 started, EHCI 1.00 usb usb1: New USB device found, idVendor=1d6b, idProduct=0002 usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb1: Product: EHCI Host Controller usb usb1: Manufacturer: Linux 2.6.32-279.9.1.el6.x86_64 ehci_hcd usb usb1: SerialNumber: 0000:00:1a.0 usb usb1: configuration #1 chosen from 1 choice hub 1-0:1.0: USB hub found hub 1-0:1.0: 2 ports detected ehci_hcd 0000:00:1d.0: PCI INT A -> GSI 23 (level, low) -> IRQ 23 ehci_hcd 0000:00:1d.0: setting latency timer to 64 ehci_hcd 0000:00:1d.0: EHCI Host Controller ehci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2 ehci_hcd 0000:00:1d.0: debug port 2 ehci_hcd 0000:00:1d.0: cache line size of 64 is not supported ehci_hcd 0000:00:1d.0: irq 23, io mem 0xfe502000 ehci_hcd 0000:00:1d.0: USB 2.0 started, EHCI 1.00 usb usb2: New USB device found, idVendor=1d6b, idProduct=0002 usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb2: Product: EHCI Host Controller usb usb2: Manufacturer: Linux 2.6.32-279.9.1.el6.x86_64 ehci_hcd usb usb2: SerialNumber: 0000:00:1d.0 usb usb2: configuration #1 chosen from 1 choice hub 2-0:1.0: USB hub found hub 2-0:1.0: 2 ports detected ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver uhci_hcd: USB Universal Host Controller Interface driver PNP: PS/2 Controller [PNP0303:PS2K] at 0x60,0x64 irq 1 PNP: PS/2 appears to have AUX port disabled, if this is incorrect please boot with i8042.nopnp serio: i8042 KBD port at 0x60,0x64 irq 1 mice: PS/2 mouse device common for all mice rtc_cmos 00:04: RTC can wake from S4 rtc_cmos 00:04: rtc core: registered rtc_cmos as rtc0 rtc0: alarms up to one month, y3k, 114 bytes nvram, hpet irqs cpuidle: using governor ladder cpuidle: using governor menu EFI Variables Facility v0.08 2004-May-17 usbcore: registered new interface driver hiddev usbcore: registered new interface driver usbhid usbhid: v2.6:USB HID core driver TCP cubic registered Initializing XFRM netlink socket NET: Registered protocol family 17 registered taskstats version 1 rtc_cmos 00:04: setting system clock to 2012-09-28 17:35:36 UTC (1348853736) Initalizing network drop monitor service Freeing unused kernel memory: 1260k freed Write protecting the kernel read-only data: 10240k Freeing unused kernel memory: 972k freed Freeing unused kernel memory: 1732k freed dracut: dracut-004-284.el6_3 device-mapper: uevent: version 1.0.3 device-mapper: ioctl: 4.22.6-ioctl (2011-10-19) initialised: dm-devel(a)redhat.com udev: starting version 147 input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input3 [drm] Initialized drm 1.1.0 20060810 i915 0000:00:02.0: PCI INT A -> GSI 16 (level, low) -> IRQ 16 i915 0000:00:02.0: setting latency timer to 64 alloc irq_desc for 33 on node -1 alloc kstat_irqs on node -1 i915 0000:00:02.0: irq 33 for MSI/MSI-X [drm] Supports vblank timestamp caching Rev 1 (10.10.2010). [drm] Driver supports precise vblank timestamp query. vgaarb: device changed decodes: PCI:0000:00:02.0,olddecodes=io+mem,decodes=io+mem:owns=io+mem No connectors reported connected with modes [drm] Cannot find any crtc or sizes - going 1024x768 fbcon: inteldrmfb (fb0) is primary device Console: switching to colour frame buffer device 128x48 fb0: inteldrmfb frame buffer device drm: registered panic notifier Slow work thread pool: Starting up Slow work thread pool: Ready fixme: max PWM is zero. acpi device:32: registered as cooling_device8 input: Video Bus as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A08:00/LNXVIDEO:00/input/input4 ACPI: Video Device [GFX0] (multi-head: yes rom: no post: no) [drm] Initialized i915 1.6.0 20080730 for 0000:00:02.0 on minor 0 usb 1-1: new high speed USB device number 2 using ehci_hcd dracut: Starting plymouth daemon usb 1-1: New USB device found, idVendor=8087, idProduct=0024 usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 usb 1-1: configuration #1 chosen from 1 choice hub 1-1:1.0: USB hub found hub 1-1:1.0: 6 ports detected ahci 0000:00:1f.2: version 3.0 alloc irq_desc for 20 on node -1 alloc kstat_irqs on node -1 ahci 0000:00:1f.2: PCI INT B -> GSI 20 (level, low) -> IRQ 20 alloc irq_desc for 34 on node -1 alloc kstat_irqs on node -1 ahci 0000:00:1f.2: irq 34 for MSI/MSI-X ahci 0000:00:1f.2: AHCI 0001.0300 32 slots 6 ports 6 Gbps 0x3 impl SATA mode ahci 0000:00:1f.2: flags: 64bit ncq sntf pm led clo pio slum part ems apst ahci 0000:00:1f.2: setting latency timer to 64 scsi0 : ahci scsi1 : ahci scsi2 : ahci scsi3 : ahci scsi4 : ahci scsi5 : ahci ata1: SATA max UDMA/133 abar m2048@0xfe501000 port 0xfe501100 irq 34 ata2: SATA max UDMA/133 irq_stat 0x00400040, connection status changed irq 34 ata3: DUMMY ata4: DUMMY ata5: DUMMY ata6: DUMMY usb 2-1: new high speed USB device number 2 using ehci_hcd usb 2-1: New USB device found, idVendor=8087, idProduct=0024 usb 2-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 usb 2-1: configuration #1 chosen from 1 choice hub 2-1:1.0: USB hub found hub 2-1:1.0: 8 ports detected ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300) ata1.00: ATA-8: ST33000651AS, CC44, max UDMA/133 ata1.00: 5860533168 sectors, multi 16: LBA48 NCQ (depth 31/32) ata1.00: configured for UDMA/133 scsi 0:0:0:0: Direct-Access ATA ST33000651AS CC44 PQ: 0 ANSI: 5 Refined TSC clocksource calibration: 3411.482 MHz. Switching to clocksource tsc ata2: SATA link up 6.0 Gbps (SStatus 133 SControl 300) ata2.00: ATA-8: ST33000651AS, CC44, max UDMA/133 ata2.00: 5860533168 sectors, multi 16: LBA48 NCQ (depth 31/32) ata2.00: configured for UDMA/133 scsi 1:0:0:0: Direct-Access ATA ST33000651AS CC44 PQ: 0 ANSI: 5 sd 0:0:0:0: [sda] 5860533168 512-byte logical blocks: (3.00 TB/2.72 TiB) sd 1:0:0:0: [sdb] 5860533168 512-byte logical blocks: (3.00 TB/2.72 TiB) sd 1:0:0:0: [sdb] Write Protect is off sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00 sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA sdb: sd 0:0:0:0: [sda] Write Protect is off sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00 sd 0:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA sda: sda1 sda2 sda3 sd 0:0:0:0: [sda] Attached SCSI disk sdb1 sdb2 sdb3 sd 1:0:0:0: [sdb] Attached SCSI disk md: bind<sda1> md: bind<sda3> md: bind<sda2> md: bind<sdb1> md: raid1 personality registered for level 1 bio: create slab <bio-1> at 1 md/raid1:md0: active with 2 out of 2 mirrors md0: detected capacity change from 0 to 2146369536 md0: unknown partition table md: bind<sdb3> md/raid1:md2: active with 2 out of 2 mirrors md2: detected capacity change from 0 to 2195265093632 md2: unknown partition table md: bind<sdb2> md/raid1:md1: active with 2 out of 2 mirrors md1: detected capacity change from 0 to 536805376 md1: unknown partition table kjournald starting. Commit interval 5 seconds EXT3-fs (md2): mounted filesystem with ordered data mode dracut: Mounted root filesystem /dev/md2 dracut: Loading SELinux policy SELinux: 2048 avtab hash slots, 251092 rules. SELinux: 2048 avtab hash slots, 251092 rules. SELinux: 9 users, 12 roles, 3761 types, 187 bools, 1 sens, 1024 cats SELinux: 81 classes, 251092 rules SELinux: Completing initialization. SELinux: Setting up existing superblocks. SELinux: initialized (dev md2, type ext3), uses xattr SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs SELinux: initialized (dev usbfs, type usbfs), uses genfs_contexts SELinux: initialized (dev selinuxfs, type selinuxfs), uses genfs_contexts SELinux: initialized (dev mqueue, type mqueue), uses transition SIDs SELinux: initialized (dev hugetlbfs, type hugetlbfs), uses transition SIDs SELinux: initialized (dev devpts, type devpts), uses transition SIDs SELinux: initialized (dev inotifyfs, type inotifyfs), uses genfs_contexts SELinux: initialized (dev anon_inodefs, type anon_inodefs), uses genfs_contexts SELinux: initialized (dev pipefs, type pipefs), uses task SIDs SELinux: initialized (dev debugfs, type debugfs), uses genfs_contexts SELinux: initialized (dev sockfs, type sockfs), uses task SIDs SELinux: initialized (dev devtmpfs, type devtmpfs), uses transition SIDs SELinux: initialized (dev tmpfs, type tmpfs), uses transition SIDs SELinux: initialized (dev proc, type proc), uses genfs_contexts SELinux: initialized (dev bdev, type bdev), uses genfs_contexts SELinux: initialized (dev rootfs, type rootfs), uses genfs_contexts SELinux: initialized (dev sysfs, type sysfs), uses genfs_contexts type=1403 audit(1348853738.977:2): policy loaded auid=4294967295 ses=4294967295 dracut: dracut: Switching root udev: starting version 147 xhci_hcd 0000:03:00.0: PCI INT A -> GSI 17 (level, low) -> IRQ 17 xhci_hcd 0000:03:00.0: setting latency timer to 64 xhci_hcd 0000:03:00.0: xHCI Host Controller xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 3 xhci_hcd 0000:03:00.0: irq 17, io mem 0xfe400000 alloc irq_desc for 35 on node -1 alloc kstat_irqs on node -1 xhci_hcd 0000:03:00.0: irq 35 for MSI/MSI-X alloc irq_desc for 36 on node -1 alloc kstat_irqs on node -1 xhci_hcd 0000:03:00.0: irq 36 for MSI/MSI-X alloc irq_desc for 37 on node -1 alloc kstat_irqs on node -1 xhci_hcd 0000:03:00.0: irq 37 for MSI/MSI-X alloc irq_desc for 38 on node -1 alloc kstat_irqs on node -1 xhci_hcd 0000:03:00.0: irq 38 for MSI/MSI-X alloc irq_desc for 39 on node -1 alloc kstat_irqs on node -1 xhci_hcd 0000:03:00.0: irq 39 for MSI/MSI-X alloc irq_desc for 40 on node -1 alloc kstat_irqs on node -1 xhci_hcd 0000:03:00.0: irq 40 for MSI/MSI-X alloc irq_desc for 41 on node -1 alloc kstat_irqs on node -1 xhci_hcd 0000:03:00.0: irq 41 for MSI/MSI-X alloc irq_desc for 42 on node -1 alloc kstat_irqs on node -1 xhci_hcd 0000:03:00.0: irq 42 for MSI/MSI-X usb usb3: New USB device found, idVendor=1d6b, idProduct=0002 usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb3: Product: xHCI Host Controller usb usb3: Manufacturer: Linux 2.6.32-279.9.1.el6.x86_64 xhci_hcd usb usb3: SerialNumber: 0000:03:00.0 usb usb3: configuration #1 chosen from 1 choice xHCI xhci_add_endpoint called for root hub xHCI xhci_check_bandwidth called for root hub hub 3-0:1.0: USB hub found hub 3-0:1.0: 2 ports detected xhci_hcd 0000:03:00.0: xHCI Host Controller xhci_hcd 0000:03:00.0: new USB bus registered, assigned bus number 4 usb usb4: config 1 interface 0 altsetting 0 endpoint 0x81 has no SuperSpeed companion descriptor usb usb4: New USB device found, idVendor=1d6b, idProduct=0003 usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1 usb usb4: Product: xHCI Host Controller usb usb4: Manufacturer: Linux 2.6.32-279.9.1.el6.x86_64 xhci_hcd usb usb4: SerialNumber: 0000:03:00.0 usb usb4: configuration #1 chosen from 1 choice xHCI xhci_add_endpoint called for root hub xHCI xhci_check_bandwidth called for root hub hub 4-0:1.0: USB hub found hub 4-0:1.0: 2 ports detected iTCO_vendor_support: vendor-support=0 iTCO_wdt: Intel TCO WatchDog Timer Driver v1.07rh iTCO_wdt: Found a Cougar Point TCO device (Version=2, TCOBASE=0x0460) iTCO_wdt: initialized. heartbeat=30 sec (nowayout=0) i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level, low) -> IRQ 18 r8169 Gigabit Ethernet driver 2.3LK-NAPI loaded r8169 0000:04:00.0: PCI INT A -> GSI 18 (level, low) -> IRQ 18 r8169 0000:04:00.0: setting latency timer to 64 alloc irq_desc for 43 on node -1 alloc kstat_irqs on node -1 r8169 0000:04:00.0: irq 43 for MSI/MSI-X r8169 0000:04:00.0: eth0: RTL8168e/8111e at 0xffffc90007d9e000, 14:da:e9:b3:99:b9, XID 0c200000 IRQ 43 r8169 0000:04:00.0: eth0: jumbo features [frames: 9200 bytes, tx checksumming: ko] sd 0:0:0:0: Attached scsi generic sg0 type 0 sd 1:0:0:0: Attached scsi generic sg1 type 0 EXT3-fs (md2): warning: maximal mount count reached, running e2fsck is recommended EXT3-fs (md2): using internal journal kjournald starting. Commit interval 5 seconds EXT3-fs (md1): warning: maximal mount count reached, running e2fsck is recommended EXT3-fs (md1): using internal journal EXT3-fs (md1): mounted filesystem with ordered data mode SELinux: initialized (dev md1, type ext3), uses xattr Adding 2096056k swap on /dev/md0. Priority:-1 extents:1 across:2096056k SELinux: initialized (dev binfmt_misc, type binfmt_misc), uses genfs_contexts ip_tables: (C) 2000-2006 Netfilter Core Team nf_conntrack version 0.5.0 (16384 buckets, 65536 max) NET: Registered protocol family 10 lo: Disabled Privacy Extensions r8169 0000:04:00.0: firmware: requesting rtl_nic/rtl8168e-2.fw r8169 0000:04:00.0: eth0: link down r8169 0000:04:00.0: eth0: link down ADDRCONF(NETDEV_UP): eth0: link is not ready r8169 0000:04:00.0: eth0: link up ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready eth0: no IPv6 routers present TCP: Peer 84.153.49.176:2024/80 unexpectedly shrunk window 1555448146:1555457576 (repaired)

2 2

Re: [CentOS] Apache : Listen on ports other than 80
by adekunleadekoya＠gmail.com 01 Oct '12

01 Oct '12

I v run d netstat and no result(i.e no process is listening on port 1010). Also I telnet 127.0.0.1 1010 just 2 confirm that no process is using the port. I can't figure out why the port(and any other except 80) would not work! ------Original Message------ From: John R Pierce Sender: centos-bounces(a)centos.org To: centos(a)centos.org ReplyTo: CentOS mailing list Subject: Re: [CentOS] Apache : Listen on ports other than 80 Sent: Sep 28, 2012 9:41 PM On 09/28/12 1:30 PM, adekunleadekoya(a)gmail.com wrote: > I saw this error msg : 'permission denied:make_sock : could not bind to address 0.0.0.0:1010. > No listening sockets available,shutting down' . is another process listening on 1010 ? netstat -an |grep :1010 look for lines with LISTEN on the end ... -- john r pierce N 37, W 122 santa cruz ca mid-left coast _______________________________________________ CentOS mailing list CentOS(a)centos.org http://lists.centos.org/mailman/listinfo/centos Sent from my BlackBerry wireless device from MTN

4 3

fsck -fvy ?
by Hadi Motamedi 01 Oct '12

01 Oct '12

Dear All My server got inconsistency from sudden power cut that I fixed it with #fsck -fvy /dev/hda at the maintenance prompt . But after reboot, one of the installed applications is preventing it from booting up (as can be seen in the boot up process list). Can you please let me know how can I remove it from the boot up list to reach the root user prompt and then re-install it again ? Thank you

3 2

Changes to inodes discovered by aide
by Jobst Schmalenbach 01 Oct '12

01 Oct '12

Hi. On one of my servers aide just reported inode changes to a large bunch of files in a variety of directories, e.g. /usr/bin, /usr/sbin etc. This machine sits behind a couple of firewalls and it would be hard to get to. The day before I updated "clam*" and updated the aide database right after that: -rw------- 1 root root 7407412 Sep 26 10:58 aide.db.gz The problem was that the changes were made when no-one was in the office, here are a few: Directory: /usr/sbin Mtime : 2012-09-26 10:55:15 , 2012-09-27 06:36:42 Ctime : 2012-09-26 10:55:15 , 2012-09-27 06:36:42 File: /usr/sbin/wpa_supplicant Ctime : 2012-09-07 06:39:44 , 2012-09-27 06:36:40 Inode : 2490595 , 2490536 MD5 : IVNJESmXwIG9XY0MowL3CA== , DUQMpFMsKqlZgjOmJIp3OQ== RMD160 : 4xuWhqqliTLM5Jx6zAvQ9f1PY1c= , AlSPQGiVe+/T8YdHDSIypI904kA= SHA256 : OaUWNIGUS9AhXEjV3p8Cg4TeIEjuQ/tu , z1c9XCKVyjDzDuN7t32B+sbj6nil90TK File: /usr/sbin/clamav-milter Size : 202453 , 206637 Ctime : 2012-09-26 10:55:15 , 2012-09-27 06:36:37 Inode : 2490507 , 2490625 MD5 : HoONWy9q+qbRzHtlTeR6Wg== , klWTxNFmL8MEAQmIPwvHxg== RMD160 : lfa72Vrh6Q2DWjf+UIxREAK4V1Y= , MPbEoKH/ws3aWA+sBuycRvU9DP0= SHA256 : aFRvKcA999IPRFJ2qByu8aKB6QmHpW5i , u0oTtBkHjchhlY8AIejOfKPoJRencpmK Yum does not report anything (last 4 lines os yum.log) Sep 21 10:40:11 Installed: ghostscript-fonts-5.50-13.1.1.noarch Sep 26 10:55:14 Updated: clamav-0.97.6-1.el5.rf.x86_64 Sep 26 10:55:15 Updated: clamd-0.97.6-1.el5.rf.x86_64 Sep 26 10:55:15 Updated: clamav-milter-0.97.6-1.el5.rf.x86_64 I ran (a fresh install) of rkhunter, did not find a thing ... Is it possible that a change to one file sets of a domino effect of indode changes? thanks Jobst -- Diplomacy: The art of saying, "Nice Doggy," until you can find a stick. | |0| | Jobst Schmalenbach, jobst(a)barrett.com.au, General Manager | | |0| Barrett Consulting Group P/L & The Meditation Room P/L |0|0|0| +61 3 9532 7677, POBox 277, Caulfield South, 3162, Australia

2 2

Prelink failure
by Lars Hecking 30 Sep '12

30 Sep '12

I have a number of CentOS6 machines, and on one of them, the daily prelink cron job aborts. Any ideas what to look for? /etc/cron.daily/prelink: /etc/cron.daily/prelink: line 47: 9381 Aborted (core dumped) /usr/sbin/prelink -av $PRELINK_OPTS >> /var/log/prelink/prelink.log 2>&1 I'm unfamiliar with prelink, so I'm not sure what to look for in the logs. Grepping out recurring entries like "Could not find one of the dependencies", "^Prelinking", "^Linking", "undefined non-weak symbols", "Could not prelink", the interesting stuff remaining is /usr/sbin/prelink: /opt/lgtonmc/bin/libcommonssl2.so is not present in any config file directories, nor was specified on command line /usr/sbin/prelink: /usr/pgsql-9.1/lib/libpq.so.5.4 is not present in any config file directories, nor was specified on command line /usr/sbin/prelink: /usr/bin/emacs-23.1: COPY relocations don't point into .bss or .sbss section /usr/sbin/prelink: /usr/bin/emacs-23.1-nox: COPY relocations don't point into .bss or .sbss section Prelink failed with return value 134

3 3

Re: [CentOS] hi
by James B. Byrne 30 Sep '12

30 Sep '12

On Fri, September 28, 2012 15:21, m.roth(a)5-cent.us wrote: > Andrei Rolando LeÃ³n Salas wrote: >> Hello how are u, im having a little problem. i use the webmin to >> install >> automatly postfix, in the other server works, but in this now says. >> >> Error while checking current Postfix configuration. Please manually >> fix >> Postfix configuration. >> >> postfix: fatal: file /etc/postfix/main.cf: parameter mail_owner: >> unknown user name value: postfix >> >> Manually edit the Postfix configuration >> file.<https://mercadeoeficaz-cr.com:10000/postfix/edit_manual.cgi> >> >> i dont know what to do, can u help me please > > You need to learn something about what you're doing. > > For one, how did webmin install postfix? If from a tarball, then > there's a > bunch of stuff to do. If, on the other hand, you'd done it the *right* > way, with a yum install, the scripts in the package would have created > the > user and group postfix. > > mark > Webmin as shipped will use Yum on CentOS to handle software management (other than Perl modules). From the way the information is presented by the OP it seems likely that the default method is used in this instance. The error is this: >> postfix: fatal: file /etc/postfix/main.cf: parameter mail_owner: >> unknown user name value: postfix You are missing the postfix user id in /etc/passwd. If that user id is missing then it is likely that your install procedure did not complete properly. I would suggest using yum remove post followed by yum install postfix issued as the root user from the command line. -- *** E-Mail is NOT a SECURE channel *** James B. Byrne mailto:ByrneJB@Harte-Lyne.ca Harte & Lyne Limited http://www.harte-lyne.ca 9 Brockley Drive vox: +1 905 561 1241 Hamilton, Ontario fax: +1 905 561 0757 Canada L8E 3C3

1 1

Jump to page:

2026

2025

2024

2023

2022

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

Discuss