[CentOS-devel] eventfd to select problem between Intel and AMD

Wed Feb 12 01:57:25 UTC 2014
lulin <f_ella at yeah.net>

Hi 
I met a strange problem, I wrote a simple program that create some eventfds and add it to select loop, create a thread to write some bytes to a random eventfd, when i run 100 instances on a Intel E5-2650, mpstat shows that we have 95% idle of CPU, but when i run same 100 instances on a AMD Opteron(TM) Processor 6272, mpstat shows 50% on sys, 50% idle. Kernel version is Linux 2.6.32-279.el6.


The attachment is the source code of my test program.


Can someone  give any suggestion?? Thanks, :)


Bellow is the environment:
=============Intel E5-2650=============================
processor: 31
vendor_id: GenuineIntel
cpu family: 6
model: 45
model name: Intel(R) Xeon(R) CPU E5-2650 0 @ 2.00GHz
stepping: 7
cpu MHz: 2000.149
cache size: 20480 KB
physical id: 1
siblings: 16
core id: 7
cpu cores: 8
apicid: 47
initial apicid: 47
fpu: yes
fpu_exception: yes
cpuid level: 13
wp: yes
flags: fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx pdpe1gb rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology nonstop_tsc aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm dca sse4_1 sse4_2 x2apic popcnt aes xsave avx lahf_lm ida arat epb xsaveopt pln pts dts tpr_shadow vnmi flexpriority ept vpid
bogomips: 3999.44
clflush size: 64
cache_alignment: 64
address sizes: 46 bits physical, 48 bits virtual
power management:




[root at OVP ~]# mpstat 2
Linux 2.6.32-279.el6.x86_64 (OVP.com) 02/11/2014 _x86_64_(32 CPU)


08:57:40 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest   %idle
08:57:42 PM  all    0.24    0.00    0.24    0.00    0.00    0.00    0.00    0.02   99.50
08:57:44 PM  all    0.00    0.00    0.19    0.00    0.00    0.00    0.00    0.10   99.71
08:57:46 PM  all    0.19    0.00    0.29    0.00    0.00    0.00    0.00    0.02   99.50
08:57:48 PM  all    0.34    0.00    0.29    0.00    0.00    0.02    0.00    0.32   99.02




==================AMD Opteron(TM) Processor 6272=========================
processor       : 63
vendor_id       : AuthenticAMD
cpu family      : 21
model           : 1
model name      : AMD Opteron(TM) Processor 6272
stepping        : 2
cpu MHz         : 2100.089
cache size      : 2048 KB
physical id     : 3
siblings        : 16
core id         : 7
cpu cores       : 8
apicid          : 111
initial apicid  : 111
fpu             : yes
fpu_exception   : yes
cpuid level     : 13
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic mtrr pge mca cmov pat pse36 clflush mmx
 fxsr sse sse2 ht syscall nx mmxext fxsr_opt pdpe1gb rdtscp lm constant_tsc rep_good nonstop_tsc
extd_apicid amd_dcm aperfmperf pni pclmulqdq monitor ssse3 cx16 sse4_1 sse4_2 popcnt aes xsave av
x lahf_lm cmp_legacy svm extapic cr8_legacy abm sse4a misalignsse 3dnowprefetch osvw ibs xop skin
it wdt lwp fma4 nodeid_msr topoext perfctr_core cpb npt lbrv svm_lock nrip_save tsc_scale vmcb_cl
ean flushbyasid decodeassists pausefilter pfthreshold
bogomips        : 4199.79
TLB size        : 1536 4K pages
clflush size    : 64
cache_alignment : 64
address sizes   : 48 bits physical, 48 bits virtual
power management: ts ttp tm 100mhzsteps hwpstate cpb
amd TM 6272


09:43:11 PM  CPU    %usr   %nice    %sys %iowait    %irq   %soft  %steal  %guest   %idle
09:43:12 AM  all    2.42    0.00   53.37    0.00    0.00    0.03    0.00    0.43   43.74
09:43:13 AM  all    2.91    0.00   55.37    0.00    0.00    0.02    0.00    0.14   41.55
09:43:14 AM  all    2.34    0.00   55.60    0.00    0.00    0.02    0.00    0.19   41.85
09:43:15 AM  all    3.81    0.00   55.32    0.00    0.00    0.02    0.00    0.16   40.69
09:43:16 AM  all    2.63    0.00   54.53    0.00    0.00    0.02    0.00    0.13   42.70
09:43:17 AM  all    2.47    0.00   55.60    0.00    0.00    0.03    0.00    0.18   41.72
09:43:18 AM  all    2.50    0.00   55.64    0.00    0.00    0.02    0.00    0.18   41.67
09:43:19 AM  all    3.34    0.00   54.61    0.00    0.00    0.02    0.00    0.22   41.81
09:43:20 AM  all    2.39    0.00   55.21    0.00    0.00    0.02    0.00    0.18   42.20
09:43:21 AM  all    2.83    0.00   53.78    0.00    0.00    0.02    0.00    0.34   43.03
09:43:22 AM  all    2.63    0.00   53.96    0.00    0.00    0.02    0.00    0.24   43.15


====Perf on AMD=====:
-  25.64%  [kernel]                    [k] fget_light                                               
   - fget_light                                                                                  
      - 96.65% do_select                                                                        
           core_sys_select                                                                     
           sys_select                                                                          
           system_call_fastpath                                                                
         - __select                                                                           
              99.98% __libc_start_main                                                                                                         
      - 2.39% core_sys_select                                                                 
           sys_select                                                                          
           system_call_fastpath                                                                
         - __select                                                                            
              100.00% __libc_start_main                                                        
-  15.88%  [kernel]                    [k] _spin_lock_irqsave                                  
   - _spin_lock_irqsave                                                                        
      - 46.50% eventfd_poll                                                                    
           do_select                                                                           
           core_sys_select                                                                     
           sys_select                                                                          
           system_call_fastpath                                                               
         - __select                                                                           
              99.97% __libc_start_main                                                        
      + 26.00% add_wait_queue                                                                 
      + 21.97% remove_wait_queue                                                              
      + 2.64% do_select                                                                      
      + 1.32% __pollwait                                                                     
      + 1.10% free_poll_entry                                                                
+   9.89%  [kernel]                    [k] _spin_unlock_irqrestore                           
+   8.09%  [kernel]                    [k] fput                                               
+   4.79%  [kernel]                    [k] __pollwait                                        
+   3.64%  [kernel]                    [k] do_select                                         
+   2.78%  [kernel]                    [k] eventfd_poll                                       
+   2.16%  [kernel]                    [k] list_del                                           
+   2.13%  efd_test2                   [.] main                                              
+   1.87%  [kernel]                    [k] __list_add
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.centos.org/pipermail/centos-devel/attachments/20140212/8a30056c/attachment-0004.html>
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: eventfd_select.c
URL: <http://lists.centos.org/pipermail/centos-devel/attachments/20140212/8a30056c/attachment-0004.c>