<div dir="ltr"><span style="font-size:12.8px">Hi All,</span><br style="font-size:12.8px"><br style="font-size:12.8px"><span style="font-size:12.8px">First time mailing here.</span><div><div style="font-size:12.8px">I have installed on a CentOS 7.0 KVM (with DPDK and OVS) one Deep Packet Inspection VM.</div><div style="font-size:12.8px">I have one channel and some virtual traffic generator.</div><div style="font-size:12.8px">The traffic is lost between dpdk vhostuser and the DPI VM.</div><div style="font-size:12.8px">The setup is attached. Any suggestions or ideas?</div><div style="font-size:12.8px">Regarding the OVS+DPDK configuration, the following configuration is already made:</div><div style="font-size:12.8px">- SELINUX is disabled</div><div style="font-size:12.8px">- QEMU 2.9.0 was downloaded from sources</div><div style="font-size:12.8px">- All Linux packages were updated for QEMU 2.9.0</div><div style="font-size:12.8px">- DPDK source was downloaded</div><div style="font-size:12.8px">- Paths were set</div><div style="font-size:12.8px">- epel repo was installed</div><div style="font-size:12.8px">- DPDK 16.11 was installed</div><div style="font-size:12.8px">- OVS 2.7.0 was installed with DPDK option</div><div style="font-size:12.8px">- Hugepages VM is set</div><div style="font-size:12.8px">- NICs are configured for DPDK</div><div style="font-size:12.8px">- Permissions are set for DPDK</div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Regarding the OVS+DPDK startup steps, the following configuration is in place:</div><div style="font-size:12.8px">B. OVS+DPDK startup and configuration</div><div style="font-size:12.8px"><span style="white-space:pre-wrap"> </span>1. Set and verify the memory hugepages<span style="white-space:pre-wrap">  </span></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">              </span>mount -t hugetlbfs nodev /mnt/huge</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">            </span>echo 64 > /sys/devices/system/node/<wbr>node0/hugepages/hugepages-<wbr>1048576kB/nr_hugepages <span style="white-space:pre-wrap">  </span></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">      </span>2. Setting the driver permissions</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">             </span>modprobe vfio-pci</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">             </span>/usr/bin/chmod a+x /dev/vfio</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">          </span>/usr/bin/chmod 0666 /dev/vfio/*<span style="white-space:pre-wrap">                                         </span></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">      </span>3. Configure the PATHS to DPDK and OVS database </div><div style="font-size:12.8px"><span style="white-space:pre-wrap">             </span>cd dpdk-stable-16.11.1/</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">               </span>export DPDK_DIR=$PWD</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">          </span>export PATH=$PATH:/usr/local/share/<wbr>openvswitch/scripts</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">             </span>export DB_SOCK=/usr/local/var/run/<wbr>openvswitch/db.sock</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">      </span>4. Re-initializing OVS</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">                </span>rm /usr/local/etc/openvswitch/<wbr>conf.db</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">              </span>mkdir -p /usr/local/etc/openvswitch</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">           </span>mkdir -p /usr/local/var/run/openvswitch</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">               </span>ovsdb-tool create /usr/local/etc/openvswitch/<wbr>conf.db                        /usr/local/share/openvswitch/<wbr>vswitch.ovsschema<span style="white-space:pre-wrap">    </span></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">      </span>5. Starting OVS + DPDK</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">                </span>ovsdb-server --remote=punix:/usr/local/var/<wbr>run/openvswitch/db.sock --remote=db:Open_vSwitch,Open_<wbr>vSwitch,manager_options --pidfile --detach</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">             </span>ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">            </span>ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="<wbr>1024,1024"</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">          </span>ovs-ctl --no-ovsdb-server --db-sock="$DB_SOCK" start<span style="white-space:pre-wrap">  </span></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">      </span>6. Cheking driver mapping for NICs</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">            </span>driverctl -v list-devices | grep -i net</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">               </span>$DPDK_DIR/tools/dpdk-devbind.<wbr>py --status</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">   </span>7. Adding OVS+DPDK bridge and ports</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">           </span>ovs-vsctl add-br ch1int_dpdk -- set bridge ch1int_dpdk datapath_type=netdev</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">           </span>ovs-vsctl add-br ch1ext_dpdk -- set bridge ch1ext_dpdk datapath_type=netdev</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">           </span>ovs-vsctl add-port ch1int_dpdk nic810 -- set Interface nic810 type=dpdk options:dpdk-devargs=0000:81:<wbr>00.0</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">          </span>ovs-vsctl add-port ch1ext_dpdk nic811 -- set Interface nic811 type=dpdk options:dpdk-devargs=0000:81:<wbr>00.1<span style="white-space:pre-wrap">    </span></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">      </span>8. Adding OVS (no DPDK) bridge and ports</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">              </span>ovs-vsctl add-br br0</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">          </span>ovs-vsctl add-port br0 myportnameone<span style="white-space:pre-wrap">    </span></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">      </span>9. Adding VM ports to OVS (or OVS+DPDK) bridge<span style="white-space:pre-wrap">  </span></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">              </span>#<interface type='bridge'></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">              </span># <mac address='52:54:00:cb:57:af'/></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">            </span># <source bridge='br0'/></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">                </span># <virtualport type='openvswitch'/></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">             </span># ...</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">         </span>#</interface><span style="white-space:pre-wrap">     </span></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">      </span>10. Adding dpdkvhostuser to the ovs switch - these are used for the socket created between the VM and the OVS bridge</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">          </span>ovs-vsctl add-port ch1int_dpdk ch1int_dvhu -- set Interface ch1int_dvhu type=dpdkvhostuser</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">            </span>ovs-vsctl add-port ch1ext_dpdk ch1ext_dvhu -- set Interface ch1ext_dvhu type=dpdkvhostuser<span style="white-space:pre-wrap">      </span></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">      </span>11. DPI deployment</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">            </span>virt-install --connect qemu:///system --name=DPI --memory=16384 --vcpus=16 --os-type=linux --os-variant=virtio26 --disk path=/path/image.qcow2,format=<wbr>qcow2,bus=virtio,cache=none --network bridge=admin,model=e1000 --nographics --noautoconsole --import<span style="white-space:pre-wrap">   </span></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">      </span>12. Adding vhostuser ports to guest VM - for this the XML file for the virtual machine found at etc/libvirt/qemu/<VM_name>.xml<span style="white-space:pre-wrap">    </span>was edited</div><div style="font-size:12.8px"><span style="white-space:pre-wrap">    </span><interface type='vhostuser'></div><div style="font-size:12.8px">    <span style="white-space:pre-wrap">             </span><mac address='52:54:00:2f:e8:de'/></div><div style="font-size:12.8px">    <span style="white-space:pre-wrap">               </span><source type='unix' path='/usr/local/var/run/<wbr>openvswitch/ch1int_dvhu' mode='client'/></div><div style="font-size:12.8px">   <span style="white-space:pre-wrap">           </span> <model type='virtio'/></div><div style="font-size:12.8px">    <span style="white-space:pre-wrap">         </span><address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">     </span></interface></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">    </span><interface type='vhostuser'></div><div style="font-size:12.8px">  <span style="white-space:pre-wrap">                </span>  <mac address='52:54:00:b9:40:d0'/></div><div style="font-size:12.8px">  <span style="white-space:pre-wrap">              </span>  <source type='unix' path='/usr/local/var/run/<wbr>openvswitch/ch1ext_dvhu' mode='client'/></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">            </span>    <model type='virtio'/></div><div style="font-size:12.8px"> <span style="white-space:pre-wrap">         </span>   <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/></div><div style="font-size:12.8px"><span style="white-space:pre-wrap">        </span></interface></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px"><br></div><div style="font-size:12.8px">Thanks, </div><div style="font-size:12.8px">Daniel</div></div></div>