Skip to content

Commit

Permalink
INSTALL.DPDK.md: Provide a little a more consistency to documentation.
Browse files Browse the repository at this point in the history
A few users (based on the reports in [email protected]) have been
literally following the instructions in INSTALL.DPDK.md and mixing up
pre-installed utilities and daemons with freshly compiled utilities
because the current documentation does not consistently call out
using utilities from the compiled sources.

This commit updates DPDK documentation to ask users to do a 'make install'
and then use the utilities and daemons directly from Linux PATH.
It also adds github markup where applicable.

Reported-by: Arkajit Ghosh <[email protected]>
Signed-off-by: Gurucharan Shetty <[email protected]>
Acked-by: Pravin B Shelar <[email protected]>
  • Loading branch information
shettyg committed Jan 26, 2015
1 parent 47ff8ab commit a52b049
Show file tree
Hide file tree
Showing 2 changed files with 51 additions and 42 deletions.
1 change: 1 addition & 0 deletions AUTHORS
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,7 @@ Anton Matsiuk [email protected]
Anup Khadka [email protected]
Anuprem Chalvadi [email protected]
Ariel Tubaltsev [email protected]
Arkajit Ghosh [email protected]
Atzm Watanabe [email protected]
Bastian Blank [email protected]
Ben Basler [email protected]
Expand Down
92 changes: 50 additions & 42 deletions INSTALL.DPDK.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,35 +91,33 @@ Using the DPDK with ovs-vswitchd:
Ref to http://www.dpdk.org/doc/quick-start for verifying DPDK setup.
4. Start ovsdb-server as discussed in [INSTALL.md] doc:
4. Follow the instructions in [INSTALL.md] to install only the
userspace daemons and utilities (via 'make install').
1. First time only db creation (or clearing):
```
mkdir -p /usr/local/etc/openvswitch
mkdir -p /usr/local/var/run/openvswitch
rm /usr/local/etc/openvswitch/conf.db
cd $OVS_DIR
./ovsdb/ovsdb-tool create /usr/local/etc/openvswitch/conf.db \
./vswitchd/vswitch.ovsschema
```
```
mkdir -p /usr/local/etc/openvswitch
mkdir -p /usr/local/var/run/openvswitch
rm /usr/local/etc/openvswitch/conf.db
ovsdb-tool create /usr/local/etc/openvswitch/conf.db \
/usr/local/share/openvswitch/vswitch.ovsschema
```
2. start ovsdb-server
2. Start ovsdb-server
```
cd $OVS_DIR
./ovsdb/ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
```
ovsdb-server --remote=punix:/usr/local/var/run/openvswitch/db.sock \
--remote=db:Open_vSwitch,Open_vSwitch,manager_options \
--private-key=db:Open_vSwitch,SSL,private_key \
--certificate=Open_vSwitch,SSL,certificate \
--bootstrap-ca-cert=db:Open_vSwitch,SSL,ca_cert --pidfile --detach
```
```
3. First time after db creation, initialize:
```
cd $OVS_DIR
./utilities/ovs-vsctl --no-wait init
```
```
ovs-vsctl --no-wait init
```
5. Start vswitchd:
Expand All @@ -128,32 +126,38 @@ Using the DPDK with ovs-vswitchd:
dpdk arg -c is ignored by ovs-dpdk, but it is a required parameter
for dpdk initialization.
export DB_SOCK=/usr/local/var/run/openvswitch/db.sock
./vswitchd/ovs-vswitchd --dpdk -c 0x1 -n 4 -- unix:$DB_SOCK --pidfile --detach
```
export DB_SOCK=/usr/local/var/run/openvswitch/db.sock
ovs-vswitchd --dpdk -c 0x1 -n 4 -- unix:$DB_SOCK --pidfile --detach
```
If allocated more than one GB hugepage (as for IVSHMEM), set amount and use NUMA
node 0 memory:
If allocated more than one GB hugepage (as for IVSHMEM), set amount and
use NUMA node 0 memory:
./vswitchd/ovs-vswitchd --dpdk -c 0x1 -n 4 --socket-mem 1024,0 \
-- unix:$DB_SOCK --pidfile --detach
```
ovs-vswitchd --dpdk -c 0x1 -n 4 --socket-mem 1024,0 \
-- unix:$DB_SOCK --pidfile --detach
```
6. Add bridge & ports
To use ovs-vswitchd with DPDK, create a bridge with datapath_type
"netdev" in the configuration database. For example:
`ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev`
`ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev`
Now you can add dpdk devices. OVS expect DPDK device name start with dpdk
and end with portid. vswitchd should print (in the log file) the number of dpdk
devices found.
and end with portid. vswitchd should print (in the log file) the number
of dpdk devices found.
ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk
```
ovs-vsctl add-port br0 dpdk0 -- set Interface dpdk0 type=dpdk
ovs-vsctl add-port br0 dpdk1 -- set Interface dpdk1 type=dpdk
```
Once first DPDK port is added to vswitchd, it creates a Polling thread and
polls dpdk device in continuous loop. Therefore CPU utilization
for that thread is always 100%.
Once first DPDK port is added to vswitchd, it creates a Polling thread and
polls dpdk device in continuous loop. Therefore CPU utilization
for that thread is always 100%.
7. Add test flows
Expand All @@ -180,14 +184,14 @@ Using the DPDK with ovs-vswitchd:
interfaces on the same numa node. The following two commands can be used
to configure the multi-threading behavior.
ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=<hex string>
`ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=<hex string>`
The command above asks for a CPU mask for setting the affinity of pmd threads.
A set bit in the mask means a pmd thread is created and pinned to the
corresponding CPU core. For more information, please refer to
The command above asks for a CPU mask for setting the affinity of pmd
threads. A set bit in the mask means a pmd thread is created and pinned
to the corresponding CPU core. For more information, please refer to
`man ovs-vswitchd.conf.db`
ovs-vsctl set Open_vSwitch . other_config:n-dpdk-rxqs=<integer>
`ovs-vsctl set Open_vSwitch . other_config:n-dpdk-rxqs=<integer>`
The command above sets the number of rx queues of each DPDK interface. The
rx queues are assigned to pmd threads on the same numa node in round-robin
Expand All @@ -208,12 +212,14 @@ Using the DPDK with ovs-vswitchd:
Configure pmd threads on core 4,6,8,10 using 'pmd-cpu-mask':
ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=00000550
`ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=00000550`
You should be able to check that pmd threads are pinned to the correct cores
via:
top -p `pidof ovs-vswitchd` -H -d1
```
top -p `pidof ovs-vswitchd` -H -d1
```
Note, the pmd threads on a numa node are only created if there is at least
one DPDK interface from the numa node that has been added to OVS.
Expand All @@ -228,7 +234,7 @@ Following the steps above to create a bridge, you can now add dpdk rings
as a port to the vswitch. OVS will expect the DPDK ring device name to
start with dpdkr and end with a portid.
ovs-vsctl add-port br0 dpdkr0 -- set Interface dpdkr0 type=dpdkr
`ovs-vsctl add-port br0 dpdkr0 -- set Interface dpdkr0 type=dpdkr`
DPDK rings client test application
Expand All @@ -240,8 +246,10 @@ location tests/ovs_client
To run the client :
cd /usr/src/ovs/tests/
ovsclient -c 1 -n 4 --proc-type=secondary -- -n "port id you gave dpdkr"
```
cd /usr/src/ovs/tests/
ovsclient -c 1 -n 4 --proc-type=secondary -- -n "port id you gave dpdkr"
```
In the case of the dpdkr example above the "port id you gave dpdkr" is 0.
Expand Down

0 comments on commit a52b049

Please sign in to comment.