upcall: Don't start new revalidation round too soon after the last one.

The execution time of 'ovs-ofctl add-flows' with a large number of
flows can be more than halved if revalidators are not running after
each flow mod separately.  This was first suspected when it was found
that 'ovs-ofctl --bundle add-flows' is about 10 times faster than the
same command without the '--bundle' option in a scenario where there
is a large set of flows being added and no datapath flows at all.  One
of the differences caused by the '--bundle' option is that the
revalidators are woken up only once, at the end of the whole set of
flow table changes, rather than after each flow table change
individually.

This patch limits the revalidation to run at most 200 times a second
by enforcing a minimum of 5ms time gap between the start times of
revalidation rounds.  If nothing happens in, say 6 milliseconds, and
then a new flow table change is signaled, the revalidator threads wake
up immediately without any further delay.  Values smaller than 5 were
found to increase the 'ovs-ofctl add-flows' execution time noticeably.

Since the revalidators are not running after each flow mod, the
overall OVS CPU utilization during the 'ovs-ofctl add-flows' run time
is reduced roughly by one core on a four core machine.

In testing the 'ovs-ofctl add-flows' execution time is not
significantly improved from this even if the revalidators are not
notified about the flow table changes at all.

Signed-off-by: Jarno Rajahalme <[email protected]>
Acked-by: Ben Pfaff <[email protected]>

ofproto/ofproto-dpif-upcall.c

@@ -939,6 +939,21 @@ udpif_revalidator(void *arg)
             latch_wait(&udpif->exit_latch);
             latch_wait(&udpif->pause_latch);
             poll_block();
+
+            if (!latch_is_set(&udpif->pause_latch) &&
+                !latch_is_set(&udpif->exit_latch)) {
+                long long int now = time_msec();
+                /* Block again if we are woken up within 5ms of the last start
+                 * time. */
+                start_time += 5;
+
+                if (now < start_time) {
+                    poll_timer_wait_until(start_time);
+                    latch_wait(&udpif->exit_latch);
+                    latch_wait(&udpif->pause_latch);
+                    poll_block();
+                }
+            }
         }
     }
 

tests/ofproto-dpif.at

@@ -4609,6 +4609,9 @@ m4_define([CHECK_CONTINUATION], [dnl
         m4_if([$3], [0], [],
             [AT_CHECK([echo "$actions1" | sed 's/pause/controller(pause)/g' | ovs-ofctl -O OpenFlow13 add-flows br1 -])])
 
+        # Make sure the datapath is up-to-date before sending the packet.
+        ovs-appctl revalidator/wait
+
         # Run a packet through the switch.
         AT_CHECK([ovs-appctl netdev-dummy/receive p1 "$flow"], [0], [stdout])