dpif-netdev: Fix PMD auto load balance with pmd-rxq-isolate.

There are currently some checks for cross-numa polling cases to
ensure that they won't effect the accuracy of the PMD ALB.

If an rxq is pinned to a PMD thread core by the user it will not
be reassigned by OVS, so even if it is non-local numa polled it
will not impact PMD ALB accuracy.

To establish this, a check was made on whether the PMD thread core
was isolated or not. However, since other_config:pmd-rxq-isolate was
introduced, rxqs may be pinned but the PMD thread core not isolated.

It means that by setting pmd-rxq-isolate=false and doing non-local
numa pinning, PMD ALB may not run where it should.

If the PMD thread core is isolated we can skip individual rxq checks
but if not, we should check the individual rxqs for pinning before we
disallow PMD ALB.

Also, update function comments to make it's operation clearer.

Fixes: 6193e03 ("dpif-netdev: Allow pin rxq and non-isolate PMD.")
Signed-off-by: Kevin Traynor <[email protected]>
Acked-by: Mike Pattrick <[email protected]>
Acked-by: David Marchand <[email protected]>
Signed-off-by: Ilya Maximets <[email protected]>

lib/dpif-netdev.c

@@ -5683,23 +5683,28 @@ sched_numa_list_put_in_place(struct sched_numa_list *numa_list)
     }
 }
 
+/* Returns 'true' if OVS rxq scheduling algorithm assigned any unpinned rxq to
+ * a PMD thread core on a non-local numa node. */
 static bool
 sched_numa_list_cross_numa_polling(struct sched_numa_list *numa_list)
 {
     struct sched_numa *numa;
 
-    /* For each numa */
     HMAP_FOR_EACH (numa, node, &numa_list->numas) {
-        /* For each pmd */
         for (int i = 0; i < numa->n_pmds; i++) {
             struct sched_pmd *sched_pmd;
 
             sched_pmd = &numa->pmds[i];
-            /* For each rxq. */
+            if (sched_pmd->isolated) {
+                /* All rxqs on this PMD thread core are pinned. */
+                continue;
+            }
             for (unsigned k = 0; k < sched_pmd->n_rxq; k++) {
                 struct dp_netdev_rxq *rxq = sched_pmd->rxqs[k];
-
-                if (!sched_pmd->isolated &&
+                /* Check if the rxq is not pinned to a specific PMD thread core
+                 * by the user AND the PMD thread core that OVS assigned is
+                 * non-local to the rxq port. */
+                if (rxq->core_id == OVS_CORE_UNSPEC &&
                     rxq->pmd->numa_id !=
                         netdev_get_numa_id(rxq->port->netdev)) {
                     return true;