Optional Rate Limiting

To better support server scale testing, an optional
rate limiting mode can be enabled and compiled in
(see README for details). This mode takes advantage
of the existing bandwidth management option
'-B mbps' by having the server limit the maximum
sending rate the rate adjustment algorithm can use.
As such, this mode only needs to be enabled (and
the software recompiled) on the server to take
effect. Otherwise, the bandwidth management option
continues to only be used for test admission control.

CHANGELOG.MD

@@ -13,6 +13,7 @@ the client and one or more server instances (i.e., distributed servers). Additio
 * Randomizing the start of load PDU generation (to better support multiple connections)
 * Enhancing socket receive processing to provide load balancing of epoll events (to better support multiple connections)
 * Adding GSO (Generic Segmentation Offload) and RecvMMsg()+Truncation performance optimizations
+* Adding optional rate limiting for server scale testing
 
 *With these changes, new fields were required in the setup and load PDUs. As a result,
 both the current and minimum protocol version needed to be bumped to 10.*

CMakeLists.txt

@@ -77,6 +77,7 @@ OPTION(DISABLE_INT_TIMER "Disable the timer for systems which don't provide the
 OPTION(HAVE_SENDMMSG "Enable/Disable use of SendMMsg()" ON)
 OPTION(HAVE_RECVMMSG "Enable/Disable use of RecvMMsg()" ON)
 OPTION(HAVE_GSO "Enable/Disable use of Generic Segmentation Offload (GSO)" ON)
+OPTION(RATE_LIMITING "Enable/Disable rate limiting via bandwidth management" OFF)
 
 add_definitions(-DSYSCONFDIR=\"${CMAKE_INSTALL_PREFIX}/etc\")
 add_definitions(-DLOCALSTATEDIR=\"${CMAKE_INSTALL_PREFIX}/var/lib\")

README.md

@@ -432,6 +432,27 @@ will proceed normally.
 algorithm in any way. It only provides test admission control to better manage
 the server's network bandwidth.*
 
+**Rate Limiting (Optional)**
+
+To assist with server scale testing, an optional mode is available where each
+test is limited to the bandwidth requested via the bandwidth management option
+`-B mbps`. This allows tests to ramp-up normally, but limits the maximum
+sending rate index possible with the rate adjustment algorithm. This simulates
+a test limited by a client's "provisioned" speed, even though it may be
+connected to the server(s) at a much higher speed. And because the rate 
+adjustment algorithm only executes on the server, this functionality only needs
+to be enabled on the server to take effect (where a notification message is
+generated for each test that is rate limited). To enable this mode of operation
+on the server(s): 
+```
+$ cmake -D RATE_LIMITING=ON .
+```
+
+*Note: Sending rates above the high-speed threshold (1 Gbps) are much less
+granular than sending rates below it. Also, aggregate rates may end up slightly
+below requested rates due to the traffic pattern and enforced limit of the
+maximum sending rate a connection is limited to.*
+
 ## Increasing the Starting Sending Rate (Considerations)
 While the `-I index` option designates a fixed sending rate, it is also possible
 to set the starting rate with load adjustment enabled. Option `-I @index` allows

config.h.cmake

@@ -17,5 +17,6 @@
 #cmakedefine HAVE_GSO
 #cmakedefine HAVE_RECVMMSG
 #cmakedefine DISABLE_INT_TIMER
+#cmakedefine RATE_LIMITING
 
 #endif /* CONFIG_H */

udpst.c

@@ -63,6 +63,7 @@
  *                                       clock updates based on rx packets
  * Len Ciavattone          03/04/2023    Load balance returned epoll events
  * Len Ciavattone          03/22/2023    Add optimization output to banner
+ * Len Ciavattone          04/04/2023    Add optional rate limiting to banner
  *
  */
 
@@ -240,8 +241,12 @@ int main(int argc, char **argv) {
                         var += sprintf(&scratch[var], ", Protocol Ver: %d-%d", PROTOCOL_MIN, PROTOCOL_VER);
                 else
                         var += sprintf(&scratch[var], ", Protocol Ver: %d", PROTOCOL_VER); // Client is always the latest
-                var += sprintf(&scratch[var], ", Built: " __DATE__ " " __TIME__ "\n");
-                var = write(outputfd, scratch, var);
+                var += sprintf(&scratch[var], ", Built: " __DATE__ " " __TIME__);
+#ifdef RATE_LIMITING
+                var += sprintf(&scratch[var], ", Rate Limiting via '-B mbps'");
+#endif // RATE_LIMITING
+                scratch[var++] = '\n';
+                var            = write(outputfd, scratch, var);
                 //
                 var = 0;
                 if (conf.ipv6Only)

udpst_data.c

@@ -68,6 +68,7 @@
  * Len Ciavattone          01/14/2023    Add multi-connection support
  * Len Ciavattone          03/22/2023    Add GSO and GRO optimizations
  * Len Ciavattone          03/25/2023    GRO replaced w/recvmmsg+truncation
+ * Len Ciavattone          04/04/2023    Add optional rate limiting
  *
  */
 
@@ -1389,6 +1390,44 @@ int adjust_sending_rate(int connindex) {
                         }
                 }
         }
+#ifdef RATE_LIMITING
+        //
+        // Perform rate limiting when bandwidth management is used (-B mbps) by limiting the maximum sending rate index
+        //
+        // NOTE: This is for test purposes only and only needs to be enabled on the server. It is intended for testing
+        // speeds below the actual achievable maximum, generally as part of server scale testing. For example, simulating
+        // low-speed tests when the client and server are actually connected via high speed.
+        //
+        if (c->maxBandwidth > 0) {
+                //
+                // Enforce limit directly when index is equal to bandwidth, else find sending rate index that covers bandwidth
+                //
+                int i = c->maxBandwidth, bw = c->maxBandwidth;
+                if (c->maxBandwidth > 1000) { // If index != bandwidth
+                        struct sendingRate *sr;
+                        for (i = 1001, sr = &repo.sendingRates[i]; i < repo.maxSendingRates; i++, sr++) {
+                                bw = 0; // Simplified bandwidth calculation (random sizes ignored)
+                                if (sr->txInterval1 > 0)
+                                        bw += ((sr->udpPayload1 + L3DG_OVERHEAD) * sr->burstSize1 * 8) / sr->txInterval1;
+                                if (sr->txInterval2 > 0) {
+                                        if (sr->udpPayload2 > 0)
+                                                bw += ((sr->udpPayload2 + L3DG_OVERHEAD) * sr->burstSize2 * 8) / sr->txInterval2;
+                                        if (sr->udpAddon2 > 0)
+                                                bw += ((sr->udpAddon2 + L3DG_OVERHEAD) * 8) / sr->txInterval2;
+                                }
+                                if (bw >= c->maxBandwidth)
+                                        break;
+                        }
+                }
+                if (c->srIndex > i) // Enforce limit
+                        c->srIndex = i;
+                if (conf.verbose && c->spduSeqNo == 1) { // Generate notification once per test
+                        var = sprintf(scratch, "[%d]RATE_LIMITING: Rate adjustment limited to sending rate index %d (%d Mbps)\n",
+                                      connindex, i, bw);
+                        send_proc(errConn, scratch, var);
+                }
+        }
+#endif // RATE_LIMITING
 
         //
         // Output debug messages if configured