Enabled SW Fallback support for FBSD

- SW fallback support for asymmetric algorithms - SW fallback support for symmetric algorithms - SW fallback support for kdf algorithms - Version bump and Readme update Signed-off-by: Premkumar Shanmugam <[email protected]> Co-authored-by: Nagha Abirami <[email protected]> Co-authored-by: Tirupatigopi Ravulapalli <[email protected]>
intel · Nov 5, 2024 · a4a741b · a4a741b
1 parent eae5f3f
commit a4a741b
Show file tree

Hide file tree

Showing 58 changed files with 2,282 additions and 1,474 deletions.
diff --git a/configure.ac b/configure.ac
@@ -2,7 +2,7 @@
 # Process this file with autoconf to produce a configure script.
 
 AC_PREREQ([2.68])
-AC_INIT([qatengine], [1.6.2], [])
+AC_INIT([qatengine], [1.7.0], [])
 AC_CONFIG_SRCDIR([config.h.in])
 AC_CONFIG_HEADERS([config.h])
 AC_CONFIG_AUX_DIR([.])
@@ -329,9 +329,16 @@ then
     AC_SUBST([with_qat_hw_dir], ["${qat_hw_dir_prefix}"])
     AC_MSG_NOTICE([QAT in-tree driver using QATlib in ${with_qat_hw_dir}])
   else
-    AC_SUBST([cflags_qat_hw_intree], ["-DQAT_HW_FBSD_INTREE"])
-    AC_SUBST([with_qat_hw_dir], ["${qat_hw_dir_prefix}"])
-    AC_MSG_NOTICE([QAT in-tree driver using FBSD QATlib in ${with_qat_hw_dir}])
+    if grep "define SAL_INFO2_DRIVER_SW_VERSION_MAJ_NUMBER 3" ${qat_hw_dir_prefix}/include/qat/icp_sal_versions.h
+    then
+      AC_SUBST([cflags_qat_hw_intree], ["-DQAT_HW_FBSD_OOT"])
+      AC_SUBST([with_qat_hw_dir], ["${qat_hw_dir_prefix}"])
+      AC_MSG_NOTICE([QAT FBSD OOT driver using libs from ${with_qat_hw_dir}])
+    else
+      AC_SUBST([cflags_qat_hw_intree], ["-DQAT_HW_FBSD_INTREE"])
+      AC_SUBST([with_qat_hw_dir], ["${qat_hw_dir_prefix}"])
+      AC_MSG_NOTICE([QAT in-tree driver using FBSD QATlib in ${with_qat_hw_dir}])
+    fi
   fi
 fi
 

diff --git a/docs/qat_hw.md b/docs/qat_hw.md
@@ -20,13 +20,15 @@ Hardware accelerator ignoring the small packet offload threshold.  Please refer
 to the OpenSSL\* manual for more information about pipelining.
 <https://www.openssl.org/docs/man1.1.0/ssl/SSL_CTX_set_split_send_fragment.html>
 
-# Intel&reg; QAT OpenSSL\* Engine Software Fallback Feature
+# Intel&reg; QAT OpenSSL\* Engine Software Fallback
 
 The Intel&reg; QuickAssist Heartbeat feature provides a mechanism for the
 Intel&reg; QAT OpenSSL\* Engine to detect unresponsive acceleration devices and
 to be notified of the start and end of any reset of the acceleration devices.
 The Heartbeat feature suspends all QAT instances associated with that
 acceleration device between these two reset-start and reset-end events.
+Application using QAT needs to have the utility/daemon that periodically checks
+device which is needed as part of heartbeat functionality.
 An acceleration device can be configured for automatic reset by the QAT
 framework upon heartbeat failure by using the `AutomaticResetOnError = 1` field
 in the `[GENERAL]` section of device configuration file `/etc/<device>.conf`.
@@ -36,7 +38,7 @@ field to be set.
 The Intel&reg; QAT OpenSSL\* Engine's software fallback feature, when enabled
 by the user, essentially provides continuity of crypto operations for the
 application between the two above-mentioned reset-start & reset-end events.
-It does this by exhibiting the following behaviour:
+It does this by exhibiting the following behavior:
 
 * Any requests that have already been submitted to the acceleration device that
 goes down but have not completed will be handled as on core requests and will
@@ -52,28 +54,52 @@ a potential slow down in performance until the acceleration device comes back on
 
 ### Requirements:
  1. This Intel&reg; QAT OpenSSL\* Engine supports the Intel&reg; QAT Driver
-Heartbeat feature starting with QAT driver version 4.6 of the following device:
+Heartbeat feature on the following QAT Enabled Devices or Platforms using
+the driver Linux Hardware v1.7, v1.8, v2.0(qatlib intree) and FreeBSD Hardware
+v2.0(OOT package)
+
     * [Intel&reg; Xeon&reg; with Intel&reg; C62X Series Chipset][1]
+    * [Intel® Xeon® Scalable Processor family with Intel® QAT Gen4/Gen4m][2]
 
 [1]:https://www.intel.com/content/www/us/en/design/products-and-solutions/processors-and-chipsets/purley/intel-xeon-scalable-processors.html
+[2]:https://www.intel.com/content/www/us/en/products/docs/processors/xeon-accelerated/4th-gen-xeon-scalable-processors.html
+
+ 2. Asymmetric PKE, Key Derivation algorithms supported by QAT Engine/provider and
+Symmetric algorithms like AES-GCM, AES-CCM, CHACHAPOLY, AES-CBC chained ciphers
+supports software fallback on device failure. Other Hash and SMx algorithms doesnt
+support software Fallback.
+
+ 3. Software fallback feature for symmetric algorithms are only supported in FreeBSD platform.
+
+### Testing the Software Fallback using OpenSSL Speed application
+
+Pre-Requisites:  Build QAT Driver using `./configure --enable-icp-hb-fail-sim` flag
+and change the driver config files `/etc/<device>.conf` to have
+`AutoResetOnError = 1` as mentioned above. Driver needs to be built with same
+version of OpenSSL as the application, to meet OpenSSL's requirement of version compatibility
+between OpenSSL 1.1.1 and 3.x. Set the below envs to compile Driver against same version of OpenSSL if needed.
 
- 2. Intel&reg; QAT OpenSSL\* Engine needs to be configured to disable Symmetric
-Chained Ciphers, PRF, HKDF & X25519/X448 acceleration by adding the below four
-flags in the configure command of Intel&reg; QAT OpenSSL\* Engine build.
+   * `setenv LDFLAGS "-L<installed OpenSSL path>/lib"`
+   * `setenv CPPFLAGS "-I<installed OpenSSL path>/include"`
 
-    ```bash
-    --disable-qat_hw_ciphers --disable-qat_hw_prf --disable-qat_hw_hkdf --disable-qat_hw_ecx
-    ```
-3. The Heartbeat feature is not supported in the FreeBSD operating system or in the
-qatlib RPM.
+ 1. Manually set the `enable_sw_fallback = 1` in the e_qat.c otherwise this needs to
+ be turned on with Engine Ctrl messages at runtime from the application.
+ 2. Build QAT Engine as per the steps mentioned in the Example build in the Installation Instructions section.
+ 3. Run OpenSSL speed application with qatengine from <path_to_OpenSSL_install>/bin
+    e.g., `./openssl speed -engine qatengine -elapsed rsa2048`
+ 4. Inject device failure using the command.
+       `sysctl dev.qat.0.heartbeat_sim_fail=1`
+ 5. Check device status(0 - device failure, 1 - device active).
+       `sysctl dev.qat.0.heartbeat`
+ 6. On successful software fallback, there will not be any errors reported from OpenSSL speed test.
 
 ### Additional Information
-Additional information on this Heartbeat feature can be found in:
-Intel&reg; QAT Software for Linux\* - Programmer's Guide: Hardware
-v1.x CE  Release - Section 3.17 Heartbeat.
+Additional information on this Heartbeat feature can be found at the Heartbeat Section
+of respective QAT Hardware programmer's Guide below
 
-This document can be found on the 01.org website at the following hyperlink:
-* [Intel&reg; QuickAssist Technology Programmer's Guide][2]
+* [Intel QAT Software for Linux—Programmer's Guide: Hardware v1.x CE Release][3]
+* [Intel QAT Software for Linux—Programmer's Guide: Hardware v2.0][4]
 
-[2]:https://01.org/sites/default/files/downloads/336210qatswpg-013.pdf
+[3]:https://cdrdv2.intel.com/v1/dl/getContent/710060
+[4]:https://cdrdv2.intel.com/v1/dl/getContent/743912
 
diff --git a/e_qat.c b/e_qat.c
@@ -55,8 +55,12 @@
 /* Defines */
 #ifdef QAT_HW
 # ifdef QAT_HW_INTREE
-#  define ENABLE_QAT_HW_SHA3
-#  define ENABLE_QAT_HW_CHACHAPOLY
+#  ifndef ENABLE_QAT_HW_SHA3
+#   define ENABLE_QAT_HW_SHA3
+#  endif
+#  ifndef ENABLE_QAT_HW_CHACHAPOLY
+#   define ENABLE_QAT_HW_CHACHAPOLY
+#  endif
 # endif
 #endif
 
@@ -164,13 +168,13 @@ int qat_fips_kat_test;
 const char *engine_qat_id = STR(QAT_ENGINE_ID);
 #if defined(QAT_HW) && defined(QAT_SW)
 const char *engine_qat_name =
-    "Reference implementation of QAT crypto engine(qat_hw & qat_sw) v1.6.2";
+    "Reference implementation of QAT crypto engine(qat_hw & qat_sw) v1.7.0";
 #elif QAT_HW
 const char *engine_qat_name =
-    "Reference implementation of QAT crypto engine(qat_hw) v1.6.2";
+    "Reference implementation of QAT crypto engine(qat_hw) v1.7.0";
 #else
 const char *engine_qat_name =
-    "Reference implementation of QAT crypto engine(qat_sw) v1.6.2";
+    "Reference implementation of QAT crypto engine(qat_sw) v1.7.0";
 #endif
 unsigned int engine_inited = 0;
 int fallback_to_openssl = 0;
@@ -340,6 +344,12 @@ __thread unsigned long long num_ecx_sw_derive_reqs = 0;
 __thread unsigned long long num_sm4_cbc_hw_cipher_reqs = 0;
 __thread unsigned long long num_sm4_cbc_sw_cipher_reqs = 0;
 
+#ifndef __FreeBSD__
+clock_t clock_id = CLOCK_MONOTONIC_RAW;
+#else
+clock_t clock_id = CLOCK_MONOTONIC_PRECISE;
+#endif
+
 #ifndef QAT_BORINGSSL
 const ENGINE_CMD_DEFN qat_cmd_defns[] = {
     {
@@ -434,7 +444,6 @@ const ENGINE_CMD_DEFN qat_cmd_defns[] = {
         "SET_CONFIGURATION_SECTION_NAME",
         "Set the configuration section to use in QAT driver configuration file",
         ENGINE_CMD_FLAG_STRING},
-# ifndef __FreeBSD__
     {
         QAT_CMD_ENABLE_SW_FALLBACK,
         "ENABLE_SW_FALLBACK",
@@ -445,7 +454,6 @@ const ENGINE_CMD_DEFN qat_cmd_defns[] = {
         "HEARTBEAT_POLL",
         "Check the acceleration devices are still functioning",
         ENGINE_CMD_FLAG_NO_INPUT},
-# endif
     {
         QAT_CMD_DISABLE_QAT_OFFLOAD,
         "DISABLE_QAT_OFFLOAD",
@@ -970,20 +978,14 @@ int qat_engine_ctrl(ENGINE *e, int cmd, long i, void *p, void (*f) (void))
         }
         break;
         case QAT_CMD_ENABLE_SW_FALLBACK:
-# if !defined(__FreeBSD__) && !defined(QAT_HW_INTREE)
         DEBUG("Enabled SW Fallback\n");
         BREAK_IF(engine_inited, \
                 "ENABLE_SW_FALLBACK failed as the engine is already initialized\n");
         enable_sw_fallback = 1;
         CRYPTO_QAT_LOG("SW Fallback enabled - %s\n", __func__);
-# else
-        WARN("QAT_CMD_ENABLE_SW_FALLBACK is not supported\n");
-        retVal = 0;
-# endif
         break;
 
         case QAT_CMD_HEARTBEAT_POLL:
-# if !defined(__FreeBSD__) && !defined(QAT_HW_INTREE)
         BREAK_IF(!engine_inited, "HEARTBEAT_POLL failed as engine is not initialized\n");
         BREAK_IF(!enable_external_polling,
                 "HEARTBEAT_POLL failed as external polling is not enabled\n");
@@ -996,10 +998,6 @@ int qat_engine_ctrl(ENGINE *e, int cmd, long i, void *p, void (*f) (void))
             WARN("HEARTBEAT_POLL failed as no instances are available\n");
             retVal = 0;
         }
-# else
-        WARN("QAT_CMD_HEARTBEAT_POLL is not supported\n");
-        retVal = 0;
-# endif
         break;
 
         case QAT_CMD_DISABLE_QAT_OFFLOAD:
@@ -1113,7 +1111,7 @@ int bind_qat(ENGINE *e, const char *id)
    int ret = 0;
 #ifdef QAT_HW
     char *config_section = NULL;
-# if defined(QAT20_OOT) || defined(__FreeBSD__)
+# if !defined(QAT_HW_INTREE) && (defined(QAT20_OOT) || defined(__FreeBSD__)) 
     Cpa32U dev_count = 0;
 # endif
 #endif

diff --git a/e_qat.h b/e_qat.h
@@ -85,7 +85,7 @@
 #  include "qat_sw_queue.h"
 #  include "qat_sw_freelist.h"
 # endif
-# include "qat_common.h"
+#include "qat_common.h"
 
 # ifndef ERR_R_RETRY
 #  define ERR_R_RETRY 57
@@ -128,6 +128,7 @@
 # define QAT_ATOMIC_DEC(qat_int)              \
             (__sync_sub_and_fetch(&(qat_int), 1))
 
+extern clock_t clock_id;
 # ifdef QAT_HW
 typedef struct {
     int qatAsymInstanceNumForThread;
@@ -931,7 +932,8 @@ int qat_sw_cpu_support(void);
 # endif
 
 # ifdef QAT_OPENSSL_PROVIDER
-typedef _Atomic int CRYPTO_REF_COUNT;
+# if OPENSSL_VERSION_NUMBER < 0x30200000
+
 static __inline__ int CRYPTO_UP_REF(int *val, int *ret, ossl_unused void *lock)
 {
     *ret = __atomic_fetch_add(val, 1, __ATOMIC_RELAXED) + 1;
@@ -946,22 +948,23 @@ static __inline__ int CRYPTO_DOWN_REF(int *val, int *ret,
 	__atomic_thread_fence(__ATOMIC_ACQUIRE);
     return 1;
 }
-# endif
 
-# if OPENSSL_VERSION_NUMBER >= 0x30200000
-static __inline__ int QAT_CRYPTO_UP_REF(QAT_CRYPTO_REF_COUNT *refcnt, int *ret)
+# else
+
+static __inline__ int QAT_CRYPTO_UP_REF(CRYPTO_REF_COUNT *refcnt, int *ret)
 {
     *ret = __atomic_fetch_add(&refcnt->val, 1, __ATOMIC_RELAXED) + 1;
     return 1;
 }
 
-static __inline__ int QAT_CRYPTO_DOWN_REF(QAT_CRYPTO_REF_COUNT *refcnt, int *ret)
+static __inline__ int QAT_CRYPTO_DOWN_REF(CRYPTO_REF_COUNT *refcnt, int *ret)
 {
     *ret = __atomic_fetch_sub(&refcnt->val, 1, __ATOMIC_RELAXED) - 1;
     if (*ret == 0)
         __atomic_thread_fence(__ATOMIC_ACQUIRE);
     return 1;
 }
 # endif
+# endif
 
 #endif   /* E_QAT_H */
diff --git a/qat_common.h b/qat_common.h
@@ -54,10 +54,12 @@
 
 # define MAX_KEYLEN  57
 
-# if OPENSSL_VERSION_NUMBER >= 0x30200000
+# if OPENSSL_VERSION_NUMBER < 0x30200000
+typedef int CRYPTO_REF_COUNT;
+# else
 typedef struct {
-    _Atomic int val;
-} QAT_CRYPTO_REF_COUNT;
+    int val;
+} CRYPTO_REF_COUNT;
 # endif
 
 /* Only for QAT_HW built with OpenSSL 1.1.1 Engine */
@@ -84,12 +86,10 @@ typedef struct ecx_key_st {
     unsigned char *privkey;
     size_t keylen;
     ECX_KEY_TYPE type;
-# if OPENSSL_VERSION_NUMBER < 0x30200000
-    int references;
+    CRYPTO_REF_COUNT references;
+#if OPENSSL_VERSION_NUMBER < 0x30200000
     CRYPTO_RWLOCK *lock;
-# else
-    QAT_CRYPTO_REF_COUNT references;
-# endif
+#endif
 } QAT_SW_ECX_KEY, ECX_KEY;
 # else
 typedef struct {

diff --git a/qat_evp.c b/qat_evp.c
@@ -107,8 +107,12 @@
 #endif
 
 #ifdef QAT_HW_INTREE
-# define ENABLE_QAT_HW_SHA3
-# define ENABLE_QAT_HW_CHACHAPOLY
+# ifndef ENABLE_QAT_HW_SHA3
+#  define ENABLE_QAT_HW_SHA3
+# endif
+# ifndef ENABLE_QAT_HW_CHACHAPOLY
+#  define ENABLE_QAT_HW_CHACHAPOLY
+# endif
 #endif
 
 #ifndef SM4_BLOCK_SIZE
@@ -776,7 +780,7 @@ int qat_pkey_methods(ENGINE *e, EVP_PKEY_METHOD **pmeth,
     return 0;
 }
 
-static inline const EVP_CIPHER *qat_gcm_cipher_sw_impl(int nid)
+const EVP_CIPHER *qat_gcm_cipher_sw_impl(int nid)
 {
     switch (nid) {
         case NID_aes_128_gcm:
@@ -933,7 +937,8 @@ const EVP_CIPHER *qat_create_ccm_cipher_meth(int nid, int keylen)
 
     if (qat_hw_offload &&
         (qat_hw_algo_enable_mask & ALGO_ENABLE_MASK_AES_CCM)) {
-#if !defined(QAT20_OOT) && !defined(QAT_HW_INTREE)
+#if !defined(QAT20_OOT) && !defined(QAT_HW_INTREE) \
+    && !defined(QAT_HW_FBSD_OOT) && !defined(QAT_HW_FBSD_INTREE)
         if (nid == NID_aes_192_ccm || nid == NID_aes_256_ccm) {
             EVP_CIPHER_meth_free(c);
             DEBUG("OpenSSL SW AES_CCM_%d registration succeeded\n", keylen*8);
@@ -1476,7 +1481,8 @@ void qat_free_ciphers(void)
 #endif
 #ifdef ENABLE_QAT_HW_CCM
             case NID_aes_128_ccm:
-#if defined(QAT20_OOT) || defined(QAT_HW_INTREE)
+#if defined(QAT20_OOT) || defined(QAT_HW_INTREE) \
+    || defined(QAT_HW_FBSD_OOT) || defined(QAT_HW_FBSD_INTREE)
             case NID_aes_192_ccm:
             case NID_aes_256_ccm:
 #endif

diff --git a/qat_evp.h b/qat_evp.h
@@ -162,6 +162,7 @@ int qat_ciphers(ENGINE *e, const EVP_CIPHER **cipher, const int **nids,
                 int nid);
 const EVP_CIPHER *qat_create_gcm_cipher_meth(int nid, int keylen);
 const EVP_CIPHER *qat_create_ccm_cipher_meth(int nid, int keylen);
+const EVP_CIPHER *qat_gcm_cipher_sw_impl(int nid);
 # ifndef ENABLE_QAT_SMALL_PKT_OFFLOAD
 #  define CRYPTO_SMALL_PACKET_OFFLOAD_THRESHOLD_DEFAULT 2048
 #  define CRYPTO_SMALL_PACKET_OFFLOAD_THRESHOLD_SM4_CBC 64