Bug 1720464 - land NSS 8f41147c2192 UPGRADE_NSS_RELEASE, r=beurdouche

``` 2021-07-22 Benjamin Beurdouche <[email protected]> * doc/rst/index.rst: Display warning on the new NSS documentation [8f41147c2192] [tip] 2021-07-20 Robert Relyea <[email protected]> * lib/softoken/sdb.c: Bug 1721476 sqlite 3.34 changed it's open semantics, causing nss failures. https://sqlite.org/forum/info/42cf8e985bb051a2 sqlite is now permissive on opening a readonly file even if you ask for the file to be opened R/W. normally sqlite is very conservative in changing it's underlying semantics, but evidently they chose convience over compatibility. NSS now needs to check the file permissions itself to preserve nss semantics. [f2d34a957599] 2021-07-15 Robert Relyea <[email protected]> * tests/common/init.sh, tests/common/parsegtestreport.sed, tests/common/parsegtestreport.sh, tests/gtests/gtests.sh, tests/ssl_gtests/ssl_gtests.sh: Bug 1720230 Gtest update changed the gtest reports, losing gtest details in all.sh reports. This patch includes the updated .sed script, and an experiment using bash instead to see how hard it would be to make a more robust parser. The robust parser generates identical output as sed, but takes about 30x longer, so instead of subsecond operations, it takes almost half a minute. With that result, I think we can stay with sed and continue to update when we get new versions of gtests. (sigh). time cat report.xml.0 | sed -f parsegtestreport.sed > r1 real 0m0.710s user 0m0.705s sys 0m0.008s time cat report.xml.0 | sh parsegtestreport.sh > r2 real 0m25.066s user 0m17.759s sys 0m9.506s [rrelyea@localhost common]$ diff r1 r2 updated: with review comments from Martin and move the report parsing to the common code so it can be shared with both ssl_gtests and gtests shell scripts. [f12856d5d2c2] 2021-07-13 Robert Relyea <[email protected]> * gtests/softoken_gtest/softoken_dh_vectors.h, lib/softoken/pkcs11c.c, lib/softoken/pkcs11i.h, lib/softoken/pkcs11u.c, lib/softoken/sftkdhverify.c: Bug 1720228 NSS incorrectly accepting 1536 bit DH primes in FIPS mode When NSS is in FIPS mode, it should reject all primes smaller than 2048. The ike 1536 prime is in the accepted primes table. In FIPS mode it should be rejected. [d2ec946e601a] 2021-07-15 Robert Relyea <[email protected]> * cmd/manifest.mn, cmd/sdbthreadtst/Makefile, cmd/sdbthreadtst/manifest.mn, cmd/sdbthreadtst/sdbthreadtst.c, cmd/sdbthreadtst/sdbthreadtst.gyp, lib/softoken/sdb.c, lib/softoken/sftkdb.c, nss.gyp, tests/dbtests/dbtests.sh: Bug 1720232 SQLite calls could timeout in starvation situations. Some of our servers could cause random failures when trying to generate many key pairs from multiple threads. This is caused because some threads would starve long enough for them to give up on getting a begin transaction on sqlite. sqlite only allows one transaction at a time. Also, there were some bugs in error handling of the broken transaction case where NSS would try to cancel a transation after the begin failed (most cases were correct, but one case in particular was problematic). [b54b0d41e51b] 2021-07-13 Robert Relyea <[email protected]> * lib/pk11wrap/pk11cxt.c, lib/pk11wrap/pk11hpke.c, lib/softoken/kbkdf.c, lib/softoken/sftkhmac.c, lib/softoken/sftkike.c: Bug 1720225 Coverity/cpp scanner errors found in nss 3.67 A number of coverity/scanner issues were found in the kdf code which was added in nss 3.44 and the fixes never upstreamed, as well as coverity/scanner errors in nss 3.66. Not all errors were fixed, those errors which were determined to be false positives were just recorded. No attempt has been made to fix coverity/scanner errors in gtests. [d1b9709d8861] ``` Differential Revision: https://phabricator.services.mozilla.com/D120624
dothq · Jul 22, 2021 · 9753f75 · 9753f75
1 parent 4367653
commit 9753f75
Show file tree

Hide file tree

Showing 27 changed files with 518 additions and 99 deletions.
diff --git a/security/nss/TAG-INFO b/security/nss/TAG-INFO
@@ -1 +1 @@
-b1eac8c86e99
+8f41147c2192
diff --git a/security/nss/cmd/manifest.mn b/security/nss/cmd/manifest.mn
@@ -65,6 +65,7 @@ NSS_SRCDIRS = \
  pwdecrypt \
  rsaperf \
  rsapoptst \
+ sdbthreadtst \
  sdrtest \
  selfserv  \
  signtool \

diff --git a/security/nss/cmd/sdbthreadtst/Makefile b/security/nss/cmd/sdbthreadtst/Makefile
@@ -0,0 +1,48 @@
+#! gmake
+#
+# This Source Code Form is subject to the terms of the Mozilla Public
+# License, v. 2.0. If a copy of the MPL was not distributed with this
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
+
+#######################################################################
+# (1) Include initial platform-independent assignments (MANDATORY).   #
+#######################################################################
+
+include manifest.mn
+
+#######################################################################
+# (2) Include "global" configuration information. (OPTIONAL)          #
+#######################################################################
+
+include $(CORE_DEPTH)/coreconf/config.mk
+
+#######################################################################
+# (3) Include "component" configuration information. (OPTIONAL)       #
+#######################################################################
+
+#######################################################################
+# (4) Include "local" platform-dependent assignments (OPTIONAL).      #
+#######################################################################
+
+include ../platlibs.mk
+
+
+#######################################################################
+# (5) Execute "global" rules. (OPTIONAL)                              #
+#######################################################################
+
+include $(CORE_DEPTH)/coreconf/rules.mk
+
+#######################################################################
+# (6) Execute "component" rules. (OPTIONAL)                           #
+#######################################################################
+
+
+
+#######################################################################
+# (7) Execute "local" rules. (OPTIONAL).                              #
+#######################################################################
+
+
+include ../platrules.mk
+
diff --git a/security/nss/cmd/sdbthreadtst/manifest.mn b/security/nss/cmd/sdbthreadtst/manifest.mn
@@ -0,0 +1,22 @@
+#
+# This Source Code Form is subject to the terms of the Mozilla Public
+# License, v. 2.0. If a copy of the MPL was not distributed with this
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
+
+CORE_DEPTH = ../..
+
+DEFINES += -DNSPR20
+
+# MODULE public and private header  directories are implicitly REQUIRED.
+MODULE = nss
+
+CSRCS = \
+	sdbthreadtst.c \
+	$(NULL)
+
+# The MODULE is always implicitly required.
+# Listing it here in REQUIRES makes it appear twice in the cc command line.
+
+PROGRAM = sdbthreadtst
+
+# USE_STATIC_LIBS = 1
diff --git a/security/nss/cmd/sdbthreadtst/sdbthreadtst.c b/security/nss/cmd/sdbthreadtst/sdbthreadtst.c
@@ -0,0 +1,213 @@
+#if defined(XP_UNIX)
+#include <unistd.h>
+#endif
+#include <stdio.h>
+#include <nss.h>
+#include <prtypes.h>
+#include <prerr.h>
+#include <prerror.h>
+#include <prthread.h>
+#include <pk11pub.h>
+#include <keyhi.h>
+
+#define MAX_THREAD_COUNT 100
+
+/* globals */
+int THREAD_COUNT = 30;
+int FAILED = 0;
+int ERROR = 0;
+int LOOP_COUNT = 100;
+int KEY_SIZE = 3072;
+int STACK_SIZE = 0;
+int VERBOSE = 0;
+char *NSSDIR = ".";
+PRBool ISTOKEN = PR_TRUE;
+CK_MECHANISM_TYPE MECHANISM = CKM_RSA_PKCS_KEY_PAIR_GEN;
+
+void
+usage(char *prog, char *error)
+{
+    if (error) {
+        fprintf(stderr, "Bad Arguments: %s", error);
+    }
+    fprintf(stderr, "usage: %s [-l loop_count] [-t thread_count] "
+                    "[-k key_size] [-s stack_size] [-d nss_dir] [-e] [-v] [-h]\n",
+            prog);
+    fprintf(stderr, "    loop_count   -- "
+                    "number of keys to generate on each thread (default=%d)\n",
+            LOOP_COUNT);
+    fprintf(stderr, "    thread_count -- "
+                    "number of of concurrent threads to run (def=%d,max=%d)\n",
+            THREAD_COUNT, MAX_THREAD_COUNT);
+    fprintf(stderr, "    key_size     -- "
+                    "rsa key size in bits (default=%d)\n",
+            KEY_SIZE);
+    fprintf(stderr, "    stack_size     -- "
+                    "thread stack size in bytes, 0=optimal (default=%d)\n",
+            STACK_SIZE);
+    fprintf(stderr, "    nss_dir     -- "
+                    "location of the nss directory (default=%s)\n",
+            NSSDIR);
+    fprintf(stderr, "    -e use session keys rather than token keys\n");
+    fprintf(stderr, "    -v verbose, print progress indicators\n");
+    fprintf(stderr, "    -h print this message\n");
+    exit(2);
+}
+
+void
+create_key_loop(void *arg)
+{
+    int i;
+    PK11SlotInfo *slot = PK11_GetInternalKeySlot();
+    PK11RSAGenParams param;
+    int threadnumber = *(int *)arg;
+    int failures = 0;
+    int progress = 5;
+    PRIntervalTime epoch = PR_IntervalNow();
+    param.keySizeInBits = KEY_SIZE;
+    param.pe = 0x10001L;
+    printf(" - thread %d starting\n", threadnumber);
+    progress = 30 / THREAD_COUNT;
+    if (progress < 2)
+        progress = 2;
+    for (i = 0; i < LOOP_COUNT; i++) {
+        SECKEYPrivateKey *privKey;
+        SECKEYPublicKey *pubKey;
+        privKey = PK11_GenerateKeyPair(slot, MECHANISM, &param, &pubKey,
+                                       ISTOKEN, PR_TRUE, NULL);
+        if (privKey == NULL) {
+            fprintf(stderr,
+                    "keypair gen in thread %d failed %s\n", threadnumber,
+                    PORT_ErrorToString(PORT_GetError()));
+            FAILED++;
+            failures++;
+        }
+        if (VERBOSE && (i % progress) == 0) {
+            PRIntervalTime current = PR_IntervalNow();
+            PRIntervalTime interval = current - epoch;
+            int seconds = (interval / PR_TicksPerSecond());
+            int mseconds = ((interval * 1000) / PR_TicksPerSecond()) - (seconds * 1000);
+            epoch = current;
+            printf(" - thread %d @ %d iterations %d.%03d sec\n", threadnumber,
+                   i, seconds, mseconds);
+        }
+        if (ISTOKEN && privKey) {
+            SECKEY_DestroyPublicKey(pubKey);
+            SECKEY_DestroyPrivateKey(privKey);
+        }
+    }
+    PK11_FreeSlot(slot);
+    printf(" * thread %d ending with %d failures\n", threadnumber, failures);
+    return;
+}
+
+int
+main(int argc, char **argv)
+{
+    PRThread *thread[MAX_THREAD_COUNT];
+    int threadnumber[MAX_THREAD_COUNT];
+    int i;
+    PRStatus status;
+    SECStatus rv;
+    char *prog = *argv++;
+    char buf[2048];
+    char *arg;
+
+    while ((arg = *argv++) != NULL) {
+        if (*arg == '-') {
+            switch (arg[1]) {
+                case 'l':
+                    if (*argv == NULL)
+                        usage(prog, "missing loop count");
+                    LOOP_COUNT = atoi(*argv++);
+                    break;
+                case 'k':
+                    if (*argv == NULL)
+                        usage(prog, "missing key size");
+                    KEY_SIZE = atoi(*argv++);
+                    break;
+                case 's':
+                    if (*argv == NULL)
+                        usage(prog, "missing stack size");
+                    STACK_SIZE = atoi(*argv++);
+                    break;
+                case 't':
+                    if (*argv == NULL)
+                        usage(prog, "missing thread count");
+                    THREAD_COUNT = atoi(*argv++);
+                    if (THREAD_COUNT > MAX_THREAD_COUNT) {
+                        usage(prog, "max thread count exceeded");
+                    }
+                    break;
+                case 'v':
+                    VERBOSE = 1;
+                    break;
+                case 'd':
+                    if (*argv == NULL)
+                        usage(prog, "missing directory");
+                    NSSDIR = *argv++;
+                    break;
+                case 'e':
+                    ISTOKEN = PR_FALSE;
+                    break;
+                case 'h':
+                    usage(prog, NULL);
+                    break;
+                default:
+                    sprintf(buf, "unknown option %c", arg[1]);
+                    usage(prog, buf);
+            }
+        } else {
+            sprintf(buf, "unknown argument %s", arg);
+            usage(prog, buf);
+        }
+    }
+    /* initialize NSS */
+    rv = NSS_InitReadWrite(NSSDIR);
+    if (rv != SECSuccess) {
+        fprintf(stderr,
+                "NSS_InitReadWrite(%s) failed(%s)\n", NSSDIR,
+                PORT_ErrorToString(PORT_GetError()));
+        exit(2);
+    }
+
+    /* need to initialize the database here if it's not already */
+
+    printf("creating %d threads\n", THREAD_COUNT);
+    for (i = 0; i < THREAD_COUNT; i++) {
+        threadnumber[i] = i;
+        thread[i] = PR_CreateThread(PR_USER_THREAD, create_key_loop,
+                                    &threadnumber[i], PR_PRIORITY_NORMAL,
+                                    PR_GLOBAL_THREAD,
+                                    PR_JOINABLE_THREAD, STACK_SIZE);
+        if (thread[i] == NULL) {
+            ERROR++;
+            fprintf(stderr,
+                    "PR_CreateThread failed iteration %d, %s\n", i,
+                    PORT_ErrorToString(PORT_GetError()));
+        }
+    }
+    printf("waiting on %d threads\n", THREAD_COUNT);
+    for (i = 0; i < THREAD_COUNT; i++) {
+        if (thread[i] == NULL) {
+            continue;
+        }
+        status = PR_JoinThread(thread[i]);
+        if (status != PR_SUCCESS) {
+            ERROR++;
+            fprintf(stderr,
+                    "PR_CreateThread filed iteration %d, %s]n", i,
+                    PORT_ErrorToString(PORT_GetError()));
+        }
+    }
+    printf("%d failures and %d errors found\n", FAILED, ERROR);
+    /* clean up */
+    NSS_Shutdown();
+    if (FAILED) {
+        exit(1);
+    }
+    if (ERROR) {
+        exit(2);
+    }
+    exit(0);
+}
diff --git a/security/nss/cmd/sdbthreadtst/sdbthreadtst.gyp b/security/nss/cmd/sdbthreadtst/sdbthreadtst.gyp
@@ -0,0 +1,29 @@
+# This Source Code Form is subject to the terms of the Mozilla Public
+# License, v. 2.0. If a copy of the MPL was not distributed with this
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
+{
+  'includes': [
+    '../../coreconf/config.gypi',
+    '../../cmd/platlibs.gypi'
+  ],
+  'targets': [
+    {
+      'target_name': 'sdbthreadtst',
+      'type': 'executable',
+      'sources': [
+        'sdbthreadtst.c'
+      ],
+      'dependencies': [
+        '<(DEPTH)/exports.gyp:nss_exports'
+      ]
+    }
+  ],
+  'target_defaults': {
+    'defines': [
+      'NSPR20'
+    ]
+  },
+  'variables': {
+    'module': 'nss'
+  }
+}
diff --git a/security/nss/coreconf/coreconf.dep b/security/nss/coreconf/coreconf.dep
@@ -10,3 +10,4 @@
  */
 
 #error "Do not include this header file."
+
diff --git a/security/nss/doc/rst/index.rst b/security/nss/doc/rst/index.rst
@@ -12,6 +12,10 @@ Network Security Services
    introduction_to_network_security_services/index.rst
    More documentation <more_docs>
 
+
+.. warning::
+   This NSS documentation was just imported from our legacy MDN repository. It currently is very deprecated and likely incorrect or broken in many places.
+
 `Documentation <#documentation>`__
 ----------------------------------
 

diff --git a/security/nss/gtests/softoken_gtest/softoken_dh_vectors.h b/security/nss/gtests/softoken_gtest/softoken_dh_vectors.h
@@ -2872,7 +2872,7 @@ static const DhTestVector DH_TEST_VECTORS[] = {
      {siBuffer, (unsigned char *)g2, sizeof(g2)},
      {siBuffer, NULL, 0},
      {siBuffer, NULL, 0},
-     IKE_APPROVED,
+     SAFE_PRIME,
      CLASS_1536},
     {"IKE 2048",
      {siBuffer, (unsigned char *)prime_ike_2048, sizeof(prime_ike_2048)},
@@ -2952,7 +2952,7 @@ static const DhTestVector DH_TEST_VECTORS[] = {
      {siBuffer, (unsigned char *)sub2_prime_ike_1536,
       sizeof(sub2_prime_ike_1536)},
      {siBuffer, NULL, 0},
-     IKE_APPROVED,
+     SAFE_PRIME,
      CLASS_1536},
     {"IKE 2048 with subprime",
      {siBuffer, (unsigned char *)prime_ike_2048, sizeof(prime_ike_2048)},

diff --git a/security/nss/lib/pk11wrap/pk11cxt.c b/security/nss/lib/pk11wrap/pk11cxt.c
@@ -382,7 +382,7 @@ pk11_CreateNewContextInSlot(CK_MECHANISM_TYPE type,
      * of the connection.*/
     context->fortezzaHack = PR_FALSE;
     if (type == CKM_SKIPJACK_CBC64) {
-        if (symKey->origin == PK11_OriginFortezzaHack) {
+        if (symKey && (symKey->origin == PK11_OriginFortezzaHack)) {
             context->fortezzaHack = PR_TRUE;
         }
     }

diff --git a/security/nss/lib/pk11wrap/pk11hpke.c b/security/nss/lib/pk11wrap/pk11hpke.c
@@ -1164,8 +1164,6 @@ PK11_HPKE_Seal(HpkeContext *cx, const SECItem *aad, const SECItem *pt,
     unsigned char tagBuf[HASH_LENGTH_MAX];
     size_t tagLen;
     unsigned int fixedBits;
-    PORT_Assert(cx->baseNonce->len == sizeof(ivOut));
-    PORT_Memcpy(ivOut, cx->baseNonce->data, cx->baseNonce->len);
 
     /* aad may be NULL, PT may be zero-length but not NULL. */
     if (!cx || !cx->aeadContext ||
@@ -1176,6 +1174,9 @@ PK11_HPKE_Seal(HpkeContext *cx, const SECItem *aad, const SECItem *pt,
         return SECFailure;
     }
 
+    PORT_Assert(cx->baseNonce->len == sizeof(ivOut));
+    PORT_Memcpy(ivOut, cx->baseNonce->data, cx->baseNonce->len);
+
     tagLen = cx->aeadParams->tagLen;
     maxOut = pt->len + tagLen;
     fixedBits = (cx->baseNonce->len - 8) * 8;
Original file line number	Diff line number	Diff line change
Expand Up		@@ -10,3 +10,4 @@
		*/

		#error "Do not include this header file."