[pthreadpool] Cap max thread count to fix TSAN issues (pytorch#83950)

Summary: Cap the thread count to 64 unconditionally to solve this tsan issue which leads to harder to debug, flaky test failures.

Test Plan: CI

Reviewed By: kimishpatel

Differential Revision: D38136212

Pull Request resolved: pytorch#83950
Approved by: https://github.com/kimishpatel

caffe2/utils/threadpool/ThreadPool.cc

@@ -100,6 +100,17 @@ size_t getDefaultNumThreads() {
     // Always give precedence to explicit setting.
     numThreads = FLAGS_pthreadpool_size;
   }
+
+  /*
+   * For llvm-tsan, holding limit for the number of locks for a single thread
+   * is 64. pthreadpool's worst case is the number of threads in a pool. So we
+   * want to limit the threadpool size to 64 when running with tsan. However,
+   * sometimes it is tricky to detect if we are running under tsan, for now
+   * capping the default threadcount to the tsan limit unconditionally.
+   */
+  int tsanThreadLimit = 64;
+  numThreads = std::min(numThreads, tsanThreadLimit);
+
   return numThreads;
 }