Skip to content

Commit

Permalink
[flaky tests] Address LSAN false positives
Browse files Browse the repository at this point in the history
Some tests, like external_mini_cluster-itest, are now only failing
in ASAN due to leaks. Upon inspection I could only find false
positives among these leaks around thread local variables.

This seems related to this issue:
google/sanitizers#757

The fix is to wrap those specific instances of false positives
with ScopedLeakDisabler so that they don't get reported.

After many tries I was unable to come up with the right incantation
to cause this in dist-tests. Running the highest failing test in
ASAN with stress was not enough to cause this. Somehow this is
more likely in jenkins where these failures are pretty common.

Change-Id: I8c8d9ff83c0cfbc11cab213a25cbd5daa3b25869
Reviewed-on: http://gerrit.cloudera.org:8080/6320
Reviewed-by: David Ribeiro Alves <[email protected]>
Tested-by: Kudu Jenkins
  • Loading branch information
dralves committed May 4, 2017
1 parent 174a058 commit dcd029e
Show file tree
Hide file tree
Showing 2 changed files with 8 additions and 0 deletions.
4 changes: 4 additions & 0 deletions src/kudu/util/kernel_stack_watchdog.cc
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
#include <string>

#include "kudu/util/debug-util.h"
#include "kudu/util/debug/leakcheck_disabler.h"
#include "kudu/util/env.h"
#include "kudu/util/faststring.h"
#include "kudu/util/flag_tags.h"
Expand Down Expand Up @@ -154,6 +155,9 @@ void KernelStackWatchdog::RunThread() {
}

KernelStackWatchdog::TLS* KernelStackWatchdog::GetTLS() {
// Disable leak check. LSAN sometimes gets false positives on thread locals.
// See: https://github.com/google/sanitizers/issues/757
debug::ScopedLeakCheckDisabler d;
INIT_STATIC_THREAD_LOCAL(KernelStackWatchdog::TLS, tls_);
return tls_;
}
Expand Down
4 changes: 4 additions & 0 deletions src/kudu/util/thread_restrictions.cc
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,7 @@
#include <glog/logging.h>
#include <gperftools/heap-checker.h>

#include "kudu/util/debug/leakcheck_disabler.h"
#include "kudu/util/thread.h"
#include "kudu/util/threadlocal.h"
#include "kudu/util/thread_restrictions.h"
Expand All @@ -41,6 +42,9 @@ struct LocalThreadRestrictions {
};

LocalThreadRestrictions* LoadTLS() {
// Disable leak check. LSAN sometimes gets false positives on thread locals.
// See: https://github.com/google/sanitizers/issues/757
debug::ScopedLeakCheckDisabler d;
BLOCK_STATIC_THREAD_LOCAL(LocalThreadRestrictions, local_thread_restrictions);
return local_thread_restrictions;
}
Expand Down

0 comments on commit dcd029e

Please sign in to comment.