Skip to content

Commit

Permalink
[core] mutable_object_test flaky tsan fix ray-project#44764
Browse files Browse the repository at this point in the history
mutable_object test is still flaky. In the recent buildkite runs, it appears that there is a data race on the access to num_readers in SetErrorUnlocked(). This PR fixes that data race.

Signed-off-by: Jack Humphries <[email protected]>
  • Loading branch information
jackhumphries authored Apr 17, 2024
1 parent 00a0b9f commit d5ebda5
Showing 1 changed file with 7 additions and 8 deletions.
15 changes: 7 additions & 8 deletions src/ray/object_manager/common.cc
Original file line number Diff line number Diff line change
Expand Up @@ -81,9 +81,11 @@ Status PlasmaObjectHeader::TryToAcquireSemaphore(sem_t *sem) const {
// section after `SetErrorUnlocked()` has been called. One thread could be in the
// critical section when that is called, but no additional thread will enter the
// critical section.
RAY_RETURN_NOT_OK(CheckHasError());

return Status::OK();
Status s = CheckHasError();
if (!s.ok()) {
RAY_CHECK_EQ(sem_post(sem), 0);
}
return s;
}

void PlasmaObjectHeader::SetErrorUnlocked(Semaphores &sem) {
Expand All @@ -97,11 +99,8 @@ void PlasmaObjectHeader::SetErrorUnlocked(Semaphores &sem) {
// be more than one writer.
RAY_CHECK_EQ(sem_post(sem.object_sem), 0);

// Increment `sem.header_sem` by `num_readers` since there could potentially be that
// many readers blocked on `sem_wait()`.
for (int64_t i = 0; i < num_readers; i++) {
RAY_CHECK_EQ(sem_post(sem.header_sem), 0);
}
// Increment `header_sem` to unblock any readers and/or the writer.
RAY_CHECK_EQ(sem_post(sem.header_sem), 0);
}

Status PlasmaObjectHeader::WriteAcquire(Semaphores &sem,
Expand Down

0 comments on commit d5ebda5

Please sign in to comment.