Skip to content

Commit

Permalink
[Core][C++ Worker]Optimize the log printing and code of the LD_LIBRAR…
Browse files Browse the repository at this point in the history
…Y_PATH environment variable (ray-project#32682)

1. C++ Worker users will dynamically load so through the ray_code_search_path parameter configuration. However, users often encounter the problem that C++ Actor/Task fails to find its own so in the cluster, resulting in startup failure. So adding the log of ray_code_search_path is convenient for users to locate. This log is not printed frequently.
2. C++ Worker users will dynamically load so often encounter the problem of "Not Found Dynamic Link Library". Therefore, the log printing of the "LD_LIBRARY_PATH" environment variable is added to facilitate users to locate this problem in the cluster. This log is not printed frequently.
3. Optimize the code of the LD_LIBRARY_PATH env
  • Loading branch information
larrylian authored Feb 22, 2023
1 parent a4ff83e commit e0af1fb
Show file tree
Hide file tree
Showing 6 changed files with 33 additions and 9 deletions.
2 changes: 2 additions & 0 deletions cpp/src/ray/config_internal.cc
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,8 @@ void ConfigInternal::Init(RayConfig &config, int argc, char **argv) {

if (!FLAGS_ray_code_search_path.CurrentValue().empty()) {
// Code search path like this "/path1/xxx.so:/path2".
RAY_LOG(DEBUG) << "The code search path is "
<< FLAGS_ray_code_search_path.CurrentValue();
code_search_path = absl::StrSplit(
FLAGS_ray_code_search_path.CurrentValue(), ':', absl::SkipEmpty());
}
Expand Down
3 changes: 3 additions & 0 deletions cpp/src/ray/util/function_helper.cc
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@
#include <boost/range/iterator_range.hpp>
#include <memory>

#include "ray/common/constants.h"
#include "ray/util/logging.h"
#include "util.h"

namespace ray {
namespace internal {
Expand Down Expand Up @@ -151,6 +153,7 @@ void FunctionHelper::LoadFunctionsFromPaths(const std::vector<std::string> &path
}
}

RAY_LOG(INFO) << std::string(kLibraryPathEnvName) << ": " << getLibraryPathEnv();
// Try to load all found libraries.
for (auto lib : dynamic_libraries) {
LoadDll(lib);
Expand Down
10 changes: 10 additions & 0 deletions cpp/src/ray/util/util.cc
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
#include <boost/algorithm/string.hpp>
#include <boost/asio.hpp>

#include "ray/common/constants.h"
#include "ray/util/logging.h"

namespace ray {
Expand All @@ -43,5 +44,14 @@ std::string GetNodeIpAddress(const std::string &address) {
return "";
}
}

std::string getLibraryPathEnv() {
auto path_env_p = std::getenv(kLibraryPathEnvName);
if (path_env_p != nullptr && strlen(path_env_p) != 0) {
return std::string(path_env_p);
}
return {};
}

} // namespace internal
} // namespace ray
3 changes: 3 additions & 0 deletions cpp/src/ray/util/util.h
Original file line number Diff line number Diff line change
Expand Up @@ -31,5 +31,8 @@ namespace internal {
/// you care about.
/// \return The IP address by which the local node can be reached from the address.
std::string GetNodeIpAddress(const std::string &address = "8.8.8.8:53");

std::string getLibraryPathEnv();

} // namespace internal
} // namespace ray
8 changes: 8 additions & 0 deletions src/ray/common/constants.h
Original file line number Diff line number Diff line change
Expand Up @@ -53,3 +53,11 @@ constexpr char kSetupWorkerFilename[] = "setup_worker.py";

/// The version of Ray
constexpr char kRayVersion[] = "3.0.0.dev0";

#if defined(__APPLE__)
constexpr char kLibraryPathEnvName[] = "DYLD_LIBRARY_PATH";
#elif defined(_WIN32)
constexpr char kLibraryPathEnvName[] = "PATH";
#else
constexpr char kLibraryPathEnvName[] = "LD_LIBRARY_PATH";
#endif
16 changes: 7 additions & 9 deletions src/ray/raylet/worker_pool.cc
Original file line number Diff line number Diff line change
Expand Up @@ -362,14 +362,7 @@ WorkerPool::BuildProcessCommandArgs(const Language &language,
// Set native library path for shared library search.
if (!native_library_path_.empty() || !code_search_path.empty()) {
#if defined(__APPLE__) || defined(__linux__) || defined(_WIN32)
#if defined(__APPLE__)
static const std::string kLibraryPathEnvName = "DYLD_LIBRARY_PATH";
#elif defined(__linux__)
static const std::string kLibraryPathEnvName = "LD_LIBRARY_PATH";
#elif defined(_WIN32)
static const std::string kLibraryPathEnvName = "PATH";
#endif
auto path_env_p = std::getenv(kLibraryPathEnvName.c_str());
auto path_env_p = std::getenv(kLibraryPathEnvName);
std::string path_env = native_library_path_;
if (path_env_p != nullptr && strlen(path_env_p) != 0) {
path_env.append(":").append(path_env_p);
Expand All @@ -378,7 +371,12 @@ WorkerPool::BuildProcessCommandArgs(const Language &language,
if (!code_search_path.empty()) {
path_env.append(":").append(code_search_path);
}
env.emplace(kLibraryPathEnvName, path_env);
auto path_env_iter = env.find(kLibraryPathEnvName);
if (path_env_iter == env.end()) {
env.emplace(kLibraryPathEnvName, path_env);
} else {
env[kLibraryPathEnvName] = path_env_iter->second.append(":").append(path_env);
}
#endif
}
}
Expand Down

0 comments on commit e0af1fb

Please sign in to comment.