Skip to content

Commit

Permalink
Merge pull request oneapi-src#1 from outoftardis/dev/emekhnet-readme
Browse files Browse the repository at this point in the history
Update README.md
  • Loading branch information
ykiryano authored Nov 21, 2019
2 parents c4b0f54 + c28f18b commit fb4bd69
Showing 1 changed file with 35 additions and 27 deletions.
62 changes: 35 additions & 27 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,14 @@
# oneccl
Intel(R) oneAPI Collective Communications Library
# Intel(R) oneAPI Collective Communications Library (oneCCL)

## Prerequisites

Ubuntu* 18
GNU*: C, C++ 4.8.5 or higher.
- Ubuntu* 18
- GNU*: C, C++ 4.8.5 or higher.

## Documentation

- [oneCCL Documentation (Beta)](https://intel.github.io/oneccl/)
- [Release notes](https://software.intel.com/en-us/articles/oneapi-collective-communication-library-ccl-release-notes)

## Installation
### General installation scenario
Expand All @@ -19,43 +24,46 @@ make -j install
If a "clear" build is needed, then one should create a new build directory and invoke `cmake` within it.

### Specify installation directory
Modify `cmake` command as follow:
Modify `cmake` command as follows:

```
cmake .. -DCMAKE_INSTALL_PREFIX=/path/to/installation/directory
```

If no `-DCMAKE_INSTALL_PREFIX` is specified then ccl will be installed into `_install` subdirectory of the current
build directory, e.g. `ccl/build/_install`
If no `-DCMAKE_INSTALL_PREFIX` is specified then oneCCL will be installed into `_install` subdirectory of the current
build directory, e.g. `ccl/build/_install`.

### Specify compiler
Modify `cmake` command as follow:
Modify `cmake` command as follows:

```
cmake .. -DCMAKE_C_COMPILER=your_c_compiler -DCMAKE_CXX_COMPILER=your_cxx_compiler
```

### Specify build type
Modify `cmake` command as follow:
Modify `cmake` command as follows:

```
cmake .. -DCMAKE_BUILD_TYPE=[Debug|Release|RelWithDebInfo|MinSizeRel]
```

### Enable `make` verbose output
Modify `make` command as follow to see all parameters used by `make` during compilation
Modify `make` command as follows to see all parameters used by `make` during compilation
and linkage:

```
make -j VERBOSE=1
```

### Build with address sanitizer
Modify `cmake` command as follow:
### Build with Address Sanitizer
Modify `cmake` command as follows:
```
cmake .. -DCMAKE_BUILD_TYPE=Debug -DWITH_ASAN=true
```
*Note:* address sanitizer only works in Debug build
---
**Note:** Address Sanitizer only works in Debug build.

---

Make sure that libasan.so exists.

Expand All @@ -69,41 +77,41 @@ $ cd <install_dir>/examples
$ mpirun -n 2 ./common/benchmark
```
### Setting workers affinity
There are two ways to set workers threads affinity - explicit and automatic
There are two ways to set workers threads affinity: explicit and automatic.

#### Setting affinity explicitly
1. Set environment variable *CCL_WORKER_COUNT* with desired number of workers threads
2. Set environment variable *CCL_WORKER_AFFINITY* with IDs of cores to be bound to
1. Set the environment variable *CCL_WORKER_COUNT* with the desired number of worker threads.
2. Set the environment variable *CCL_WORKER_AFFINITY* with the IDs of the cores to be bound to.

Example:
```
export CCL_WORKER_COUNT=4
export CCL_WORKER_AFFINITY=3,4,5,6
```
With variables above CCL will create 4 threads and pin them to cores with numbers 3,4,5 and 6 accordingly
With the variables above oneCCL will create 4 threads and pin them to the cores under the numbers 3, 4, 5 and 6 accordingly

#### Setting affinity automatically
*NOTE:* automatic pinning only works if application has been launched using *mpirun* provided by CCL distribution package.
---
**NOTE:** Automatic pinning only works if application has been launched using *mpirun* provided by oneCCL distribution package.

1. Set environment variable *CCL_WORKER_COUNT* with desired number of workers threads
2. Set environment variable *CCL_WORKER_AFFINITY* with value *auto*
---
1. Set the environment variable *CCL_WORKER_COUNT* with the desired number of worker threads.
2. Set the environment variable *CCL_WORKER_AFFINITY* with the value *auto*.

Example:
```
export CCL_WORKER_COUNT=4
export CCL_WORKER_AFFINITY=auto
```
With variables above CCL will create 4 threads and pin them to the last 4 cores available for the launched process.
With the variables above oneCCL will create 4 threads and pin them to the last 4 cores available for the launched process.

The exact IDs of CPU cores depend on parameters passed to *mpirun*
The exact IDs of CPU cores depend on parameters passed to *mpirun*.

## FAQ

### When do I need a clean build? When should I remove my favorite build directory?

In the most cases there is no need in removal of the current build directory. Just run `make` to
compile and link changed files. Only if one sees some suspicious build errors after significant
change in the code (e.g. after rebase or change of branch) then it is a hint to clean build directory.

Release notes available by link: https://software.intel.com/en-us/articles/oneapi-collective-communication-library-ccl-release-notes
In most cases, there is no need to remove the current build directory. Just run `make` to
compile and link changed files. Only if you see some suspicious build errors after a significant
change in your code (e.g. after rebase or change of branch), you may want to clean the build directory.

0 comments on commit fb4bd69

Please sign in to comment.