Skip to content

Commit

Permalink
[Core][Doc] add docs for out of disk prevention. (ray-project#26291)
Browse files Browse the repository at this point in the history
Update docs to reflect the out of disk prevention feature.
  • Loading branch information
scv119 authored Jul 6, 2022
1 parent 0bc465f commit 29358f9
Showing 1 changed file with 22 additions and 0 deletions.
22 changes: 22 additions & 0 deletions doc/source/ray-core/objects/object-spilling.rst
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,28 @@ If you are using an HDD, it is recommended that you specify a large buffer size
},
)
To prevent running out of disk space, local object spilling will throw ``OutOfDiskError`` if the disk utilization exceeds the predefined threshold.
If multiple physical devices are used, any physical device's over-usage will trigger the ``OutOfDiskError``.
The default threshold is 0.95 (95%). You can adjust the threshold by setting ``local_fs_capacity_threshold``, or set it to 1 to disable the protection.

.. code-block:: python
ray.init(
_system_config={
# Allow spilling until the local disk is 99% utilized.
# This only affects spilling to the local file system.
"local_fs_capacity_threshold": 0.99,
"object_spilling_config": json.dumps(
{
"type": "filesystem",
"params": {
"directory_path": "/tmp/spill",
},
)
},
)
To enable object spilling to remote storage (any URI supported by `smart_open <https://pypi.org/project/smart-open/>`__):
.. code-block:: python
Expand Down

0 comments on commit 29358f9

Please sign in to comment.