Skip to content

Commit

Permalink
yarn scheduler faq moved to hadoop docs
Browse files Browse the repository at this point in the history
  • Loading branch information
Amy Wang committed Jan 13, 2015
1 parent 5edb09f commit 9b76235
Show file tree
Hide file tree
Showing 10 changed files with 123 additions and 100 deletions.
6 changes: 4 additions & 2 deletions h2o-docs/source/deployment/hadoop.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.. _Hadoop:

H\ :sub:`2`\ O on Hadoop
=========================
========================

H\ :sub:`2`\ O is the open source math & machine learning engine for big data that brings distribution and parallelism to powerful
algorithms while keeping the widely used languages of R and JSON as an API. H\ :sub:`2`\ O brings and elegant lego-like infrastructure
Expand All @@ -14,4 +14,6 @@ TaskTrackers on all major distros.

hadoop_glossary
hadoop_tutorial
hadoop_white_paper
hadoop_yarn
hadoop_white_paper
../faq/hadoop_related
5 changes: 2 additions & 3 deletions h2o-docs/source/deployment/hadoop_tutorial.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.. _Hadoop_Tutorial:

Running H2O on Hadoop
================================
=====================

The following tutorial will walk the user through the download or build of H2O and the parameters involved in launching H2O from the command line.

Expand Down Expand Up @@ -44,5 +44,4 @@ review the output from your command after the nodes has clouded up and formed a

.. image:: hadoop_cmd_output.png
:width: 100 %

""""

109 changes: 109 additions & 0 deletions h2o-docs/source/deployment/hadoop_yarn.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
.. _Hadoop_Yarn:


H2O on Yarn
===========

When you launch H2O on Hadoop with the ``hadoop jar`` command, Yarn will allocate
the resources necessary to launch the nodes requested. Then, H2O will be launched
as a MapReduce (V2) task such that each Mapper will be a H2O node with the
designated mapper size. A problem that some users might encounter will be Yarn's
rejection of a job request either because the user do not actually have the
machine memory necessary to launch the job or there is a configuration problem.

If the problem is a lack of memory then try launching the job with a less memory.
If the issue is a configuration problem, usually to fix this, the user will need
to adjust the maximum memory that Yarn will allow the user to launch for
each mapper.

**How to launch H2O with less memory?**

Try the job again but this time with a smaller heap size ``-mapperXmx`` and less
nodes ``-nodes`` to verify that a small launch can proceed at all. The default
test case is to run a single 1g node.

**How to increase the maximum memory Yarn will allow for each mapper?**

If the cluster manager settings are configured for the default maximum memory
size but the memory required for the request exceeds that amount, YARN will not
launch and H2O will time out. If you have a default configuration, change the
configuration settings in your cluster manager to enable launching of mapper tasks
for specific memory sizes. Use the following formula to calculate the amount of
memory required:

::

YARN container size
== mapreduce.map.memory.mb
== mapperXmx + (mapperXmx * extramempercent [default is 10%])

Output from an H2O launch is shown below:

::

$ hadoop jar h2odriver_hdp2.1.jar water.hadoop.h2odriver
-libjars ../h2o.jar -mapperXmx 30g -extramempercent 20 -nodes 4 -output hdfsOutputDir
Determining driver host interface for mapper->driver callback...
[Possible callback IP address: 172.16.2.181]
[Possible callback IP address: 127.0.0.1]
Using mapper->driver callback IP address and port: 172.16.2.181:58280
(You can override these with -driverif and -driverport.)
Driver program compiled with MapReduce V1 (Classic)
14/10/10 18:39:53 INFO Configuration.deprecation: mapred.map.child.java.opts is deprecated.
Instead, use mapreduce.map.java.opts
Memory Settings:
mapred.child.java.opts: -Xms30g -Xmx30g
mapred.map.child.java.opts: -Xms30g -Xmx30g
Extra memory percent: 20
mapreduce.map.memory.mb: 36864


``mapreduce.map.memory.mb`` must be less than the YARN memory configuration values for the launch to succeed. See the examples below for how to change the memory configuration values for your version of Hadoop.


**For Cloudera, configure the settings in Cloudera Manager. Depending on how the cluster is configured, you may need to change the settings for more than one role group.**

1. Click **Configuration** and enter the following search term in quotes: **yarn.nodemanager.resource.memory-mb**.

2. Enter the amount of memory (in GB) to allocate in the **Value** field. If more than one group is listed, change the values for all listed groups.

.. image:: TroubleshootingHadoopClouderayarnnodemgr.png
:width: 100 %

3. Click the **Save Changes** button in the upper-right corner.
4. Enter the following search term in quotes: **yarn.scheduler.maximum-allocation-mb**
5. Change the value, click the **Save Changes** button in the upper-right corner, and redeploy.

.. image:: TroubleshootingHadoopClouderayarnscheduler.png
:width: 100%


**For Hortonworks,** `configure <http://docs.hortonworks.com/HDPDocuments/Ambari-1.6.0.0/bk_Monitoring_Hadoop_Book/content/monitor-chap2-3-3_2x.html>`_ **the settings in Ambari.**

1. Select **YARN**, then click the **Configs** tab.
2. Select the group.
3. In the **Node Manager** section, enter the amount of memory (in MB) to allocate in the **yarn.nodemanager.resource.memory-mb** entry field.

.. image:: TroubleshootingHadoopAmbariNodeMgr.png
:width: 100 %

4. In the **Scheduler** section, enter the amount of memory (in MB)to allocate in the **yarn.scheduler.maximum-allocation-mb** entry field.

.. image:: TroubleshootingHadoopAmbariyarnscheduler.png
:width: 100 %

5. Click the **Save** button at the bottom of the page and redeploy the cluster.


**For MapR:**

1. Edit the **yarn-site.xml** file for the node running the ResourceManager.
2. Change the values for the `yarn.nodemanager.resource.memory-mb` and `yarn.scheduler.maximum-allocation-mb` properties.
3. Restart the ResourceManager and redeploy the cluster.


To verify the values were changed, check the values for the following properties:

- `<name>yarn.nodemanager.resource.memory-mb</name>`
- `<name>yarn.scheduler.maximum-allocation-mb</name>`

4 changes: 2 additions & 2 deletions h2o-docs/source/faq/ec2_related.rst
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
.. _EC2_Related:


EC2 and H\ :sub:`2`\ O
======================
Amazon's EC2 FAQ
================

**How do you import data from a S3 bucket?**

Expand Down
95 changes: 4 additions & 91 deletions h2o-docs/source/faq/hadoop_related.rst
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
.. _Hadoop_Related:


Hadoop and H\ :sub:`2`\ O
=========================
Hadoop FAQ
==========

Sending Logs for Hadoop Troubleshooting
""""""""""""""""""""""""""""""""""""""""""""
"""""""""""""""""""""""""""""""""""""""
If H2O does not launch properly on Hadoop, send us the logs.

There are a number of ways to get the logs:
Expand Down Expand Up @@ -83,7 +83,7 @@ Copy and email the logs to [email protected] or submit them to h2ostream@googlegrou
""""""""

Common Hadoop Questions
""""""""""""""""""""""""
"""""""""""""""""""""""

**What versions of Hadoop are supported?**

Expand Down Expand Up @@ -157,90 +157,3 @@ the `-output hdfsOutputDir` argument to `-output hdfsOutputDir1` and the task sh

The likely cause is a driver mismatch - check to make sure the Hadoop distribution matches the driver jar file used to launch H2O. If your distribution is not currently
available in the package, `email us <[email protected]>`_ for a new driver file.

---

**What should I do if I want to create a job with a bigger heap size but YARN doesn't launch and H2O times out?**

First, try the job again but with a smaller heap size (`-mapperXmx`) and a smaller number of nodes (`-nodes`) to verify that a small launch can proceed at all.

If the cluster manager settings are configured for the default maximum memory size but the memory required for the request exceeds that amount, YARN will not launch and H2O will time out.
If you have a default configuration, change the configuration settings in your cluster manager to enable launching of mapper tasks for specific memory sizes. Use the following formula to calculate the amount of memory required:

::

YARN container size
== mapreduce.map.memory.mb
== mapperXmx + (mapperXmx * extramempercent [default is 10%])

Output from an H2O launch is shown below:

::

$ hadoop jar h2odriver_hdp2.1.jar water.hadoop.h2odriver
-libjars ../h2o.jar -mapperXmx 30g -extramempercent 20 -nodes 4 -output hdfsOutputDir
Determining driver host interface for mapper->driver callback...
[Possible callback IP address: 172.16.2.181]
[Possible callback IP address: 127.0.0.1]
Using mapper->driver callback IP address and port: 172.16.2.181:58280
(You can override these with -driverif and -driverport.)
Driver program compiled with MapReduce V1 (Classic)
14/10/10 18:39:53 INFO Configuration.deprecation: mapred.map.child.java.opts is deprecated.
Instead, use mapreduce.map.java.opts
Memory Settings:
mapred.child.java.opts: -Xms30g -Xmx30g
mapred.map.child.java.opts: -Xms30g -Xmx30g
Extra memory percent: 20
mapreduce.map.memory.mb: 36864


`mapreduce.map.memory.mb` must be less than the YARN memory configuration values for the launch to succeed. See the examples below for how to change the memory configuration values for your version of Hadoop.


**For Cloudera, configure the settings in Cloudera Manager. Depending on how the cluster is configured, you may need to change the settings for more than one role group.**

1. Click **Configuration** and enter the following search term in quotes: **yarn.nodemanager.resource.memory-mb**.

2. Enter the amount of memory (in GB) to allocate in the **Value** field. If more than one group is listed, change the values for all listed groups.

.. image:: TroubleshootingHadoopClouderayarnnodemgr.png
:width: 100 %

3. Click the **Save Changes** button in the upper-right corner.
4. Enter the following search term in quotes: **yarn.scheduler.maximum-allocation-mb**
5. Change the value, click the **Save Changes** button in the upper-right corner, and redeploy.

.. image:: TroubleshootingHadoopClouderayarnscheduler.png
:width: 100%


**For Hortonworks,** `configure <http://docs.hortonworks.com/HDPDocuments/Ambari-1.6.0.0/bk_Monitoring_Hadoop_Book/content/monitor-chap2-3-3_2x.html>`_ **the settings in Ambari.**

1. Select **YARN**, then click the **Configs** tab.
2. Select the group.
3. In the **Node Manager** section, enter the amount of memory (in MB) to allocate in the **yarn.nodemanager.resource.memory-mb** entry field.

.. image:: TroubleshootingHadoopAmbariNodeMgr.png
:width: 100 %

4. In the **Scheduler** section, enter the amount of memory (in MB)to allocate in the **yarn.scheduler.maximum-allocation-mb** entry field.

.. image:: TroubleshootingHadoopAmbariyarnscheduler.png
:width: 100 %

5. Click the **Save** button at the bottom of the page and redeploy the cluster.


**For MapR:**

1. Edit the **yarn-site.xml** file for the node running the ResourceManager.
2. Change the values for the `yarn.nodemanager.resource.memory-mb` and `yarn.scheduler.maximum-allocation-mb` properties.
3. Restart the ResourceManager and redeploy the cluster.


To verify the values were changed, check the values for the following properties:

- `<name>yarn.nodemanager.resource.memory-mb</name>`
- `<name>yarn.scheduler.maximum-allocation-mb</name>`

---
4 changes: 2 additions & 2 deletions h2o-docs/source/faq/r_related.rst
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
.. _R_Related:

R and H\ :sub:`2`\ O
====================
R FAQ
=====

In order for H2O and R to work together, an H2O instance that is specified in the R workspace must be running. If the H2O instance is terminated, the H2O package in R will no longer work because R cannot send or receive information to or from H2O's distributed analysis. Even if a new instance of H2O with the exact same IP and port number is started, users must re-establish the connection between H2O and R using the command `h2o.init()` and restart their H2O work session.

Expand Down

0 comments on commit 9b76235

Please sign in to comment.