Remove resolv.conf and associated docs (apache-spark-on-k8s#20)

* Remove resolv.conf

As it’s no longer needed due to apache-spark-on-k8s#16

Fixes apache-spark-on-k8s#19

* Update docs to reflect k8s 1.6 requirement

charts/README.md

@@ -1,9 +1,13 @@
+### Prerequisite
+
+Requires Kubernetes 1.6 as the `namenode` and `datanodes` are using `ClusterFirstWithHostNet`, which was introduced in Kubernetes 1.6
+
+### Usage
+
 Helm charts for launching HDFS in a K8s cluster. They should be launched in
 the following order.
 
-  1. `hdfs-resolv-conf`: Creates a config map containing resolv.conf used by
-     the HDFS daemons. See `hdfs-resolv-conf/README.md` for how to launch.
-  2. `hdfs-namenode-k8s`: Launches the hdfs namenode. See
+  1. `hdfs-namenode-k8s`: Launches the hdfs namenode. See
      `hdfs-namenode-k8s/README.md` for how to launch.
-  3. `hdfs-datanode-k8s`: Launches the hdfs datanode daemons. See
+  2. `hdfs-datanode-k8s`: Launches the hdfs datanode daemons. See
      `hdfs-datanode-k8s/README.md` for how to launch.

charts/hdfs-datanode-k8s/README.md

@@ -1,15 +1,6 @@
 HDFS `datanodes` running inside a kubernetes cluster. See the other chart for
 `namenode`.
 
-### Prerequisite
-
-  Requires Kubernetes version 1.5 and beyond, because `namenode` is using
-  `StatefulSet`, which is available only in version 1.5 and later.
-
-  Make sure the `hdfs-resolv-conf` chart is launched. Also ensure `namenode` is
-  fully launched using the corresponding chart. `Datanodes` rely
-  on DNS to resolve the hostname of the namenode when they start up.
-
 ### Usage
 
   1. In some setup, the master node may launch a datanode. To prevent this,

charts/hdfs-datanode-k8s/templates/datanode-daemonset.yaml

@@ -32,6 +32,8 @@ spec:
         - name: datanode
           image: uhopper/hadoop-datanode:2.7.2
           env:
+            - name: CORE_CONF_fs_defaultFS
+              value: hdfs://hdfs-namenode-0.hdfs-namenode.default.svc.cluster.local:8020
             # The below uses two loops to make sure the last item does not have comma. It uses index 0
             # for the last item since that is the only special index that helm template gives us.
             - name: HDFS_CONF_dfs_datanode_data_dir
@@ -46,10 +48,6 @@ spec:
                   /hadoop/dfs/data/{{ $index }}
                 {{- end }}
               {{- end }}
-            # This works only with /etc/resolv.conf mounted from the config map.
-            # K8s version 1.6 will fix this, per https://github.com/kubernetes/kubernetes/pull/29378.
-            - name: CORE_CONF_fs_defaultFS
-              value: hdfs://hdfs-namenode-0.hdfs-namenode.default.svc.cluster.local:8020
             # We now add custom hadoop configuration provided
             {{- range $key, $value := .Values.customHadoopConfig }}
             {{- if and (ne $key "HDFS_CONF_dfs_datanode_data_dir") (ne $key "CORE_CONF_fs_defaultFS") }}

charts/hdfs-namenode-k8s/README.md

@@ -1,13 +1,6 @@
 HDFS `namenode` running inside a kubernetes cluster. See the other chart for
 `datanodes`.
 
-### Prerequisite
-
-  Requires Kubernetes version 1.5 and beyond, because `namenode` is using
-  `StatefulSet`, which is available only in version 1.5 and later.
-
-  Make sure the `hdfs-resolv-conf` chart is launched.
-
 ### Usage
 
   1. Attach a label to one of your k8s cluster host that will run the `namenode`