Skip to content

Commit

Permalink
Merge pull request #1 from yxl/master
Browse files Browse the repository at this point in the history
fix README to correct quicker start for spark and hive
  • Loading branch information
Marcel-Jan authored Nov 9, 2021
2 parents 249195e + 5d450c9 commit 1a7a9a6
Show file tree
Hide file tree
Showing 2 changed files with 5 additions and 7 deletions.
10 changes: 4 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,9 +43,7 @@ Go to the bash shell on the namenode with that same Container ID of the namenode
Create a HDFS directory /data//openbeer/breweries.

```
hdfs dfs -mkdir /data
hdfs dfs -mkdir /data/openbeer
hdfs dfs -mkdir /data/openbeer/breweries
hdfs dfs -mkdir -p /data/openbeer/breweries
```

Copy breweries.csv to HDFS:
Expand All @@ -67,7 +65,7 @@ Go to the command line of the Spark master and start PySpark.

Load breweries.csv from HDFS.
```
brewfile = spark.read.csv("hdfs://namenode:8020/data/openbeer/breweries/breweries.csv")
brewfile = spark.read.csv("hdfs://namenode:9000/data/openbeer/breweries/breweries.csv")
brewfile.show()
+----+--------------------+-------------+-----+---+
Expand Down Expand Up @@ -113,7 +111,7 @@ Go to the command line of the Spark master and start spark-shell.

Load breweries.csv from HDFS.
```
val df = spark.read.csv("hdfs://namenode:8020/data/openbeer/breweries/breweries.csv")
val df = spark.read.csv("hdfs://namenode:9000/data/openbeer/breweries/breweries.csv")
df.show()
+----+--------------------+-------------+-----+---+
Expand Down Expand Up @@ -167,7 +165,7 @@ tcp 0 0 0.0.0.0:10000 0.0.0.0:* LISTEN
Okay. Beeline is the command line interface with Hive. Let's connect to hiveserver2 now.

```
beeline
beeline -u jdbc:hive2://localhost:10000 -n root
!connect jdbc:hive2://127.0.0.1:10000 scott tiger
```
Expand Down
2 changes: 1 addition & 1 deletion hadoop-hive.env
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ HIVE_SITE_CONF_datanucleus_autoCreateSchema=false
HIVE_SITE_CONF_hive_metastore_uris=thrift://hive-metastore:9083
HDFS_CONF_dfs_namenode_datanode_registration_ip___hostname___check=false

CORE_CONF_fs_defaultFS=hdfs://namenode:8020
CORE_CONF_fs_defaultFS=hdfs://namenode:9000
CORE_CONF_hadoop_http_staticuser_user=root
CORE_CONF_hadoop_proxyuser_hue_hosts=*
CORE_CONF_hadoop_proxyuser_hue_groups=*
Expand Down

0 comments on commit 1a7a9a6

Please sign in to comment.