Skip to content

Commit

Permalink
Update PYTHONPATH (databricks#51)
Browse files Browse the repository at this point in the history
* Update Dockerfile

* Remove redudant SPARK_HOME
  • Loading branch information
Fokko authored Jan 24, 2023
1 parent ec91669 commit 0512a8c
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion spark/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ RUN curl https://github.com/SpencerPark/IJava/releases/download/v1.3.0/ijava-1.3
# Optional env variables
ENV SPARK_HOME=${SPARK_HOME:-"/opt/spark"}
ENV HADOOP_HOME=${HADOOP_HOME:-"/opt/hadoop"}
ENV PYTHONPATH=$SPARK_HOME/python:$SPARK_HOME/python/lib/py4j-0.10.9.5-src.zip:$PYTHONPATH

RUN mkdir -p ${HADOOP_HOME} && mkdir -p ${SPARK_HOME}
WORKDIR ${SPARK_HOME}
Expand Down Expand Up @@ -110,7 +111,6 @@ COPY ipython/startup/README /root/.ipython/profile_default/startup

COPY spark-defaults.conf /opt/spark/conf
ENV PATH="/opt/spark/sbin:/opt/spark/bin:${PATH}"
ENV SPARK_HOME="/opt/spark"

RUN chmod u+x /opt/spark/sbin/* && \
chmod u+x /opt/spark/bin/*
Expand Down

0 comments on commit 0512a8c

Please sign in to comment.