Skip to content

Commit

Permalink
[SPARK-50609][INFRA] Eliminate warnings in docker image building
Browse files Browse the repository at this point in the history 
### What changes were proposed in this pull request?
The pr aims to eliminate warnings in docker image building
```shell
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 75)
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 27)
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 29)
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 30)
```

### Why are the changes needed?
- Eliminate warnings
- Alignment of environment variable declarations for other `dockerfile` files in the code repository
https://github.com/apache/spark/blob/229118ca7a127753635543909efdb27601985d42/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/Dockerfile#L63
https://github.com/apache/spark/blob/229118ca7a127753635543909efdb27601985d42/resource-managers/kubernetes/docker/src/main/dockerfiles/spark/bindings/R/Dockerfile#L40

### Does this PR introduce _any_ user-facing change?
No.

### How was this patch tested?
- Pass GA.
- Manually check.

### Was this patch authored or co-authored using generative AI tooling?
No.

Closes apache#49224 from panbingkun/SPARK-50609.

Authored-by: panbingkun <[email protected]>
Signed-off-by: Max Gekk <[email protected]>
  • Loading branch information
@panbingkun @MaxGekk
panbingkun authored and MaxGekk committed Dec 18, 2024
1 parent 0dd90d9 commit 788fa5a
Show file tree
Hide file tree
Showing 12 changed files with 40 additions and 40 deletions.
4 changes: 2 additions & 2 deletions binder/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,8 @@ RUN pip install --no-cache notebook jupyterlab
# create user with a home directory
ARG NB_USER
ARG NB_UID
ENV USER ${NB_USER}
ENV HOME /home/${NB_USER}
ENV USER=${NB_USER}
ENV HOME=/home/${NB_USER}

RUN adduser --disabled-password \
--gecos "Default user" \
Expand Down
8 changes: 4 additions & 4 deletions dev/create-release/spark-rm/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,10 @@ LABEL org.opencontainers.image.ref.name="Apache Spark Release Manager Image"
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""

ENV FULL_REFRESH_DATE 20240318
ENV FULL_REFRESH_DATE=20240318

ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true

RUN apt-get update && apt-get install -y \
build-essential \
Expand Down Expand Up @@ -88,7 +88,7 @@ RUN Rscript -e "install.packages(c('devtools', 'knitr', 'markdown', \
Rscript -e "devtools::install_version('preferably', version='0.4', repos='https://cloud.r-project.org')"

# See more in SPARK-39735
ENV R_LIBS_SITE "/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library"
ENV R_LIBS_SITE="/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library"


RUN add-apt-repository ppa:pypy/ppa
Expand Down
8 changes: 4 additions & 4 deletions dev/infra/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image"
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""

ENV FULL_REFRESH_DATE 20241119
ENV FULL_REFRESH_DATE=20241119

ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true

RUN apt-get update && apt-get install -y \
build-essential \
Expand Down Expand Up @@ -82,7 +82,7 @@ RUN Rscript -e "install.packages(c('devtools', 'knitr', 'markdown', \
Rscript -e "devtools::install_version('preferably', version='0.4', repos='https://cloud.r-project.org')"

# See more in SPARK-39735
ENV R_LIBS_SITE "/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library"
ENV R_LIBS_SITE="/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library"


RUN add-apt-repository ppa:pypy/ppa
Expand Down
8 changes: 4 additions & 4 deletions dev/spark-test-image/docs/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image for Documentat
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""

ENV FULL_REFRESH_DATE 20241029
ENV FULL_REFRESH_DATE=20241029

ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true

RUN apt-get update && apt-get install -y \
build-essential \
Expand Down Expand Up @@ -72,7 +72,7 @@ RUN Rscript -e "install.packages(c('devtools', 'knitr', 'markdown', 'rmarkdown',
Rscript -e "devtools::install_version('preferably', version='0.4', repos='https://cloud.r-project.org')"

# See more in SPARK-39735
ENV R_LIBS_SITE "/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library"
ENV R_LIBS_SITE="/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library"

# Install Python 3.9
RUN add-apt-repository ppa:deadsnakes/ppa
Expand Down
8 changes: 4 additions & 4 deletions dev/spark-test-image/lint/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image for Linter"
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""

ENV FULL_REFRESH_DATE 20241112
ENV FULL_REFRESH_DATE=20241112

ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true

RUN apt-get update && apt-get install -y \
build-essential \
Expand Down Expand Up @@ -63,7 +63,7 @@ RUN Rscript -e "install.packages(c('devtools', 'knitr', 'markdown', 'rmarkdown',
&& Rscript -e "devtools::install_version('lintr', version='2.0.1', repos='https://cloud.r-project.org')" \

# See more in SPARK-39735
ENV R_LIBS_SITE "/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library"
ENV R_LIBS_SITE="/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library"

# Install Python 3.9
RUN add-apt-repository ppa:deadsnakes/ppa
Expand Down
6 changes: 3 additions & 3 deletions dev/spark-test-image/pypy-310/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image For PySpark wi
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""

ENV FULL_REFRESH_DATE 20241212
ENV FULL_REFRESH_DATE=20241212

ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true

RUN apt-get update && apt-get install -y \
build-essential \
Expand Down
6 changes: 3 additions & 3 deletions dev/spark-test-image/python-309/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image For PySpark wi
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""

ENV FULL_REFRESH_DATE 20241205
ENV FULL_REFRESH_DATE=20241205

ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true

RUN apt-get update && apt-get install -y \
build-essential \
Expand Down
6 changes: 3 additions & 3 deletions dev/spark-test-image/python-310/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image For PySpark wi
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""

ENV FULL_REFRESH_DATE 20241205
ENV FULL_REFRESH_DATE=20241205

ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true

RUN apt-get update && apt-get install -y \
build-essential \
Expand Down
6 changes: 3 additions & 3 deletions dev/spark-test-image/python-311/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image For PySpark wi
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""

ENV FULL_REFRESH_DATE 20241212
ENV FULL_REFRESH_DATE=20241212

ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true

RUN apt-get update && apt-get install -y \
build-essential \
Expand Down
6 changes: 3 additions & 3 deletions dev/spark-test-image/python-312/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image For PySpark wi
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""

ENV FULL_REFRESH_DATE 20241206
ENV FULL_REFRESH_DATE=20241206

ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true

RUN apt-get update && apt-get install -y \
build-essential \
Expand Down
6 changes: 3 additions & 3 deletions dev/spark-test-image/python-313/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image For PySpark wi
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""

ENV FULL_REFRESH_DATE 20241210
ENV FULL_REFRESH_DATE=20241210

ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true

RUN apt-get update && apt-get install -y \
build-essential \
Expand Down
8 changes: 4 additions & 4 deletions dev/spark-test-image/sparkr/Dockerfile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -24,10 +24,10 @@ LABEL org.opencontainers.image.ref.name="Apache Spark Infra Image for SparkR"
# Overwrite this label to avoid exposing the underlying Ubuntu OS version label
LABEL org.opencontainers.image.version=""

ENV FULL_REFRESH_DATE 20241114
ENV FULL_REFRESH_DATE=20241114

ENV DEBIAN_FRONTEND noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN true
ENV DEBIAN_FRONTEND=noninteractive
ENV DEBCONF_NONINTERACTIVE_SEEN=true

RUN apt-get update && apt-get install -y \
build-essential \
Expand Down Expand Up @@ -74,4 +74,4 @@ RUN Rscript -e "install.packages(c('devtools', 'knitr', 'markdown', \
Rscript -e "devtools::install_version('preferably', version='0.4', repos='https://cloud.r-project.org')"

# See more in SPARK-39735
ENV R_LIBS_SITE "/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library"
ENV R_LIBS_SITE="/usr/local/lib/R/site-library:${R_LIBS_SITE}:/usr/lib/R/library"

0 comments on commit 788fa5a

Please sign in to comment.