Skip to content

Commit

Permalink
The .pypirc file is read from docker-context-files (apache#11779)
Browse files Browse the repository at this point in the history
If you used context from git repo, the .piprc file was missing and
COPY in Dockerfile is not conditional.

This change copies the .pypirc conditionally from the
docker-context-files folder instead.

Also it was needlessly copied in the main image where it is not
needed and it was even dangerous to do so.
  • Loading branch information
potiuk authored Oct 23, 2020
1 parent cc9c4c6 commit 53e5d8f
Show file tree
Hide file tree
Showing 4 changed files with 6 additions and 14 deletions.
1 change: 0 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,6 @@
!NOTICE
!.github
!empty
!.pypirc

# This folder is for you if you want to add any files to the docker context when you build your own
# docker image. most of other files and any new folder you add will be excluded by default
Expand Down
6 changes: 3 additions & 3 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -164,7 +164,9 @@ RUN mkdir -p /root/.local/bin
ARG AIRFLOW_PRE_CACHED_PIP_PACKAGES="true"
ENV AIRFLOW_PRE_CACHED_PIP_PACKAGES=${AIRFLOW_PRE_CACHED_PIP_PACKAGES}

COPY .pypirc /root/.pypirc
RUN if [[ -f /docker-context-files/.pypirc ]]; then \
cp /docker-context-files/.pypirc /root/.pypirc; \
fi

# In case of Production build image segment we want to pre-install master version of airflow
# dependencies from GitHub so that we do not have to always reinstall it from the scratch.
Expand Down Expand Up @@ -396,8 +398,6 @@ RUN chmod a+x /entrypoint /clean-logs
# See https://github.com/apache/airflow/issues/9248
RUN chmod g=u /etc/passwd

COPY .pypirc ${AIRFLOW_USER_HOME_DIR}/.pypirc

ENV PATH="${AIRFLOW_USER_HOME_DIR}/.local/bin:${PATH}"
ENV GUNICORN_CMD_ARGS="--worker-tmp-dir /dev/shm"

Expand Down
7 changes: 3 additions & 4 deletions docs/production-deployment.rst
Original file line number Diff line number Diff line change
Expand Up @@ -265,10 +265,9 @@ the resulting image using ``FROM`` any dependencies you want.
Customizing PYPI installation
.............................

You can customize PYPI sources used during image build by modifying .pypirc file that should be
placed in the root of Airflow Directory. This .pypirc will never be committed to the repository
and will not be present in the final production image. It is added and used only in the build
segment of the image so it is never copied to the final image.
You can customize PYPI sources used during image build by adding a docker-context-files/.pypirc file
This .pypirc will never be committed to the repository and will not be present in the final production image.
It is added and used only in the build segment of the image so it is never copied to the final image.

External sources for dependencies
---------------------------------
Expand Down
6 changes: 0 additions & 6 deletions scripts/ci/libraries/_initialization.sh
Original file line number Diff line number Diff line change
Expand Up @@ -34,12 +34,6 @@ function initialization::create_directories() {
export FILES_DIR="${AIRFLOW_SOURCES}/files"
readonly FILES_DIR

# Create an empty .pypirc file that you can customise. It is .gitignored so it will never
# land in the repository - it is only added to the "build image" of production image
# So you can keep your credentials safe as long as you do not push the build image.
# The final image does not contain it.
touch "${AIRFLOW_SOURCES}/.pypirc"

# Directory where all the build cache is stored - we keep there status of all the docker images
# As well as hashes of the important files, but also we generate build scripts there that are
# Used to execute the commands for breeze
Expand Down

0 comments on commit 53e5d8f

Please sign in to comment.