Skip to content

Commit

Permalink
Remove python2 from the pulsar images build (apache#7936)
Browse files Browse the repository at this point in the history
The pulsar docker and pulsar standalone docker image build failed by:
```
Processing ./pulsar-client/pulsar_client-2.6.0-cp27-cp27mu-manylinux1_x86_64.whl
2020-08-25T02:23:28.9567844Z [INFO] Collecting fastavro
2020-08-25T02:23:29.0080316Z [INFO]   Downloading fastavro-1.0.0.tar.gz (658 kB)
2020-08-25T02:23:29.3108074Z [INFO]     ERROR: Command errored out with exit status 1:
2020-08-25T02:23:29.3109489Z [INFO]      command: /usr/bin/python2.7 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-jvZueR/fastavro/setup.py'"'"'; __file__='"'"'/tmp/pip-install-jvZueR/fastavro/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' egg_info --egg-base /tmp/pip-pip-egg-info-iH1mMy
2020-08-25T02:23:29.3110061Z [INFO]          cwd: /tmp/pip-install-jvZueR/fastavro/
2020-08-25T02:23:29.3110303Z [INFO]     Complete output (5 lines):
2020-08-25T02:23:29.3110520Z [INFO]     Traceback (most recent call last):
2020-08-25T02:23:29.3110741Z [INFO]       File "<string>", line 1, in <module>
2020-08-25T02:23:29.3111190Z [INFO]       File "/tmp/pip-install-jvZueR/fastavro/setup.py", line 58, in <module>
2020-08-25T02:23:29.3111437Z [INFO]         if sys.implementation.name != "pypy":
2020-08-25T02:23:29.3111886Z [INFO]     AttributeError: 'module' object has no attribute 'implementation'
```

I remove the python2.7 and using the python3 to build images.

* Remove python2 in images

* Remove python2 stuff

* Update the default python version
  • Loading branch information
zymap authored Sep 8, 2020
1 parent 98233c0 commit ed356ad
Show file tree
Hide file tree
Showing 4 changed files with 13 additions and 25 deletions.
2 changes: 1 addition & 1 deletion dashboard/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ RUN bash -c "echo deb http://apt.postgresql.org/pub/repos/apt/ stretch-pgdg main
RUN wget --quiet -O - https://www.postgresql.org/media/keys/ACCC4CF8.asc | apt-key add -

RUN apt-get update
RUN apt-get -y install postgresql-11 postgresql-contrib libpq-dev python sudo nginx supervisor
RUN apt-get -y install postgresql-11 postgresql-contrib libpq-dev sudo nginx supervisor

# Postgres configuration
COPY conf/postgresql.conf /etc/postgresql/11/main/
Expand Down
14 changes: 10 additions & 4 deletions docker/pulsar-standalone/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,10 @@ FROM openjdk:8-jdk
# Note that the libpq-dev package is needed here in order to install
# the required python psycopg2 package (for postgresql) later
RUN apt-get update \
&& apt-get -y install python2.7 python2.7-dev python-pip postgresql sudo nginx supervisor libpq-dev
&& apt-get -y install python3.7 python3.7-dev python3-pip postgresql sudo nginx supervisor libpq-dev

RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 10
RUN ln -s /usr/bin/pip3 /usr/bin/pip

# Postgres configuration
COPY --from=dashboard /etc/postgresql/11/main/postgresql.conf /etc/postgresql/11/main/postgresql.conf
Expand Down Expand Up @@ -59,15 +62,17 @@ COPY --from=dashboard /pulsar/django /pulsar/django
COPY --from=dashboard /pulsar/requirements.txt /pulsar/django
COPY --from=dashboard /pulsar/conf/* /pulsar/conf/

RUN pip -V

# Python dependencies
RUN pip2 install -r /pulsar/django/requirements.txt
RUN pip install -r /pulsar/django/requirements.txt

# Copy web-app sources
# Setup database and create tables
COPY --from=dashboard /pulsar/init-postgres.sh /pulsar/django/init-postgres.sh
RUN mkdir /data
RUN /pulsar/django/init-postgres.sh
RUN sudo -u postgres /etc/init.d/postgresql stop
RUN sudo -u postgres /etc/init.d/postgresql stop
# Add postgresql to supervisord. Redirect logs to stdout
RUN echo "\n[program:postgresql]\n\
command = /etc/init.d/postgresql start\n\
Expand All @@ -78,9 +83,10 @@ redirect_stderr=true" >> /etc/supervisor/conf.d/supervisor-app.conf
# Collect all static files needed by Django in a
# single place. Needed to run the app outside the
# Django test web server
RUN cd /pulsar/django && ./manage.py collectstatic --no-input
RUN cd /pulsar/django && python manage.py collectstatic --no-input

ENV SERVICE_URL http://127.0.0.1:8080
EXPOSE 80 8080 6650

CMD ["supervisord", "-n"]

6 changes: 2 additions & 4 deletions docker/pulsar/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,6 @@ COPY scripts/generate-zookeeper-config.sh /pulsar/bin
COPY scripts/pulsar-zookeeper-ruok.sh /pulsar/bin
COPY scripts/watch-znode.py /pulsar/bin
COPY scripts/set_python_version.sh /pulsar/bin
COPY scripts/install-pulsar-client-27.sh /pulsar/bin
COPY scripts/install-pulsar-client-37.sh /pulsar/bin


Expand All @@ -43,17 +42,17 @@ FROM openjdk:8-jdk-slim
# Install some utilities
RUN apt-get update \
&& apt-get install -y netcat dnsutils less procps iputils-ping \
python2.7 python2.7-dev python-setuptools python-yaml python-kazoo \
python3.7 python3.7-dev python3-setuptools python3-yaml python3-kazoo \
libreadline-gplv2-dev libncursesw5-dev libssl-dev libsqlite3-dev tk-dev libgdbm-dev libc6-dev libbz2-dev \
curl \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*

RUN curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
RUN python2.7 get-pip.py
RUN python3.7 get-pip.py

RUN update-alternatives --install /usr/bin/python python /usr/bin/python3 10

ADD target/python-client/ /pulsar/pulsar-client
ADD target/cpp-client/ /pulsar/cpp-client
RUN echo networkaddress.cache.ttl=1 >> $JAVA_HOME/jre/lib/security/java.security
Expand All @@ -70,6 +69,5 @@ ENV PULSAR_ROOT_LOGGER=INFO,CONSOLE
COPY --from=pulsar /pulsar /pulsar
WORKDIR /pulsar

RUN /pulsar/bin/install-pulsar-client-27.sh
RUN /pulsar/bin/install-pulsar-client-37.sh

16 changes: 0 additions & 16 deletions docker/pulsar/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -52,22 +52,6 @@
<artifactId>exec-maven-plugin</artifactId>
<version>${exec-maven-plugin.version}</version>
<executions>
<execution>
<id>build-pulsar-clients-python-27</id>
<phase>compile</phase>
<goals>
<goal>exec</goal>
</goals>
<configuration>
<skip>${skipBuildPythonClient}</skip>
<workingDirectory>${project.basedir}/target</workingDirectory>
<executable>${project.basedir}/../../pulsar-client-cpp/docker/build-wheels.sh</executable>
<arguments>
<!-- build python 2.7 -->
<argument>2.7 cp27-cp27mu</argument>
</arguments>
</configuration>
</execution>
<execution>
<id>build-pulsar-clients-python-35</id>
<phase>compile</phase>
Expand Down

0 comments on commit ed356ad

Please sign in to comment.