Skip to content

Commit

Permalink
first commit of CompreFace in one image
Browse files Browse the repository at this point in the history
  • Loading branch information
@pospielov
pospielov committed Nov 17, 2021
1 parent b4b4c0a commit 8bf2ca5
Show file tree
Hide file tree
Showing 4 changed files with 160 additions and 9 deletions.
3 changes: 1 addition & 2 deletions .dockerignore
View file
Original file line number Diff line number Diff line change
@@ -1,2 +1 @@
.git
/embedding-calculator/sample_images
.git
15 changes: 8 additions & 7 deletions custom-builds/README.md
View file
Original file line number Diff line number Diff line change
@@ -1,9 +1,10 @@
# List of custom-builds

| Custom-build | Base library | CPU | GPU | Face detection model / accuracy on [WIDER Face (Hard)](https://paperswithcode.com/sota/face-detection-on-wider-face-hard) | Face recognition model / accuracy on [LFW](https://paperswithcode.com/sota/face-verification-on-labeled-faces-in-the) | Age and gender detection | Face mask detection | Comment |
| -------------------------- | --------------------------------------------------------- | -------------------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | ------------------------------------------------ | ---------------------------------------------- |
| FaceNet (default) | [FaceNet](https://github.com/davidsandberg/facenet) | x86 (AVX instructions) | not supported | MTCNN / 80.9% | FaceNet (20180402-114759) / 99.63% | Custom, the model is taken [here](https://github.com/GilLevi/AgeGenderDeepLearning) | [Custom model](../docs/Mask-detection-plugin.md) | For general purposes. Support CPU without AVX2 |
| Mobilenet | [InsightFace](https://github.com/deepinsight/insightface) | x86 (AVX2 instructions) | not supported | RetinaFace-MobileNet0.25 / 82.5% | MobileFaceNet,ArcFace / 99.50% | InsightFace | [Custom model](../docs/Mask-detection-plugin.md) | The fastest model among CPU only models |
| Mobilenet-gpu | [InsightFace](https://github.com/deepinsight/insightface) | x86 (AVX2 instructions) | GPU (CUDA required) | RetinaFace-MobileNet0.25 / 82.5% | MobileFaceNet,ArcFace / 99.50% | InsightFace | [Custom model](../docs/Mask-detection-plugin.md) | The fastest model |
| SubCenter-ArcFace-r100 | [InsightFace](https://github.com/deepinsight/insightface) | x86 (AVX2 instructions) | not supported | retinaface_r50_v1 / 91.4% | arcface-r100-msfdrop75 / 99.80% | InsightFace | [Custom model](../docs/Mask-detection-plugin.md) | The most accurate model, but the most slow |
| SubCenter-ArcFace-r100-gpu | [InsightFace](https://github.com/deepinsight/insightface) | x86 (AVX2 instructions) | GPU (CUDA required) | retinaface_r50_v1 / 91.4% | arcface-r100-msfdrop75 / 99.80% | InsightFace | [Custom model](../docs/Mask-detection-plugin.md) | The most accurate model |
| Custom-build | Base library | CPU | GPU | Face detection model / accuracy on [WIDER Face (Hard)](https://paperswithcode.com/sota/face-detection-on-wider-face-hard) | Face recognition model / accuracy on [LFW](https://paperswithcode.com/sota/face-verification-on-labeled-faces-in-the) | Age and gender detection | Face mask detection | Comment |
|-------------------------------|-----------------------------------------------------------|-------------------------|---------------------|---------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------------------|--------------------------------------------------|------------------------------------------------|
| FaceNet (default) | [FaceNet](https://github.com/davidsandberg/facenet) | x86 (AVX instructions) | not supported | MTCNN / 80.9% | FaceNet (20180402-114759) / 99.63% | Custom, the model is taken [here](https://github.com/GilLevi/AgeGenderDeepLearning) | [Custom model](../docs/Mask-detection-plugin.md) | For general purposes. Support CPU without AVX2 |
| FaceNet Masked (Experimental) | [FaceNet](https://github.com/davidsandberg/facenet) | x86 (AVX instructions) | not supported | MTCNN / 80.9% | inception_v3_on_mafa_kaggle123 / 98.73% | Custom, the model is taken [here](https://github.com/GilLevi/AgeGenderDeepLearning) | [Custom model](../docs/Mask-detection-plugin.md) | For general purposes. Support CPU without AVX2 |
| Mobilenet | [InsightFace](https://github.com/deepinsight/insightface) | x86 (AVX2 instructions) | not supported | RetinaFace-MobileNet0.25 / 82.5% | MobileFaceNet,ArcFace / 99.50% | InsightFace | [Custom model](../docs/Mask-detection-plugin.md) | The fastest model among CPU only models |
| Mobilenet-gpu | [InsightFace](https://github.com/deepinsight/insightface) | x86 (AVX2 instructions) | GPU (CUDA required) | RetinaFace-MobileNet0.25 / 82.5% | MobileFaceNet,ArcFace / 99.50% | InsightFace | [Custom model](../docs/Mask-detection-plugin.md) | The fastest model |
| SubCenter-ArcFace-r100 | [InsightFace](https://github.com/deepinsight/insightface) | x86 (AVX2 instructions) | not supported | retinaface_r50_v1 / 91.4% | arcface-r100-msfdrop75 / 99.80% | InsightFace | [Custom model](../docs/Mask-detection-plugin.md) | The most accurate model, but the most slow |
| SubCenter-ArcFace-r100-gpu | [InsightFace](https://github.com/deepinsight/insightface) | x86 (AVX2 instructions) | GPU (CUDA required) | retinaface_r50_v1 / 91.4% | arcface-r100-msfdrop75 / 99.80% | InsightFace | [Custom model](../docs/Mask-detection-plugin.md) | The most accurate model |
112 changes: 112 additions & 0 deletions custom-builds/Single-Docker-File/FaceNet/Dockerfile
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,112 @@
################# build compreface-api and compreface-admin start ####################
FROM maven:3.6.3-jdk-11-slim as build
ARG ND4J_CLASSIFIER
WORKDIR /workspace/compreface
LABEL intermidiate_frs=true
COPY java/pom.xml .
COPY java/api/pom.xml api/
COPY java/admin/pom.xml admin/
COPY java/common/pom.xml common/
RUN mvn -B clean install -DskipTests -Dcheckstyle.skip -Dasciidoctor.skip -Djacoco.skip -Dmaven.gitcommitid.skip -Dspring-boot.repackage.skip -Dmaven.exec.skip=true -Dmaven.install.skip -Dmaven.resources.skip
COPY java/api api
COPY java/admin admin
COPY java/common common
RUN mvn package -Dmaven.test.skip=true -Dmaven.site.skip=true -Dmaven.javadoc.skip=true -Dnd4j.classifier=$ND4J_CLASSIFIER
################# build compreface-api and compreface-admin end ####################

ARG BASE_IMAGE
FROM ${BASE_IMAGE:-python:3.7-slim}

################# compreface-core start ####################
RUN apt-get update && apt-get install -y build-essential cmake git wget unzip \
curl yasm pkg-config libswscale-dev libtbb2 libtbb-dev libjpeg-dev \
libpng-dev libtiff-dev libavformat-dev libpq-dev libfreeimage3 mc \
&& rm -rf /var/lib/apt/lists/*

# install common python packages
SHELL ["/bin/bash", "-c"]
WORKDIR /app/ml
COPY embedding-calculator/requirements.txt .
RUN pip --no-cache-dir install -r requirements.txt

ARG BE_VERSION
ARG APP_VERSION_STRING
ENV BE_VERSION=$BE_VERSION
ENV APP_VERSION_STRING=$APP_VERSION_STRING
ENV HOME=/app/ml
ENV LC_ALL=C.UTF-8
ENV LANG=C.UTF-8
ENV PYTHONUNBUFFERED=0
ENV JOBLIB_MULTIPROCESSING=0

# download ML models
ARG INTEL_OPTIMIZATION=false
ARG GPU_IDX=-1
ENV GPU_IDX=$GPU_IDX INTEL_OPTIMIZATION=$INTEL_OPTIMIZATION
ARG FACE_DETECTION_PLUGIN="facenet.FaceDetector"
ARG CALCULATION_PLUGIN="facenet.Calculator"
ARG EXTRA_PLUGINS="facenet.LandmarksDetector,agegender.AgeDetector,agegender.GenderDetector,facenet.facemask.MaskDetector"
ENV FACE_DETECTION_PLUGIN=$FACE_DETECTION_PLUGIN CALCULATION_PLUGIN=$CALCULATION_PLUGIN \
EXTRA_PLUGINS=$EXTRA_PLUGINS
COPY embedding-calculator/src src
COPY embedding-calculator/srcext srcext
RUN python -m src.services.facescan.plugins.setup

# copy rest of the code
COPY embedding-calculator/tools tools
COPY embedding-calculator/sample_images sample_images

# run tests
ARG SKIP_TESTS
COPY embedding-calculator/pytest.ini .
RUN if [ -z $SKIP_TESTS ]; then pytest -m "not performance" /app/ml/src; fi

ENV ML_PORT=3000
EXPOSE 3000

COPY embedding-calculator/uwsgi.ini .
ENV UWSGI_PROCESSES=${UWSGI_PROCESSES:-2}
ENV UWSGI_THREADS=1
################# compreface-core end ####################

################# compreface-postgres-db start ####################
ENV POSTGRES_USER=compreface
ENV POSTGRES_PASSWORD=M7yfTsBscdqvZs49
ENV POSTGRES_DB=frs

RUN apt-get update && apt-get install -y postgresql-13 \
&& rm -rf /var/lib/apt/lists/*

USER postgres

RUN /etc/init.d/postgresql start && \
psql --command "CREATE USER ${POSTGRES_USER} WITH SUPERUSER PASSWORD '${POSTGRES_PASSWORD}';" &&\
createdb -O ${POSTGRES_USER} ${POSTGRES_DB}

USER root

################# compreface-postgres-db end ####################

################# compreface-admin start ####################

ENV POSTGRES_URL=jdbc:postgresql://localhost:5432/${POSTGRES_DB}
ENV SPRING_PROFILES_ACTIVE=dev
ENV ENABLE_EMAIL_SERVER=false
ENV ADMIN_JAVA_OPTS=-Xmx1g

RUN apt-get update && apt-get install -y openjdk-11-jre-headless \
&& rm -rf /var/lib/apt/lists/*

ARG DIR=/workspace/compreface
COPY --from=build ${DIR}/admin/target/*.jar /java/admin/app.jar
ARG APPERY_API_KEY
ENV APPERY_API_KEY ${APPERY_API_KEY}

################# compreface-admin end ####################

################# supervisord ####################
RUN apt-get update && apt-get install -y supervisor && rm -rf /var/lib/apt/lists/*
RUN mkdir -p /var/log/supervisor
COPY custom-builds/Single-Docker-File/FaceNet/supervisord.conf /etc/supervisor/conf.d/supervisord.conf

CMD ["/usr/bin/supervisord"]
39 changes: 39 additions & 0 deletions custom-builds/Single-Docker-File/FaceNet/supervisord.conf
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
[supervisord]
nodaemon=true

[program:compreface-postgres-db]
command=/etc/init.d/postgresql start
user=postgres
priority=1
killasgroup=true
stopasgroup=true
autostart=true
autorestart=true
stdout_logfile=/dev/fd/1
stdout_logfile_maxbytes=0
redirect_stderr=true

[program:compreface-core]
command=uwsgi --ini uwsgi.ini
directory=/app/ml
priority=2
killasgroup=true
stopasgroup=true
autostart=true
autorestart=true
stdout_logfile=/dev/fd/2
stdout_logfile_maxbytes=0
redirect_stderr=true

[program:compreface-admin]
command=sh -c java $ADMIN_JAVA_OPTS -jar /java/admin/app.jar
directory=/java/admin
priority=2
killasgroup=true
stopasgroup=true
autostart=true
autorestart=true
stdout_logfile=/dev/fd/3
stdout_logfile_maxbytes=0
redirect_stderr=true

0 comments on commit 8bf2ca5

Please sign in to comment.