Install Docker. You should manage Docker as a non-root user.
Install Docker Desktop. It is available for Mac with Intel, and Mac with Apple silicon.
There are two different localsettings configurations, depending on whether HQ is running inside a Docker container or on your local machine. If you are planning on doing local development, it is recommended to run HQ on your local machine, and use Docker only for supporting services.
If you want to run the HQ web worker, Celery and Pillowtop in Docker, see :ref:`running-services-and-hq`.
If you have any HQ services currently running (Couch, Postgres, Redis, etc.), you should stop them now.
Run
$ scripts/docker up -d postgres couch redis elasticsearch5 zookeeper kafka minio
to build and start those Docker services in the background. (Omit -d
to run them in the foreground.)
Once the services are all up (./scripts/docker ps to check) you can
return to the CommCare HQ DEV_SETUP and set up your Django environment.
This setup is useful for running a CommCare HQ test environment, for example, if you want to build an integration with CommCare.
This setup is not recommended for local development, since you'll typically want more direct access to the Django process.
Note
This setup is also not appropriate for production use because it runs a development web server, a development Celery worker, and the PostgreSQL database is not optimized for production.
Clone commcare-hq, if you have not done so already:
$ git clone https://github.com/dimagi/commcare-hq.git $ cd commcare-hq $ git submodule update --init --recursive
Build a Docker image using
Dockerfile_incl, and tag it "commcare_incl":$ docker build -f Dockerfile_incl -t commcarehq_incl .
Create and run service and HQ containers:
$ scripts/docker runserver
This will do the following:
- Build required images
- Run all the service containers
- Migrate the DB and sync the Couch views
- Bootstrap a superuser and domain if a superuser does not already exist:
- username: [email protected]
- password: Passw0rd!
- domain: demo
- Run the Django dev server
You should be able to log into CommCare at http://localhost:8000 using the login details above.
On Mac, run docker-machine ip to get the VM's IP address, which replaces localhost in the URL.
You can create another user and domain with $ ./manage.py make_superuser <email>
$ ./scripts/docker --help
$ ./scripts/docker start $ ./scripts/docker stop $ ./scripts/docker logs postgres
The following services are included. Their ports are mapped to localhost so you can connect to them directly.
- Elasticsearch (9200 & 9300)
- PostgreSQL (5432)
- CouchDB (5984)
- Redis (6397)
- Zookeeper (2181)
- Kafka (9092)
- MinIO (9980)
$ ./scripts/docker runserver
Your Docker data gets mounted based on the DOCKER_DATA_HOME variable.
By default on nix systems this will be **~/.local/share/dockerhq/* so if you need to manually manipulate data in your Docker volumes this is the place to do it.
Note
You can destabilize your system if you manually edit data in this directory, so do so with care!
Travis also uses Docker to run the HQ test suite. To simulate the Travis build you can use the scripts/docker script.
Run Python tests:
$ JS_SETUP=yes ./scripts/docker test
Run the JavaScript tests:
$ TEST=javascript ./scripts/docker test
Run the Python sharded tests:
$ TEST=python-sharded ./scripts/docker test
Run only
corehq.apps.app_manager.tests.test_suite.SuiteTest:$ ./scripts/docker test corehq/apps/app_manager/tests/test_suite.py:SuiteTest
Drop into a bash shell in the Docker web container from where you can run any other commands:
$ ./scripts/docker bash
Remove all test containers and volumes:
$ ./scripts/docker hqtest teardown
- JS_SETUP=[ yes | no ]
- Run
yarninstalls. (Default: "no") - TEST=[ javascript | python | python-sharded | python-sharded-and-javascript ]
javascript: Extra setup and config for JS tests. Also only run JS testspython[default]: Run default testspython-elasticsearch-v5: Configure Django for ES5 testspython-sharded: Configure Django for sharded setup and only run subset of testspython-sharded-and-javascript: Combinespython-shardedandjavascript. Also sends static analysis to Datadog if a job is a Travis "cron" event.
- DIVIDED_WE_RUN
- Only runs a subset of tests. See the ``pytest dividedwerun plugin``_ for exact options.
- REUSE_DB
- Same as normal
REUSE_DB - DOCKER_HQ_OVERLAY=[ none | overlayfs | aufs ]
none: Mounts the commcare-hq directory read/write in the Docker container for direct access. This is the default when running in Travis.overlayfs: Mounts the commcare-hq directory read-only in the Docker container and uses it as the "lowerdir" in anoverlayfsmount to insulate the host OS data from being modified by the container.aufs: [deprecated] Same behavior asoverlayfs, only using Docker'saufsoverlay engine instead ofoverlayfs. This is the default when not running in Travis.
- DOCKER_HQ_OVERLAYFS_CHMOD=[ yes | no ]
- Perform a recursive chmod on the commcare-hq overlay to ensure read access for cchq user. (Default: "no")
- DOCKER_HQ_OVERLAYFS_METACOPY=[ on | off ]
- Set the
metacopy=onmount option for the overlayfs mount (performance optimization, has security implications). (Default: "off")
See .travis.yml for environment variable options used on Travis.
Podman 4.3 or later can be used to run HQ containers. Unlike docker, podman is daemonless and runs containers in rootless mode by default. Podman 4.x is available on recent versions of Ubuntu. Older versions, such as Ubuntu 22.04, require a third-party package repository.
sudo apt install podman podman-docker
echo 'export DOCKER_HOST=unix://$XDG_RUNTIME_DIR/podman/podman.sock' >> ~/.bashrc
echo 'export DOCKER_SOCK=$XDG_RUNTIME_DIR/podman/podman.sock' >> ~/.bashrcCreate a podman wrapper script named docker with the following content
somewhere on your PATH (~/.local/bin/docker may be a good place if it
is on your PATH).
#! /usr/bin/bash
if [[ "$1" == compose ]]; then
shift
/usr/bin/docker-compose "$@" # v1, installed by podman-docker
else
podman "$@"
fi./scripts/docker up -d