Skip to content

Latest commit

 

History

History

docker

README.md

CommCare HQ docker

Initial setup

  • Linux

  • OS X

    • Preferred: install Docker for Mac.
    • Alternate for older Macs that do not support Docker for Mac:
      • Install Docker Toolbox. Go through the full tutorial, which will create a default machine.
      • To create a new VM manually, run docker-machine create default --driver=virtualbox (not necessary if you followed the Docker Toolbox tutorial).
      • If not using the Quick Start terminal, run eval $(docker-machine env default) to set up Docker's environment variables.

  • If you have any HQ services currently running (couch, postgres, redis, etc.), you should stop them now.

There are two different localsettings configurations, depending on whether HQ is running inside a docker container or on your local machine. If you are planning on doing local development, it is recommended to run HQ on your local machine, and use docker only for supporting services

Run only services in docker

This is the recommended setup for local development. If you want to run the server process in docker, see below.

  • If you are using Docker Toolbox (not Docker for Mac): change all service host settings (DATABASES HOST, COUCH_SERVER_ROOT, etc.) in your localsettings.py file to point to the IP address of your virtualbox docker VM.
  • Run ./scripts/docker up -d postgres couch redis elasticsearch zookeeper kafka minio to build and start those docker services in the background.
  • Once the services are all up (./scripts/docker ps to check) you can return to the CommCare HQ DEV_SETUP and Setup your Django environment.

Run services and HQ in docker

This setup is not recommended for local development, since you'll typically want more direct access to the django process.

Bootstrap the setup.

$ ./scripts/docker runserver --bootstrap

This will do the following:

  • build all the images (if not already built)
  • run all the service containers
  • migrate the DB and sync the Couch views
  • bootstrap a superuser and domain:
  • run the Django dev server

If all goes according to plan you should be able to log into CommCare: http://localhost:8000 using the login details above.

You can create another user and domain with $ ./manage.py make_superuser <email>

On Mac, run docker-machine ip to get the VM's IP address, which replaces localhost in the URL.

General usage

  $ ./scripts/docker --help

The services (couch, postgres, elastic, redis, zookeeper, kafka)

  $ ./scripts/docker start
  $ ./scripts/docker stop
  $ ./scripts/docker logs postgres

The following services are included. Their ports are mapped to the local host so you can connect to them directly.

  • Elasticsearch (9200 & 9300)
  • PostgreSQL (5432)
  • CouchDB (5984)
  • Redis (6397)
  • Zookeeper (2181)
  • Kafka (9092)
  • Riak CS (9980)

Run the django server

  $ ./scripts/docker runserver

Caveats

  • CloudCare is not currently part of this set up. It should probably be another docker image, different from CommCareHQ.
  • Celery, rabbitmq and other components not strictly necessary for a laptop install are not part of this setup.

Your Data

Your docker data gets mounted based on the DOCKER_DATA_HOME variable.

By default on *nix systems this will be ~/.local/share/dockerhq/ - so if you need to manually manipulate data in your Docker volumes this is the place to do it.

Note that you can destabilize your system if you manually edit data in this directory, so do so with care!

Travis

Travis also uses Docker to run the HQ test suite. To simulate the travis build you can use the ./scripts/docker script:

  $ JS_SETUP=yes ./scripts/docker test
  runs python tests

  $ TEST=javascript ./scripts/docker test
  runs the javascript tests

  $ TEST=python-sharded ./scripts/docker test
  runs the python sharded tests
  
  $ ./scripts/docker test corehq/apps/app_manager/tests/test_suite.py:SuiteTest
  runs only the corehq.apps.app_manager.tests.test_suite.SuiteTest
  
  $ ./scripts/docker bash
  drops you into a bash shell in the docker web container from where you can
  run any other commands
  
  $ ./scripts/docker hqtest teardown
  remove all test containers and volumes

ENV Vars

  • JS_SETUP=[yes|no]
    • Run yarn installs
  • TEST=[javascript|python|python-sharded|python-sharded-and-javascript]
    • javascript: extra setup and config for JS tests. Also only run JS tests
    • python: default tests
    • python-elasticsearch-v5: configure django for ES5 tests
    • python-sharded: configure django for sharded setup and only run subset of tests
    • python-sharded-and-javascript: combines python-sharded and javascript. Also sends static analysis to datadog if job is travis "cron" event.
  • NOSE_DIVIDED_WE_RUN
    • used to only run a subset of tests
    • see .travis.yml for exact options
  • REUSE_DB
    • Same as normal REUSE_DB
  • DOCKER_HQ_OVERLAY=[none|overlayfs|aufs]
    • none: mounts commcare-hq directory in docker container read/write for direct access. This is the default when running in Travis.
    • overlayfs: mounts commcare-hq directory in docker container read-only and uses it as the "lowerdir" in an overlayfs mount to insulate the host OS data from being modified by the container.
    • aufs: [deprecated] same behavior as overlayfs, only using Docker's aufs overlay engine instead of overlayfs. This is the default when not running in Travis.
  • DOCKER_HQ_OVERLAYFS_CHMOD=[yes|no]
    • Perform a recursive chmod on the commcare-hq overlay to ensure read access for cchq user.
  • DOCKER_HQ_OVERLAYFS_METACOPY=[on|off]
    • Set the metacopy=on mount option for the overlayfs mount (performance optimization, has security implications).

See .travis.yml for env variable options used on travis.