[AIRFLOW-2547] Describe how to run tests using Docker

For Airflow it can be a bit tricky to test the
code. Add a description
on how to build a clean enviroment from scratch

Closes apache#3445 from Fokko/fd-local-devel-environment

CONTRIBUTING.md

@@ -79,7 +79,44 @@ extras to build the full API reference.
 
 ## Development and Testing
 
-### Setting up a development environment
+### Set up a development env using Docker
+
+Go to your Airflow directory and start a new docker container. You can choose between Python 2 or 3, whatever you prefer.
+
+```
+# Start docker in your Airflow directory
+docker run -t -i -v `pwd`:/airflow/ python:2 bash
+
+# Go to the Airflow directory
+cd /airflow/
+
+# Install Airflow with all the required dependencies,
+# including the devel which will provide the development tools
+pip install -e ".[hdfs,hive,druid,devel]"
+
+# Init the database
+airflow initdb
+
+nosetests -v tests/hooks/test_druid_hook.py
+
+  test_get_first_record (tests.hooks.test_druid_hook.TestDruidDbApiHook) ... ok
+  test_get_records (tests.hooks.test_druid_hook.TestDruidDbApiHook) ... ok
+  test_get_uri (tests.hooks.test_druid_hook.TestDruidDbApiHook) ... ok
+  test_get_conn_url (tests.hooks.test_druid_hook.TestDruidHook) ... ok
+  test_submit_gone_wrong (tests.hooks.test_druid_hook.TestDruidHook) ... ok
+  test_submit_ok (tests.hooks.test_druid_hook.TestDruidHook) ... ok
+  test_submit_timeout (tests.hooks.test_druid_hook.TestDruidHook) ... ok
+  test_submit_unknown_response (tests.hooks.test_druid_hook.TestDruidHook) ... ok
+
+  ----------------------------------------------------------------------
+  Ran 8 tests in 3.036s
+
+  OK
+```
+
+The Airflow code is mounted inside of the Docker container, so if you change something using your favorite IDE, you can directly test is in the container.
+
+### Set up a development env using Virtualenv
 
 Please install python(2.7.x or 3.4.x), mysql, and libxml by using system-level package
 managers like yum, apt-get for Linux, or homebrew for Mac OS at first.

setup.py

@@ -157,7 +157,6 @@ def write_version(filename=os.path.join(*['airflow',
     'pyhive>=0.1.3',
     'impyla>=0.13.3',
     'thrift_sasl==0.2.1',
-    'unicodecsv>=0.14.1'
 ]
 jdbc = ['jaydebeapi>=1.1.1']
 jenkins = ['python-jenkins>=0.4.15']
@@ -169,7 +168,7 @@ def write_version(filename=os.path.join(*['airflow',
 kubernetes = ['kubernetes>=3.0.0',
               'cryptography>=2.0.0']
 ldap = ['ldap3>=0.9.9.1']
-mssql = ['pymssql>=2.1.1', 'unicodecsv>=0.14.1']
+mssql = ['pymssql>=2.1.1']
 mysql = ['mysqlclient>=1.3.6']
 oracle = ['cx_Oracle>=5.1.2']
 password = [
@@ -279,6 +278,7 @@ def do_setup():
             'tenacity==4.8.0',
             'thrift>=0.9.2',
             'tzlocal>=1.4',
+            'unicodecsv>=0.14.1',
             'werkzeug>=0.14.1, <0.15.0',
             'zope.deprecation>=4.0, <5.0',
         ],