A setup_py task for the new python pipeline. (pantsbuild#4308)

Copied from the old one, with the previous special-casing of
codegen removed (the new python tasks don't invoke
codegen directly like the old ones did).

However this task does still require some special handling of
codegen: when selecting the targets to operate on we have to
replace any codegen target roots with their corresponding
synthetic targets, as those are the ones with the actual sources
to publish.

This change also copies the old task's unit tests, except the
ones pertaining to codegen (which would no longer work without
setting up and running codegen tasks in the tests).
Instead, this change adds an integration test that ensures that we
create an appropriate sdist from codegen targets, thus exercising
the special handling mentioned above (and incidentally also verifying
that our new pipeline does do the right thing regarding synthesizing
library targets for generated code).

examples/src/thrift/org/pantsbuild/example/distance/BUILD

@@ -13,4 +13,8 @@ java_thrift_library(name='distance-java',
 
 python_thrift_library(name='distance-python',
   sources=['distance.thrift'],
+  provides=setup_py(
+    name='pantsbuild.pants.distance-thrift-python',
+    version='0.0.1',
+  )
 )

src/python/pants/backend/python/register.py

@@ -25,6 +25,7 @@
 from pants.backend.python.tasks.python_repl import PythonRepl
 from pants.backend.python.tasks.python_run import PythonRun
 from pants.backend.python.tasks.setup_py import SetupPy
+from pants.backend.python.tasks2.setup_py import SetupPy as SetupPy2
 from pants.build_graph.build_file_aliases import BuildFileAliases
 from pants.build_graph.resources import Resources
 from pants.goal.task_registrar import TaskRegistrar as task
@@ -65,3 +66,4 @@ def register_goals():
   task(name='py', action=PythonRun2).install('run2')
   task(name='pytest', action=PytestRun2).install('test2')
   task(name='py', action=PythonRepl2).install('repl2')
+  task(name='setup-py2', action=SetupPy2).install()

src/python/pants/backend/python/tasks2/BUILD

@@ -5,12 +5,17 @@ python_library(
   dependencies=[
     '3rdparty/python:pex',
     '3rdparty/python/twitter/commons:twitter.common.collections',
+    '3rdparty/python/twitter/commons:twitter.common.dirutil',
     'src/python/pants/backend/python:interpreter_cache',
     'src/python/pants/backend/python:python_setup',
     'src/python/pants/backend/python/targets',
+    'src/python/pants/base:build_environment',
+    'src/python/pants/base:exceptions',
+    'src/python/pants/base:specs',
     'src/python/pants/build_graph',
     'src/python/pants/invalidation',
     'src/python/pants/task',
     'src/python/pants/util:dirutil',
+    'src/python/pants/util:meta',
   ]
 )