Skip to content

Commit

Permalink
Document Pants features for Organizations (pantsbuild#5673)
Browse files Browse the repository at this point in the history
### Problem

Organizations that are interested in Pants do not have a simple way of learning what features are available for larger organizations using Pants, nor a place to learn best-practices for how to setup their site.

### Solution

Here we create a new docs page focused on organizations, especially targeted at Developer Experience or equivalent teams who are setting up Pants for their org.

### Result

Organizations should be better able to setup Pants at their site as they have a simple way to understand which features to setup.
  • Loading branch information
traviscrawford authored and Stu Hood committed Aug 10, 2018
1 parent aad3a3c commit 4c31167
Show file tree
Hide file tree
Showing 9 changed files with 121 additions and 2 deletions.
5 changes: 5 additions & 0 deletions src/docs/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,11 @@ page(
]
)

page(
name='orgs',
source='orgs.md',
)

page(
name='options',
source='options.md',
Expand Down
8 changes: 6 additions & 2 deletions src/docs/docsite.json
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@
"notes-1.9.x": "dist/markdown/html/src/python/pants/notes/1.9.x.html",
"notes-master": "dist/markdown/html/src/python/pants/notes/master.html",
"notes-master": "dist/markdown/html/src/python/pants/notes/master.html",
"orgs": "dist/markdown/html/src/docs/orgs.html",
"options": "dist/markdown/html/src/docs/options.html",
"options_reference": "dist/reference/pants_reference_body.html",
"page": "dist/markdown/html/examples/src/java/org/pantsbuild/example/page.html",
Expand Down Expand Up @@ -102,6 +103,9 @@
"images/intellij-project-structure-project.png": "src/docs/images/intellij-project-structure-project.png",
"images/intellij-run.png": "src/docs/images/intellij-run.png",
"images/intellij-select-venv.png": "src/docs/images/intellij-select-venv.png",
"images/orgs.png": "src/docs/images/orgs.png",
"images/orgs_be.png": "src/docs/images/orgs_be.png",
"images/orgs_fe.png": "src/docs/images/orgs_fe.png",
"images/ossrh-publish-close-staging-repo.png": "src/docs/images/ossrh-publish-close-staging-repo.png",
"images/ossrh-publish-find-staging-repo.png": "src/docs/images/ossrh-publish-find-staging-repo.png",
"images/ossrh-publish-release-staging-repo.png": "src/docs/images/ossrh-publish-release-staging-repo.png",
Expand All @@ -118,13 +122,13 @@
"tree": [
{ "page": "index",
"children": [

{"heading": "Getting Started"},
{"pages": [
"install",
"setup_repo",
"first_tutorial",
"common_tasks"
"common_tasks",
"orgs"
]

},
Expand Down
17 changes: 17 additions & 0 deletions src/docs/images/orgs.dot
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
digraph G {
common [label="org.pantsbuild.common"];
fe_common [label="org.pantsbuild.fe.common"];
be_common [label="org.pantsbuild.be.common"];

fe_common -> common;
be_common -> common;

service_common [label="org.pantsbuild.common.service"];
fe_service [label="org.pantsbuild.fe.service"];
be_service [label="org.pantsbuild.be.service"];

fe_service -> service_common;
fe_service -> fe_common;
be_service -> service_common;
be_service -> be_common;
}
Binary file added src/docs/images/orgs.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
17 changes: 17 additions & 0 deletions src/docs/images/orgs_be.dot
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
digraph G {
common [label="org.pantsbuild.common", color="red"];
fe_common [label="org.pantsbuild.fe.common"];
be_common [label="org.pantsbuild.be.common", color="red"];

fe_common -> common;
be_common -> common;

service_common [label="org.pantsbuild.common.service", color="red"];
fe_service [label="org.pantsbuild.fe.service"];
be_service [label="org.pantsbuild.be.service", color="red"];

fe_service -> service_common;
fe_service -> fe_common;
be_service -> service_common;
be_service -> be_common;
}
Binary file added src/docs/images/orgs_be.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
17 changes: 17 additions & 0 deletions src/docs/images/orgs_fe.dot
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
digraph G {
common [label="org.pantsbuild.common", color="blue"];
fe_common [label="org.pantsbuild.fe.common", color="blue"];
be_common [label="org.pantsbuild.be.common"];

fe_common -> common;
be_common -> common;

service_common [label="org.pantsbuild.common.service", color="blue"];
fe_service [label="org.pantsbuild.fe.service", color="blue"];
be_service [label="org.pantsbuild.be.service"];

fe_service -> service_common;
fe_service -> fe_common;
be_service -> service_common;
be_service -> be_common;
}
Binary file added src/docs/images/orgs_fe.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
59 changes: 59 additions & 0 deletions src/docs/orgs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
# Pants for Organizations

Pants works well for organizations with a large number of developers working in a multi-tenant repo.
Typically, repos become slower over time as more source and test files are added because most
build systems build and test everything every time. Pants enables the benefits of a single repo
with the speed of per-project repos by only operating on repo "slices".

## Frontend & Backend Example

Imagine a simple repo that contains both a frontend and backend service, as illustrated by the
following dependency graph.

![orgs](images/orgs.png)

Consider a frontend developer that only works on the frontend service. With most build systems, the
developer would need to compile everything every time, slowing them down over time as more and more
code is added to the repo. With Pants, the frontend developer only needs to compile the code directly
used by their service. For example, consider building `org.pantsbuild.fe.service`. Regardless of
how complex the backend code grows over time, the frontend build will not slow down.

![orgs_fe](images/orgs_fe.png)

Now consider a backend developer who only works on the backend service. When building the backend
service, only the code required by the backend service is built, regardless of what other code
is in the repo. The ability to build "slices" of the repo is fundamental to enabling multiple users
and teams to work together in a single repo. Note how only the backend code would be built when
building `org.pantsbuild.be.service`:

![orgs_be](images/orgs_be.png)

## Continuous Integration

Most organizations require test pass before merging a change to master. Pants enables fast CI by
only running tests _affected_ by a change. That is, only targets that transitively depend on a
changed target have their tests run. In practice, a change to `README.md` might not trigger *any*
tests, a change to a top-level node such as `org.pantsbuild.fe.service` in the dependency graph
would only trigger it's direct tests, and a change to a low-level node
(e.g.: protobuf/thrift IDL file) might trigger most tests.

For most users, the following script would work well for a pull-request builder type CI job:

```bash
#!/bin/bash

set -x
set -o
set -e

# Disable Zinc incremental compilation to ensure no historical cruft pollutes the current build.
export PANTS_COMPILE_ZINC_INCREMENTAL=false

changed=$(./pants --changed-parent=origin/master list)
dependees=$(./pants dependees --dependees-transitive --dependees-closed $changed)
minimized=$(./pants minimize $dependees)
./pants filter --filter-type=-jvm_binary $minimized | sort > minimized.txt
for target in $(cat minimized.txt); do
./pants test $target
done
```

0 comments on commit 4c31167

Please sign in to comment.