Skip to content

Commit

Permalink
docs: tidy up Writing plugins : rules-api directory files (pantsbuild…
Browse files Browse the repository at this point in the history
…#17710)

[ci skip-rust]

[ci skip-build-wheels]
  • Loading branch information
AlexTereshenkov authored Dec 5, 2022
1 parent 11bca3f commit a801ced
Show file tree
Hide file tree
Showing 8 changed files with 38 additions and 38 deletions.
20 changes: 10 additions & 10 deletions docs/markdown/Writing Plugins/rules-api/rules-api-and-target-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ if target.has_fields([PythonSourceField, PythonTestsTimeoutField]):

As explained in [Concepts](doc:target-api-concepts), subclassing `Field`s is key to how the Target API works.

The `Target` methods `[MyField]`, `.has_field()` and `.get()` understand when a `Field` is subclassesd, as follows:
The `Target` methods `[MyField]`, `.has_field()` and `.get()` understand when a `Field` is subclassed, as follows:

```python
>>> docker_tgt.has_field(DockerSourceField)
Expand All @@ -70,7 +70,7 @@ This allows you to express specifically which types of `Field`s you need to work

### A Target's `Address`

Every target is identifed by its `Address`, from `pants.engine.adddresses`. Many types used in the Plugin API will use `Address` objects as fields, and it's also often useful to use the `Address` when writing the description for a `Process` you run.
Every target is identified by its `Address`, from `pants.engine.addresses`. Many types used in the Plugin API will use `Address` objects as fields, and it's also often useful to use the `Address` when writing the description for a `Process` you run.

A `Target` has a field `address: Address`, e.g. `my_tgt.address`.

Expand All @@ -79,9 +79,9 @@ You can also create an `Address` object directly, which is often useful in tests
- `project:tgt` -> `Address("project", target_name="tgt")`
- `project/` -> `Address("project")`
- `//:top-level` -> `Address("", target_name="top_level")`
- `project/app.py:tgt` -> \`Address("project", target_name="tgt", relative_file_name="app.py")
- `project/app.py:tgt` -> `Address("project", target_name="tgt", relative_file_name="app.py")`
- `project:tgt#generated` -> `Address("project", target_name="tgt", generated_name="generated")`
- `project:tgt@shell=zsh`: `Address("project", target_name="tgt", parameters={"shell": "zsh"})`
- `project:tgt@shell=zsh` -> `Address("project", target_name="tgt", parameters={"shell": "zsh"})`

You can use `str(address)` or `address.spec` to get the normalized string representation. `address.spec_path` will give the path to the parent directory of the target's original BUILD file.

Expand All @@ -90,7 +90,7 @@ How to resolve targets

How do you get `Target`s in the first place in your plugin?

As explained in [Goal rules](doc:rules-api-goal-rules), to get all of the targets specified on the command line by a user, you can request the type `Targets` as a parameter to your `@rule` or `@goal_rule`. From there, you can optionally filter out the targets you want, such as by using `target.has_field()`.
As explained in [Goal rules](doc:rules-api-goal-rules), to get all the targets specified on the command line by a user, you can request the type `Targets` as a parameter to your `@rule` or `@goal_rule`. From there, you can optionally filter out the targets you want, such as by using `target.has_field()`.

```python
from pants.engine.target import Targets
Expand Down Expand Up @@ -192,7 +192,7 @@ from pants.engine.rules import Get, rule
@rule
async def demo(...) -> Foo:
...
direct_deps = await Get(Targets, DependenciesRequest(target.get(Dependencies))
direct_deps = await Get(Targets, DependenciesRequest(target.get(Dependencies)))
```

`DependenciesRequest` takes a single argument: `field: Dependencies`. The return type `Targets` is a `Collection` of individual `Target` objects corresponding to each direct dependency of the original target.
Expand All @@ -210,7 +210,7 @@ from pants.engine.rules import Get, rule
@rule
async def demo(...) -> Foo:
...
transitive_targets = await Get(TransitiveTargets, TransitiveTargetsRequest([target.address])
transitive_targets = await Get(TransitiveTargets, TransitiveTargetsRequest([target.address]))
```

`TransitiveTargetsRequest` takes an iterable of `Address`es.
Expand Down Expand Up @@ -272,7 +272,7 @@ from pants.engine.rules import Get, rule
@rule
async def demo(...) -> Foo:
...
sources = await Get(HydratedSources, HydrateSourcesRequest(target[SourcesField])
sources = await Get(HydratedSources, HydrateSourcesRequest(target[SourcesField]))
```

`HydrateSourcesRequest` expects a `SourcesField` object. This can be a subclass, such as `PythonSourceField` or `GoPackageSourcesField`.
Expand Down Expand Up @@ -317,7 +317,7 @@ async def demo(...) -> Foo:
)
```

If the provided `SourcesField` object is already a subclass of one of the `for_sources_types`or it can be generated into one of those types—then the sources will be hydrated; otherwise, you'll get back a `HydratedSources` object with an empty snapshot and the field `sources_type=None`.
If the provided `SourcesField` object is already a subclass of one of the `for_sources_types`—or it can be generated into one of those types—then the sources will be hydrated; otherwise, you'll get back a `HydratedSources` object with an empty snapshot and the field `sources_type=None`.

`SourceFilesRequest` also accepts the `enable_codegen` and `for_source_types` arguments. This will filter out any inputted `Sources` field that are not compatible with `for_sources_type`.

Expand Down Expand Up @@ -373,7 +373,7 @@ Normally, your rule should simply use `tgt.get()` and `tgt.has_field()` instead

To create a `FieldSet`, create a new dataclass with `@dataclass(frozen=True)`. You will sometimes directly subclass `FieldSet`, but will often subclass something like `BinaryFieldSet` or `TestFieldSet`. Refer to the instructions in [Common plugin tasks](doc:common-plugin-tasks).

List every `Field` that your plugin will use as a field of your dataclass. The types hints you specify will be used by Pants to identify what `Field`s to use, e.g. `PythonSourceField` or `Dependencies`.
List every `Field` that your plugin will use as a field of your dataclass. The type hints you specify will be used by Pants to identify what `Field`s to use, e.g. `PythonSourceField` or `Dependencies`.

Finally, set the class property `required_fields` as a tuple of the `Field`s that your plugin requires. Pants will use this to filter out irrelevant targets that your plugin does not know how to operate on. Often, this will be the same as the `Field`s that you listed as dataclass fields, but it does not need to be. If a target type does not have registered one of the `Field`s that are in the dataclass fields, and it isn't a required `Field`, then Pants will use a default value as if the user left it off from their BUILD file.

Expand Down
6 changes: 3 additions & 3 deletions docs/markdown/Writing Plugins/rules-api/rules-api-concepts.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,12 @@ async def run_shellcheck(target: Target, shellcheck: Shellcheck) -> LintResult:

You do not call a rule like you would a normal function. In the above examples, you would not say `int_to_str(26)` or `run_shellcheck(tgt, shellcheck)`. Instead, the Pants engine determines when rules are used and calls the rules for you.

Each rule should be pure; you should not use side-effects like `subprocess.run()`, `print()`, or the `requests` library. Instead, the Rules API has its own alternatives that are understood by the Pants engine and which work properly with its caching and parallelism.
Each rule should be pure; you should not use side effects like `subprocess.run()`, `print()`, or the `requests` library. Instead, the Rules API has its own alternatives that are understood by the Pants engine and which work properly with its caching and parallelism.

The rule graph
--------------

All of the registered rules create a rule graph, with each type as a node and the edges being dependencies used to compute those types.
All the registered rules create a rule graph, with each type as a node and the edges being dependencies used to compute those types.

For example, the `list` goal uses this rule definition and results in the below graph:

Expand Down Expand Up @@ -140,7 +140,7 @@ async def call_fibonacci(...) -> Foo:
>
> Currently, you can only give a single input. It is not possible to do something like `Get(OutputType, InputType1(...), InputType2(...))`.
>
> Instead, it's common for rules to create a "Request" data class, such as `PexRequest` or `SourceFilesRequest`. This request centralizes all of the data it needs to operate into one data structure, which allows for call sites to say `await Get(SourceFiles, SourceFilesRequest, my_request)`, for example.
> Instead, it's common for rules to create a "Request" data class, such as `PexRequest` or `SourceFilesRequest`. This request centralizes all the data it needs to operate into one data structure, which allows for call sites to say `await Get(SourceFiles, SourceFilesRequest, my_request)`, for example.
>
> See <https://github.com/pantsbuild/pants/issues/7490> for the tracking issue.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -90,11 +90,11 @@ from pants.engine.rules import Get, rule
@rule
async def demo(...) -> Foo:
...
digest = await Get(Digest, PathGlobs(["**/*.txt", "!ignore_me.txt"])
digest = await Get(Digest, PathGlobs(["**/*.txt", "!ignore_me.txt"]))
```

- All globs must be relative paths, relative to the build root.
- `PathGlobs` uses the same syntax as the `sources` field, which is roughly Git's syntax. Use `*` for globs over just the current working directory, `**` for recursive globs over everything below (at any level the current working directory, and prefix with `!` for ignores.
- `PathGlobs` uses the same syntax as the `sources` field, which is roughly Git's syntax. Use `*` for globs over just the current working directory, `**` for recursive globs over everything below (at any level the current working directory), and prefix with `!` for ignores.
- `PathGlobs` will ignore all values from the global option `pants_ignore`.

By default, the engine will no-op for any globs that are unmatched. If you want to instead warn or error, set `glob_match_error_behavior=GlobMatchErrorBehavior.warn` or `GlobMatchErrorBehavior.error`. This will require that you also set `description_of_origin`, which is a human-friendly description of where the `PathGlobs` is coming from so that the error message is helpful. For example:
Expand Down Expand Up @@ -131,7 +131,7 @@ from pants.engine.rules import Get, rule
@rule
async def demo(...) -> Foo:
...
paths = await Get(Paths, PathGlobs(["**/*.txt", "!ignore_me.txt"])
paths = await Get(Paths, PathGlobs(["**/*.txt", "!ignore_me.txt"]))
logger.info(paths.files)
```

Expand Down Expand Up @@ -211,7 +211,7 @@ async def demo(...) -> Foo:

- It is okay if multiple digests include the same file, so long as they have identical content.
- If any digests have different content for the same file, the engine will error. Unlike Git, the engine does not attempt to resolve merge conflicts.
- It is okay if some of the digests are empty, i.e. `EMPTY_DIGEST`.
- It is okay if some digests are empty, i.e. `EMPTY_DIGEST`.

`DigestSubset`: extract certain files from a `Digest`
-----------------------------------------------------
Expand Down
14 changes: 7 additions & 7 deletions docs/markdown/Writing Plugins/rules-api/rules-api-goal-rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ updatedAt: "2022-07-25T14:45:07.539Z"
---
For many [plugin tasks](doc:common-plugin-tasks), you will be extending existing goals, such as adding a new linter to the `lint` goal. However, you may instead want to create a new goal, such as a `publish` goal. This page explains how to create a new goal.

As explained in [Concepts](doc:rules-api-concepts), `@goal_rule`s are the entry-point into the rule graph. When a user runs `./pants my-goal`, the Pants engine will look for the respective `@goal_rule`. That `@goal_rule` will usually request other types, either as parameters in the `@goal_rule` signature or through `await Get`. But unlike a `@rule`, a `@goal_rule` may also trigger side-effects (such as running interactive processes, writing to the filesystem, etc) via `await Effect`.
As explained in [Concepts](doc:rules-api-concepts), `@goal_rule`s are the entry-point into the rule graph. When a user runs `./pants my-goal`, the Pants engine will look for the respective `@goal_rule`. That `@goal_rule` will usually request other types, either as parameters in the `@goal_rule` signature or through `await Get`. But unlike a `@rule`, a `@goal_rule` may also trigger side effects (such as running interactive processes, writing to the filesystem, etc) via `await Effect`.

Often, you can keep all of your logic inline in the `@goal_rule`. As your `@goal_rule` gets more complex, you may end up factoring out helper `@rule`s, but you do not need to start with writing helper `@rule`s.

Expand Down Expand Up @@ -91,7 +91,7 @@ If your goal's purpose is to emit output, it may be helpful to use the mixin `Ou
from pants.engine.goal import Goal, GoalSubsystem, Outputting
from pants.engine.rules import goal_rule

class HelloWorldSubsystem(Outputting, GoalSubystem):
class HelloWorldSubsystem(Outputting, GoalSubsystem):
name = "hello-world"
help = "An example goal."

Expand All @@ -108,13 +108,13 @@ async def hello_world(

### `LineOriented` mixin (optional)

If your goal's purpose is to emit outputand that output is naturally split by new linesit may be helpful to use the mixin `LineOriented`. This subclasses `Outputting`, so will register both the options `--output-file` and `--sep`, which allows the user to change the separator to not be `\n`.
If your goal's purpose is to emit output -- and that output is naturally split by new lines -- it may be helpful to use the mixin `LineOriented`. This subclasses `Outputting`, so will register both the options `--output-file` and `--sep`, which allows the user to change the separator to not be `\n`.

```python
from pants.engine.goal import Goal, GoalSubsystem, LineOriented
from pants.engine.rules import goal_rule

class HelloWorldSubsystem(LineOriented, GoalSubystem):
class HelloWorldSubsystem(LineOriented, GoalSubsystem):
name = "hello-world"
help = "An example goal."""

Expand Down Expand Up @@ -168,9 +168,9 @@ See [Rules and the Target API](doc:rules-api-and-target-api) for detailed infor
>
> This will not work because the engine has no path in the rule graph to resolve a `PythonDistribution` type given the initial input types to the rule graph (the "roots").
>
> Instead, request `Targets`, which will give you all of the targets that the user specified on the command line. The engine knows how to resolve this type because it can go from `Specs` -> `Addresses` -> `Targets`.
> Instead, request `Targets`, which will give you all the targets that the user specified on the command line. The engine knows how to resolve this type because it can go from `Specs` -> `Addresses` -> `Targets`.
>
> From here, filter out the relevant targets you want using the Target API (see [Rules and the Target API](doc:rules-api-and-target-api).
> From here, filter out the relevant targets you want using the Target API (see [Rules and the Target API](doc:rules-api-and-target-api)).
>
> ```python
> from pants.engine.target import Targets
Expand Down Expand Up @@ -200,4 +200,4 @@ async def hello_world(console: Console, specs_paths: SpecsPaths) -> HelloWorld:
`SpecsPaths.files` will list all files matched by the specs, e.g. `::` will match every file in the project (regardless of if targets own the files).

To convert `SpecsPaths` into a [`Digest`](doc:rules-api-file-system), use `await Get(Digest, PathGlobs(globs=specs_paths.files))`.
To convert `SpecsPaths` into a [`Digest`](doc:rules-api-file-system), use `await Get(Digest, PathGlobs(globs=specs_paths.files))`.
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ async def demo(...) -> Foo:
docker_bin = docker_paths.first_path
if docker_bin is None:
raise OSError("Could not find 'docker'.")
result = await Get(ProcessResult, Process(argv=[docker_bin.path, ...], ...)
result = await Get(ProcessResult, Process(argv=[docker_bin.path, ...], ...))
```

`BinaryPaths` has a field called `paths: tuple[BinaryPath, ...]`, which stores all the discovered absolute paths to the specified binary. Each `BinaryPath` object has the fields `path: str`, such as `/usr/bin/docker`, and `fingerprint: str`, which is used to invalidate the cache if the binary changes. The results will be ordered by the order of `search_path`, meaning that earlier entries in `search_path` will show up earlier in the result.
Expand Down
6 changes: 3 additions & 3 deletions docs/markdown/Writing Plugins/rules-api/rules-api-process.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,7 +61,7 @@ from pants.engine.rules import Get, rule
@rule
async def partial_env(...) -> Foo:
relevant_env_vars = await Get(EnvironmentVars, EnvironmentVarsRequest(["RELEVANT_VAR", "PATH"]))
..
...
```

### Output Files
Expand All @@ -78,7 +78,7 @@ To use a timeout, set the `timeout_seconds: int` field. Otherwise, the process w
>
> By default, a `Process` will be cached to `~/.cache/pants/lmdb_store` if the `exit_code` is `0`.
>
> If it not safe to cache your `Process`usually the case when you know that a process accesses files outside of its sandboxyou can change the cacheability of your `Process` using the `ProcessCacheScope` parameter:
> If it's not safe to cache your `Process` -- usually the case when you know that a process accesses files outside its sandbox -- you can change the cacheability of your `Process` using the `ProcessCacheScope` parameter:
>
> ```python
> from pants.engine.process import Process, ProcessCacheScope, ProcessResult
Expand All @@ -90,7 +90,7 @@ To use a timeout, set the `timeout_seconds: int` field. Otherwise, the process w
> description="Not persisted between Pants runs ('sessions').",
> cache_scope=ProcessCacheScope.PER_SESSION,
> )
> ..
> ...
> ```
>
> `ProcessCacheScope` supports other options as well, including `ALWAYS`.
Expand Down
10 changes: 5 additions & 5 deletions docs/markdown/Writing Plugins/rules-api/rules-api-testing.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,7 +126,7 @@ async def find_needle_in_haystack(find_needle: FindNeedle) -> TargetsWithNeedle:
We can write this test:
```python
from pants.engine.addresses import address
from pants.engine.addresses import Address
from pants.engine.fs import EMPTY_DIGEST, Snapshot
from pants.engine.target import HydratedSources, HydrateSourcesRequest, Target, Sources
from pants.testutil.rule_runner import MockGet, run_rule_with_mocks
Expand Down Expand Up @@ -206,7 +206,7 @@ After setting up your isolated environment, you can run `rule_runner.request(Out
### Setting up the `RuleRunner`
First, you must set up a `RuleRunner` instance and activate the rules and target types you'll use in your tests. Set the argument `target_types` with a list of the `Target` types used in in your tests, and set `rules` with a list of all the rules used transitively.
First, you must set up a `RuleRunner` instance and activate the rules and target types you'll use in your tests. Set the argument `target_types` with a list of the `Target` types used in your tests, and set `rules` with a list of all the rules used transitively.
This means that you must register the rules you directly wrote, and also any rules that they depend on. Pants will automatically register some core rules for you, but leaves off most of them for better isolation of tests. If you're missing some rules, the rule graph will fail to be built.
Expand Down Expand Up @@ -289,7 +289,7 @@ def second_rule_runner() -> RuleRunner:
def test_example3(second_rule_runner: RuleRunner) -> None:
second_rule_runner.write_files(..)
second_rule_runner.write_files(...)
...
```
Expand Down Expand Up @@ -492,7 +492,7 @@ def test_one_target_one_source(rule_runner: RuleRunner) -> None:
}
)
result = rule_runner.run_goal_rule(Filedeps, args=["project/example.ext"])
assert result.stdout.splitlines() = ["project/BUILD", "project/example.ext"]
assert result.stdout.splitlines() == ["project/BUILD", "project/example.ext"]
```
Unlike when testing normal `@rules`, you do not need to define a `QueryRule` when using `rule_runner.run_goal_rule()`. This is already set up for you. However, you do need to make sure that your `@goal_rule` and all the rules it depends on are registered with the `RuleRunner` instance.
Expand Down Expand Up @@ -543,7 +543,7 @@ To read any files that were created, use `get_buildroot()` as the first part of
```python
from pathlib import Path
from pants.base.build_environment import get_buildroot()
from pants.base.build_environment import get_buildroot
from pants.testutil.pants_integration_test import run_pants, setup_tmpdir
def test_junit_report() -> None:
Expand Down
Loading

0 comments on commit a801ced

Please sign in to comment.