Skip to content

Commit

Permalink
[improve][doc] Add more concepts/tasks for bookie isolation (apache#1…
Browse files Browse the repository at this point in the history
  • Loading branch information
momo-jun authored Aug 19, 2022
1 parent 5da9be9 commit dcf3059
Show file tree
Hide file tree
Showing 10 changed files with 247 additions and 25 deletions.
227 changes: 216 additions & 11 deletions site2/docs/administration-isolation-bookie.md

Large diffs are not rendered by default.

19 changes: 11 additions & 8 deletions site2/docs/administration-isolation-broker.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,33 +12,29 @@ import TabItem from '@theme/TabItem';

In Pulsar, when namespaces (more specifically, namespace bundles) are assigned dynamically to brokers, the namespace isolation policy limits the set of brokers that can be used for assignment. Before topics are assigned to brokers, you can set the namespace isolation policy with a primary or a secondary regex to select desired brokers.

You can set a namespace isolation policy for a cluster using one of the following methods.
To set a namespace isolation policy for a broker cluster, you can use one of the following methods.

````mdx-code-block
<Tabs
defaultValue="Admin CLI"
values={[{"label":"Admin CLI","value":"Admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java admin API","value":"Java admin API"}]}>
defaultValue="Pulsar-admin CLI"
values={[{"label":"Pulsar-admin CLI","value":"Pulsar-admin CLI"},{"label":"REST API","value":"REST API"},{"label":"Java admin API","value":"Java admin API"}]}>
<TabItem value="Admin CLI">
<TabItem value="Pulsar-admin CLI">
```
pulsar-admin ns-isolation-policy set options
```
For more information about the command `pulsar-admin ns-isolation-policy set options`, see [Pulsar admin docs](https://pulsar.apache.org/tools/pulsar-admin/).
**Example**
```shell
bin/pulsar-admin ns-isolation-policy set \
--auto-failover-policy-type min_available \
--auto-failover-policy-params min_limit=1,usage_threshold=80 \
--namespaces my-tenant/my-namespace \
--primary 10.193.216.* my-cluster policy-name
```
</TabItem>
Expand All @@ -55,3 +51,10 @@ For how to set namespace isolation policy using Java admin API, see [code](https
</Tabs>
````


:::tip

To guarantee all the data that belongs to a namespace is stored in desired bookies, you can isolate the data of the namespace into user-defined groups of bookies. See [configure bookie affinity groups](#configure-bookie-affinity-groups) for more details.

:::
19 changes: 14 additions & 5 deletions site2/docs/administration-isolation.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,19 @@ In an organization, a Pulsar instance provides services to multiple teams. When
The multi-layer and segment-centric architecture and hierarchical resource management of Pulsar provide a solid foundation for isolation, which allows you to isolate resources in your desired manner, prevent resource competition, and attain stability.


## Isolation methods
To enforce resource isolation within Pulsar, you can use the following methods of isolating broker and bookie resources:
* [Isolate brokers](administration-isolation-broker.md)
* [Isolate bookies](administration-isolation-bookie.md)
## Isolation levels

Pulsar supports isolation at either of the following two levels or both.
* [Broker-level isolation](administration-isolation-broker.md) divides brokers into different groups and assigns broker groups to different namespaces. In this way, you can bind topics in a namespace to a set of brokers that belong to the specific groups.
* [Bookie-level isolation](administration-isolation-bookie.md) divides bookies into different racks/regions and assigns data replicas to bookies based on a specified data placement policy for disaster tolerance.

![Isolation levels](/assets/admin-isolation.svg)

:::tip

On top of [broker-level isolation](administration-isolation-broker.md) and [bookie-level isolation](administration-isolation-bookie.md), if you want to guarantee all the data that belongs to a namespace is stored in desired bookies, you can define and configure [bookie affinity groups](administration-isolation-bookie.md#configure-bookie-affinity-groups). See [shared BookKeeper cluster deployment](#shared-bookkeeper-cluster) for more details.

:::


## Deployments to achieve isolation within Pulsar
Expand Down Expand Up @@ -48,7 +57,7 @@ Here are some key points for understanding how it works:
- Each Pulsar cluster has one or multiple brokers.
- Each Pulsar cluster has one metadata store.

As illustrated below, storage isolation is achieved by setting [bookie affinity groups](administration-isolation-bookie.md). All bookie groups use a shared BookKeeper cluster and a metadata store, and each bookie isolation group has one or several bookies. You can specify one or multiple primary/secondary groups for a namespace. Topics under the namespace are created on the bookies in the primary group firstly and then created on the bookies in the secondary group.
As illustrated below, all bookie groups use a shared BookKeeper cluster and a metadata store, and each [bookie affinity group](administration-isolation-bookie.md#configure-bookie-affinity-groups) has one or several bookies. You can specify one or multiple primary/secondary groups for a namespace. Topics under the namespace are created on the bookies in the primary group firstly and then created on the bookies in the secondary group.

![Storage isolation achieved by bookie affinity groups](/assets/isolation-3.png)

Expand Down
1 change: 1 addition & 0 deletions site2/docs/assets/admin-isolation.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions site2/docs/assets/rack-aware-placement-policy-1.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions site2/docs/assets/rack-aware-placement-policy-2.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions site2/docs/assets/rack-aware-placement-policy-3.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions site2/docs/assets/region-aware-placement-policy-1.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions site2/docs/assets/region-aware-placement-policy-2.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 0 additions & 1 deletion site2/website/sidebars.json
Original file line number Diff line number Diff line change
Expand Up @@ -232,7 +232,6 @@
"administration-stats",
"administration-load-balance",
"administration-proxy",
"administration-isolation",
"administration-upgrade",
{
"type": "category",
Expand Down

0 comments on commit dcf3059

Please sign in to comment.