Skip to content

Commit

Permalink
[Doc] Add descriptions for namespace isolation policy, configuration …
Browse files Browse the repository at this point in the history
…store, and metadata store (apache#9422)

* add

* Update pulsar-client-tools/src/main/java/org/apache/pulsar/admin/cli/CmdNamespaceIsolationPolicy.java

Co-authored-by: Huanli Meng <[email protected]>

* update

* update

* update

* test2

Co-authored-by: Anonymitaet <anonymitaet_hotmail.com>
Co-authored-by: Huanli Meng <[email protected]>
Co-authored-by: Yong Zhang <[email protected]>
  • Loading branch information
3 people authored Feb 5, 2021
1 parent 70d9f00 commit 56767de
Show file tree
Hide file tree
Showing 3 changed files with 10 additions and 4 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ private class SetPolicy extends CliCommand {
@Parameter(names = "--namespaces", description = "comma separated namespaces-regex list", required = true, splitter = CommaParameterSplitter.class)
private List<String> namespaces;

@Parameter(names = "--primary", description = "comma separated primary-broker-regex list", required = true, splitter = CommaParameterSplitter.class)
@Parameter(names = "--primary", description = "comma separated primary-broker-regex list. In Pulsar, when namespaces (more specifically, namespace bundles) are assigned dynamically to brokers, the namespace isolation policy limits the set of brokers that can be used for assignment. Before topics are assigned to brokers, you can set the namespace isolation policy with a primary or a secondary regex to select desired brokers. If no broker matches the specified regex, you cannot create a topic. If there are not enough primary brokers, topics are assigned to secondary brokers. If there are not enough secondary brokers, topics are assigned to other brokers which do not have any isolation policies.", required = true, splitter = CommaParameterSplitter.class)
private List<String> primary;

@Parameter(names = "--secondary", description = "comma separated secondary-broker-regex list", required = false, splitter = CommaParameterSplitter.class)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -607,7 +607,7 @@ private class SetBookieAffinityGroup extends CliCommand {
"-pg" }, description = "Bookie-affinity primary-groups (comma separated) name where namespace messages should be written", required = true)
private String bookieAffinityGroupNamePrimary;
@Parameter(names = { "--secondary-group",
"-sg" }, description = "Bookie-affinity secondary-group (comma separated) name where namespace messages should be written", required = false)
"-sg" }, description = "Bookie-affinity secondary-group (comma separated) name where namespace messages should be written. If you want to verify whether there are enough bookies in groups, use `--secondary-group` flag. Messages in this namespace are stored in secondary groups. If a group does not contain enough bookies, a topic cannot be created.", required = false)
private String bookieAffinityGroupNameSecondary;


Expand Down
10 changes: 8 additions & 2 deletions site2/docs/concepts-architecture-overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ At the broader instance level, an instance-wide ZooKeeper cluster called the con

The Pulsar message broker is a stateless component that's primarily responsible for running two other components:

* An HTTP server that exposes a {@inject: rest:REST:/} API for both administrative tasks and [topic lookup](concepts-clients.md#client-setup-phase) for producers and consumers
* An HTTP server that exposes a {@inject: rest:REST:/} API for both administrative tasks and [topic lookup](concepts-clients.md#client-setup-phase) for producers and consumers. The producers connect to the brokers to publish messages and the consumers connect to the brokers to consume the messages.
* A dispatcher, which is an asynchronous TCP server over a custom [binary protocol](developing-binary-protocol.md) used for all data transfers

Messages are typically dispatched out of a [managed ledger](#managed-ledgers) cache for the sake of performance, *unless* the backlog exceeds the cache size. If the backlog grows too large for the cache, the broker will start reading entries from BookKeeper.
Expand All @@ -45,11 +45,17 @@ Clusters can replicate amongst themselves using [geo-replication](concepts-repli
## Metadata store

Pulsar uses [Apache Zookeeper](https://zookeeper.apache.org/) for metadata storage, cluster configuration, and coordination. In a Pulsar instance:
The Pulsar metadata store maintains all the metadata of a Pulsar cluster, such as topic metadata, schema, broker load data, and so on. Pulsar uses [Apache ZooKeeper](https://zookeeper.apache.org/) for metadata storage, cluster configuration, and coordination. The Pulsar metadata store can be deployed on a separate ZooKeeper cluster or deployed on an existing ZooKeeper cluster. You can use one ZooKeeper cluster for both Pulsar metadata store and [BookKeeper metadata store](https://bookkeeper.apache.org/docs/latest/getting-started/concepts/#metadata-storage). If you want to deploy Pulsar brokers connected to an existing BookKeeper cluster, you need to deploy separate ZooKeeper clusters for Pulsar metadata store and BookKeeper metadata store respectively.

In a Pulsar instance:

* A configuration store quorum stores configuration for tenants, namespaces, and other entities that need to be globally consistent.
* Each cluster has its own local ZooKeeper ensemble that stores cluster-specific configuration and coordination such as which brokers are responsible for which topics as well as ownership metadata, broker load reports, BookKeeper ledger metadata, and more.

## Configuration store

The configuration store maintains all the configurations of a Pulsar instance, such as clusters, tenants, namespaces, partitioned topic related configurations, and so on. A Pulsar instance can have a single local cluster, multiple local clusters, or multiple cross-region clusters. Consequently, the configuration store can share the configurations across multiple clusters under a Pulsar instance. The configuration store can be deployed on a separate ZooKeeper cluster or deployed on an existing ZooKeeper cluster.

## Persistent storage

Pulsar provides guaranteed message delivery for applications. If a message successfully reaches a Pulsar broker, it will be delivered to its intended target.
Expand Down

0 comments on commit 56767de

Please sign in to comment.