Skip to content

Commit

Permalink
[Issue 8130] Refine BookKeeper metadata service uri when init metadata (
Browse files Browse the repository at this point in the history
apache#8269)

Motivation
Fixes apache#8130

Modifications
rename --bookkeeper-metadata-service-uri by --existing-bk-metadata-service-uri to avoid misunderstanding
add some specification of this parameter in corresponding md files

* Refine BookKeeper metadata service uri when init metadata

* address the review comments

* address Jennifer's comment
  • Loading branch information
murong00 authored Oct 21, 2020
1 parent 80e2fec commit 3147e9e
Show file tree
Hide file tree
Showing 4 changed files with 22 additions and 6 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -111,8 +111,8 @@ private static class Arguments {
private int numTransactionCoordinators = 16;

@Parameter(names = {
"--bookkeeper-metadata-service-uri" }, description = "Metadata service uri of BookKeeper")
private String bkMetadataServiceUri;
"--existing-bk-metadata-service-uri" }, description = "The metadata service URI of the existing BookKeeper cluster that you want to use")
private String existingBkMetadataServiceUri;

@Parameter(names = { "-h", "--help" }, description = "Show this help message")
private boolean help = false;
Expand Down Expand Up @@ -176,7 +176,7 @@ public static void main(String[] args) throws Exception {

// Format BookKeeper ledger storage metadata
ServerConfiguration bkConf = new ServerConfiguration();
if (arguments.bkMetadataServiceUri == null) {
if (arguments.existingBkMetadataServiceUri == null) {
bkConf.setZkServers(arguments.zookeeper);
bkConf.setZkTimeout(arguments.zkSessionTimeoutMillis);
if (localZk.exists("/ledgers", false) == null // only format if /ledgers doesn't exist
Expand All @@ -187,7 +187,7 @@ public static void main(String[] args) throws Exception {

// Format BookKeeper stream storage metadata
if (arguments.numStreamStorageContainers > 0) {
String uriStr = arguments.bkMetadataServiceUri == null ? bkConf.getMetadataServiceUri() : arguments.bkMetadataServiceUri;
String uriStr = arguments.existingBkMetadataServiceUri == null ? bkConf.getMetadataServiceUri() : arguments.existingBkMetadataServiceUri;
ServiceURI bkMetadataServiceUri = ServiceURI.create(uriStr);
ClusterInitializer initializer = new ZkClusterInitializer(arguments.zookeeper);
initializer.initializeCluster(bkMetadataServiceUri.getUri(), arguments.numStreamStorageContainers);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ public void testSetupWithBkMetadataServiceUri() throws Exception {
"--cluster", "testReSetupClusterMetadata-cluster",
"--zookeeper", zkConnection,
"--configuration-store", zkConnection,
"--bookkeeper-metadata-service-uri", "zk+null://" + zkConnection + "/chroot/ledgers",
"--existing-bk-metadata-service-uri", "zk+null://" + zkConnection + "/chroot/ledgers",
"--web-service-url", "http://127.0.0.1:8080",
"--web-service-url-tls", "https://127.0.0.1:8443",
"--broker-service-url", "pulsar://127.0.0.1:6650",
Expand Down
11 changes: 11 additions & 0 deletions site2/docs/deploy-bare-metal.md
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,17 @@ Flag | Description
> --broker-service-url-tls pulsar+ssl://host1:6651,host2:6651,host3:6651
> ```
> If you want to use an existing BookKeeper cluster, you can add the `--existing-bk-metadata-service-uri` flag as follows:
>
> ```properties
> --existing-bk-metadata-service-uri "zk+null://zk1:2181;zk2:2181/ledgers" \
> --web-service-url http://host1:8080,host2:8080,host3:8080 \
> --web-service-url-tls https://host1:8443,host2:8443,host3:8443 \
> --broker-service-url pulsar://host1:6650,host2:6650,host3:6650 \
> --broker-service-url-tls pulsar+ssl://host1:6651,host2:6651,host3:6651
> ```
> You can obtain the metadata service URI of the existing BookKeeper cluster by using the `bin/bookkeeper shell whatisinstanceid` command. You must enclose the value in double quotes since the multiple metadata service URIs are separated with semicolons.
## Deploy a BookKeeper cluster
[BookKeeper](https://bookkeeper.apache.org) handles all persistent data storage in Pulsar. You need to deploy a cluster of BookKeeper bookies to use Pulsar. You can choose to run a **3-bookie BookKeeper cluster**.
Expand Down
7 changes: 6 additions & 1 deletion site2/docs/reference-cli-tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -174,10 +174,15 @@ Options
|`-ub` , `--broker-service-url`|The broker service URL for the new cluster||
|`-tb` , `--broker-service-url-tls`|The broker service URL for the new cluster with TLS encryption||
|`-c` , `--cluster`|Cluster name||
|`--configuration-store`|The configuration store quorum connection string||
|`-cs` , `--configuration-store`|The configuration store quorum connection string||
|`--existing-bk-metadata-service-uri`|The metadata service URI of the existing BookKeeper cluster that you want to use||
|`-h` , `--help`|Cluster name|false|
|`--initial-num-stream-storage-containers`|The number of storage containers of BookKeeper stream storage|16|
|`--initial-num-transaction-coordinators`|The number of transaction coordinators assigned in a cluster|16|
|`-uw` , `--web-service-url`|The web service URL for the new cluster||
|`-tw` , `--web-service-url-tls`|The web service URL for the new cluster with TLS encryption||
|`-zk` , `--zookeeper`|The local ZooKeeper quorum connection string||
|`--zookeeper-session-timeout-ms`|The local ZooKeeper session timeout. The time unit is in millisecond(ms)|30000|


### `proxy`
Expand Down

0 comments on commit 3147e9e

Please sign in to comment.