Skip to content

Commit

Permalink
[pulsar-admin-tools] Support delete all data associated with a cluster (
Browse files Browse the repository at this point in the history
apache#8133)

### Motivation

When multiple broker clusters shared the same bookie cluster, if user wanted to remove a broker cluster, the associated ledgers in bookies should also be deleted.

### Modifications

- Add an option to `cluster delete` command to delete all the data associated with the cluster.

Currently there's no way to delete the ledgers of schema from client's side, a new REST API to delete schema's ledgers should be exposed or the existed `SchemaRegistryServiceImpl#deleteSchema`'s semantics should be change. 

### Verifying this change
The new option has been tested in local environment by:
1. Deploy a ZK which acts as both local ZK and the configuration store;
2. Init metadata of 2 clusters, the 2nd cluster use `--bookkeeper-metadata-service` option to share the same BK metadata position;
3. Deploy a BK;
4. Deploy 2 brokers associated with the 2 clusters' metadata;
5. Create some topics and produce some messages to 2 clusters;
6. Run `bin/pulsar cluster -all <cluster>` to delete the whole cluster, use `bin/bookkeeper shell list ledgers` to verify the change of ledgers count.

**An integration test is needed to simulate the procedures above.**
  • Loading branch information
BewareMyPower authored Sep 29, 2020
1 parent 73e5bc4 commit 0b041a2
Showing 1 changed file with 19 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -119,8 +119,27 @@ private class Delete extends CliCommand {
@Parameter(description = "cluster-name\n", required = true)
private java.util.List<String> params;

@Parameter(names = { "-a", "--all" }, description = "Delete all data (tenants) of the cluster\n", required = false)
private boolean deleteAll = false;

void run() throws PulsarAdminException {
String cluster = getOneArgument(params);

if (deleteAll) {
for (String tenant : admin.tenants().getTenants()) {
for (String namespace : admin.namespaces().getNamespaces(tenant)) {
for (String topic : admin.topics().getList(namespace)) {
admin.topics().delete(topic, true);
// TODO: Delete all the ledgers of the SchemaStorage
// admin.schemas().deleteSchema(topic) won't delete the schema's ledger. Instead a new ledger will be created.
// https://github.com/apache/pulsar/issues/8134
}
admin.namespaces().deleteNamespace(namespace);
}
admin.tenants().deleteTenant(tenant);
}
}

admin.clusters().deleteCluster(cluster);
}
}
Expand Down

0 comments on commit 0b041a2

Please sign in to comment.