update website for 2.4.1 release (apache#5080)

Update website for 2.4.1 release
JamieCressey · Sep 3, 2019 · 5212497 · 5212497
1 parent d0bde68
commit 5212497
Show file tree

Hide file tree

Showing 43 changed files with 15,075 additions and 0 deletions.
diff --git a/site2/website/versioned_docs/version-2.4.1/admin-api-non-persistent-topics.md b/site2/website/versioned_docs/version-2.4.1/admin-api-non-persistent-topics.md
@@ -0,0 +1,264 @@
+---
+id: version-2.4.1-admin-api-non-persistent-topics
+title: Managing non-persistent topics
+sidebar_label: Non-Persistent topics
+original_id: admin-api-non-persistent-topics
+---
+
+Non-persistent can be used in applications that only want to consume real time published messages and
+do not need persistent guarantee that can also reduce message-publish latency by removing overhead of
+persisting messages.
+
+In all of the instructions and commands below, the topic name structure is:
+
+```shell
+non-persistent://tenant/namespace/topic
+```
+
+## Non-persistent topics resources
+
+### Get stats
+
+It shows current statistics of a given non-partitioned topic.
+
+  -   **msgRateIn**: The sum of all local and replication publishers' publish rates in messages per second
+
+  -   **msgThroughputIn**: Same as above, but in bytes per second instead of messages per second
+
+  -   **msgRateOut**: The sum of all local and replication consumers' dispatch rates in messages per second
+
+  -   **msgThroughputOut**: Same as above, but in bytes per second instead of messages per second
+
+  -   **averageMsgSize**: The average size in bytes of messages published within the last interval
+
+  -   **publishers**: The list of all local publishers into the topic. There can be zero or thousands
+
+  -   **averageMsgSize**: Average message size in bytes from this publisher within the last interval
+
+  -   **producerId**: Internal identifier for this producer on this topic
+
+  -   **producerName**: Internal identifier for this producer, generated by the client library
+
+  -   **address**: IP address and source port for the connection of this producer
+
+  -   **connectedSince**: Timestamp this producer was created or last reconnected
+
+  -   **subscriptions**: The list of all local subscriptions to the topic
+
+  -   **my-subscription**: The name of this subscription (client defined)
+
+  -   **type**: This subscription type
+
+  -   **consumers**: The list of connected consumers for this subscription
+
+  -   **consumerName**: Internal identifier for this consumer, generated by the client library
+
+  -   **availablePermits**: The number of messages this consumer has space for in the client library's listen queue. A value less than 1 means the client library's queue is full and receive() isn't being called. A non-negative value means this consumer is ready to be dispatched messages.
+
+  -   **replication**: This section gives the stats for cross-colo replication of this topic
+
+  -   **connected**: Whether the outbound replicator is connected
+
+  -   **inboundConnection**: The IP and port of the broker in the remote cluster's publisher connection to this broker
+
+  -   **inboundConnectedSince**: The TCP connection being used to publish messages to the remote cluster. If there are no local publishers connected, this connection is automatically closed after a minute.
+
+  -   **msgDropRate**: for publisher: publish: broker only allows configured number of in flight per connection, and drops all other published messages above the threshold. Broker also drops messages for subscriptions in case of unavailable limit and connection is not writable.
+
+
+```json
+{
+  "msgRateIn": 4641.528542257553,
+  "msgThroughputIn": 44663039.74947473,
+  "msgRateOut": 0,
+  "msgThroughputOut": 0,
+  "averageMsgSize": 1232439.816728665,
+  "storageSize": 135532389160,
+  "msgDropRate" : 0.0,
+  "publishers": [
+    {
+      "msgRateIn": 57.855383881403576,
+      "msgThroughputIn": 558994.7078932219,
+      "averageMsgSize": 613135,
+      "producerId": 0,
+      "producerName": null,
+      "address": null,
+      "connectedSince": null,
+      "msgDropRate" : 0.0
+    }
+  ],
+  "subscriptions": {
+    "my-topic_subscription": {
+      "msgRateOut": 0,
+      "msgThroughputOut": 0,
+      "msgBacklog": 116632,
+      "type": null,
+      "msgRateExpired": 36.98245516804671,
+       "consumers" : [ {
+        "msgRateOut" : 20343.506296021893,
+        "msgThroughputOut" : 2.0979855364233278E7,
+        "msgRateRedeliver" : 0.0,
+        "consumerName" : "fe3c0",
+        "availablePermits" : 950,
+        "unackedMessages" : 0,
+        "blockedConsumerOnUnackedMsgs" : false,
+        "address" : "/10.73.210.249:60578",
+        "connectedSince" : "2017-07-26 15:13:48.026-0700",
+        "clientVersion" : "1.19-incubating-SNAPSHOT"
+      } ],
+      "msgDropRate" : 432.2390921571593
+
+    }
+  },
+  "replication": {}
+}
+```
+
+#### pulsar-admin
+
+Topic stats can be fetched using [`stats`](reference-pulsar-admin.md#stats) command.
+
+```shell
+$ pulsar-admin non-persistent stats \
+  non-persistent://test-tenant/ns1/tp1 \
+```
+
+#### REST API
+
+{@inject: endpoint|GET|/admin/v2/non-persistent/:tenant/:namespace/:topic/stats|operation/getStats}
+
+
+#### Java
+
+```java
+String topic = "non-persistent://my-tenant/my-namespace/my-topic";
+admin.nonPersistentTopics().getStats(topic);
+```
+
+### Get internal stats
+
+It shows detailed statistics of a topic.
+
+#### pulsar-admin
+
+Topic internal-stats can be fetched using [`stats-internal`](reference-pulsar-admin.md#stats-internal) command.
+
+```shell
+$ pulsar-admin non-persistent stats-internal \
+  non-persistent://test-tenant/ns1/tp1 \
+
+{
+  "entriesAddedCounter" : 48834,
+  "numberOfEntries" : 0,
+  "totalSize" : 0,
+  "cursors" : {
+    "s1" : {
+      "waitingReadOp" : false,
+      "pendingReadOps" : 0,
+      "messagesConsumedCounter" : 0,
+      "cursorLedger" : 0,
+      "cursorLedgerLastEntry" : 0
+    }
+  }
+}
+
+```
+
+#### REST API
+
+{@inject: endpoint|GET|/admin/v2/non-persistent/:tenant/:namespace/:topic/internalStats|operation/getInternalStats}
+
+#### Java
+
+```java
+String topic = "non-persistent://my-tenant/my-namespace/my-topic";
+admin.nonPersistentTopics().getInternalStats(topic);
+```
+
+### Create partitioned topic
+
+Partitioned topics in Pulsar must be explicitly created. When creating a new partitioned topic you need to provide a name for the topic as well as the desired number of partitions.
+
+> #### Note
+>
+> By default, after 60 seconds of creation, topics are considered inactive and deleted automatically to prevent from generating trash data.
+>
+> To disable this feature, set `brokerDeleteInactiveTopicsEnabled` to `false`.
+>
+> To change the frequency of checking inactive topics, set `brokerDeleteInactiveTopicsFrequencySeconds` to your desired value.
+>
+> For more information about these two parameters, see [here](reference-configuration.md#broker).
+
+#### pulsar-admin
+
+```shell
+$ bin/pulsar-admin non-persistent create-partitioned-topic \
+  non-persistent://my-tenant/my-namespace/my-topic \
+  --partitions 4
+```
+
+#### REST API
+
+{@inject: endpoint|PUT|/admin/v2/non-persistent/:tenant/:namespace/:topic/partitions|operation/createPartitionedTopic}
+
+#### Java
+
+```java
+String topicName = "non-persistent://my-tenant/my-namespace/my-topic";
+int numPartitions = 4;
+admin.nonPersistentTopics().createPartitionedTopic(topicName, numPartitions);
+```
+
+### Get metadata
+
+Partitioned topics have metadata associated with them that you can fetch as a JSON object. The following metadata fields are currently available:
+
+Field | Meaning
+:-----|:-------
+`partitions` | The number of partitions into which the topic is divided
+
+#### pulsar-admin
+
+```shell
+$ pulsar-admin non-persistent get-partitioned-topic-metadata \
+  non-persistent://my-tenant/my-namespace/my-topic
+{
+  "partitions": 4
+}
+```
+
+#### REST API
+
+{@inject: endpoint|GET|/admin/v2/non-persistent/:tenant/:namespace/:topic/partitions|operation/getPartitionedMetadata}
+
+
+#### Java
+
+```java
+String topicName = "non-persistent://my-tenant/my-namespace/my-topic";
+admin.nonPersistentTopics().getPartitionedTopicMetadata(topicName);
+```
+
+### Unload topic
+
+It unloads a topic.
+
+#### pulsar-admin
+
+Topic can be unloaded using [`unload`](reference-pulsar-admin.md#unload) command.
+
+```shell
+$ pulsar-admin non-persistent unload \
+  non-persistent://test-tenant/ns1/tp1 \
+```
+
+#### REST API
+
+{@inject: endpoint|PUT|/admin/v2/non-persistent/:tenant/:namespace/:topic/unload|operation/unloadTopic}
+
+#### Java
+
+```java
+String topic = "non-persistent://my-tenantmy-namespace/my-topic";
+admin.nonPersistentTopics().unload(topic);
+```
diff --git a/site2/website/versioned_docs/version-2.4.1/admin-api-overview.md b/site2/website/versioned_docs/version-2.4.1/admin-api-overview.md
@@ -0,0 +1,89 @@
+---
+id: version-2.4.1-admin-api-overview
+title: The Pulsar admin interface
+sidebar_label: Overview
+original_id: admin-api-overview
+---
+
+The Pulsar admin interface enables you to manage all of the important entities in a Pulsar [instance](reference-terminology.md#instance), such as [tenants](reference-terminology.md#tenant), [topics](reference-terminology.md#topic), and [namespaces](reference-terminology.md#namespace).
+
+You can currently interact with the admin interface via:
+
+- Making HTTP calls against the admin {@inject: rest:REST:/} API provided by Pulsar [brokers](reference-terminology.md#broker). For some restful apis, they might be redirected to topic owner brokers for serving
+   with [`307 Temporary Redirect`](https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/307), hence the HTTP callers should handle `307 Temporary Redirect`. If you are using `curl`, you should specify `-L`
+   to handle redirections.
+- The `pulsar-admin` CLI tool, which is available in the `bin` folder of your [Pulsar installation](getting-started-standalone.md):
+
+```shell
+$ bin/pulsar-admin
+```
+
+Full documentation for this tool can be found in the [Pulsar command-line tools](reference-pulsar-admin.md) doc.
+
+- A Java client interface.
+
+> #### The REST API is the admin interface
+> Under the hood, both the `pulsar-admin` CLI tool and the Java client both use the REST API. If you’d like to implement your own admin interface client, you should use the REST API as well. Full documentation can be found here.
+
+In this document, examples from each of the three available interfaces will be shown.
+
+## Admin setup
+
+Each of Pulsar's three admin interfaces---the [`pulsar-admin`](reference-pulsar-admin.md) CLI tool, the [Java admin API](/api/admin), and the {@inject: rest:REST:/} API ---requires some special setup if you have [authentication](security-overview.md#authentication-providers) enabled in your Pulsar [instance](reference-terminology.md#instance).
+
+### pulsar-admin
+
+If you have [authentication](security-overview.md#authentication-providers) enabled, you will need to provide an auth configuration to use the [`pulsar-admin`](reference-pulsar-admin.md) tool. By default, the configuration for the `pulsar-admin` tool is found in the [`conf/client.conf`](reference-configuration.md#client) file. Here are the available parameters:
+
+|Name|Description|Default|
+|----|-----------|-------|
+|webServiceUrl|The web URL for the cluster.|http://localhost:8080/|
+|brokerServiceUrl|The Pulsar protocol URL for the cluster.|pulsar://localhost:6650/|
+|authPlugin|The authentication plugin.| |
+|authParams|The authentication parameters for the cluster, as a comma-separated string.| |
+|useTls|Whether or not TLS authentication will be enforced in the cluster.|false|
+|tlsAllowInsecureConnection|Accept untrusted TLS certificate from client.|false|
+|tlsTrustCertsFilePath|Path for the trusted TLS certificate file.| |
+
+### REST API
+
+You can find documentation for the REST API exposed by Pulsar [brokers](reference-terminology.md#broker) in this reference {@inject: rest:document:/}.
+
+### Java admin client
+
+To use the Java admin API, instantiate a {@inject: javadoc:PulsarAdmin:/admin/org/apache/pulsar/client/admin/PulsarAdmin} object, specifying a URL for a Pulsar [broker](reference-terminology.md#broker) and a {@inject: javadoc:PulsarAdminBuilder:/admin/org/apache/pulsar/client/admin/PulsarAdminBuilder}. Here's a minimal example using `localhost`:
+
+```java
+String url = "http://localhost:8080";
+// Pass auth-plugin class fully-qualified name if Pulsar-security enabled
+String authPluginClassName = "com.org.MyAuthPluginClass";
+// Pass auth-param if auth-plugin class requires it
+String authParams = "param1=value1";
+boolean useTls = false;
+boolean tlsAllowInsecureConnection = false;
+String tlsTrustCertsFilePath = null;
+PulsarAdmin admin = PulsarAdmin.builder()
+.authentication(authPluginClassName,authParams)
+.serviceHttpUrl(url)
+.tlsTrustCertsFilePath(tlsTrustCertsFilePath)
+.allowTlsInsecureConnection(tlsAllowInsecureConnection)
+.build();
+```
+
+If you have multiple brokers to use, you can use multi-host like Pulsar service. For example,
+```java
+String url = "http://localhost:8080,localhost:8081,localhost:8082";
+// Pass auth-plugin class fully-qualified name if Pulsar-security enabled
+String authPluginClassName = "com.org.MyAuthPluginClass";
+// Pass auth-param if auth-plugin class requires it
+String authParams = "param1=value1";
+boolean useTls = false;
+boolean tlsAllowInsecureConnection = false;
+String tlsTrustCertsFilePath = null;
+PulsarAdmin admin = PulsarAdmin.builder()
+.authentication(authPluginClassName,authParams)
+.serviceHttpUrl(url)
+.tlsTrustCertsFilePath(tlsTrustCertsFilePath)
+.allowTlsInsecureConnection(tlsAllowInsecureConnection)
+.build();
+```