Skip to content

Commit

Permalink
Add binary-protocol lookup url in documentation (apache#129)
Browse files Browse the repository at this point in the history
* Add binary-protocol lookup url in documentation

* add webServiceUrl and brokerServiceUrl to clientTools

* Fallback to old version serviceUrl property for clientTools
  • Loading branch information
rdhabalia authored and merlimat committed Feb 8, 2017
1 parent 9833b19 commit d278601
Show file tree
Hide file tree
Showing 11 changed files with 59 additions and 21 deletions.
3 changes: 2 additions & 1 deletion conf/client.conf
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
#

# Pulsar Client configuration
serviceUrl=http://localhost:8080/
webServiceUrl=http://localhost:8080/
brokerServiceUrl=pulsar://localhost:6650/
#authPlugin=
#authParams=
#useTls=
Expand Down
4 changes: 2 additions & 2 deletions docs/Architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -197,7 +197,7 @@ Pulsar enables messages to be produced and consumed in different geo-locations.

### Authentication and Authorization

TODO
Pulsar supports a pluggable [authentication](https://github.com/yahoo/pulsar/blob/master/docs/Authentication.md) mechanism which can be configured at broker and it also supports [authorization](https://github.com/yahoo/pulsar/blob/master/docs/Authorization.md) to identify client and its access rights on topics and properties.


# Client Library
Expand All @@ -219,7 +219,7 @@ A PulsarClient (TODO: javadocs) instance is needed before producing/consuming me

```java
ClientConfiguration config = new ClientConfiguration();
PulsarClient pulsarClient = PulsarClient.create("http://broker.example.com:8080", config);
PulsarClient pulsarClient = PulsarClient.create("pulsar://broker.example.com:6650", config);
...
pulsarClient.close();
```
Expand Down
2 changes: 1 addition & 1 deletion docs/Authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,7 +136,7 @@ authParams.put("tlsKeyFile", "/path/to/client-cert.pem");
conf.setAuthentication(AuthenticationTls.class.getName(), authParams);

PulsarClient client = PulsarClient.create(
"https://my-broker.com:4443", conf);
"pulsar+ssl://my-broker.com:6651", conf);
```

#### Configure CLI tools
Expand Down
9 changes: 5 additions & 4 deletions docs/ClusterSetup.md
Original file line number Diff line number Diff line change
Expand Up @@ -136,8 +136,10 @@ as well as the Pulsar metadata.
$ bin/pulsar initialize-cluster-metadata --cluster us-west \
--zookeeper zk1.us-west.example.com:2181 \
--global-zookeeper zk1.us-west.example.com:2184 \
--service-url http://pulsar.us-west.example.com:8080/ \
--service-url-tls https://pulsar.us-west.example.com:8443/
--web-service-url http://pulsar.us-west.example.com:8080/ \
--web-service-url-tls https://pulsar.us-west.example.com:8443/ \
--broker-service-url pulsar://pulsar.us-west.example.com:6650/ \
--broker-service-url-tls pulsar+ssl://pulsar.us-west.example.com:6651/
```

#### BookKeeper
Expand Down Expand Up @@ -241,8 +243,7 @@ only requirement is that when the client does a HTTP request on
`http://pulsar.us-west.example.com:8080/` it must be redirected (through DNS, IP
or HTTP redirect) to an active broker, without preference.

The included discovery service works with HTTP redirect and it maintains the
list of active brokers from ZooKeeper.
The included discovery service maintains the list of active brokers from ZooKeeper and it supports lookup redirection with HTTP and also with [binary protocol](https://github.com/yahoo/pulsar/blob/master/docs/BinaryProtocol.md#service-discovery).

Add the ZK servers in `conf/discovery.conf`:
```shell
Expand Down
6 changes: 3 additions & 3 deletions docs/GettingStarted.md
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ $ bin/pulsar standalone
```

The Pulsar service is now ready to use and we can point
clients to use service URL `http://localhost:8080/`
clients to use service URL as either `http://localhost:8080/` or `pulsar://localhost:6650`

A sample namespace, `sample/standalone/ns1`, is already available.

Expand All @@ -109,7 +109,7 @@ Latest version is [![Maven Central](https://maven-badges.herokuapp.com/maven-cen
### Consumer

```java
PulsarClient client = PulsarClient.create("http://localhost:8080");
PulsarClient client = PulsarClient.create("pulsar://localhost:6650");

Consumer consumer = client.subscribe(
"persistent://sample/standalone/ns1/my-topic",
Expand All @@ -132,7 +132,7 @@ client.close();
### Producer

```java
PulsarClient client = PulsarClient.create("http://localhost:8080");
PulsarClient client = PulsarClient.create("pulsar://localhost:6650");

Producer producer = client.createProducer(
"persistent://sample/standalone/ns1/my-topic");
Expand Down
4 changes: 2 additions & 2 deletions docs/PulsarStorm.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ MessageToValuesMapper messageToValuesMapper = new MessageToValuesMapper() {

// Configure a Pulsar Spout
PulsarSpoutConfiguration spoutConf = new PulsarSpoutConfiguration();
spoutConf.setServiceUrl("http://broker.messaging.usw.example.com:8080");
spoutConf.setServiceUrl("pulsar://broker.messaging.usw.example.com:6650");
spoutConf.setTopic("persistent://my-property/usw/my-ns/my-topic1");
spoutConf.setSubscriptionName("my-subscriber-name1");
spoutConf.setMessageToValuesMapper(messageToValuesMapper);
Expand Down Expand Up @@ -95,7 +95,7 @@ TupleToMessageMapper tupleToMessageMapper = new TupleToMessageMapper() {

// Configure a Pulsar Bolt
PulsarBoltConfiguration boltConf = new PulsarBoltConfiguration();
boltConf.setServiceUrl("http://broker.messaging.usw.example.com:8080");
boltConf.setServiceUrl("pulsar://broker.messaging.usw.example.com:6650");
boltConf.setTopic("persistent://my-property/usw/my-ns/my-topic2");
boltConf.setTupleToMessageMapper(tupleToMessageMapper);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -44,12 +44,19 @@ private static class Arguments {
@Parameter(names = { "-c", "--cluster" }, description = "Cluster name", required = true)
private String cluster;

@Parameter(names = { "-u", "--service-url" }, description = "Service URL for new cluster", required = true)
private String clusterServiceUrl;
@Parameter(names = { "-uw", "--web-service-url" }, description = "Web-service URL for new cluster", required = true)
private String clusterWebServiceUrl;

@Parameter(names = { "-t",
"--service-url-tls" }, description = "Service URL for new cluster with TLS encryption", required = false)
private String clusterServiceUrlTls;
@Parameter(names = { "-tw",
"--web-service-url-tls" }, description = "Web-service URL for new cluster with TLS encryption", required = false)
private String clusterWebServiceUrlTls;

@Parameter(names = { "-ub", "--broker-service-url" }, description = "Broker-service URL for new cluster", required = false)
private String clusterBrokerServiceUrl;

@Parameter(names = { "-tb",
"--broker-service-url-tls" }, description = "Broker-service URL for new cluster with TLS encryption", required = false)
private String clusterBrokerServiceUrlTls;

@Parameter(names = { "-zk",
"--zookeeper" }, description = "Local ZooKeeper quorum connection string", required = true)
Expand Down Expand Up @@ -101,7 +108,8 @@ public static void main(String[] args) throws Exception {
ZkUtils.createFullPathOptimistic(globalZk, "/admin/clusters", new byte[0], ZooDefs.Ids.OPEN_ACL_UNSAFE,
CreateMode.PERSISTENT);

ClusterData clusterData = new ClusterData(arguments.clusterServiceUrl, arguments.clusterServiceUrlTls);
ClusterData clusterData = new ClusterData(arguments.clusterWebServiceUrl, arguments.clusterWebServiceUrlTls,
arguments.clusterBrokerServiceUrl, arguments.clusterBrokerServiceUrlTls);
byte[] clusterDataJson = ObjectMapperFactory.getThreadLocal().writeValueAsBytes(clusterData);

globalZk.create("/admin/clusters/" + arguments.cluster, clusterDataJson, ZooDefs.Ids.OPEN_ACL_UNSAFE,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,8 @@
import java.util.Arrays;
import java.util.Properties;

import org.apache.commons.lang3.StringUtils;

import com.beust.jcommander.JCommander;
import com.beust.jcommander.Parameter;
import com.yahoo.pulsar.client.admin.PulsarAdmin;
Expand All @@ -34,7 +36,9 @@ public class PulsarAdminTool {
private boolean help;

PulsarAdminTool(Properties properties) throws Exception {
String serviceUrl = properties.getProperty("serviceUrl");
// fallback to previous-version serviceUrl property to maintain backward-compatibility
String serviceUrl = StringUtils.isNotBlank(properties.getProperty("webServiceUrl"))
? properties.getProperty("webServiceUrl") : properties.getProperty("serviceUrl");
String authPluginClassName = properties.getProperty("authPlugin");
String authParams = properties.getProperty("authParams");
boolean useTls = Boolean.parseBoolean(properties.getProperty("useTls"));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@
import java.util.Arrays;
import java.util.Properties;

import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

Expand Down Expand Up @@ -59,7 +60,12 @@ public class PulsarClientTool {
CmdConsume consumeCommand;

public PulsarClientTool(Properties properties) throws MalformedURLException {
this.serviceURL = properties.getProperty("serviceUrl");
this.serviceURL = StringUtils.isNotBlank(properties.getProperty("brokerServiceUrl"))
? properties.getProperty("brokerServiceUrl") : properties.getProperty("webServiceUrl");
// fallback to previous-version serviceUrl property to maintain backward-compatibility
if (StringUtils.isBlank(this.serviceURL)) {
this.serviceURL = properties.getProperty("serviceUrl");
}
this.authPluginClassName = properties.getProperty("authPlugin");
this.authParams = properties.getProperty("authParams");
this.useTls = Boolean.parseBoolean(properties.getProperty("useTls"));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -129,6 +129,15 @@ public static void main(String[] args) throws Exception {
Properties prop = new Properties(System.getProperties());
prop.load(new FileInputStream(arguments.confFile));

if (arguments.serviceURL == null) {
arguments.serviceURL = prop.getProperty("brokerServiceUrl");
}

if (arguments.serviceURL == null) {
arguments.serviceURL = prop.getProperty("webServiceUrl");
}

// fallback to previous-version serviceUrl property to maintain backward-compatibility
if (arguments.serviceURL == null) {
arguments.serviceURL = prop.getProperty("serviceUrl", "http://localhost:8080/");
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -163,6 +163,15 @@ public static void main(String[] args) throws Exception {
Properties prop = new Properties(System.getProperties());
prop.load(new FileInputStream(arguments.confFile));

if (arguments.serviceURL == null) {
arguments.serviceURL = prop.getProperty("brokerServiceUrl");
}

if (arguments.serviceURL == null) {
arguments.serviceURL = prop.getProperty("webServiceUrl");
}

// fallback to previous-version serviceUrl property to maintain backward-compatibility
if (arguments.serviceURL == null) {
arguments.serviceURL = prop.getProperty("serviceUrl", "http://localhost:8080/");
}
Expand Down

0 comments on commit d278601

Please sign in to comment.