Skip to content

Commit

Permalink
[Issue 3681][docs] Update client-libraries in Get Started chapter (ap…
Browse files Browse the repository at this point in the history 
…ache#6050)

Fixes apache#3681 

### Motivation
The client library doc content is out of date.

### Modifications
1. Pulsar supports more clients, add links to the new clients.
2. Remove redundant information, and will update each client respectively.
3. Update Pulsar client go info.
4. Proofread parts of the Python/Go client content while reading.
5. Fixes apache#3681 by adding examples in Python client.
6. Update Python API.
  • Loading branch information
@Jennifer88huang-zz @codelipenghui
Jennifer88huang-zz authored and codelipenghui committed Jan 22, 2020
1 parent cb19ee8 commit 992fc1e
Show file tree
Hide file tree
Showing 5 changed files with 73 additions and 79 deletions.
2 changes: 1 addition & 1 deletion pulsar-client-cpp/python/pulsar/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -549,7 +549,7 @@ def subscribe(self, topic, subscription_name,
This method will accept these forms:
- `topic='my-topic'`
- `topic=['topic-1', 'topic-2', 'topic-3']`
- `topic=re.compile('topic-.*')`
- `topic=re.compile('persistent://public/default/topic-*')`
* `subscription`: The name of the subscription.
**Options**
Expand Down
2 changes: 2 additions & 0 deletions site2/docs/client-libraries-cpp.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ sidebar_label: C++

Pulsar C++ client is supported on **Linux** and **MacOS** platforms.

[Doxygen](http://www.doxygen.nl/)-generated API docs for the C++ client are available [here](/api/cpp).

## Linux

> Since 2.1.0 release, Pulsar ships pre-built RPM and Debian packages. You can download and install those packages directly.
Expand Down
22 changes: 14 additions & 8 deletions site2/docs/client-libraries-go.md
View file
Original file line number Diff line number Diff line change
@@ -1,10 +1,17 @@
---
id: client-libraries-go
title: The Pulsar Go client
title: Pulsar Go client
sidebar_label: Go
---

The Pulsar Go client can be used to create Pulsar [producers](#producers), [consumers](#consumers), and [readers](#readers) in Go (aka Golang).
You can use Pulsar Go client to create Pulsar [producers](#producers), [consumers](#consumers), and [readers](#readers) in Go (aka Golang).

Currently, the following Go clients are maintained in two repositories.

| Language | Project | Maintainer | License | Description |
|----------|---------|------------|---------|-------------|
| CGo | [pulsar-client-go](https://github.com/apache/pulsar/tree/master/pulsar-client-go) | [Apache Pulsar](https://github.com/apache/pulsar) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | CGo client that depends on C++ client library |
| Go | [pulsar-client-go](https://github.com/apache/pulsar-client-go) | [Apache Pulsar](https://github.com/apache/pulsar) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | A native golang client |

> #### API docs available as well
> For standard API docs, consult the [Godoc](https://godoc.org/github.com/apache/pulsar/pulsar-client-go/pulsar).
Expand All @@ -15,10 +22,9 @@ The Pulsar Go client can be used to create Pulsar [producers](#producers), [cons
### Requirements

Pulsar Go client library is based on the C++ client library. Follow
the instructions for [C++ library](client-libraries-cpp.md) for installing the binaries
through [RPM](client-libraries-cpp.md#rpm), [Deb](client-libraries-cpp.md#deb) or [Homebrew packages](client-libraries-cpp.md#macos).
the instructions for [C++ library](client-libraries-cpp.md) for installing the binaries through [RPM](client-libraries-cpp.md#rpm), [Deb](client-libraries-cpp.md#deb) or [Homebrew packages](client-libraries-cpp.md#macos).

### Installing go package
### Install go package

> #### Compatibility Warning
> The version number of the Go client **must match** the version number of the Pulsar C++ client library.
Expand Down Expand Up @@ -63,7 +69,7 @@ If you're using [TLS](security-tls-authentication.md) authentication, the URL wi
pulsar+ssl://pulsar.us-west.example.com:6651
```

## Creating a client
## Create a client

In order to interact with Pulsar, you'll first need a `Client` object. You can create a client object using the `NewClient` function, passing in a `ClientOptions` object (more on configuration [below](#client-configuration)). Here's an example:

Expand Down Expand Up @@ -433,8 +439,8 @@ reader, err := client.CreateReader(pulsar.ReaderOptions{

Parameter | Description | Default
:---------|:------------|:-------
`Topic` | The Pulsar [topic](reference-terminology.md#topic) on which the reader will establish a subscription and listen for messages |
`Name` | The name of the reader |
`Topic` | The Pulsar [topic](reference-terminology.md#topic) on which the reader will establish a subscription and listen for messages
`Name` | The name of the reader
`StartMessageID` | THe initial reader position, i.e. the message at which the reader begins processing messages. The options are `pulsar.EarliestMessage` (the earliest available message on the topic), `pulsar.LatestMessage` (the latest available message on the topic), or a `MessageID` object for a position that isn't earliest or latest. |
`MessageChannel` | The Go channel used by the reader. Messages that arrive from the Pulsar topic(s) will be passed to this channel. |
`ReceiverQueueSize` | Sets the size of the reader's receiver queue, i.e. the number of messages that can be accumulated by the reader before the application calls `Next`. A value higher than the default of 1000 could increase reader throughput, though at the expense of more memory utilization. | 1000
Expand Down
77 changes: 45 additions & 32 deletions site2/docs/client-libraries-python.md
View file
Original file line number Diff line number Diff line change
@@ -1,16 +1,18 @@
---
id: client-libraries-python
title: The Pulsar Python client
title: Pulsar Python client
sidebar_label: Python
---

The Pulsar Python client library is a wrapper over the existing [C++ client library](client-libraries-cpp.md) and exposes all of the [same features](/api/cpp). You can find the code in the [`python` subdirectory](https://github.com/apache/pulsar/tree/master/pulsar-client-cpp/python) of the C++ client code.
Pulsar Python client library is a wrapper over the existing [C++ client library](client-libraries-cpp.md) and exposes all of the [same features](/api/cpp). You can find the code in the [`python` subdirectory](https://github.com/apache/pulsar/tree/master/pulsar-client-cpp/python) of the C++ client code.

## Installation
[pdoc](https://github.com/BurntSushi/pdoc)-generated API docs for the Python client are available [here](/api/python).

## Install

You can install the [`pulsar-client`](https://pypi.python.org/pypi/pulsar-client) library either via [PyPi](https://pypi.python.org/pypi), using [pip](#installation-using-pip), or by building the library from source.

### Installation using pip
### Install using pip

To install the `pulsar-client` library as a pre-built package using the [pip](https://pip.pypa.io/en/stable/) package manager:

Expand All @@ -25,9 +27,9 @@ Platform | Supported Python versions
MacOS <br /> 10.13 (High Sierra), 10.14 (Mojave) <br /> | 2.7, 3.7
Linux | 2.7, 3.4, 3.5, 3.6, 3.7

### Installing from source
### Install from source

To install the `pulsar-client` library by building from source, follow [these instructions](client-libraries-cpp.md#compilation) and compile the Pulsar C++ client library. That will also build the Python binding for the library.
To install the `pulsar-client` library by building from source, follow [instructions](client-libraries-cpp.md#compilation) and compile the Pulsar C++ client library. That builds the Python binding for the library.

To install the built Python bindings:

Expand All @@ -43,11 +45,11 @@ The complete Python API reference is available at [api/python](/api/python).

## Examples

Below you'll find a variety of Python code examples for the `pulsar-client` library.
You can find a variety of Python code examples for the `pulsar-client` library.

### Producer example

This creates a Python producer for the `my-topic` topic and send 10 messages on that topic:
The following example creates a Python producer for the `my-topic` topic and sends 10 messages on that topic:

```python
import pulsar
Expand All @@ -64,7 +66,7 @@ client.close()

### Consumer example

This creates a consumer with the `my-subscription` subscription on the `my-topic` topic, listen for incoming messages, print the content and ID of messages that arrive, and acknowledge each message to the Pulsar broker:
The following example creates a consumer with the `my-subscription` subscription name on the `my-topic` topic, receives incoming messages, prints the content and ID of messages that arrive, and acknowledges each message to the Pulsar broker.

```python
consumer = client.subscribe('my-topic', 'my-subscription')
Expand Down Expand Up @@ -97,13 +99,32 @@ while True:
print("Received message '{}' id='{}'".format(msg.data(), msg.message_id()))
# No acknowledgment
```
### Multi-topic subscriptions

In addition to subscribing a consumer to a single Pulsar topic, you can also subscribe to multiple topics simultaneously. To use multi-topic subscriptions, you can supply a regular expression (regex) or a `List` of topics. If you select topics via regex, all topics must be within the same Pulsar namespace.

The following is an example.

```python
import re
consumer = client.subscribe(re.compile('persistent://public/default/topic-*'), 'my-subscription')
while True:
msg = consumer.receive()
try:
print("Received message '{}' id='{}'".format(msg.data(), msg.message_id()))
# Acknowledge successful processing of the message
consumer.acknowledge(msg)
except:
# Message failed to be processed
consumer.negative_acknowledge(msg)
client.close()
```

## Schema

### Declaring and validating schema
### Declare and validate schema

A schema can be declared by passing a class that inherits
You can declare a schema by passing a class that inherits
from `pulsar.schema.Record` and defines the fields as
class variables. For example:

Expand All @@ -116,8 +137,7 @@ class Example(Record):
c = Boolean()
```

With this simple schema definition we can then create producers,
consumers and readers instances that will be referring to that.
With this simple schema definition, you can create producers, consumers and readers instances that refer to that.

```python
producer = client.create_producer(
Expand All @@ -127,13 +147,9 @@ producer = client.create_producer(
producer.send(Example(a='Hello', b=1))
```

When the producer is created, the Pulsar broker will validate that
the existing topic schema is indeed of "Avro" type and that the
format is compatible with the schema definition of the `Example`
class.
After creating the producer, the Pulsar broker validates that the existing topic schema is indeed of "Avro" type and that the format is compatible with the schema definition of the `Example` class.

If there is a mismatch, the producer creation will raise an
exception.
If there is a mismatch, an exception occurs in the producer creation.

Once a producer is created with a certain schema definition,
it will only accept objects that are instances of the declared
Expand Down Expand Up @@ -163,8 +179,7 @@ while True:

### Supported schema types

There are different builtin schema types that can be used in Pulsar.
All the definitions are in the `pulsar.schema` package.
You can use different builtin schema types in Pulsar. All the definitions are in the `pulsar.schema` package.

| Schema | Notes |
| ------ | ----- |
Expand All @@ -175,12 +190,11 @@ All the definitions are in the `pulsar.schema` package.

### Schema definition reference

The schema definition is done through a class that inherits from
`pulsar.schema.Record`.
The schema definition is done through a class that inherits from `pulsar.schema.Record`.

This class can have a number of fields which can be of either
`pulsar.schema.Field` type or even another nested `Record`. All the
fields are also specified in the `pulsar.schema` package. The fields
This class has a number of fields which can be of either
`pulsar.schema.Field` type or another nested `Record`. All the
fields are specified in the `pulsar.schema` package. The fields
are matching the AVRO fields types.

| Field Type | Python Type | Notes |
Expand All @@ -192,20 +206,19 @@ are matching the AVRO fields types.
| `Double` | `float` | |
| `Bytes` | `bytes` | |
| `String` | `str` | |
| `Array` | `list` | Need to specify record type for items |
| `Map` | `dict` | Key is always `String`. Need to specify value type |
| `Array` | `list` | Need to specify record type for items. |
| `Map` | `dict` | Key is always `String`. Need to specify value type. |

Additionally, any Python `Enum` type can be used as a valid field
type
Additionally, any Python `Enum` type can be used as a valid field type.

#### Fields parameters

When adding a field these parameters can be used in the constructor:
When adding a field, you can use these parameters in the constructor.

| Argument | Default | Notes |
| ---------- | --------| ----- |
| `default` | `None` | Set a default value for the field. Eg: `a = Integer(default=5)` |
| `required` | `False` | Mark the field as "required". This will set it in the schema accordingly. |
| `required` | `False` | Mark the field as "required". It is set in the schema accordingly. |

#### Schema definition examples

Expand Down
49 changes: 11 additions & 38 deletions site2/docs/getting-started-clients.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,46 +6,19 @@ sidebar_label: Use Pulsar with client libraries

Pulsar supports the following client libraries:

- [Java client](#java-client)
- [Go client](#go-client)
- [Python client](#python-client)
- [C++ client](#c-client)

## Java client

For instructions on how to use the Pulsar Java client to produce and consume messages, see [Pulsar Java client](client-libraries-java.md).

Two independent sets of Javadoc API docs are available.

Library | Purpose
:-------|:-------
[`org.apache.pulsar.client.api`](/api/client) | The [Pulsar Java client](client-libraries-java.md) is used to produce and consume messages on Pulsar topics.
[`org.apache.pulsar.client.admin`](/api/admin) | The Java client for the [Pulsar admin interface](admin-api-overview.md).


## Go client

For a tutorial on using the Pulsar Go client, see [Pulsar Go client](client-libraries-go.md).


## Python client

For a tutorial on using the Pulsar Python client, see [Pulsar Python client](client-libraries-python.md).

There are also [pdoc](https://github.com/BurntSushi/pdoc)-generated API docs for the Python client [here](/api/python).

## C++ client

For a tutorial on using the Pulsar C++ clent, see [Pulsar C++ client](client-libraries-cpp.md).

There are also [Doxygen](http://www.doxygen.nl/)-generated API docs for the C++ client [here](/api/cpp).

## Feature Matrix
- [Java client](client-libraries-java.md)
- [Go client](client-libraries-go.md)
- [Python client](client-libraries-python.md)
- [C++ client](client-libraries-cpp.md)
- [Node.js client](client-libraries-node.md)
- [WebSocket client](client-libraries-websocket.md)

## Feature matrix
Pulsar client feature matrix for different languages is listed on [Client Features Matrix](https://github.com/apache/pulsar/wiki/Client-Features-Matrix) page.

## Thirdparty Clients
## Third-party clients

Besides the official released clients, there are also multiple projects on developing a Pulsar client in different languages.
Besides the official released clients, multiple projects on developing Pulsar clients are available in different languages.

> If you have developed a new Pulsar client, feel free to submit a pull request and add your client to the list below.
Expand All @@ -55,4 +28,4 @@ Besides the official released clients, there are also multiple projects on devel
| Go | [go-pulsar](https://github.com/t2y/go-pulsar) | [t2y](https://github.com/t2y) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | |
| Scala | [pulsar4s](https://github.com/sksamuel/pulsar4s) | [sksamuel](https://github.com/sksamuel) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | Idomatic, typesafe, and reactive Scala client for Apache Pulsar |
| Rust | [pulsar-rs](https://github.com/wyyerd/pulsar-rs) | [Wyyerd Group](https://github.com/wyyerd) | [![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0) | Future-based Rust bindings for Apache Pulsar |
| .NET | [pulsar-client-dotnet](https://github.com/fsharplang-ru/pulsar-client-dotnet) | [Lanayx](https://github.com/Lanayx) | [![GitHub](https://img.shields.io/badge/license-MIT-green.svg)](https://opensource.org/licenses/MIT) | Native .NET client for C#/F#/VB |
| .NET | [pulsar-client-dotnet](https://github.com/fsharplang-ru/pulsar-client-dotnet) | [Lanayx](https://github.com/Lanayx) | [![GitHub](https://img.shields.io/badge/license-MIT-green.svg)](https://opensource.org/licenses/MIT) | Native .NET client for C#/F#/VB |

0 comments on commit 992fc1e

Please sign in to comment.