Skip to content

Commit

Permalink
Move Pulsar IO as a top-category section at the sidebar (apache#2050)
Browse files Browse the repository at this point in the history
*Motivation*

Pulsar I/O is too deep in the documentation. It is hard for people to realize the connectors.

*Changes*

Move Pulsar I/O documentation to a top-category section.

Also renaming "pulsar administration" to "Administration"
  • Loading branch information
sijie authored Jul 5, 2018
1 parent c9d1066 commit ec04dc0
Show file tree
Hide file tree
Showing 4 changed files with 64 additions and 32 deletions.
13 changes: 10 additions & 3 deletions site/_data/sidebar.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,15 @@ groups:
- title: Metrics for Pulsar Functions
endpoint: metrics

- title: Pulsar IO
dir: io
new: true
docs:
- title: Pulsar IO overview
endpoint: overview
- title: Getting started with Pulsar IO
endpoint: quickstart

- title: Deployment
dir: deployment
docs:
Expand All @@ -69,7 +78,7 @@ groups:
- title: Monitoring
endpoint: Monitoring

- title: Pulsar administration
- title: Administration
dir: admin
docs:
- title: ZooKeeper and BookKeeper
Expand Down Expand Up @@ -136,8 +145,6 @@ groups:
- title: Cookbooks
dir: cookbooks
docs:
- title: Pulsar IO
endpoint: pulsar-io
- title: Tiered Storage
endpoint: tiered-storage
- title: Topic compaction
Expand Down
28 changes: 0 additions & 28 deletions site/docs/latest/getting-started/ConceptsAndArchitecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -585,34 +585,6 @@ Pulsar currently supports S3 as a long term store. Offloading to S3 triggered vi

{% include admonition.html type="info" content="For a guide for setting up tiered storage, see the [Tiered storage cookbook](../../cookbooks/tiered-storage)." %}

## Pulsar IO

Messaging systems are most powerful when you can easily use them in conjunction with external systems like databases and other messaging systems. **Pulsar IO** is a feature of Pulsar that enables you to easily create, deploy, and manage Pulsar **connectors** that interact with external systems, such as [Apache Cassandra](https://cassandra.apache.org), [Aerospike](https://www.aerospike.com), and many others.

{% include admonition.html type="info" title="Pulsar IO and Pulsar Functions"
content="Under the hood, Pulsar IO connectors are specialized [Pulsar Functions](#pulsar-functions) purpose-built to interface with external systems. The [administrative interface](../../cookbooks/pulsar-io) for Pulsar IO is, in fact, quite similar to that of Pulsar Functions." %}

### Sources and sinks

Pulsar IO connectors come in two types:

* **Sources** feed data *into* Pulsar from other systems. Common sources include other messaging systems and "firehose"-style data pipeline APIs.
* **Sinks** are fed data *from* Pulsar. Common sinks include other messaging systems and SQL and NoSQL databases.

This diagram illustrates the relationship between sources, sinks, and Pulsar:

{% include figure.html src="/img/pulsar-io.png" alt="Pulsar IO diagram" caption="Pulsar IO connectors (sources and sinks)" width="80" %}

### Working with connectors

Pulsar IO connectors can be managed via the [`pulsar-admin`](../../reference/CliTools#pulsar-admin) CLI tool, in particular the [`source`](../../reference/CliTools#pulsar-admin-source) and [`sink`](../../reference/CliTools#pulsar-admin-sink) commands.

{% include admonition.html type="info" content="For a guide to managing connectors in your Pulsar installation, see the [Pulsar IO cookbook](../../cookbooks/pulsar-io#managing-connectors)." %}

The following connectors are currently available for Pulsar:

{% include connectors.html %}

## Schema registry

Type safety is extremely important in any application built around a message bus like Pulsar. {% popover Producers %} and {% popover consumers %} need some kind of mechanism for coordinating types at the {% popover topic %} level lest a wide variety of potential problems arise (for example serialization and deserialization issues). Applications typically adopt one of two basic approaches to type safety in messaging:
Expand Down
51 changes: 51 additions & 0 deletions site/docs/latest/io/overview.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
---
title: Pulsar IO overview
lead: Connecting the world using Pulsar Functions
---

<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

Messaging systems are most powerful when you can easily use them in conjunction with external systems like databases and other messaging systems. **Pulsar IO** is a feature of Pulsar that enables you to easily create, deploy, and manage Pulsar **connectors** that interact with external systems, such as [Apache Cassandra](https://cassandra.apache.org), [Aerospike](https://www.aerospike.com), and many others.

{% include admonition.html type="info" title="Pulsar IO and Pulsar Functions"
content="Under the hood, Pulsar IO connectors are specialized [Pulsar Functions](../../functions/overview) purpose-built to interface with external systems. The [administrative interface](../quickstart) for Pulsar IO is, in fact, quite similar to that of Pulsar Functions." %}

## Sources and sinks

Pulsar IO connectors come in two types:

* **Sources** feed data *into* Pulsar from other systems. Common sources include other messaging systems and "firehose"-style data pipeline APIs.
* **Sinks** are fed data *from* Pulsar. Common sinks include other messaging systems and SQL and NoSQL databases.

This diagram illustrates the relationship between sources, sinks, and Pulsar:

{% include figure.html src="/img/pulsar-io.png" alt="Pulsar IO diagram" caption="Pulsar IO connectors (sources and sinks)" width="80" %}

## Working with connectors

Pulsar IO connectors can be managed via the [`pulsar-admin`](../../reference/CliTools#pulsar-admin) CLI tool, in particular the [`source`](../../reference/CliTools#pulsar-admin-source) and [`sink`](../../reference/CliTools#pulsar-admin-sink) commands.

{% include admonition.html type="info" content="For a guide to managing connectors in your Pulsar installation, see the [Getting started with Pulsar IO](../quickstart)." %}

The following connectors are currently available for Pulsar:

{% include connectors.html %}
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
---
title: The Pulsar IO cookbook
title: Getting started with Pulsar IO
lead: Connecting your systems with Pulsar using Pulsar IO
---

<!--
Expand Down Expand Up @@ -80,3 +81,4 @@ Instead of submitting a sink to run on an existing Pulsar cluster, you alternati
At the moment, the following connectors are available for Pulsar:

{% include connectors.html %}

0 comments on commit ec04dc0

Please sign in to comment.