Skip to content

Commit

Permalink
ARUHA-371: working on Subscription API documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
Vyacheslav Stepanov committed Sep 30, 2016
1 parent 7abcd2e commit ec72915
Showing 1 changed file with 86 additions and 11 deletions.
97 changes: 86 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -438,10 +438,13 @@ one subscription* so that within one connection it is possible to read events fr

_(\* This possibility will be enabled soon)_

The typical workflow when using subscriptions if following:
1. create subscription specifying the event-types you want to read;
2. start reading events from your subscription;
3. periodically commit the cursors you get in event batches;
The typical workflow when using subscriptions is following:

1) create subscription specifying the event-types you want to read;

2) start reading events from your subscription;

3) commit the cursors you get in event batches;

If you closed the connection and after some time started reading again - you get events from the
point of your latest commit.
Expand Down Expand Up @@ -497,11 +500,11 @@ The response is a stream that groups events into JSON batches separated by endli

The output looks like this:
```sh
{"cursor":{"partition":"5","offset":"543","event_type":"aruha-test-hila","cursor_token":"b75c3102-98a4-4385-a5fd-b96f1d7872f2"},"events":[{"metadata":{"occurred_at":"1996-10-15T16:39:57+07:00","eid":"1f5a76d8-db49-4144-ace7-e683e8ff4ba4","event_type":"aruha-test-hila","partition":"5","received_at":"2016-09-30T09:19:00.525Z","flow_id":"blahbloh"},"data_op":"C","data":{"order_number":"abc","id":"111"},"data_type":"blah"},"info":{"debug":"Stream started"}]}
{"cursor":{"partition":"5","offset":"544","event_type":"aruha-test-hila","cursor_token":"a28568a9-1ca0-4d9f-b519-dd6dd4b7a610"},"events":[{"metadata":{"occurred_at":"1996-10-15T16:39:57+07:00","eid":"1f5a76d8-db49-4144-ace7-e683e8ff4ba4","event_type":"aruha-test-hila","partition":"5","received_at":"2016-09-30T09:19:00.741Z","flow_id":"blahbloh"},"data_op":"C","data":{"order_number":"abc","id":"111"},"data_type":"blah"}]}
{"cursor":{"partition":"5","offset":"545","event_type":"aruha-test-hila","cursor_token":"a241c147-c186-49ad-a96e-f1e8566de738"},"events":[{"metadata":{"occurred_at":"1996-10-15T16:39:57+07:00","eid":"1f5a76d8-db49-4144-ace7-e683e8ff4ba4","event_type":"aruha-test-hila","partition":"5","received_at":"2016-09-30T09:19:00.741Z","flow_id":"blahbloh"},"data_op":"C","data":{"order_number":"abc","id":"111"},"data_type":"blah"}]}
{"cursor":{"partition":"0","offset":"545","event_type":"aruha-test-hila","cursor_token":"bf6ee7a9-0fe5-4946-b6d6-30895baf0599"}}
{"cursor":{"partition":"1","offset":"545","event_type":"aruha-test-hila","cursor_token":"9ed8058a-95be-4611-a33d-f862d6dc4af5"}}
{"cursor":{"partition":"5","offset":"543","event_type":"order.ORDER_RECEIVED","cursor_token":"b75c3102-98a4-4385-a5fd-b96f1d7872f2"},"events":[{"metadata":{"occurred_at":"1996-10-15T16:39:57+07:00","eid":"1f5a76d8-db49-4144-ace7-e683e8ff4ba4","event_type":"aruha-test-hila","partition":"5","received_at":"2016-09-30T09:19:00.525Z","flow_id":"blahbloh"},"data_op":"C","data":{"order_number":"abc","id":"111"},"data_type":"blah"},"info":{"debug":"Stream started"}]}
{"cursor":{"partition":"5","offset":"544","event_type":"order.ORDER_RECEIVED","cursor_token":"a28568a9-1ca0-4d9f-b519-dd6dd4b7a610"},"events":[{"metadata":{"occurred_at":"1996-10-15T16:39:57+07:00","eid":"1f5a76d8-db49-4144-ace7-e683e8ff4ba4","event_type":"aruha-test-hila","partition":"5","received_at":"2016-09-30T09:19:00.741Z","flow_id":"blahbloh"},"data_op":"C","data":{"order_number":"abc","id":"111"},"data_type":"blah"}]}
{"cursor":{"partition":"5","offset":"545","event_type":"order.ORDER_RECEIVED","cursor_token":"a241c147-c186-49ad-a96e-f1e8566de738"},"events":[{"metadata":{"occurred_at":"1996-10-15T16:39:57+07:00","eid":"1f5a76d8-db49-4144-ace7-e683e8ff4ba4","event_type":"aruha-test-hila","partition":"5","received_at":"2016-09-30T09:19:00.741Z","flow_id":"blahbloh"},"data_op":"C","data":{"order_number":"abc","id":"111"},"data_type":"blah"}]}
{"cursor":{"partition":"0","offset":"545","event_type":"order.ORDER_RECEIVED","cursor_token":"bf6ee7a9-0fe5-4946-b6d6-30895baf0599"}}
{"cursor":{"partition":"1","offset":"545","event_type":"order.ORDER_RECEIVED","cursor_token":"9ed8058a-95be-4611-a33d-f862d6dc4af5"}}
```
Each batch contains the following fields:

Expand All @@ -515,7 +518,7 @@ In Subscription API cursors have the following structure:
{
"partition": "5",
"offset": "543",
"event_type": "aruha-test-hila",
"event_type": "order.ORDER_RECEIVED",
"cursor_token": "b75c3102-98a4-4385-a5fd-b96f1d7872f2"
}
```
Expand All @@ -525,12 +528,84 @@ Fields are:
- `offset`: the offset of this batch; user should not do any operations with offset, for him it should be just a string;
- `event_type`: specifies the event-type of the cursor (as in one stream there can be events of different event-types);
- `cursor_token`: cursor token generated by Nakadi; useless for the user;
### Committing

### Committing cursors
```sh
curl -v -X POST \
-H "Content-type: application/json" \
-H "X-Nakadi-StreamId: ae1e39c3-219d-49a9-b444-777b4b03e84c" \
"http://localhost:8080/subscriptions/038fc871-1d2c-4e2e-aa29-1579e8f2e71f/cursors" \
-d '{
"items": [
{
"partition": "5",
"offset": "543",
"event_type": "order.ORDER_RECEIVED",
"cursor_token": "b75c3102-98a4-4385-a5fd-b96f1d7872f2"
},
{
"partition": "2",
"offset": "923",
"event_type": "order.ORDER_RECEIVED",
"cursor_token": "a28568a9-1ca0-4d9f-b519-dd6dd4b7a610"
}
]
}'
```

### Checking currently committed cursors
To see what is current position of subscription it's possible to run the request:
```sh
curl -v -X GET "http://localhost:8080/subscriptions/038fc871-1d2c-4e2e-aa29-1579e8f2e71f/cursors"
```
The response will be a list of current cursors that reflect the last committed offsets.

### Rebalance

### Subscription statistics
To get statistics of subscription the folowing request should be used:
```sh
curl -v -X GET "http://localhost:8080/subscriptions/038fc871-1d2c-4e2e-aa29-1579e8f2e71f/stats"
```

The output will contain the statistics for all partitions of the stream. Like this:
```sh
{
"items": [
{
"event_type": "order.ORDER_RECEIVED",
"partitions": [
{
"partition": "0",
"state": "reassigning",
"unconsumed_events": 2115,
"client_id": "b75c3102-98a4-4385-a5fd-b96f1d7872f2"
},
{
"partition": "1",
"state": "assigned",
"unconsumed_events": 1029,
"client_id": "ae1e39c3-219d-49a9-b444-777b4b03e84c"
}
]
}
]
}
```

### Deleting subscription
```sh
curl -v -X DELETE "http://localhost:8080/subscriptions/038fc871-1d2c-4e2e-aa29-1579e8f2e71f"
```

### Getting/listing subscriptions
```sh
curl -v -X GET "http://localhost:8080/subscriptions"
```

It's possible to filter the list with following parameters: `event_type`, `owning_application`.
Also, pagination parameters are available: `offset`, `limit`.

## Build and Development

### Building
Expand Down

0 comments on commit ec72915

Please sign in to comment.