Skip to content

Commit

Permalink
[C++] Wait for all seek operations completed (apache#7216)
Browse files Browse the repository at this point in the history
### Motivation

When a partitioned consumer calls `seek`, it waits for only one partition's seek operation completed  because each internal consumer calls `callback(result)` to complete the same `Promise`.

### Modifications

- Add a `MultiResultCallback` implementation, the callback completes only if all N events completes successfully or one of N events failed.
- Use `MultiResultCallback` to wrap `callback` from `PartitionedConsumerImpl::seekAsync`.


* Wait for all seek operations completed

* Avoid iterator invalidation during partitions update
  • Loading branch information
BewareMyPower authored Jun 10, 2020
1 parent f475703 commit 3da6452
Show file tree
Hide file tree
Showing 2 changed files with 59 additions and 2 deletions.
51 changes: 51 additions & 0 deletions pulsar-client-cpp/lib/MultiResultCallback.h
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
/**
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional information
* regarding copyright ownership. The ASF licenses this file
* to you under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance
* with the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing,
* software distributed under the License is distributed on an
* "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
* KIND, either express or implied. See the License for the
* specific language governing permissions and limitations
* under the License.
*/
#pragma once

#include <pulsar/ConsumerConfiguration.h> // for ResultCallback

#include <atomic>
#include <memory>

namespace pulsar {

class MultiResultCallback {
public:
MultiResultCallback(ResultCallback callback, int numToComplete)
: callback_(callback),
numToComplete_(numToComplete),
numCompletedPtr_(std::make_shared<std::atomic_int>(0)) {}

void operator()(Result result) {
if (result == ResultOk) {
if (++(*numCompletedPtr_) == numToComplete_) {
callback_(result);
}
} else {
callback_(result);
}
}

private:
ResultCallback callback_;
const int numToComplete_;
const std::shared_ptr<std::atomic_int> numCompletedPtr_;
};

} // namespace pulsar
10 changes: 8 additions & 2 deletions pulsar-client-cpp/lib/PartitionedConsumerImpl.cc
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
* under the License.
*/
#include "PartitionedConsumerImpl.h"
#include "MultiResultCallback.h"

DECLARE_LOG_OBJECT()

Expand Down Expand Up @@ -552,9 +553,14 @@ void PartitionedConsumerImpl::seekAsync(uint64_t timestamp, ResultCallback callb
callback(ResultAlreadyClosed);
return;
}

// consumers_ could only be modified when state_ is Ready, so we needn't lock consumersMutex_ here
ConsumerList consumerList = consumers_;
stateLock.unlock();
for (ConsumerList::const_iterator i = consumers_.begin(); i != consumers_.end(); i++) {
(*i)->seekAsync(timestamp, callback);

MultiResultCallback multiResultCallback(callback, consumers_.size());
for (ConsumerList::const_iterator i = consumerList.begin(); i != consumerList.end(); i++) {
(*i)->seekAsync(timestamp, multiResultCallback);
}
}

Expand Down

0 comments on commit 3da6452

Please sign in to comment.