Skip to content

Commit

Permalink
[PROTOBUF] Fix protobuf generation on handling repeated long number … (
Browse files Browse the repository at this point in the history
…apache#7540)

*Motivation*

The code generation for `repeated long` is not handled properly. (I am not sure how changes were made to PulsarApi.proto)

*Modification*

This pull request adds the code to handle generating code for `repeated long`.

*Test*

Add unit test to ensure `repeated long` is processed. Add test cases to cover both packed and non-package serialization for `repeated long`.

See more details about packed serialization: https://developers.google.com/protocol-buffers/docs/encoding#optional
  • Loading branch information
sijie authored Jul 16, 2020
1 parent b6ceec4 commit 4e358ef
Show file tree
Hide file tree
Showing 7 changed files with 853 additions and 0 deletions.
2 changes: 2 additions & 0 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -1268,6 +1268,7 @@ flexible messaging model and an intuitive client API.</description>
<exclude>src/main/java/org/apache/bookkeeper/mledger/proto/MLDataFormats.java</exclude>
<exclude>src/main/java/org/apache/pulsar/broker/service/schema/proto/SchemaRegistryFormat.java</exclude>
<exclude>src/main/java/org/apache/pulsar/common/api/proto/*.java</exclude>
<exclude>src/test/java/org/apache/pulsar/common/api/proto/*.java</exclude>
<exclude>src/main/java/org/apache/pulsar/io/kinesis/fbs/CompressionType.java</exclude>
<exclude>src/main/java/org/apache/pulsar/io/kinesis/fbs/EncryptionCtx.java</exclude>
<exclude>src/main/java/org/apache/pulsar/io/kinesis/fbs/EncryptionKey.java</exclude>
Expand Down Expand Up @@ -1343,6 +1344,7 @@ flexible messaging model and an intuitive client API.</description>
and are included in source tree for convenience -->
<exclude>src/main/java/org/apache/bookkeeper/mledger/proto/MLDataFormats.java</exclude>
<exclude>src/main/java/org/apache/pulsar/common/api/proto/PulsarApi.java</exclude>
<exclude>src/test/java/org/apache/pulsar/common/api/proto/TestApi.java</exclude>
<exclude>src/main/java/org/apache/pulsar/common/api/proto/PulsarMarkers.java</exclude>
<exclude>src/main/java/org/apache/pulsar/broker/service/schema/proto/SchemaRegistryFormat.java</exclude>
<exclude>bin/proto/MLDataFormats_pb2.py</exclude>
Expand Down

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -346,4 +346,34 @@ public void skipRawBytes(final int size) throws IOException {

buf.readerIndex(buf.readerIndex() + size);
}

public int pushLimit(int byteLimit) throws InvalidProtocolBufferException {
if (byteLimit < 0) {
throw new InvalidProtocolBufferException("CodedInputStream encountered an embedded string or message"
+ " which claimed to have negative size.");
}

byteLimit += buf.readerIndex();
final int oldLimit = buf.writerIndex();
if (byteLimit > oldLimit) {
throw new InvalidProtocolBufferException("While parsing a protocol message, the input ended unexpectedly"
+ " in the middle of a field. This could mean either than the input has been truncated or that an"
+ " embedded message misreported its own length.");
}
buf.writerIndex(byteLimit);
return oldLimit;
}

/**
* Discards the current limit, returning to the previous limit.
*
* @param oldLimit The old limit, as returned by {@code pushLimit}.
*/
public void popLimit(final int oldLimit) {
buf.writerIndex(oldLimit);
}

public int getBytesUntilLimit() {
return buf.readableBytes();
}
}
Loading

0 comments on commit 4e358ef

Please sign in to comment.