Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[C++] Fix undefined behavior caused by uninitialized variables #10892

Merged

Conversation

BewareMyPower
Copy link
Contributor

Motivation

Sometimes Windows C++ client could fail at Consumer::subscribe, the server side's log is

21:57:54.265 [pulsar-io-50-10] WARN  org.apache.pulsar.broker.service.ServerCnx - [/192.168.233.1:7131] Got exception java.lang.IndexOutOfBoundsException: readerIndex(77) + length(1) exceeds writerIndex(77): PooledSlicedByteBuf(ridx: 77, widx: 77, cap: 77/77, unwrapped: PooledUnsafeDirectByteBuf(ridx: 81, widx: 81, cap: 2048))
	at io.netty.buffer.AbstractByteBuf.checkReadableBytes0(AbstractByteBuf.java:1442)
	at io.netty.buffer.AbstractByteBuf.readByte(AbstractByteBuf.java:730)
	at org.apache.pulsar.common.util.protobuf.ByteBufCodedInputStream.readRawVarint32(ByteBufCodedInputStream.java:273)
	at org.apache.pulsar.common.util.protobuf.ByteBufCodedInputStream.readBool(ByteBufCodedInputStream.java:246)
	at org.apache.pulsar.common.api.proto.PulsarApi$CommandSubscribe$Builder.mergeFrom(PulsarApi.java:12006)
	at org.apache.pulsar.common.api.proto.PulsarApi$CommandSubscribe$Builder.mergeFrom(PulsarApi.java:11618)
	at org.apache.pulsar.common.util.protobuf.ByteBufCodedInputStream.readMessage(ByteBufCodedInputStream.java:127)
	at org.apache.pulsar.common.api.proto.PulsarApi$BaseCommand$Builder.mergeFrom(PulsarApi.java:42271)

It's caused by the corrupted CommandSubscribe from C++ client. The bug was introduced from #10243, which added a bool field replicateSubscriptionStateEnabled to consumer's config but not initialized in the constructor.

It's a undefined behavior in C++. Though GCC and Clang may initialize this bool variable to false, MSVC may have different behavior. I've added some logs to print the replicateSubscriptionStateEnabled argument in Command::newSubscribe, the uninitialized bool field is actually stored as an integer, which is a random value like 208, 116, 128, etc. Then after being serialized by protobuf, the last byte of CommandSubscribe could be a random value. If it was a negative 8 bits integer, the server side would fail to parse the command and then close the connection.

Modifications

  • Initialize replicateSubscriptionStateEnabled with false as the default value.
  • Use C++11's in-class member initialisation to initialize configurations of client, producer and consumer. It's more readable and easier to maintain than initializing each member in the constructor.
  • Add unit tests to ensure configurations' getters and setters work well after this change and add some missed default values in C++ header's docs as well.

Verifying this change

  • Make sure that the change passes the CI checks.

This change added tests and can be verified as follows:

  • ClientConfigurationTest
  • ProducerConfigurationTest
  • ConsumerConfigurationTest

@BewareMyPower
Copy link
Contributor Author

I dumped the bytes of CommandSubscribe from client. Here're some results that failed.

Error

Error2

Error3

We can see the last byte is negative, but it should be zero.

Here's a CommandSubscribe with replicateSubscriptionStateEnabled is false:

00 00 00 4B 00 00 00 47
08 04 22 43 0A 24 70 65
72 73 69 73 74 65 6E 74
3A 2F 2F 70 75 62 6C 69
63 2F 64 65 66 61 75 6C
74 2F 6D 79 2D 74 6F 70
69 63 12 03 73 75 62 18
00 20 00 28 00 32 08 63
6F 6E 73 75 6D 65 72 40
01 58 00 68 00 70 00

I keep other fields unchanged (like consumer name), then run with an uninitialized replicateSubscriptionStateEnabled, the CommandSubscribe became:

00 00 00 4B 00 00 00 47
08 04 22 43 0A 24 70 65
72 73 69 73 74 65 6E 74
3A 2F 2F 70 75 62 6C 69
63 2F 64 65 66 61 75 6C
74 2F 6D 79 2D 74 6F 70
69 63 12 03 73 75 62 18
00 20 00 28 00 32 08 63
6F 6E 73 75 6D 65 72 40
01 58 00 68 00 70 FFFFFF80

The only difference is the last byte: 0x00 vs 0x80 (-128)

From my debug code:

std::cout << "replicateSubscriptionStateEnabled: " << replicateSubscriptionStateEnabled << std::endl;

It printed

replicateSubscriptionState: 128

@merlimat merlimat added component/c++ type/bug The PR fixed a bug or issue reported a bug labels Jun 10, 2021
@merlimat merlimat added this to the 2.9.0 milestone Jun 10, 2021
@merlimat merlimat merged commit e77fa36 into apache:master Jun 10, 2021
@BewareMyPower BewareMyPower deleted the bewaremypower/fix-config-not-init branch June 11, 2021 01:58
@merlimat merlimat modified the milestones: 2.9.0, 2.8.0 Jun 11, 2021
merlimat pushed a commit that referenced this pull request Jun 14, 2021
* Use brace initializer to initialize configurations

* Add config tests
@codelipenghui codelipenghui added the cherry-picked/branch-2.8 Archived: 2.8 is end of life label Jun 15, 2021
yangl pushed a commit to yangl/pulsar that referenced this pull request Jun 23, 2021
…e#10892)

* Use brace initializer to initialize configurations

* Add config tests
bharanic-dev pushed a commit to bharanic-dev/pulsar that referenced this pull request Mar 18, 2022
…e#10892)

* Use brace initializer to initialize configurations

* Add config tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cherry-picked/branch-2.8 Archived: 2.8 is end of life type/bug The PR fixed a bug or issue reported a bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants