Skip to content

Commit

Permalink
raft_consensus-itest: address flakiness in config-change tests
Browse files Browse the repository at this point in the history
Several tests were waiting until all replicas had set their
LAST_RECEIVED operation to '1' and assuming that a config change could
then proceed. However, even if all servers have received index 1, this
doesn't necessarily mean that the leader has received the corresponding
acknowledgement, and thus the leader may refuse the config change.

Change-Id: I3eed4ca551a3342dcb07330c43ef94e169db7ab8
Reviewed-on: http://gerrit.cloudera.org:8080/4637
Tested-by: Kudu Jenkins
Reviewed-by: Mike Percy <[email protected]>
  • Loading branch information
toddlipcon committed Oct 7, 2016
1 parent 833abea commit 5233e49
Showing 1 changed file with 5 additions and 3 deletions.
8 changes: 5 additions & 3 deletions src/kudu/integration-tests/raft_consensus-itest.cc
Original file line number Diff line number Diff line change
Expand Up @@ -1546,7 +1546,7 @@ TEST_F(RaftConsensusITest, TestAddRemoveServer) {
TServerDetails* leader_tserver = tservers[0];
const string& leader_uuid = tservers[0]->uuid();
ASSERT_OK(StartElection(leader_tserver, tablet_id_, kTimeout));
ASSERT_OK(WaitForServersToAgree(kTimeout, tablet_servers_, tablet_id_, 1));
ASSERT_OK(WaitUntilCommittedOpIdIndexIs(1, leader_tserver, tablet_id_, kTimeout));

// Make sure the server rejects removal of itself from the configuration.
Status s = RemoveServer(leader_tserver, tablet_id_, leader_tserver, boost::none, kTimeout);
Expand Down Expand Up @@ -1692,7 +1692,8 @@ TEST_F(RaftConsensusITest, TestAtomicAddRemoveServer) {
// Elect server 0 as leader and wait for log index 1 to propagate to all servers.
TServerDetails* leader_tserver = tservers[0];
ASSERT_OK(StartElection(leader_tserver, tablet_id_, MonoDelta::FromSeconds(10)));
ASSERT_OK(WaitForServersToAgree(MonoDelta::FromSeconds(10), tablet_servers_, tablet_id_, 1));
ASSERT_OK(WaitUntilCommittedOpIdIndexIs(1, leader_tserver, tablet_id_,
MonoDelta::FromSeconds(10)));
int64_t cur_log_index = 1;

TabletServerMap active_tablet_servers = tablet_servers_;
Expand Down Expand Up @@ -1776,7 +1777,8 @@ TEST_F(RaftConsensusITest, TestElectPendingVoter) {
// Elect server 0 as leader and wait for log index 1 to propagate to all servers.
TServerDetails* initial_leader = tservers[0];
ASSERT_OK(StartElection(initial_leader, tablet_id_, MonoDelta::FromSeconds(10)));
ASSERT_OK(WaitForServersToAgree(MonoDelta::FromSeconds(10), tablet_servers_, tablet_id_, 1));
ASSERT_OK(WaitUntilCommittedOpIdIndexIs(1, initial_leader, tablet_id_,
MonoDelta::FromSeconds(10)));

// The server we will remove and then bring back.
TServerDetails* final_leader = tservers[4];
Expand Down

0 comments on commit 5233e49

Please sign in to comment.