Tags: zhipwang/kudu
Tags
log_block_manager: fix corruption after re-opening compacted metadata This fixes an issue discovered on a cluster due to the following sequence of events: - a block manager compacts a metadata file while starting up - when it reopens the metadata file after replacing it with the compacted one, it gets a file_cache hit. Thus, the WritablePBContainer continues to write to the _deleted_ file instead of the compacted one. Metadata entries at this point are lost (which could cause block loss in the case of lost CREATE records, or dangling blocks in the case of lost DELETEs) - if the server continues to run for a while, the FD will be evicted from the cache and eventually re-opened. At that point, a further DELETE record could end up writing to an offset past the end of the file, since the write offset was incremented by the "lost" records above. - on the next restart, the metadata file would have a "gap" of zero bytes, which would surface as a checksum failure and failure to start up. The fix is relatively simple: when we replace the metadata file we need to invalidate and evict the cache entry so that when we "reopen", it actually starts appending to the _new_ file and not the old deleted one. The bulk of the changes here are to tests: - the stress test now enforces a minimum number of live blocks before it starts deleting them. It also more aggressively compacts, and has a smaller cache. With these changes, I was sometimes able to reproduce the issue. - A more targeted test issues a canned sequence of block creations and deletions that can reliably reproduce the above issue. Change-Id: I491eacbad4750efedea854a2cc35b8ec994f9077 Reviewed-on: http://gerrit.cloudera.org:8080/7113 Reviewed-by: Adar Dembo <[email protected]> Tested-by: Kudu Jenkins Reviewed-on: http://gerrit.cloudera.org:8080/7125
Release notes for 1.3.1 I chose not to move 1.3.0 release notes to "prior version" since the release notes for the dot-release are small. Change-Id: Ia4e587d11ec8c63c731e96cb5e142abc89b2c688 Reviewed-on: http://gerrit.cloudera.org:8080/6622 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin <[email protected]>
[macOS] fix security test failures Change-Id: I5c4d464edc09f41151f95ad2f0e694f2bc00b0f0 Reviewed-on: http://gerrit.cloudera.org:8080/6346 Tested-by: Kudu Jenkins Reviewed-by: Alexey Serbin <[email protected]> Reviewed-on: http://gerrit.cloudera.org:8080/6353 Reviewed-by: Todd Lipcon <[email protected]>
KUDU-1923: rpc_encryption flag is not enforced The rpc_encryption flag wasn't being taken into account during negotiation, so TLS encryption was in effect optional regardless of configuration. Change-Id: Ifdf17c6bb59ca4af1215376c5221a90bd1d93b64 Reviewed-on: http://gerrit.cloudera.org:8080/6342 Reviewed-by: Todd Lipcon <[email protected]> Tested-by: Kudu Jenkins
Update version to non-snapshot 1.2 Change-Id: I9730ba902956d03c0baf244d51555ff339da45c9 Reviewed-on: http://gerrit.cloudera.org:8080/5700 Tested-by: Kudu Jenkins Reviewed-by: Jean-Daniel Cryans <[email protected]>
PreviousNext