Skip to content

Commit

Permalink
update DB::AddFile to ingest list of sst files
Browse files Browse the repository at this point in the history
Summary:
DB::AddFile(std::string file_path) API that allow them to ingest an SST file created using SstFileWriter
We want to update this interface to be able to accept a list of files that will be ingested, DB::AddFile(std::vector<std::string> file_path_list).

Test Plan:
Add test case `AddExternalSstFileList` in `DBSSTTest`. To make sure:
1. files key ranges are not overlapping with each other
2. each file key range dont overlap with the DB key range
3. make sure no snapshots are held

Reviewers: andrewkr, sdong, IslamAbdelRahman

Reviewed By: IslamAbdelRahman

Subscribers: andrewkr, dhruba, leveldb

Differential Revision: https://reviews.facebook.net/D58587
  • Loading branch information
lightmark committed Jul 11, 2016
1 parent 296545a commit 8e6b38d
Show file tree
Hide file tree
Showing 8 changed files with 534 additions and 160 deletions.
1 change: 0 additions & 1 deletion db/db_impl.cc
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
// Copyright (c) 2011 The LevelDB Authors. All rights reserved.
// Use of this source code is governed by a BSD-style license that can be
// found in the LICENSE file. See the AUTHORS file for names of contributors.

#include "db/db_impl.h"

#ifndef __STDC_FORMAT_MACROS
Expand Down
11 changes: 8 additions & 3 deletions db/db_impl.h
Original file line number Diff line number Diff line change
Expand Up @@ -258,10 +258,11 @@ class DBImpl : public DB {

using DB::AddFile;
virtual Status AddFile(ColumnFamilyHandle* column_family,
const ExternalSstFileInfo* file_info,
const std::vector<ExternalSstFileInfo>& file_info_list,
bool move_file) override;
virtual Status AddFile(ColumnFamilyHandle* column_family,
const std::string& file_path, bool move_file) override;
const std::vector<std::string>& file_path_list,
bool move_file) override;

#endif // ROCKSDB_LITE

Expand Down Expand Up @@ -637,13 +638,17 @@ class DBImpl : public DB {
// Finds the lowest level in the DB that the ingested file can be added to
// REQUIRES: mutex_ held
int PickLevelForIngestedFile(ColumnFamilyData* cfd,
const ExternalSstFileInfo* file_info);
const ExternalSstFileInfo& file_info);

Status CompactFilesImpl(
const CompactionOptions& compact_options, ColumnFamilyData* cfd,
Version* version, const std::vector<std::string>& input_file_names,
const int output_level, int output_path_id, JobContext* job_context,
LogBuffer* log_buffer);
Status ReadExternalSstFileInfo(ColumnFamilyHandle* column_family,
const std::string& file_path,
ExternalSstFileInfo* file_info);

#endif // ROCKSDB_LITE

ColumnFamilyData* GetColumnFamilyDataByName(const std::string& cf_name);
Expand Down
Loading

0 comments on commit 8e6b38d

Please sign in to comment.