datanode: Introducing data partition selector which allow users to customize the client's selection logic for data partition when reading and writing, and can switch at any time. #853
client: Add enablePosixACL configuration to enable POSIX ACL feature. #906datanode: Improved data recovery speed. #899
datanode: Fix the issue that the information in the memory is not released immediately after deleting the extent, which causes the status information of the data partition to be incorrect. #938datanode: Fix the issue that repeatedly sending requests to the failed data partitions. #937objectnode: Fix the issue that recursive making directory concurrent with sample multi-layer path prefix. #904
master: Fix the concurrency safe issue when removing raft member. #893sdk: Fix the panic issue while remove data partition concurrently. #894
object: Implemented S3 api 'DeleteBucketPolicy' for bucket deletion. #757master: Introducing management API for volume capacity expanding and shrinking. New capacity must be set more than 20% lager than used. #764master: Introducing management API for updating node address. #813master: Introducing management API for checking nodes. #813metanode: Add header and checksum verification for EXTENT_DEL files. #813cli: Add logging for CLI tool, and added several commands to call master API. #764 #801compile:Support direct compile or docker cross compile on Arm64 platform. #779client: Introducing nearRead option that allow client read data priority from the nearest DataNode to improve reading performance. #810client: Update volume follower read config from master periodicity. #837
object: Improved compatibility for ListObjects and ListObjectsV2 interfaces. #769master: Check whether the used space between the replicas is consistent when performing the load partition operation. #813metanodedatanode: Optimize batch delete Extent; add autoRepairLimitRater on DataNode. #781datanode: Introducing autoRepair option for limit data repair speed on DataNode. #842datanode: The data repair task will skip process extent file which has been already deleted. #842
master: When loading metadata of cluster, using the current ID instead of the loaded ID. #821metanode: Fix the deadlock problem while MetaNode deleting dentry. #785metanode: Fixes #760 Add inode to freeList when NLink is 0, and delete inode 7 days later. #767datanodemetanode: When DataNode and MetaNode start, the partitions not exist in cluster view will be renamed to expiredPartition and skip loading. #824datanodemetanode: Fixes issue#698, Raft instance delete itself by applying ConfChange raft log. #866metanode: MetaNode may not free space. #838client: Fix the problem that client opening file with outdated extents information. #783client: Batch inode get mechanism is out of service. #847client: Fix the problem that makes authorized user mount volume failure. #828datanode: Fix several issue in automatic repair process for tiny extent. #855 #857datanode: Fix the problem that the client still applies to create a new extent to a data partition that has no space. #867
- Updated Q&A, environment&capacity planing documentation. #801
console: ChubaoFS Console is a web application, which provides management for volume, file, s3, user, node, monitoring, and alarm. The ChubaoFS Console makes it much easier to access services provided by ChubaoFS. #728
Please refer to https://chubaofs.readthedocs.io/en/latest/user-guide/console.html to start the Console.metanode: Provide compatibility verification tool for meta partition. #684object: CORS access control. #507fuse: Introduce fsyncOnClose mount option. Choose if closing system call shall trigger fsync. #494
Release2.1.0 did a lot of work to optimize memory usage.
- Modify the max write size to 128k; limit the rate of Forget requests. #533
- If concurrent write operations are more than 256, bufferPool is no longer used. #538
- Eliminates the unnecessary memory allocations for Inode struct by using proto InodeInfo struct directly. #545
- Use sync Pool and rate limiter to limit the memory usage. #639
- Use total count instead of rate. #642
- Uses sorted extents instead of BTree. #646
master: Add set/get deleteBatchCount interfaces. #608master: Delete partitions on DataNode/MetaNode concurrently in decommission action. #724master: Added api for getting & setting parameters for batch deleting extents. #726metanode: Filter inode candidates that will be sent to a partition in batch-inode-get to save memory. #481metanode: Batch delete inode, unlink&evict dentry. #586datanode: Prioritize healthy data node when sending message to data partition. #562metanodedatanode: Check expired partition before loading. #624object: Implemented several conditional parameters in GetObject action. #471object: Making S3 credential compatible with earlier version of ChubaoFS. #508object: Totally support presigned url; Partial support user-defined metadata; Making ETag versioned. #540object: CopyObject across volumes; Support coping directory; Support coping xattr. #563object: Parallel downloading. #548object: Add 'Expires' and 'Cache-Control' in putObject/copyObject/getObject/headObject. #589object: Modify part system metadata. #636fuse: Decrease request channel size to save memory. #512fuse: Let the client daemon inherit all the environment variables but not just PATH. #529fuse: Introduces a debug interface /debug/freeosmemory to trigger garbage collection manually. #539fuse: Add configuration MaxCPUs. #546fuse: View client log through http proto. #552fuse: Support modifying ModifyTime of inode by setAttr. #733deployment: Add log map in docker-compose.yaml. #478deployment: Introduce nginx to improve compatibility. #534test: Add testing script for S3 APIs in python. #514 #595monitor: Add detailed vol info in grafana dashboard. #522cli: Some new features for CLI tool: bash completions, configuration setting, decommission, diagnose etc. #555 #695
metanode: Accelerate deletion. #582 #600datanode: Limit replica number of data partition to at least 3. #587datanode: Set if auto-repair by http interface. #672metanodedatanodefuse: Optimizations on port checking. #543 #531metanodedatanode: Validate creating partition request from master. #611object: Refactor copy object function. #563fuse: Replace process of token validation from meta wrapper to client. #498fuse: Adjust rlimit config for client. #521
metanode: When overwriting a file, if the inode in a dentry is updated successfully, ignore unlink and evict inode errors. #500metanode: Fix incorrect metrics collection for volume usedGB in monitor system. #503metanode: VolStat can correctly update when force update metaPartitions frequently. #537metanode: The MaxInodeID of meta partition is not synchronized when recovered from snapshot. #571metanode: Free Inodes by raft protocol. #582metanode: Painc with deleteMarkedInodes. #597object: Fix the parsing issue of the two preconditions of If-Match and If-None-Match. #516object: Change the time format in list buckets API to UTC time. #525object: Fix xml element of DeleteResult. #532object: Empty result in list operation when run in parallel with delete. #509object: Change from hard link to soft link in CopyObject action. #563object: Solved parallel-safety issue; Clean up useless data on failure in upload part. #553object: Fixed a problem in listing multipart uploads. #595object: Solve the problem that back-end report “NotExistErr” error when uploading files with the same key in parallel. #685fuse: Evict inode cache when dealing with forget. #523
- Update related documents of ObjectNode. #554
- Added user and CLI introduction, synchronize documentation according to the latest code. #564
- Added F&Q section. #573
- Multi-Zone replication & Create volume on a specified zone. #407 #416
- Support token authentication for readwrite-mount & readonly-mount of fuse client. #435
- A command line tool for cluster operations. #441
- Implemented user security and authorization system to improve resource access control. #441
- Implemented extend attributes (xattr) for metadata and posix-compatible file system interface (mountable client). #441
- Support folder operations in S3 APIs. #450
- Reduce blocking under concurrency. #458
- Implemented more Amazon S3-compatible object storage interfaces to improve compatibility. #441
- Make replicas of the data partition a specific option when creating volume. #377
- If meta node reaches threshold,set meta partition status to readonly. #411
- Add cluster status API. #457
- Checks only file type instead of the whole mode. #381
- Make
followerReada client specific option. #382 - Support command line argument for fuse client. #418
- Introduce disable
dentry-cacheto client option. #453 - Filter target meta partitions in batch iget. #472
- Yum tool for deploying ChubaoFS cluster. #385
- Fix the signature algorithm issues. #369 #476
- Avoid inode unlink due to net error. #402
- A map structure locked during serialization. #413
- Wait for data sync in close syscall. #419
- Fix empty result on list objects. #433
- Set lookup valid duration for newly created file. #437
- Fix
igeterror due to metapartition split. #446 - Fix mount fail when volume is full. #453
- Fix offline strategy for raft peers of
data partitionandmeta partition. #467
- Add guide for running ChubaoFS by yum tools. #386
- Update FUSE client mount options. #439
- Add documentation for client token. #449
- Support building docker image that contains both cfs-server and cfs-client. #353
- Only one replica of meta partition can be taken offline at the same time. #345
- Check if server port is open before raft leader change. #348
- Solved several issue in signature algorithm version 2. #357
- Solved the issue related to copying files across folder through object storage interface. #361
- Update dashboard configuration of grafana. #347
- Unified the configuration of master address and listening port in documentation. #362
- Added benchmark data and guidelines for deploying ChubaoFS cluster with Helm in README file. #350
- Add a general Authentication & Authorization framework for ChubaoFS. commit
- Object storage interface. Add ObjectNode to provide S3-compatible APIs. commit
- Check disk path size in run-docker script. commit
- Support building ChubaoFS docker image that contains cfs-server and cfs-client. commit
- Add go test in docker-compose. commit
- Add authorization to master api getVol. commit
- Support building under the Darwin(Apple MacOS) and Microsoft Windows operating system environment. commit
- Set DataNode disk size in docker script to be compatible with lower version of
df. commit - The
reservedSpaceparameter invalid when gt 30GB. commit - The mtime of parent inode does not change when create or delete a
dentry. commit - MetaNode
opResponseLoadPartitionremoves duplicate locks. commit - MetaNode leak memory on DeleteMetaPartition operator. commit
- Change configuration file of Master daemon. commit
- Remove unnecessary function in raft store. commit
- Change metaNode
loadSnapshotSigncommand aboutinodeCountanddentryCount. commit - Rename
racktocell. commit - Improve meta partition replicas verification by
MaxInodeandDentryCnt. commit - Change parameter name in MetaNode configuration file. commit
- Using current applyID to replace snapshot applyID in MetaNode LoadMetaPartition response action. commit
- Delete data partition and meta partition synchronously. commit
- Add design and user guide document for AuthNode. commit
- Update configuration file sample in document. commit
- Add design and user guide document for object subsystem. commit
- Add IO, small file and metadata performance benchmark data to document. commit
- Datanode : support read from follower and if packet is tinyExtent,then do write it once commit
- Vol add followerRead field to support reading data from foll owner commit
- Support read from raft follower instead of just leader commit
- Support to modify whether vol supports reading data from a replica commit
- Introduce read and write iops rate limit commit
- Add metrics commit
- If vol has been marked deleted,data partitions, meta partition information reported by heartbeat will no longer be accepted commit
- Use static ip for meta and data nodes commit
- Improve debug environment using docker commit
- Support custom meta node reserved memory commit
- Data partition and meta partition must have three replicas except reducing replicas to 2 commit
- Adjust demo config parameters commit
- Update grafana dashbord for disk error metric commit
- OpFollowerRead if read eof,return error commit
- Get follower read option in init commit
- Stream traverse process never gets triggered in some situation commit
- Check LoadConfigFile before starting daemon commit
- Return error from function LoadConfigFile to the caller commit
- ExtentStorage engine :autoComputeCrc compute crc error commit
- Clean up async delete process of metanode commit
- Set default port to non-system reserved port commit
- Leader change not warning on raft commit
- Remove go module files for now commit
- Clean up response of get all inodes info commit
- Master, DataNode and MetaNode Fix dp or mp offline process commit
- Use AddNodeWithPort replace AddNode,and delete AddNode API commit
- Delete reserved space on DataNode config file commit
- Refine labels of the disk error metric commit
- Optimize auto compute crc commit
- Add use cases commit
- Introduce writecache mount option. commit
- Introduce keepcache mount option. commit
- Add admin API for get all meta parititons under vol commit
- Support for truncating raft log. commit
- Dynamiclly reduce the num of replicas for vol. commit
- The specified number of replica num is supported when creating vol. commit
- Feature: daemonize server commit
- Support log module change loglevel on line. commit
- Extent_store LoadTinyDeleteFileOffset return s.baseTinyDeleteOffset. commit
- Enable async read by default. commit
- Improve log message details for clientv2. commit
- Compatible with string when get bool config. commit
- Add performance tracepoint for clientv2. commit
- Align out message buffer size with max read size. commit
- For splitting meta partition,updating meta partition and creating new meta partition are persisted within a single transaction. commit
- If metanode used memory is full,then the partition must set to readonly. commit
- Set report time to now after creating data partition. commit
- Set writeDeadLineTime to one minute,avoid metanode gc reset conn which snapshot used as much as possible. commit
- Add raft monitor. commit
- If the creation of a data partition fails, the successfully created replica is deleted. commit
- If the creation of a meta partition fails, the successfully created replica is deleted. commit
- Add unit test case. commit
- Passing create data partition type to datanode. commit
- If create dp is normal,must start Raft else backend start raft. commit
- The tickInterval and electionTick support reading from a configuration file commit
- Fix: add del vol step after ltptest in travis-ci test script. commit
- Clientv2 file handle memory leak. commit
- Redirect stderr to an output file in daemon. commit
- Exclude data partition only when connection refused. commit
- When delete DataParittion,the forwardToLeader mayBe painc. commit
- Metanode load error mayme shield. commit
- Truncate raft corrupt data. commit
- When meta node memory usage arrive threshold, split meta partition occurred dead lock. commit
- SplitMetaPartition race lock with updateViewCache. commit
- After the vol is created and before the heartbeat report, the status of the data partition is set to read only after the check dp operation is performed. commit
- When disk error,the raft cannot start on new data server first. commit
- OpDecommissionDataPartition delete dataPartition on new server. commit
- Datanode may be painc. commit
- Datanode auto compute crc. commit
- DataNode: when dataPartition load,if applyId ==0 ,then start Raft. commit
- The reported data partition usage decreased, and the statistical usage did not decrease accordingly. commit
- Docker metanode.cfg add totalMem parameter. commit
- Datanode deadlock on deletePartition. commit
- DataNode may be painc. commit
- Exclude dir inode in the orphan list. commit
- Evict inode cache after successful deletion. commit
- The actual reduction in the number of replicas exceeds the expected reduction in the number of replicas. commit
- Compatible with old heartbeat mode, old heartbeat mode does not report volname. commit
- Metanode mistakenly delete empty dir inode. commit
- Treat ddelete not exist error as successful. commit
- Fuse directIO read size can exceeds buffer size. commit
- Fix Datanode retain RaftLog commit
- Fix: Datanode: when tinyExtentRepair auto repair,it has been commit
- Fix: Storage :when write tinyExtent,if offset!=e.datasize,return error commit
- The buf is not reset and the old data is repeatedly written to the ump warning log, resulting in the overall performance degradation of the cluster commit
- DataPartitionMap occurred error which is concurrent map iteration and map write commit
- Client gets stale file size if streamer is auto evicted. commit
- Update export init for consul register commit
- Err is shadowed in server main function commit
- If master only create DataPartitionCnt is 10,then cannnot mount commit
- Update export init for consul register commit
- DataPartition disk error ,not recvoery raft log on new datanode commit
- Datanode register hang bug commit
- Sdk When creating a datapartition, select the datapartition retry strategy. commit
- Refactoring SDK: when write datapartition num greater 10 ,then trust master. commit
- When disk error,the datapartition recover only recover avali data on tinyExtent. commit
- Decommission Meta or Data Partition must sync response to master. commit
- Synchronized decommission the data partition. commit
- Datanode api /partition add raftStatus. commit
- StartRaftLoggingSchedule not use goroutine. commit
- Metanode must config totalMem. commit
- Sync tinyDeleteExtent time change to 1 days. commit
- Change partitionId varliable to partitionID. commit
- Datanode delete SnapshotFile Pool. commit
- Exporter add ump. commit
- DataNode: compatible old dataPartition Meta info. commit
- Add ltptest log. commit
- Docker metanode config.json change totalMem to 6GB. commit
- Datanode create dataPartition select disk function change. commit
- Datanode start must start StartRaftLoggingSchedule func. commit
- Master create vol min default DataPartition set to 10. commit
- Docker: run ltptest print errorinfo. commit
- When load dp,if dp status is normal,then start raft ,else wait snapshot has recover. commit
- Datanode delete unused func. commit
- If not config warnLogDir,then donnot write umplog to disk. commit
- DataNode: if not config raftDir,then not start server. commit
- Add log on metanode delete extent. commit
- Doc :delete warnLogDir config. commit
- Keep mount point in error stataus when client is killed commit
- Log checkroration checkTime change to 1 second commit
- Metanode change deleteDentry or deleteInode api to log.LogDebugf commit
- Increase the judgment condition of disk error,add syscall.EROFS commit
- Sync code from git.jd.com/chubaofs/chubaofs commit
- When datanode or metanode start,must compare partition with master and gofmt project. commit
- Datanode add backend check disk status func. commit
- Update rocksdb build depends. commit
- Metanode panic when deleting meta partition. commit
- When raft member become leader,must apply from appliyID to commitID. commit
- Clientv2: add dentry cache. commit
- Vendor: introduce jacobsa daemonize package. commit
- Client: start client as a daemon. commit
- Clientv2: daemonize client.commit
- Update docker helper script. commit
- Data sdk: use ip address instead of partition id to rule out unavailable data partitions. commit
- Add monitor add prometheus, grafana in docker helper script.commit
- Add GOPATH check in build.sh. commit
- Integrated rocksdb compilation. commit
- Update makefile. commit
- Docs: start client as a daemon. commit
- Update docs: add make build in readme and docs. commit
- Use one thread to send and recive from follower on primary backup replication protocol .commit
- Change random write raft serialize not use json. commit
- Fix: when datanode is killed, maxBaseExtentId is corrupted.commit
- Once vol is created successfully, the state of the data partition it has should be writable. commit
- Metanode: fix create dentry exist error. commit
- Add privileged for docker-compose server node. commit
- The retain space field in config file is not allowed to be larger than the default value. commit
- Fix: when data node gets a disk error, the disk status does not changed. commit
- Fix: build error. commit
- Fix build with lua bug. commit
- Check whether vol is valid. If there is no meta partition with start being. commit
- Build: update build.sh for user permission. commit
- Clientv2: fix to get the latest file size. commit
- When create dp failed,then Check all dp host if the host is alive. commit
- Fix bug: when read tinyDeleteFile return eof error. commit
- Build: fix multi users build bug. [commit](https://github.com/chubaofs/chubaofs/commit/a885777b9da97957cc1eddfee102be3707763b64[commit](https://github.com/chubaofs/chubaofs/commit/a885777b9da97957cc1eddfee102be3707763b64)
- Build: fix no root user permission bug. commit
- Init replicas info from create data partition request. commit
- Add cfs-base dockerfile. commit
- Specification code and metanode config file. commit
- Add go mod. commit
- Docs: update according to recent changes. commit
- If warnLogDir is not specified in the config files, server or client will return an error. commit
- Rename RestSize to ReservedSize. commit
- Print more detailed error information when server starts failed. commit
- Remove docker client security opt. commit
- Add disk config detailed description. commit
- By default create 10 data partitions and 3 meta partitions when creating volume. commit
- Doc: add clarification that resource manager is AKA master. commit
- Normalize exporter cluster name. commit
- Set the value of rack for data node to default. commit
- Refine decommission disk. commit
- Change document about metanode config. commit
- Doc: update docker helper section. commit
- Update quick-start-guide.rst. commit
- Change dataPartition IntervalToUpdateReplica to 600 seconds. commit
- Add third-party directory. commit
- Write data partition decommission url message to log. commit
- Go fmt project. commit
- Data sdk: check amount of writable data partitions when mount. commit
- Delete third-party directory. commit
- Update docs: remove server and client. commit
- Pass the hosts to data node when creating the data partition. commit
- Warn log message unicode encoding is converted to utf8. commit
- Ignore generated build files. commit
- Build: update build.sh and Makefile. commit
- Change packet attr func and ReadTinyDelete impl log. commit
- Add log when read tinyDeleteFile failed. commit
- Add log when streamRead and ExtentRepairRead. commit
- Remove unused source code. commit
- Build: update build.sh. commit
- Update README.md. commit
- Build: remove client unused depends. commit
- Client: enable support to fifo and socket file types. commit
- Clientv2: use jacobsa fuse package instead of bazil. commit
- Docker: introduce docker compose to create a local testing ChubaoFS cluster. commit
- Meta: update atime in inode get operation. [metanode] commit
- Fix: potential panic if send returns nil resp with no error. [client] commit
- Fix: raft election takes a long time and timeout; issue a panic if raft server is down. [raft] commit
- Fix: potential deadlock if applyHandler. [master] commit
- Fix: put vol to cache after it is persistent. [master] commit
- Fix: partition is nil when apply remove raft node. [datanode] commit
- Fix: metanode painc. [metanode] commit
- Fix: panic when pprof does not start. commit
- Sdk: retry if mount failed in case master is unavailable temporarily. commit
- Build: add verbose build info. commit
- Master: introduce data partition over-provision. commit
- Master: reserve writable data partition amount according to capacity instead of a const. commit
- Monitor: Use UMP performance monitor if exporter is not enabled. commit
- Rename the repository from cfs to chubaofs.
- Use own errors module instead of juju errors due to license incompatibility.
- Metanode: Change not raft leader error to tryOtherAddr error.
- Master: Partition recovered but the status not changed.
- Datanode: Report to client with proto.OptryAgain resultcode when datapartition does not exsit.
- Raft: A member must apply playback from old apply id to commit id after elected as leader.
- Metanode: generate identical inode number under extreme conditions. commit
- Master: Add ump warn packet.
- Master: Remove redundant calling of loadMetaData method.
- Master: Reload meta data after leader changed.
- Master: make dataPartition disk Path persistent. commit
- Master: Volume creation supports specifying the amount of meta partitions. commit
- Metanode: Add totalMem in configFile.
- Datanode: Change default disk reserved space.
- Datanode: Add volname in heartbeat report.
- Raft: Use raft.ErrNotLeader instead of ErrNotLeader.
- Client: Create a dummy node instance if inode does not exist.
- Client: Add UMP monitor alarms for read/write/fsync errors.
- Client: Suppress some error messages. commit
- Log: Automatically create subdirectory under the log directory.
The initial release version.