Skip to content

Commit

Permalink
feat(doc): update doc for v3.4.0
Browse files Browse the repository at this point in the history 
Signed-off-by: chihe <[email protected]>
  • Loading branch information
@leonrayang
chihe authored and leonrayang committed Oct 30, 2024
1 parent 206d5dd commit 189c7dd
Show file tree
Hide file tree
Showing 7 changed files with 165 additions and 58 deletions.
56 changes: 56 additions & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
@@ -1,3 +1,59 @@
## Release v3.4.0 - 2024/10/28

### **UPGRAGDE NOTICE**
UPGRAGDE NOTICE
If your CubeFS version is v2.3.x or before, please refer to the UPGRADE NOTICE in v2.4.0 for upgrading steps.
if you CubeFS version is v3.3.2 or before, and need to upgrade to v3.3.*,you must follow these upgrade steps:
1. When upgrading the metanode node, you need to add "raftSyncSnapFormatVersion": 0 to the configuration file.
2. After all metanodes are upgraded, remove this configuration item so that raftSyncSnapFormatVersion defaults to 1.
3. Restart all metanodes.
4. Please upgrade the client at the end, the master has enhanced the check of the client, otherwise it will cause the client to mount abnormally

If your Blobstore version is v1.1.0 or before which built with cubefs-blobstore (https://github.com/cubefs/cubefs-blobstore) , please refer to UPGRADE to v3.2.0 following these steps [#1556](https://github.com/cubefs/cubefs/issues/1556).

Note: Provides basic capabilities of the snapshot version in v3.4.0(beta version, disabled by default)

### **Main Feature**
* `master`: Auto decommission for bad disk. （#3494，@bboyCH4）
* `master`: Meta for automatic repair of data replicas. (#3495, @bboyCH4)
* `meta`: Synchronize accessTime between replicas for meta. (#3496, @bboyCH4)
* `cli`: Cli support configuration and querying for decommission operations. (#3497, @NaturalSelect)
* `master/datanode/cli`: Datanode bad disk report to master adds IoErrPartititionCnt and TotalPartititionCnt. (#2679, @true1064)
* `master,datanode`:Retain the data of replicas that have been deleted by RaftForce for a period of time. (#3499, @bboyCH4)

### **Enhance**
* `master`: Atomicity of data partition decommission operation. (#3500, @bboyCH4)
* `master`: Cli display the repair progress of data replica. (#3502, @bboyCH4)
* `master`: Querying the decommission progress of datanode is too slow. (#3504, @bboyCH4)
* `master`: Simplify the return results of the datanode/disk query decomission status interface. (#3507, @bboyCH4)
* `master`: Add a new interface to query the status of decommission tokens. (#3509, @bboyCH4)
* `master`: The decommission progress can display data partitions have not undergone the decommission operation. (#3512, @bboyCH4)
* `master/datanode`: Audit log for dp decommission (#3513, @NaturalSelect)
* `master/data/meta/client`: Add version information to the metrics reported by each subsystem of CubeFS. (#3516, @NaturalSelect)
* `data`: The ReloadSnapshot process of the data partition is taking too long. (#3517, @Victor1319)
* `meta`: Under heavy pressure from deleting a large number of extents, the deletion efficiency is too low. (#3518, @NaturalSelect)
* `master`: Simplify the return results of the datanode/disk query decomission status interface. (#3507, @bboyCH4)
* `client`: when volume is deleted, cfs-client need to exit. (#3512, @longerfly)
* `master`: Add config to control http pool size. (#3567, @Victor1319)
* `metanode`: Persist access time for inode in meta node . (#3565, @bboyCH4)
* `client`: Support libsdk to determine whether it is a file or directory based on mode.(#3566 , @longerfly)


### **Bugfix**
* `master`: Decommission both replicas of the dp results in both replicas failing to decommission.. (#3498, @bboyCH4）
* `master/data`:After the cluster performs decommission operations, many data partitions are backed up to old directories with dates on the datanode. (#3501, @bboyCH4)
* `master/data`:The replica repair process is continuously retried . (#3503, @bboyCH4)
* `data`: Decommission operations often fail due to nodes being inactive. (#3506, @bboyCH4)
* `master`: Using RaftForce to decommission two replicas is stuck. (#3510, @bboyCH4)
* `master`: After executing the cancellation of the decommission operation, the disk remains in a disabled state. (#3511, @bboyCH4)
* `master/data`:The interface for recovering bad disk is timing out. (#3514, @bboyCH4)
* `data`: Encountering an I/O error while writing WAL logs can lead to a panic in the data service.(#3515, @bboyCH4)
* `master`: Only choose from the specific zone in function canWriteForNode. (#3519, @true1064)
* `client`: client update extents from meta and drop extents in cache leadto ek conflict. (#3520, @longerfly)
* `client`: if volume name not match regexp when mounted, return failed immediately. (#3522, @longerfly)
* `master`: The data node decommission has no response. (#3528, @bboyCH4)


## Release v3.3.2 - 2024/04/23

### **UPGRAGDE NOTICE**
Expand Down
34 changes: 23 additions & 11 deletions docs-zh/source/dev-guide/admin-api/master/datanode.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,8 +12,8 @@ curl -v "http://192.168.0.11:17010/dataNode/add?addr=192.168.0.33:17310&zoneName

| 参数 | 类型 | 描述 |
|----------|--------|----------------------------------|
| addr | string | 数据节点和 master 的交互地址 |
| zoneName | string | 指定区域，如果为空则默认值为 default |
| addr | string | 数据节点和master的交互地址 |
| zoneName | string | 指定区域，如果为空则默认值为default |

## 查询

Expand All @@ -27,7 +27,7 @@ curl -v "http://10.196.59.198:17010/dataNode/get?addr=10.196.59.201:17310" | py

| 参数 | 类型 | 描述 |
|------|--------|--------------------------|
| addr | string | 数据节点和 master 的交互地址 |
| addr | string | 数据节点和master的交互地址 |

响应示例

Expand Down Expand Up @@ -65,7 +65,7 @@ curl -v "http://192.168.0.11:17010/dataNode/decommission?addr=192.168.0.33:17310

| 参数 | 类型 | 描述 |
|------|--------|--------------------------|
| addr | string | 数据节点和 master 的交互地址 |
| addr | string | 数据节点和master的交互地址 |

## 获取磁盘信息

Expand Down Expand Up @@ -95,7 +95,9 @@ curl -v "http://192.168.0.11:17010/disk/decommission"
|-------|--------|-------------------------------|
| addr | string | 要下线的磁盘的节点地址 |
| disk | string | 故障磁盘 |
| count | int | 每次下线个数，默认 0，代表全部下线 |
| count | int | 每次下线个数，默认0，代表全部下线 |
| diskDisable | bool | 是否禁用磁盘，即禁止在下线的磁盘上新建dp，默认为true |
| decommissionType | int | 下线类型，默认0,代表手动下线；1表示自动下线 |

## 迁移

Expand All @@ -111,7 +113,7 @@ curl -v "http://10.196.59.198:17010/dataNode/migrate?srcAddr=src&targetAddr=dst&
|------------|--------|----------------------------------|
| srcAddr | string | 迁出数据节点地址 |
| targetAddr | string | 迁入数据节点地址 |
| count | int | 迁移数据分区的个数，非必填，默认 50 个 |
| count | int | 迁移数据分区的个数，非必填，默认50个 |

## 设置磁盘下线控制速度

Expand All @@ -123,7 +125,7 @@ curl -v "http://192.168.0.11:17320/admin/updateDecommissionLimit?decommissionLim

| 参数 | 类型 | 描述 |
|-------------------|------|----------------|
| decommissionLimit | int | 并发下线的 dp 个数 |
| decommissionLimit | int | 并发下线的dp个数 |

::: tip 提示
v3.2.1新增接口
Expand All @@ -136,7 +138,7 @@ curl -v "http://192.168.0.11:17320/admin/queryDecommissionLimit"
```

::: tip 提示
v3.2.1 新增接口
v3.2.1新增接口
:::

## 查询磁盘下线进度
Expand Down Expand Up @@ -175,7 +177,7 @@ v3.2.1新增接口
## 取消磁盘下线

``` bash
curl -v "http://192.168.0.11:17320/disk/cancelDecommission?addr=192.168.0.12:17310&disk=/home/service/var/data1"
curl -v "http://192.168.0.11:17320/disk/pauseDecommission?addr=192.168.0.12:17310&disk=/home/service/var/data1"
```

参数列表
Expand All @@ -186,7 +188,7 @@ curl -v "http://192.168.0.11:17320/disk/cancelDecommission?addr=192.168.0.12:173
| disk | string | 下线磁盘地址 |

::: tip 提示
v3.2.1新增接口
v3.4.0新增接口
:::

## 取消节点下线
Expand All @@ -199,8 +201,18 @@ curl -v "http://192.168.0.11:17010/dataNode/cancelDecommission?addr=192.168.0.33

| 参数 | 类型 | 描述 |
|------|--------|--------------------------|
| addr | string | 数据节点和 master 的交互地址 |
| addr | string | 数据节点地址 |

::: tip 提示
v3.2.1新增接口
:::

## 查询所有下线磁盘

``` bash
curl -v "http://192.168.0.11:17010/disk/queryAllDecommissionDisk"
```

::: tip 提示
v3.4.0新增接口
:::
37 changes: 29 additions & 8 deletions docs-zh/source/dev-guide/admin-api/master/management.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,29 +3,50 @@
## 增加节点

``` bash
curl -v "http://10.196.59.198:17010/raftNode/add?addr=10.196.59.197:17010&id=3"
curl -v "http://192.168.0.1:17010/raftNode/add?addr=10.196.59.197:17010&id=3"
```

增加新的 master 节点到 raft 复制组。
增加新的master节点到raft复制组。

参数列表

| 参数 | 类型 | 描述 |
|------|--------|-------------------------|
| addr | string | master 的 ip 地址, 格式为 ip:port |
| id | uint64 | master 的节点标识 |
| addr | string | master的ip地址, 格式为ip:port |
| id | uint64 | master的节点标识 |

## 删除节点

``` bash
curl -v "http://10.196.59.198:17010/raftNode/remove?addr=10.196.59.197:17010&id=3"
curl -v "http://192.168.0.1:17010/raftNode/remove?addr=10.196.59.197:17010&id=3"
```

从 raft 复制组中移除某个节点。
从raft复制组中移除某个节点。

参数列表

| 参数 | 类型 | 描述 |
|------|--------|-------------------------|
| addr | string | master 的 ip 地址, 格式为 ip:port |
| id | uint64 | master 的节点标识 |
| addr | string | master的ip地址, 格式为ip:port |
| id | uint64 | master的节点标识 |

## 获取nodeset列表

``` bash
curl -v "http://192.168.0.1:17010/nodeSet/list"
```
获取集群的nodeset列表。

## 获取nodeset详细信息

``` bash
curl -v "http://192.168.0.1:17010//nodeSet/get?nodesetId=1"
```

获取集群中nodeset的详细信息

参数列表

| 参数 | 类型 | 描述 |
|------|--------|-------------------------|
| nodesetId | string | nodeset编号 |
31 changes: 11 additions & 20 deletions docs-zh/source/dev-guide/admin-api/master/meta-partition.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,40 +3,31 @@
## 创建

``` bash
curl -v "http://10.196.59.198:17010/metaPartition/create?name=test&start=10000"
curl -v "http://192.168.0.1:17010/metaPartition/create?count=10&name=test"
```

手动切分元数据分片，如果卷的最大的元数据分片 **inode** 的范围是
`[begin, end)`:

- 若 **start** 大于 **begin** 小于 **end**参数，原来最大的元数据分片的 inode 范围变为 `[begin, start]`，新创建的元数据分片的范围是 `[start+1,+inf)`
- 若 **start** 小于 **begin**，max 为当前分片上最大的 inode 编号，则 inode 范围变为 `[begin, max+16777216]`，新创建的元数据分片的范围是 `[max+16777217,+inf)` ;
- 若 **start** 大于 **end**，max 为当前分片上最大的 inode 编号，则 inode 范围变为 `[begin, start]`，新创建的元数据分片的范围是 `[start+1, +inf)` ;

::: warning 注意
start 过大会导致单个分片上 inode 过大，占用较大内存，当最后一个分片上 inode 过多时，也会触发 mp 自动分裂的。
:::
批量创建元数据分区

参数列表

| 参数 | 类型 | 描述 |
|-------|--------|-------------|
| name | string | 卷的名字 |
| start | uint64 | 根据此值切分元数据分片 |
| count | uint64 | 新增的mp数目 |

## 查询

``` bash
curl -v "http://10.196.59.198:17010/metaPartition/get?id=1" | python -m json.tool
curl -v "http://192.168.0.1:17010/metaPartition/get?id=1" | python -m json.tool
```

展示元数据分片的详细信息，包括分片 ID，分片的起始范围等等。
展示元数据分片的详细信息，包括分片ID，分片的起始范围等等。

参数列表

| 参数 | 类型 | 描述 |
|-----|--------|---------|
| id | uint64 | 元数据分片 ID |
| id | uint64 | 元数据分片ID |

响应示例

Expand All @@ -62,7 +53,7 @@ curl -v "http://10.196.59.198:17010/metaPartition/get?id=1" | python -m json.too
## 下线副本

``` bash
curl -v "http://10.196.59.198:17010/metaPartition/decommission?id=13&addr=10.196.59.202:17210"
curl -v "http://192.168.0.1:17010/metaPartition/decommission?id=13&addr=10.196.59.202:17210"
```

下线元数据分片的某个副本，并且创建一个新的副本。
Expand All @@ -71,19 +62,19 @@ curl -v "http://10.196.59.198:17010/metaPartition/decommission?id=13&addr=10.196

| 参数 | 类型 | 描述 |
|------|--------|----------|
| id | uint64 | 元数据分片 ID |
| id | uint64 | 元数据分片ID |
| addr | string | 要下线副本的地址 |

## 比对副本

``` bash
curl -v "http://10.196.59.198:17010/metaPartition/load?id=1"
curl -v "http://192.168.0.1:17010/metaPartition/load?id=1"
```

发送比对副本任务到各个副本，然后检查各个副本的 Crc 是否一致。
发送比对副本任务到各个副本，然后检查各个副本的Crc是否一致。

参数列表

| 参数 | 类型 | 描述 |
|-----|--------|---------|
| id | uint64 | 元数据分片 ID |
| id | uint64 | 元数据分片ID |
20 changes: 17 additions & 3 deletions docs/source/dev-guide/admin-api/master/datanode.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,10 @@ Parameter List
| Parameter | Type | Description |
|-----------|--------|--------------------------------------------------|
| addr | string | The node address of the disk to be taken offline |
| disk | string | Faulty disk address |
| count | int | Number of dp to take offline each time, default is 0, which represents all dp offline. |
| diskDisable | bool | Whether to disable the disk, i.e., prohibit creating new data partitions on the offline disks. Default is true |
| decommissionType | int | Offline type, default is 0, which represents manual offline; 1 indicates automatic offline |
## Migration
Expand Down Expand Up @@ -169,7 +173,7 @@ New interface in v3.2.1
## Cancel Disk Decommission
``` bash
curl -v "http://192.168.0.11:17320/disk/cancelDecommission?addr=192.168.0.12:17310&disk=/home/service/var/data1"
curl -v "http://192.168.0.11:17320/disk/pauseDecommission?addr=192.168.0.12:17310&disk=/home/service/var/data1"
```
Parameter List
Expand All @@ -180,7 +184,7 @@ Parameter List
| disk | string | Address of the disk to be decommissioned |
::: tip Note
New interface in v3.2.1
New interface in v3.4.0
:::
## Cancel Node Decommission
Expand All @@ -193,8 +197,18 @@ Parameter List
| Parameter | Type | Description |
|-----------|--------|------------------------------------------------------|
| addr | string | Address for interaction between data node and master |
| addr | string | Address for data node |
::: tip Note
New interface in v3.2.1
:::
## Query all offline disks
``` bash
curl -v "http://192.168.0.11:17010/disk/queryAllDecommissionDisk"
```
::: tip Note
New interface in v3.4.0
:::
25 changes: 23 additions & 2 deletions docs/source/dev-guide/admin-api/master/management.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
## Add Node

``` bash
curl -v "http://10.196.59.198:17010/raftNode/add?addr=10.196.59.197:17010&id=3"
curl -v "http://192.168.0.1:17010/raftNode/add?addr=10.196.59.197:17010&id=3"
```

Adds a new master node to the raft replication group.
Expand All @@ -18,7 +18,7 @@ Parameter List
## Remove Node

``` bash
curl -v "http://10.196.59.198:17010/raftNode/remove?addr=10.196.59.197:17010&id=3"
curl -v "http://192.168.0.1:17010/raftNode/remove?addr=10.196.59.197:17010&id=3"
```

Removes a node from the raft replication group.
Expand All @@ -29,3 +29,24 @@ Parameter List
|-----------|--------|----------------------------------------------------|
| addr | string | IP address of the master, in the format of ip:port |
| id | uint64 | Node identifier of the master |

## Retrieve the nodeset list of the cluster

``` bash
curl -v "http://192.168.0.1:17010/nodeSet/list"
```
Retrieve the nodeset list of the cluster.

## Retrieve detailed information about the nodeset

``` bash
curl -v "http://192.168.0.1:17010//nodeSet/get?nodesetId=1"
```

Retrieve detailed information about the nodeset

Parameter List

| Parameter | Type | Description |
|------|--------|-------------------------|
| nodesetId | string | odeset identifier |
Loading

0 comments on commit 189c7dd

Please sign in to comment.