forked from Meituan-Dianping/Leaf
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
zhangzhitong
committed
Dec 11, 2018
1 parent
d7db9b0
commit b73e0dc
Showing
49 changed files
with
2,382 additions
and
1 deletion.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,14 @@ | ||
*.class | ||
|
||
# Package Files # | ||
*.jar | ||
*.war | ||
*.ear | ||
*.versionsBackup | ||
|
||
|
||
*.iml | ||
.idea/ | ||
target/ | ||
logs/ | ||
log/ |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,110 @@ | ||
# Leaf | ||
|
||
> There are no two identical leaves in the world. | ||
> | ||
> 世界上没有两片完全相同的树叶。 | ||
> | ||
> — 莱布尼茨 | ||
## Introduction | ||
|
||
Leaf 最早期需求是各个业务线的订单ID生成需求。在美团早期,有的业务直接通过DB自增的方式生成ID,有的业务通过redis缓存来生成ID,也有的业务直接用UUID这种方式来生成ID。以上的方式各自有各自的问题,因此我们决定实现一套分布式ID生成服务来满足需求。具体Leaf 设计文档见:[ leaf 美团分布式ID生成服务 ](https://tech.meituan.com/MT_Leaf.html ) | ||
|
||
目前Leaf覆盖了美团点评公司内部金融、餐饮、外卖、酒店旅游、猫眼电影等众多业务线。在4C8G VM基础上,通过公司RPC方式调用,QPS压测结果近5w/s,TP999 1ms。 | ||
|
||
## Quick Start | ||
|
||
### Leaf Server | ||
|
||
我们提供了一个基于spring boot的HTTP服务来获取ID | ||
|
||
#### 运行Leaf Server | ||
|
||
##### 打包服务 | ||
|
||
```shell | ||
cd leaf | ||
mvn clean install -DskipTests | ||
cd leaf-server | ||
``` | ||
|
||
##### 运行服务 | ||
###### mvn方式 | ||
|
||
```shell | ||
mvn spring-boot:run | ||
``` | ||
|
||
###### 脚本方式 | ||
|
||
```shell | ||
sh deploy/run.sh | ||
``` | ||
##### 测试 | ||
|
||
```shell | ||
#segment | ||
curl http://localhost:8080/api/segment/get/leaf-segment-test | ||
#snowflake | ||
curl http://localhost:8080/api/snowflake/get/test | ||
``` | ||
#### 配置介绍 | ||
|
||
Leaf 提供两种生成的ID的方式(号段模式和snowflake模式),你可以同时开启两种方式,也可以指定开启某种方式(默认两种方式都会开启)。 | ||
|
||
Leaf Server的配置都在leaf-server/src/main/resources/leaf.properties中 | ||
|
||
| 配置项 | 含义 | 默认值 | | ||
| ------------------------- | ----------------------------- | ------ | | ||
| leaf.name | leaf 服务名 | | | ||
| leaf.segment.enable | 是否开启号段模式 | false | | ||
| leaf.jdbc.url | mysql 库地址 | | | ||
| leaf.jdbc.username | mysql 用户名 | | | ||
| leaf.jdbc.password | mysql 密码 | | | ||
| leaf.snowflake.enable | 是否开启snowflake模式 | false | | ||
| leaf.snowflake.zk.address | snowflake模式下的zk地址 | | | ||
| leaf.snowflake.port | snowflake模式下的服务注册端口 | | | ||
|
||
#### 号段模式 | ||
|
||
如果使用号段模式,需要建立DB表,并配置leaf.jdbc.url, leaf.jdbc.username, leaf.jdbc.password | ||
|
||
如果不想使用该模式配置leaf.segment.enable=false即可。 | ||
|
||
##### 创建数据表 | ||
|
||
```sql | ||
CREATE DATABASE leaf | ||
CREATE TABLE `leaf_alloc` ( | ||
`biz_tag` varchar(128) NOT NULL DEFAULT '', | ||
`max_id` bigint(20) NOT NULL DEFAULT '1', | ||
`step` int(11) NOT NULL, | ||
`description` varchar(256) DEFAULT NULL, | ||
`update_time` timestamp NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, | ||
PRIMARY KEY (`biz_tag`) | ||
) ENGINE=InnoDB; | ||
|
||
insert into leaf_alloc(biz_tag, max_id, step, description) values('leaf-segment-test', 1, 2000, 'Test leaf Segment Mode Get Id') | ||
``` | ||
|
||
##### 配置相关数据项 | ||
|
||
在leaf.properties中配置leaf.jdbc.url, leaf.jdbc.username, leaf.jdbc.password参数 | ||
|
||
#### Snowflake模式 | ||
|
||
算法取自twitter开源的snowflake算法。 | ||
|
||
如果不想使用该模式配置leaf.snowflake.enable=false即可。 | ||
|
||
##### 配置zookeeper地址 | ||
|
||
在leaf.properties中配置leaf.snowflake.zk.address,配置leaf 服务监听的端口leaf.snowflake.port。 | ||
|
||
##### 监控页面 | ||
|
||
号段模式:http://localhost:8080/cache | ||
|
||
### Leaf Core | ||
|
||
当然,为了追求更高的性能,需要通过RPC Server来部署Leaf 服务,那仅需要引入leaf-core的包,把生成ID的API封装到指定的RPC框架中即可。 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,115 @@ | ||
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" | ||
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd"> | ||
<modelVersion>4.0.0</modelVersion> | ||
<parent> | ||
<groupId>com.sankuai.inf.leaf</groupId> | ||
<artifactId>leaf-parent</artifactId> | ||
<version>1.0.1</version> | ||
</parent> | ||
<groupId>com.sankuai.inf.leaf</groupId> | ||
<artifactId>leaf-core</artifactId> | ||
<packaging>jar</packaging> | ||
<version>1.0.1</version> | ||
<name>leaf-core</name> | ||
<properties> | ||
<mysql-connector-java.version>5.1.38</mysql-connector-java.version> | ||
<commons-io.version>2.4</commons-io.version> | ||
<log4j.version>2.7</log4j.version> | ||
<mybatis-spring.version>1.2.5</mybatis-spring.version> | ||
</properties> | ||
<dependencies> | ||
<dependency> | ||
<groupId>org.mybatis</groupId> | ||
<artifactId>mybatis</artifactId> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>org.perf4j</groupId> | ||
<artifactId>perf4j</artifactId> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.slf4j</groupId> | ||
<artifactId>slf4j-api</artifactId> | ||
</dependency> | ||
<dependency> | ||
<groupId>commons-io</groupId> | ||
<artifactId>commons-io</artifactId> | ||
</dependency> | ||
<!--zk--> | ||
<dependency> | ||
<groupId>org.apache.curator</groupId> | ||
<artifactId>curator-recipes</artifactId> | ||
<scope>provided</scope> | ||
</dependency> | ||
<dependency> | ||
<groupId>com.fasterxml.jackson.core</groupId> | ||
<artifactId>jackson-databind</artifactId> | ||
<version>${jackson-databind.version}</version> | ||
<scope>provided</scope> | ||
</dependency> | ||
<!-- test scope --> | ||
<dependency> | ||
<groupId>org.springframework</groupId> | ||
<artifactId>spring-core</artifactId> | ||
<scope>test</scope> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.springframework</groupId> | ||
<artifactId>spring-beans</artifactId> | ||
<scope>test</scope> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.springframework</groupId> | ||
<artifactId>spring-jdbc</artifactId> | ||
<scope>test</scope> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.springframework</groupId> | ||
<artifactId>spring-context</artifactId> | ||
<scope>test</scope> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.springframework</groupId> | ||
<artifactId>spring-test</artifactId> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>org.apache.logging.log4j</groupId> | ||
<artifactId>log4j-slf4j-impl</artifactId> | ||
<scope>test</scope> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.apache.logging.log4j</groupId> | ||
<artifactId>log4j-api</artifactId> | ||
<scope>test</scope> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.apache.logging.log4j</groupId> | ||
<artifactId>log4j-core</artifactId> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>com.alibaba</groupId> | ||
<artifactId>druid</artifactId> | ||
<scope>test</scope> | ||
</dependency> | ||
<dependency> | ||
<groupId>mysql</groupId> | ||
<artifactId>mysql-connector-java</artifactId> | ||
</dependency> | ||
<dependency> | ||
<groupId>org.mybatis</groupId> | ||
<artifactId>mybatis-spring</artifactId> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
<dependency> | ||
<groupId>junit</groupId> | ||
<artifactId>junit</artifactId> | ||
<scope>test</scope> | ||
</dependency> | ||
|
||
</dependencies> | ||
</project> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
package com.sankuai.inf.leaf; | ||
|
||
import com.sankuai.inf.leaf.common.Result; | ||
|
||
public interface IDGen { | ||
Result get(String key); | ||
boolean init(); | ||
} |
27 changes: 27 additions & 0 deletions
27
leaf-core/src/main/java/com/sankuai/inf/leaf/common/CheckVO.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,27 @@ | ||
package com.sankuai.inf.leaf.common; | ||
|
||
public class CheckVO { | ||
private long timestamp; | ||
private int workID; | ||
|
||
public CheckVO(long timestamp, int workID) { | ||
this.timestamp = timestamp; | ||
this.workID = workID; | ||
} | ||
|
||
public long getTimestamp() { | ||
return timestamp; | ||
} | ||
|
||
public void setTimestamp(long timestamp) { | ||
this.timestamp = timestamp; | ||
} | ||
|
||
public int getWorkID() { | ||
return workID; | ||
} | ||
|
||
public void setWorkID(int workID) { | ||
this.workID = workID; | ||
} | ||
} |
22 changes: 22 additions & 0 deletions
22
leaf-core/src/main/java/com/sankuai/inf/leaf/common/PropertyFactory.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,22 @@ | ||
package com.sankuai.inf.leaf.common; | ||
|
||
import org.slf4j.Logger; | ||
import org.slf4j.LoggerFactory; | ||
|
||
import java.io.IOException; | ||
import java.util.Properties; | ||
|
||
public class PropertyFactory { | ||
private static final Logger logger = LoggerFactory.getLogger(PropertyFactory.class); | ||
private static final Properties prop = new Properties(); | ||
static { | ||
try { | ||
prop.load(PropertyFactory.class.getClassLoader().getResourceAsStream("leaf.properties")); | ||
} catch (IOException e) { | ||
logger.warn("Load Properties Ex", e); | ||
} | ||
} | ||
public static Properties getProperties() { | ||
return prop; | ||
} | ||
} |
39 changes: 39 additions & 0 deletions
39
leaf-core/src/main/java/com/sankuai/inf/leaf/common/Result.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,39 @@ | ||
package com.sankuai.inf.leaf.common; | ||
|
||
public class Result { | ||
private long id; | ||
private Status status; | ||
|
||
public Result() { | ||
|
||
} | ||
public Result(long id, Status status) { | ||
this.id = id; | ||
this.status = status; | ||
} | ||
|
||
public long getId() { | ||
return id; | ||
} | ||
|
||
public void setId(long id) { | ||
this.id = id; | ||
} | ||
|
||
public Status getStatus() { | ||
return status; | ||
} | ||
|
||
public void setStatus(Status status) { | ||
this.status = status; | ||
} | ||
|
||
@Override | ||
public String toString() { | ||
final StringBuilder sb = new StringBuilder("Result{"); | ||
sb.append("id=").append(id); | ||
sb.append(", status=").append(status); | ||
sb.append('}'); | ||
return sb.toString(); | ||
} | ||
} |
6 changes: 6 additions & 0 deletions
6
leaf-core/src/main/java/com/sankuai/inf/leaf/common/Status.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,6 @@ | ||
package com.sankuai.inf.leaf.common; | ||
|
||
public enum Status { | ||
SUCCESS, | ||
EXCEPTION | ||
} |
21 changes: 21 additions & 0 deletions
21
leaf-core/src/main/java/com/sankuai/inf/leaf/common/Utils.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
package com.sankuai.inf.leaf.common; | ||
|
||
import org.slf4j.Logger; | ||
import org.slf4j.LoggerFactory; | ||
|
||
import java.net.InetAddress; | ||
|
||
public class Utils { | ||
private static final Logger logger = LoggerFactory.getLogger(Utils.class); | ||
public static String getIp() { | ||
String ip; | ||
try { | ||
InetAddress addr = InetAddress.getLocalHost(); | ||
ip = addr.getHostAddress(); | ||
} catch(Exception ex) { | ||
ip = ""; | ||
logger.warn("Utils get IP warn", ex); | ||
} | ||
return ip; | ||
} | ||
} |
15 changes: 15 additions & 0 deletions
15
leaf-core/src/main/java/com/sankuai/inf/leaf/common/ZeroIDGen.java
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,15 @@ | ||
package com.sankuai.inf.leaf.common; | ||
|
||
import com.sankuai.inf.leaf.IDGen; | ||
|
||
public class ZeroIDGen implements IDGen { | ||
@Override | ||
public Result get(String key) { | ||
return new Result(0, Status.SUCCESS); | ||
} | ||
|
||
@Override | ||
public boolean init() { | ||
return true; | ||
} | ||
} |
Oops, something went wrong.