Skip to content

Commit

Permalink
Update docs since v1.1.7 will be released
Browse files Browse the repository at this point in the history
  • Loading branch information
allwefantasy committed Dec 24, 2018
1 parent 097242d commit 9bbf9fe
Show file tree
Hide file tree
Showing 5 changed files with 57 additions and 37 deletions.
34 changes: 20 additions & 14 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,17 +22,32 @@ This is an active open source project for everyone,
and we are always open to people who want to use this system or contribute to it.
This guide document introduce [how to contribute to MLSQL](https://github.com/allwefantasy/streamingpro/blob/master/docs/docv2/contribute/contribute.md).

## Roadmap
## RoadMap

1. [Roadmap](https://github.com/allwefantasy/streamingpro/blob/master/docs/docv2/contribute/roadmap.md)
1. [MLSQL v1.1.7 RoadMap](https://www.jianshu.com/p/622ff0c7beb1) MLSQL v1.1.7 plans to release in Mid Jan 2019
2. [Versioning Policy](https://github.com/allwefantasy/streamingpro/blob/master/docs/docv2/contribute/release.md)



## Get PreBuild Distribution

[Releases Page](https://github.com/allwefantasy/streamingpro/releases)
The lasted version is [MLSQL-v1.1.6](https://github.com/allwefantasy/streamingpro/releases/tag/v1.1.6).
You can also download from [MLSQL Website](http://download.mlsql.tech/mlsql-1.1.6/)
* [Release Notes](https://github.com/allwefantasy/streamingpro/tree/master/docs/docv2/release_notes)
* [Releases Page](https://github.com/allwefantasy/streamingpro/releases)
* The lasted version is [MLSQL-v1.1.6](https://github.com/allwefantasy/streamingpro/releases/tag/v1.1.6).
* You can also download from [MLSQL Website](http://download.mlsql.tech/mlsql-1.1.6/)


Run PreBuild Distribution:

```shell
cp streamingpro-spark_2.x-x.x.x.tar.gz /tmp
cd /tmp && tar xzvf streamingpro-spark_2.x-x.x.x.tar.gz
cd /tmp/streamingpro-spark_2.x-x.x.x

## make sure spark distribution is available
## visit http://127.0.0.1:9003
export SPARK_HOME="....." ; ./start-local.sh
```

## Build Distribution

Expand All @@ -49,15 +64,6 @@ export DISTRIBUTION=false

## build
./dev/package.sh


cp streamingpro-bin-x.x.x.tgz /tmp
cd /tmp && tar xzvf streamingpro-bin-x.x.x.tgz
cd /tmp/streamingpro

## make sure spark distribution is available
## visit http://127.0.0.1:9003
export SPARK_HOME="....." ; ./start-local.sh
```

## Contributors
Expand Down
6 changes: 2 additions & 4 deletions dev/create-release/release-build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -55,8 +55,6 @@ export LANG=C.UTF-8
# Commit ref to checkout when building
GIT_REF=${GIT_REF:-master}

RELEASE_STAGING_LOCATION="https://dist.apache.org/repos/dist/dev/spark"

BASE_DIR=$(pwd)

init_java
Expand All @@ -76,7 +74,7 @@ git clone "$MLSQL_GIT_REPO"
cd streamingpro
git checkout $GIT_REF
git_hash=`git rev-parse --short HEAD`
echo "Checked out Spark git hash $git_hash"
echo "Checked out MLSQL git hash $git_hash"

if [ -z "$MLSQL_VERSION" ]; then
# Run $MVN in a separate command so that 'set -e' does the right thing.
Expand Down Expand Up @@ -147,7 +145,7 @@ mvn -DskipTests clean package -pl streamingpro-mlsql -am $BASE_PROFILES
EOF
mvn -DskipTests clean package -pl streamingpro-mlsql -am $BASE_PROFILES
cd ..
PNAME="streamingpro-spark_$MLSQL_BIG_SPARK_VERSIOIN-$MLSQL_PACKAGE_VERSION"
PNAME="mlsql-spark_$MLSQL_BIG_SPARK_VERSIOIN-$MLSQL_PACKAGE_VERSION"
mkdir -p $PNAME/libs
cp streamingpro/streamingpro-mlsql/target/streamingpro-mlsql-spark_$MLSQL_BIG_SPARK_VERSIOIN-$MLSQL_VERSION.jar $PNAME/libs/
cp ../start-local.sh $PNAME/
Expand Down
10 changes: 0 additions & 10 deletions dev/package.sh
Original file line number Diff line number Diff line change
Expand Up @@ -28,16 +28,6 @@ for env in MLSQL_SPARK_VERSION DRY_RUN DISTRIBUTION; do
fi
done

if [[ ${DRY_RUN} != "true" ]];then
rm -rf /tmp/temp_ServiceFramework
git clone --depth 1 https://github.com/allwefantasy/ServiceFramework.git /tmp/temp_ServiceFramework
cd /tmp/temp_ServiceFramework
mvn install -DskipTests -Pjetty-9 -Pweb-include-jetty-9

cd -
fi


BASE_PROFILES="-Pscala-2.11 -Ponline -Phive-thrift-server -Pcarbondata -Pcrawler"

if [[ "$MLSQL_SPARK_VERSION" > "2.2" ]]; then
Expand Down
9 changes: 0 additions & 9 deletions docs/docv2/release_notes/release_1.1.4.md

This file was deleted.

35 changes: 35 additions & 0 deletions docs/docv2/release_notes/release_1.1.6.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
## MLSQL v1.1.6 Release

We are glad to announce that [MLSQL v1.1.6](http://download.mlsql.tech/mlsql-1.1.6) is released. There are huge new features, bug fix and code refractor in this half month' development:

1. Almost 163 commits.
2. 28 [ISSUE](https://github.com/allwefantasy/streamingpro/issues?page=1&q=is%3Aissue+is%3Aclosed+label%3AMLSQL-1.1.6)
3. 41 [PR](https://github.com/allwefantasy/streamingpro/pulls?q=is%3Apr+is%3Aclosed+label%3AMLSQL-1.1.6). 25 of them are BugFix, 26 of them are improvements.

## Improvements:

* [PR-747](https://github.com/allwefantasy/streamingpro/pull/744): Load balance, Dynamic Resource Allocation,Instances manager based on labels。More details please check:[mlsql-cluster doc](https://github.com/allwefantasy/streamingpro/blob/master/docs/docv2/cluster/cluster.md).
* [PR-735](https://github.com/allwefantasy/streamingpro/pull/735): Supports Using load statement to view API doc and Configuration of MLSQL。
* [PR-728](https://github.com/allwefantasy/streamingpro/pull/728): Upgrade to latest carbondata
* [PR-781](https://github.com/allwefantasy/streamingpro/pull/781): A new transformer supports complex father-children relation computing which is hard if you use SQL to do this.
* [PR-794](https://github.com/allwefantasy/streamingpro/pull/794): Provides shell/cmd script to make you compile/package MLSQL more easy.

## BugFix:

* [PR-773](https://github.com/allwefantasy/streamingpro/pull/773): The last As keyword is upper will cause exception。
* [PR-777](https://github.com/allwefantasy/streamingpro/pull/777): Save partition by can not support multi columns.
* [PR-760](https://github.com/allwefantasy/streamingpro/pull/760): conflict in multi kafka-client.(MLSQLsupport kafka 0.8/0.9/1.x at the same time)
* [PR-757](https://github.com/allwefantasy/streamingpro/pull/757): MLSQL Auth bug
* [PR-795](https://github.com/allwefantasy/streamingpro/pull/795): PS cluster enabled will fail the exetuor in yarn mode.

## Code refractor:

* [PR-727](https://github.com/allwefantasy/streamingpro/pull/727): Remove the support for Spark 1.6.x
* [PR-732](https://github.com/allwefantasy/streamingpro/pull/732): Move some non-core modules to external/contri directory.







0 comments on commit 9bbf9fe

Please sign in to comment.