Skip to content

Commit

Permalink
Merge branch 'master' of github.com:allwefantasy/streamingpro
Browse files Browse the repository at this point in the history
  • Loading branch information
allwefantasy committed Sep 26, 2016
2 parents 7a4b3d2 + 6af27fe commit 3d3b4d6
Show file tree
Hide file tree
Showing 2 changed files with 280 additions and 67 deletions.
31 changes: 11 additions & 20 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,31 +1,22 @@


Declarative workflows for building Spark Streaming / Spark batch programe / Spark SQL Server
StreamingPro is a fast,expresivive,and convenient cluster system running on Spark with streaming,batch,interactive,machine learning support.

## Spark Streaming
It make devlopers more easy to build spark application without writing any code by means of:

Spark Streaming is an extension of the core Spark API that enables stream processing
from a variety of sources.
Spark is a extensible and programmable framework for massive distributed processing of datasets,
called Resilient Distributed Datasets (RDD). Spark Streaming receives input data streams and
divides the data into batches, which are then processed by the Spark engine to generate the results.

Spark Streaming data is organized into a sequence of DStreams,
represented internally as a sequence of RDDs.

## StreamingPro
* Using json file in combination with many powerful modules which are easy to be reused provides the user declarative configurations to build your spark application.
* Data processing are SQL-Based
* Script support


StreamingPro is not only a complete
application, but also a extensible and programmable framework for spark streaming (also include spark,storm)
that can easily be used to build your streaming application.
StreamingPro is not only a out of box complete
application, but also a extensible and programmable framework for spark since you can develop you ower compositors (moduler).


StreamingPro also make it possible that all you should do to build streaming program is
assembling components(eg. SQL Component) in configuration file.

## Features

* Pure Spark Streaming(Or normal Spark) program (Storm in future)
* Pure Spark Streaming(Or normal Spark) program
* No need of coding, only declarative workflows
* Rest API for interactive
* SQL-Oriented workflows support
Expand All @@ -38,9 +29,9 @@ assembling components(eg. SQL Component) in configuration file.
## downloads


> Precomile version based on Spark 1.6.1 : http://share.weiyun.com/9dabb088c3643957798927de6928027e
> Precomile version based on Spark 1.6.1 : http://share.weiyun.com/b1d38eb9436d4e05e60b44caee42ce10
>
> Precomile version based on Spark 2.0.0: http://share.weiyun.com/503b9f6f2c4fb61e7e3868cc57ef2e36
> Precomile version based on Spark 2.0.0: http://share.weiyun.com/62297a8023d6809efa6346370d4b39ea
>
> [Spark 2.0 based on Scala 2.10](http://share.weiyun.com/91dc6441759e61825ee25fd8f2404067)
Expand Down
Loading

0 comments on commit 3d3b4d6

Please sign in to comment.