Skip to content

Latest commit

 

History

History
94 lines (53 loc) · 3.98 KB

README.md

File metadata and controls

94 lines (53 loc) · 3.98 KB

StreamingPro is a fast,expresivive,and convenient cluster system running on Spark with streaming,batch,interactive,machine learning support.

It make devlopers more easy to build spark application without writing any code by means of:

  • Using json file in combination with many powerful modules which are easy to be reused provides the user declarative configurations to build your spark application.
  • Data processing are SQL-Based
  • Script support

StreamingPro is not only a out of box complete application, but also a extensible and programmable framework for spark since you can develop you ower compositors (moduler).

Features

  • Pure Spark Streaming(Or normal Spark) program
  • No need of coding, only declarative workflows
  • Rest API for interactive
  • SQL-Oriented workflows support
  • Data continuously streamed in & processed in near real-time
  • dynamically CURD of workflows at runtime via Rest API
  • Flexible workflows (input, output, parsers, etc...)
  • High performance
  • Scalable

downloads

Precomile version based on Spark 1.6.1 : http://pan.baidu.com/s/1pKBWa7L

Precomile version based on Spark 2.0.0: http://pan.baidu.com/s/1mijmgi8

[Spark 2.0 based on Scala 2.10] http://pan.baidu.com/s/1mhH1NQS

Precomile version based on Spark 2.0.2 supporting structrued streaming: https://pan.baidu.com/s/1geNIq1X  Spark 2.0.2 based on Scala 2.10

Documents

Architecture

If no picture show,please click me

If github is too slow to view ,please click me

Declarative workflows for building Spark Streaming

If no picture show,please click me

Implementation

If no picture show,please click me