Skip to content

hope-onely/streamingpro

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

StreamingPro is a fast,expresivive,and convenient cluster system running on Spark with streaming,batch,interactive,machine learning support.

It make devlopers more easy to build spark application without writing any code by means of:

  • Using json file in combination with many powerful modules which are easy to be reused provides the user declarative configurations to build your spark application.
  • Data processing are SQL-Based
  • Script support

StreamingPro is not only a out of box complete application, but also a extensible and programmable framework for spark since you can develop you ower compositors (moduler).

Features

  • Pure Spark Streaming(Or normal Spark) program
  • No need of coding, only declarative workflows
  • Rest API for interactive
  • SQL-Oriented workflows support
  • Data continuously streamed in & processed in near real-time
  • dynamically CURD of workflows at runtime via Rest API
  • Flexible workflows (input, output, parsers, etc...)
  • High performance
  • Scalable

downloads

Precomile version based on Spark 1.6.1 : http://pan.baidu.com/s/1pKBWa7L

Precomile version based on Spark 2.0.0: http://pan.baidu.com/s/1mijmgi8

[Spark 2.0 based on Scala 2.10] http://pan.baidu.com/s/1mhH1NQS

Precomile version based on Spark 2.0.2 supporting structrued streaming: https://pan.baidu.com/s/1geNIq1X  Spark 2.0.2 based on Scala 2.10

Documents

Architecture

If no picture show,please click me

If github is too slow to view ,please click me

Declarative workflows for building Spark Streaming

If no picture show,please click me

Implementation

If no picture show,please click me

About

Build Spark Streaming Application by SQL

Resources

Stars

Watchers

Forks

Packages

No packages published

Languages

  • JavaScript 67.1%
  • CSS 20.2%
  • HTML 11.6%
  • Other 1.1%