Skip to content

Commit

Permalink
[docs] Update obsolate cluster execution guide
Browse files Browse the repository at this point in the history
Closes apache#835
  • Loading branch information
mbalassi committed Jun 16, 2015
1 parent be50795 commit 20d7e5a
Showing 1 changed file with 1 addition and 66 deletions.
67 changes: 1 addition & 66 deletions docs/apis/cluster_execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ The following illustrates the use of the `RemoteEnvironment`:
~~~java
public static void main(String[] args) throws Exception {
ExecutionEnvironment env = ExecutionEnvironment
.createRemoteEnvironment("strato-master", "7661", "/home/user/udfs.jar");
.createRemoteEnvironment("flink-master", 6123, "/home/user/udfs.jar");

DataSet<String> data = env.readTextFile("hdfs://path/to/file");

Expand All @@ -80,71 +80,6 @@ Note that the program contains custom user code and hence requires a JAR file wi
the classes of the code attached. The constructor of the remote environment
takes the path(s) to the JAR file(s).

## Remote Executor

Similar to the RemoteEnvironment, the RemoteExecutor lets you execute
Flink programs on a cluster directly. The remote executor accepts a
*Plan* object, which describes the program as a single executable unit.

### Maven Dependency

If you are developing your program in a Maven project, you have to add the
`flink-clients` module using this dependency:

~~~xml
<dependency>
<groupId>org.apache.flink</groupId>
<artifactId>flink-clients</artifactId>
<version>{{ site.version }}</version>
</dependency>
~~~

### Example

The following illustrates the use of the `RemoteExecutor` with the Scala API:

~~~scala
def main(args: Array[String]) {
val input = TextFile("hdfs://path/to/file")

val words = input flatMap { _.toLowerCase().split("""\W+""") filter { _ != "" } }
val counts = words groupBy { x => x } count()

val output = counts.write(wordsOutput, CsvOutputFormat())

val plan = new ScalaPlan(Seq(output), "Word Count")
val executor = new RemoteExecutor("strato-master", 7881, "/path/to/jarfile.jar")
executor.executePlan(p);
}
~~~

The following illustrates the use of the `RemoteExecutor` with the Java API (as
an alternative to the RemoteEnvironment):

~~~java
public static void main(String[] args) throws Exception {
ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();

DataSet<String> data = env.readTextFile("hdfs://path/to/file");

data
.filter(new FilterFunction<String>() {
public boolean filter(String value) {
return value.startsWith("http://");
}
})
.writeAsText("hdfs://path/to/result");

Plan p = env.createProgramPlan();
RemoteExecutor e = new RemoteExecutor("strato-master", 7881, "/path/to/jarfile.jar");
e.executePlan(p);
}
~~~

Note that the program contains custom UDFs and hence requires a JAR file with
the classes of the code attached. The constructor of the remote executor takes
the path(s) to the JAR file(s).

## Linking with modules not contained in the binary distribution

The binary distribution contains jar packages in the `lib` folder that are automatically
Expand Down

0 comments on commit 20d7e5a

Please sign in to comment.