Storm Hackathon

Learn to use Storm!

---

Table of Contents

• Getting started
• Using storm-hack with Maven
• Using storm-hack with IntelliJ IDEA

---

Getting started

Prerequisites

First, you will need a laptop that connect to internet using Wi-FI. In the laptop, make sure that you have java and git installed and in your user's PATH.

Second, you will need a Twitter account. If you do not have a Twitter account, please create one. From your Twitter account, follow the Storm experts

@karthikz, @sanjeevrk, @staneja, @challenger_nik, @jason_j, @ankitoshniwal

You will be using Twitter direct messaging capability to get help from them.

Third, if you need to use Twitter firehose/Tweet stream for your idea, create a set of credentials by following the instructions at

https://dev.twitter.com/discussions/631

Fourth, partner with someone to form a group. We encourage 2/3 students per group.

Finally, make sure you have the storm-hack code available on your machine. Git/GitHub beginners may want to use the following command to download the latest storm-hack code and change to the new directory that contains the downloaded code.

$ git clone https://github.com/kramasamy/storm-hack.git && cd storm-hack

Overview

storm-hack contains a variety of examples of using Storm. If this is your first time working with Storm, check out these topologies first:

1. ExclamationTopology: Basic topology written in all Java
2. WordCountTopology: Basic topology for counting words all written in Java

After you have familiarized yourself with these topologies, take a look at the other topopologies in src/jvm/storm/starter/ such as RollingTopWords for more advanced implementations.

If you want to learn more about how Storm works, please head over to the Storm project page.

Using storm-hack with Maven

Install Maven

Maven is an alternative to Leiningen. Install Maven (preferably version 3.x) by following the Maven installation instructions.

Packaging storm-hack for use on a Storm cluster

You can package a jar suitable for submitting to a Storm cluster with the command:

$ mvn -f m2-pom.xml package

This will package your code and all the non-Storm dependencies into a single "uberjar" at the path target/storm-hack-{version}-jar-with-dependencies.jar.

Submitting your jobs to the Storm cluster

After compiling your package, you can submit your job to real cluster using the following command

$ storm jar -c nimbus.host=<name-of-the-host> <jar-name> storm.starter.WordCountTopology <topology-name>

Running unit tests

Use the following Maven command to run the unit tests that ship with storm-hack.

$ mvn -f m2-pom.xml test

Using storm-hack with IntelliJ IDEA

Importing storm-hack as a project in IDEA

The following instructions will import storm-hack as a new project in IntelliJ IDEA.

• Copy m2-pom.xml to pom.xml. This is requried so that IDEA (or Eclipse) can properly detect the maven configuration.
• Open File > Import Project... and navigate to the top-level directory of your storm-hack clone (e.g. ~/git/storm-hack).
• Select Import project from external model, select "Maven", and click Next.
• In the following screen, enable the checkbox Import Maven projects automatically. Leave all other values at their defaults. Click Next.
• Click Next on the following screen about selecting Maven projects to import.
• Select the JDK to be used by IDEA for storm-hack, then click Next.
  • At the time of this writing you should use JDK 6.
  • It is strongly recommended to use Sun/Oracle JDK 6 rather than OpenJDK 6.
• You may now optionally change the name of the project in IDEA. The default name suggested by IDEA is "storm-hack". Click Finish once you are done.