A tool to record and visualize metrics captured from Cascading (Scalding) workflows at runtime.
Designed to target the pain points of analysts and end users of Cascading, Sahale provides insight into a workflow's runtime resource usage and makes job debugging and locating relevant Hadoop logs easy. The tool reveals optimization opportunities by exposing inefficient MapReduce jobs in a larger workflow, and enables users to track the execution history of their workflows.
Sahale has been verified to work with several Cascading DSLs, but the example projects and several features are tailored to Twitter's Scalding DSL, which is the DSL we use at Etsy. Pull requests for better support for other flavors of Cascading are welcome!
There are two components to Sahale, a flowtracker JAR that contains the code that tracks running jobs and a Node.JS dashboard for visualizing the tracked jobs.
The flowtracker JAR is published to Maven Central for easy inclusion in your projects:
<dependency>
<groupId>com.etsy.sahale</groupId>
<artifactId>flowtracker_2.10</artifactId>
<version>1.0.1</version>
</dependency>You will need add some configuration for the flow tracker. Create the file src/main/resources/flow-tracker.properties in your project with the following contents:
host=http://host.where.nodejs.server.is.running
port=5735
The host setting will be the hostname of the server where you install the dashboard (see below)
You can also add the setting sahale.flow.selected.configs in this file. This is a comma-delimited list of Hadoop configuration properties that will be tracked along with the other job information.
If this configuration step is not performed job tracking will fail.
-
A MySQL instance that Sahale can use.
-
A server with Node.JS and
npminstalled.
-
Set up the MySQL database. Follow the instructions in
create_db_tables.sqlto do this. -
Clone the Sahale repo onto the server where you would like to run the dashboard.
-
Modify
db-config.jsonto point to your database. -
Run
npm installfrom the root directory of the Sahale repo checkout.
Now you can run node app to start the Sahale dashboard. It will be running on port 5735, so go to <hostname>:5735 in your browser to see the dashboard.
Sahale server (NodeJS app) and FlowTracker (Scala client jar) must always maintain parity between clients and server versions. An upgrade deployment must coordinate the distribution of the new client jar and restart of the updated server. In rare cases (tagged in the Git repo) Sahale will make breaking changes that will require addition steps. The two notable cases are listed below:
There were no data model or other incompatibility changes, but the flowtracker JAR is now being published to Maven Central. It is no longer required to manually build this JAR; see the above instructions for its use and configuration.
The data model has changed. Please recreate your backing MySQL tables using the script in src/main/sql before using the new client jar or restarting the NodeJS app. The old tables and data will remain, the new tables will be suffixed with _new unless you opt to alias them. No additional change is required.
Mark incompatible changes between older and newer versions of Scala/Scalding. If your org still uses older versions of Scala/Scalding, please see this commit. All other changes and feature upgrades in the 0.6 line will work as expected with this commit reverted and your own choice of versions applied to the pom.xml.
Sahale was handmade at Etsy.com and is named for Sahale Mountain, which is a wonderful vantage point from which to view the Cascades