Spark-POT

IBM Proof of Technology - Spark

Description:

Use IBM’s Data Science Experience (DSX) and BlueMix cloud services to learn and explore Spark. Participants will be led through a series of three labs that showcase various aspects of the Spark architechure.

1. Lab-1 - The first lab will begin with basic Spark fundamentals such as RDDs and map-reduce paradigms using a Jupyter notebook in DSX with python.

2. Lab-2 - The second lab will explore Spark SQL working with the data in Spark and connecting to an external database in Bluemix.

3. Lab-3 - The final lab will guide participants through a Machine Learning lab where users create a model which predicts survival probabilities for passengers on the Titanic.

Instructions:

Step 1. Sign up for the IBM Data Science Experience (DSX). We recommend using the Firefox browser.

NOTE: If you already have a DSX account, please skip to Step 2.

Create an account.

1. Go to http://datascience.ibm.com/

2. Click the signup button on the top right

3. Enter your email and press Continue. Note: If you already have an IBM id you can click Sign in with your IBM id -- if you don't have a DSX account one will be created for you.

4. You should get an email from "[email protected]" with your IBMid Confirmation code

5. Then, on the next page fill in the corresponding fields and click CREATE ACCOUNT

6. In the new page, write your email and click CONTINUE

7. Write your recently generated password and click on SIGN IN

8. It will take a minute to create your account. When ready, click on Get Started.

You are now in the Data Science Experience landing page. Your environment is automatically set up with one Apache Spark instance and 5 GB of object storage. From here you can explore any of the tutorials, videos, sample notebooks, totorials or articles in the community.

Step 2. Create a project

1. Click on Projects > View All Projects to see a list of your projects. You should only see a default project.

2. Click on the create project icon on the top right of the project list.

3. Type a name for your project. For instance, "DSX Lab". A Spark service and an object storage will be automatically selected as well as a container with a default name. A container is a directory on the object storage. Click on Create.

You are now in your new project where you can create notebooks and data assets as well as add collaborators.

You are now ready to begin Lab-1.