Databricks Community Cloud

An overview on the Databricks Community Cloud platform offered by Databricks at: https://community.cloud.databricks.com/

Provides step by step instructions on how to create a Spark Standalone Cluster and how to use notebooks.

1. Databricks Community Cloud By: Robert Sanders

2. Databricks Community Cloud Free/Paid Standalone Spark Cluster Online Notebook Python R Scala SQL Tutorials and Guides Shareable Notebooks

3. Why is it useful? Learning about Spark Testing different versions of Spark Rapid Prototyping Data Analysis Saved Code Others …

4. Forums https:// forums.databricks.com /

5. Login/Sign Up https:// community.cloud.databricks.com / login.html

6. Home Page

7. Active Clusters

8. Create a Cluster - Steps From the Active Clusters page, click the “+ Create Cluster” button Fill in the cluster name Select the version of Apache Spark Click “Create Cluster” Wait for the Cluster to start up and be in a “Running” state

9. Create a Cluster

10. Active Clusters

11. Active Clusters – Spark Cluster UI - Master

12. Workspaces

13. Create a Notebook - Steps Right click within a Workspace and click Create -> Notebook Fill in the Name Select the programming language Select the running cluster you’ve created that you want to attach to the Notebook Click the “Create” button

14. Create a Notebook

15. Notebook

16. Using the Notebook

17. Using the Notebook – Code Snippets > sc > sc. parallelize ( 1 to 5 ). collect ()

18. Using the Notebook - Shortcuts Short Cut Action Shift + Enter Run Selected Cell and Move to nex t Cell Ctrl + Enter Run Selected Cell Option + Enter Run Selected Cell and Insert Cell Bellow Ctrl + Alt + P Create Cell Above Current Cell Ctrl + Alt + N Create Cell Bellow Selected Cell

19. Tables

20. Create a Table - Steps From the Tables section, click “+ Create Table” Select the Data Source (bellow steps assume you’re using File as the Data Source) Upload a file from your local file system Supported file types: CSV, JSON, Avro, Parquet Click Preview Table Fill in the Table Name Select the File Type and other Options depending on the File Type Change Column Names and Types as desired Click “Create Table”

21. Create a Table – Upload File

22. Create a Table – Configure Table

23. Create a Table – Review Table

24. Notebook – Access Table

25. Notebook – Access Table – Code Snippets > sqlContext > sqlContext. sql ( " show tables" ). collect ( ) > v al got = sqlContext. sql ( "select * from got" ) > got. limit ( 10 ) . collect ( )

26. Notebook – Display

27. Notebook – Data Cleaning for Charting

28. Notebook – Plot Options

29. Notebook – Charting

30. Notebook – Display and Charting – Code Snippets > filter (got) > val got = sqlContext. sql ( "select * from got" ) > got. limit ( 10 ). collect () > import org.apache.spark.sql.functions . _ > val allegiancesCleanupUDF = udf [String, String ] ( _ . toLowerCase (). replace ( "house " , "" ) ) > val isDeathUDF = udf { deathYear : Integer => if ( deathYear != null ) 1 else } > val gotCleaned = got. filter ( "Allegiances != \"None\"" ). withColumn ( "Allegiances" , allegiancesCleanupUDF ($ "Allegiances" )). withColumn ( " isDeath " , isDeathUDF ($ "Death Year" )) > display( gotCleaned )

31. Publish Notebook - Steps While in a Notebook, click “Publish” on the top right Click “Publish” on the pop up Copy the link and send it out

32. Publish Notebook