Examples for the Learning Spark book. These examples require a number of libraries and as such have long build files. We have also added a stand alone example with minimal dependencies and a small build file in the mini-complete-example directory.

These examples have been updated to run against Spark 1.3 so they may be slightly different than the versions in your copy of "Learning Spark".

• JDK 1.7 or higher
• Scala 2.10.3

• scala-lang.org

• Spark 1.3
• Protobuf compiler

• On debian you can install with sudo apt-get install protobuf-compiler

• R & the CRAN package Imap are required for the ChapterSixExample
• The Python examples require urllib3

From spark just run ./bin/pyspark ./src/python/[example]

You can also create an assembly jar with all of the dependencies for running either the java or scala versions of the code and run the job with the spark-submit script

./sbt/sbt assembly OR mvn package cd $SPARK_HOME; ./bin/spark-submit --class com.oreilly.learningsparkexamples.[lang].[example] ../learning-spark-examples/target/scala-2.10/learning-spark-examples-assembly-0.0.1.jar