Scribd Logo
Upload

Welcome to Scribd!

  • Exercise 8

    Uploaded by

    aditi13goyal
    3 views1 page
    The document provides instructions for a Spark exercise to be completed individually. It outlines installing Spark locally, downloading lab files and datasets, examining Spark versions and configurations through the web UI, running a sample code to load diabetes data from a CSV file into a DataFrame and analyzing the web UI tabs. Students are asked to create a 3 page maximum Word report answering exercise questions and including screenshots and diagrams.

    Original Description:

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    The document provides instructions for a Spark exercise to be completed individually. It outlines installing Spark locally, downloading lab files and datasets, examining Spark versions and configurations through the web UI, running a sample code to load diabetes data from a CSV file into a DataFrame and analyzing the web UI tabs. Students are asked to create a 3 page maximum Word report answering exercise questions and including screenshots and diagrams.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    3 views1 page

    Exercise 8

    Uploaded by

    aditi13goyal
    The document provides instructions for a Spark exercise to be completed individually. It outlines installing Spark locally, downloading lab files and datasets, examining Spark versions and configurations through the web UI, running a sample code to load diabetes data from a CSV file into a DataFrame and analyzing the web UI tabs. Students are asked to create a 3 page maximum Word report answering exercise questions and including screenshots and diagrams.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 1

    BAN 5753, Big Data & Spark Exercise (10 Points)

    You must do it alone (it is not a group activity)

    1. Install Spark setup on local machine – Standalone cluster installation


    2. Download the lab files - notebooks and datasets to the desired location(Documents or
    Downloads ); paths that you can access via read/write operations
    3. Examine the Spark versions, configurations and Web UI using the below screenshot

    4. Run the below command and examine the spark web UI different tabs and share your
    findings.

    //Modify the path based on the file location

    scala> val rawDF=spark.read.option("inferSchema","true") .option("header","true").csv("/Users


    /Downloads/spark_modules/Lab/data/diabetes.csv")
    rawDF: org.apache.spark.sql.DataFrame = [Pregnancies: int, Glucose: int ... 7 more fields]

    Deliverables:
    As you complete the exercise, create a short report in Microsoft Word (max 3 pages) and
    in this report answer the questions in the exercise description. Copy and paste supporting
    documents/diagrams/screenshots as needed to justify your answer. Make sure you print
    your name, section number, student ID# on the report and turn-in the report as
    communicated by your instructor.

    You might also like