Scribd Logo
Upload

Welcome to Scribd!

  • Data Cleaning With PySpark

    Uploaded by

    vikas gupta
    16 views21 pages
    The document discusses various techniques for cleaning data with Apache Spark, including handling null/missing values, removing duplicates, filtering data, and transforming data types. It covers functions like dropDuplicates(), fill(), and cast() that can be used to clean datasets in a distributed manner using Spark.

    Original Description:

    Original Title

    Data Cleaning with PySpark

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    The document discusses various techniques for cleaning data with Apache Spark, including handling null/missing values, removing duplicates, filtering data, and transforming data types. It covers functions like dropDuplicates(), fill(), and cast() that can be used to clean datasets in a distributed manner using Spark.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    16 views21 pages

    Data Cleaning With PySpark

    Uploaded by

    vikas gupta
    The document discusses various techniques for cleaning data with Apache Spark, including handling null/missing values, removing duplicates, filtering data, and transforming data types. It covers functions like dropDuplicates(), fill(), and cast() that can be used to clean datasets in a distributed manner using Spark.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 21

    Firefox https://louisazhou.gitbook.

    io/notes/spark/data-cleaning-with-apache-spark

    1 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    （ ）

    2 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    3 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    4 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    5 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    6 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    7 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    8 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    9 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    10 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    每个文件 行，字符，名字叫 ，生成 开始

    11 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    12 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    13 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    14 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    15 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    16 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    17 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    18 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    19 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    20 of 21 26-06-2023, 09:12
    Firefox https://louisazhou.gitbook.io/notes/spark/data-cleaning-with-apache-spark

    21 of 21 26-06-2023, 09:12

    You might also like