Professional Documents
Culture Documents
or macOS
Kazi Aminul Islam
Department of Computer Science
Kennesaw State University
Acknowledgment:
Dr. Dan Lo
Install Spark on Windows 10
Steps
• Download and install 7Zip if not in your computer
• Download and install JVM if not in your computer
• Download and install Spark
• Download and install Hadoop
• Configure environment variables
• Grant permission to temp folder
• Test it
Download and Install 7Zip (latest version)
https://www.7-zip.org/download.html
Download and Install JVM v1.8.0_221-b11
• Check the Java version your have: java –version
• Other Java versions may not work!
Download and Install Spark
http://spark.apache.org/downloads.html
Password:
==> This script will install:
/usr/local/bin/brew
/usr/local/share/doc/homebrew
/usr/local/share/man/man1/brew.1
/usr/local/share/zsh/site-functions/_brew
/usr/local/etc/bash_completion.d/brew
/usr/local/Homebrew
==> The following existing directories will be made group writable:
/usr/local/bin
/usr/local/include
/usr/local/lib
/usr/local/share
/usr/local/lib/pkgconfig
/usr/local/share/doc
==> The following existing directories will have their owner set to dlo2:
/usr/local/bin
/usr/local/include
/usr/local/lib
/usr/local/share
Install Java
(base) ltksup39868mac:~ dlo2$ brew cask install java
==> Tapping homebrew/cask
Cloning into '/usr/local/Homebrew/Library/Taps/homebrew/homebrew-cask'...
remote: Enumerating objects: 543760, done.
remote: Total 543760 (delta 0), reused 0 (delta 0), pack-reused 543760
Receiving objects: 100% (543760/543760), 238.43 MiB | 29.48 MiB/s, done.
Resolving deltas: 100% (383842/383842), done.
Tapped 3790 casks (3,911 files, 255.9MB).
Error: Calling brew cask install is disabled! Use brew install [--cask] instead.
(base) ltksup39868mac:~ dlo2$ brew install java
==> Downloading https://homebrew.bintray.com/bottles/openjdk-15.0.1.catalina.bot
==> Downloading from https://d29vzk4ow07wi7.cloudfront.net/9376a1c6fdf8b0268b6cb
######################################################################## 100.0%
==> Pouring openjdk-15.0.1.catalina.bottle.tar.gz
==> Caveats
For the system Java wrappers to find this JDK, symlink it with
sudo ln -sfn /usr/local/opt/openjdk/libexec/openjdk.jdk /Library/Java/JavaVirtualMachines/openjdk.jdk
/ / //
_\ \/ _ \/ _ `/ / '_/
/ / . /\_,_/_/ /_/\_\ version 3.0.1
Copy Sample Dataset
(base) ltksup39868mac:Downloads dlo2$ ls spark*
spark-3.0.1-bin-hadoop2.7.tgz
spark-3.0.1-bin-hadoop2.7:
LICENSE README.md conf jars python
NOTICE RELEASE data kubernetes sbin
R bin examples licenses yarn
(base) ltksup39868mac:Downloads dlo2$ mv spark-3.0.1-bin-hadoop2.7 /usr/local/spark
mv: rename spark-3.0.1-bin-hadoop2.7 to /usr/local/spark: Permission denied
(base) ltksup39868mac:Downloads dlo2$ sudo mv spark-3.0.1-bin-hadoop2.7 /usr/local/spark
Password:
(base) ltksup39868mac:Downloads dlo2$ pwd
/Users/dlo2/Downloads
(base) ltksup39868mac:Downloads dlo2$ cd /usr/local
(base) ltksup39868mac:local dlo2$ ls
Caskroom bin jamf outset share
Cellar dockutil lib remotedesktop spark
Frameworks etc munki sal var
Homebrew include opt sbin
(base) ltksup39868mac:local dlo2$ cd spark/
(base) ltksup39868mac:spark dlo2$ ls
LICENSE README.md conf jars python