Distributed Data Processing on the Cloud

Quiz and Exercises - 3 Total points: 10

Working with Heroku Platform as a Service

In this practice session, we will look into Heroku platform as a service (PaaS). We will
explore the Heroku Command-Line Interface (CLI) to deploy an application on Heroku
platform.

Heroku supports applications written in Ruby, Node.js, Java, Python, Clojure, Scala, Go and
PHP, where users can create, deploy and scale the applications. It also supports testing and
deploying apps locally and then pushing them into the Heroku environment.

The overall architecture of Heroku:

Figure source link: https://www.oreilly.com/library/view/heroku-up-

and/9781449341381/ch02.html

 Dynos: All Heroku applications run in a collection of lightweight Linux containers called
dynos. (For more information refer: https://devcenter.heroku.com/articles/dynos)
 Routers: The Heroku router is a piece of software (written using Erlang, called Hermes)
which acts as a gateway between your users and the dynos that are running your code.
 Logplex: The Logplex is an open-source tool that replaces the logfile concept by providing a
constant stream of information flowing from your application with no beginning and no end.
 Add-ons: Set of other services provided or developed by other service providers such
databases and so on belong to Add-on services. One of the most popular add-ons is one
 provided by Heroku itself. Called Heroku Postgres, this add-on provides PostgreSQL services
to all applications.

For more information, please refer to the following links:

https://devcenter.heroku.com/categories/reference#

Applications can be deployed on to Heroku in three ways:

 Using Heroku CLI (We are using this method in Exercise 3.2)
 Github based
 Using docker images

The glossary of keywords need to be familiar while using Heroku:

 App: An application that runs on heroku using a set of dynos and has unique URL
.herokuapp.com
 Dyno: A container that runs a Heroku app’s code. When a dyno starts up, it runs a single
command that is usually specified in the app’s Procfile.
 Free dyno: A dyno that enables to host your app freely on the Heroku platform.
 Web dyno: A dyno that receives HTTP traffic.
 Worker dyno: A dyno does not receive HTTP traffic.
 Procfile: A plaintext file that declares the commands that an app's dynos run when they
startup.
 Heroku CLI: The command-line interface for interacting with Heroku apps.
 Heroku Dashboard: The web interface for interacting with Heroku apps. Available at
dashboard.heroku.com.

References
 Heroku documentation: https://devcenter.heroku.com/categories/reference
 Using Python Flask and SQLAlchemy together: https://flask-
sqlalchemy.palletsprojects.com/en/2.x/

1. Getting familiar with the Heroku platform and setting up Heroku CLI

In this exercise, you will learn how to create an account in the Heroku platform and get
familiar with the Heroku dashboard. Apart from this, you are going to set up Heroku CLI for
application deployment.

 Create a free Heroku account using the link https://signup.heroku.com/dc

 Now, Sign in to the Heroku dashboard and create an application to get familiar with basic
components.
o Click on New -> Create new app, and create an application with the name :
yourname-v1 as shown in figure below
  Open your app and get familiar with the application dashboard.
 Now, click on the Deploy tab, which shows the different modes of deploying the app to the
Heroku platform. We will work with Heroku Git-based deployment from the CLI.

 Setting up Heroku CLI

o Install Python 3.7 or higher on your computer.
 Linux: https://docs.python-guide.org/starting/install3/linux/
 Windows: https://www.python.org/downloads/windows/
 Mac - OSx: https://www.python.org/downloads/mac-osx/
o Installation Heroku CLI: https://devcenter.heroku.com/articles/heroku-cli#install-
the-heroku-cli
 Note that installing Git is also needed for Heroku CLI!
o After completing the above steps, login into Heroku CLI
 In Windows: Open Powershell and enter heroku login and follow
subsequent steps.
 In Linux/OSx: Open terminal and enter heroku login.

You should see the login successful as shown below

 2. Creating and deploying python flask application using Heroku CLI

This task mainly helps you to learn the deployment of python applications locally and on the
Heroku platform using Heroku CLI. We will deploy our text-file based Flask message board
application.

 Fetch the application source code

o git clone https://bitbucket.org/jaks6/cloud-computing-2022-lab-
1.git lab1app
 To make the application work in Heroku, we need to create a Procfile. The Procfile will
instruct Heroku how exactly this application should be run (e.g. what command starts this
software, are additional arguments necessary, etc)
 Go inside the application's src directory (containing app.py and other files) and create a file
called Procfile
o Contents of Procfile should be

web: gunicorn app:app.

 web indicates the process type. In case of "web" type, Heroku routers will
forward HTTP traffic to this process.
 gunicorn indicates the web interface to use. Gunicorn is a Web Server
Gateway Interface. It acts as a lightweight server who delegates incoming
requests to the actual Python server ( based on Flask in our case). We use
Gunicorn, because it allows to handle multiple requests concurrently, by
default a Flask will process only handle 1 request at a time. Read here for
more on why we use Gunicorn with Heroku.
 app:app indicates which application gunicorn should use - "app:app" refers
to module name and filename ( module app, file app.py, your flask
application).
 Update the requirements.txt, adding Gunicorn to the list of packages.
o gunicorn==20.1.0
Local deployment

Now let's run the application locally (on our own development machine instead of Cloud)
using Heroku and Gunicorn. Gunicorn actually is not available on Windows, so in case of
using Windows, we will use an alternative Procfile.

 First, install the libraries defined in requirements.txt using pip in a venv:

o On Windows:
 py -m venv env or python3 -m venv env
 .\env\Scripts\activate
 pip install -r requirements.txt
o On Linux:
 python3 -m venv env
 source env/bin/activate
 pip3 install -r requirements.txt
o Then, run the app with Heroku (adjusting for Windows OS, if necessary):
 In Windows
 create a file Procfile.windows with contents: web: flask run
 Run: heroku local -f Procfile.windows
 We are using an alternate Procfile which runs the app with python
directly (like we did in lab 1)
 In Linux : heroku local

You can open the local app in the browser using the link http://127.0.0.1:5000

Deploying on Heroku platform

Here, you will learn how to deploy the flask application into the Heroku platform using
Heroku CLI and Git. Heroku manages the application code in Heroku's Git repository. Each
created Heroku application will have its own git repository.

 Update the current git project by adding Heroku's repository as a remote

heroku git:remote -a appname.

o appname - the name of the application you created on Heroku in the 1st
task
o This allows us to start deploying updates to the code to Heroku by pushing
git commits.
 Add current changes code changes to git git add .
 Commit the code

git commit -m "Added Heroku Procfile and Gunicorn"

 Deploy application git push heroku master

o You should start seeing the application being deployed - requirements being
installed, etc.

 Open the deployed app

heroku open
 Make sure everything works!

o You can check the logs

heroku logs --tail

 Provide the exact URL (...herokuapp.com) of your app as a deliverable

 Make a screenshot of the browser showing your deployed application (1 point)
o NB! Your app-name (URL) must be clearly visible on the screenshot!
 Experiment 1: Perform the following experiment:
o Enter a few messages to the app, then let it run idle for at least 30 minutes (nobody
should load the page for at least 30 minutes). After at least 30 min has passed,
access the app again. Answer the below question:
 What happened to the state / data of the app? Why did this happen? (1
point)
 Note: if you think somebody else has somehow accessed your app within
the time, you can use heroku logs to double-check if the app really was idle
for 30 minutes or not.

3. Working with datastore on Heroku platform

In this task, you will learn about working with datastore by provisioning managed Postgres
SQL provided by Heroku and how to use this datastore with the database version of our
application.

 Exit the previous tasks Flask application directory, and fetch the database version of the
code (the one introduced in Exercise 2). This version of the app uses SQLAlchemy to connect
with PostgreSQL, more information about SQLAlchemy can found here
https://www.sqlalchemy.org/. :
o git clone https://bitbucket.org/shivupoojar87/task3lab2app.git
lab2app
o Enter the directory of lab2app
o Create a 2nd Heroku application, this time using CLI instead of the web dashboard.
As app name use yourname-v2 heroku create yourname-v2 --buildpack
heroku/python (Make sure that you have logged in using heroku login and you
are in lab2app application directory)

 Add a Procfile, and add gunicorn to requirements.txt (just like last exercise)

 Provision Postgres SQL service using the command heroku addons:create heroku-
postgresql:hobby-dev --app appname, appname--> your application name
o This creates a postgres database service running in Heroku for our app.
o hobby-dev indicates postgres license and we are using a free license in this
experiment. For more information refer here.
o deliverable: take a screenshot of the output of heroku addons command (1 point)
  If you inspect the code of app.py, you will see that the application is configured to use a
database specified by the environment variable "DATABASE_URL":

app.config['SQLALCHEMY_DATABASE_URI'] = os.environ['DATABASE_URL']

o DATABASE_URL defines where the database is located and its authentication details.
o In Exercise 2 with Docker, we passed the value of DATABASE_URL when starting the
container. With Heroku, because we have added the Postgres addon, this value is
automatically defined on the Heroku platform.
o You can inspect the value of DATABASE_URL using heroku config. You will need
this value to try running the database-version locally.
 Let's try running this version of the application.
o heroku git:remote -a appname
o git add .
o git commit -m 'Deploy'
o git push heroku master
 Now you can use heroku open command to automatically open the browser window with
a correct address of your application
 If you experience error such as UniqueViolation: duplicate key value violates
unique constraint "pg_type_typname_nsp_index , try running heroku restart
or simply wait a few minutes. Sometimes applying database changes can take time.
 You can also explore your Cloud database at https://data.heroku.com/
 If you want to test locally (it is OK to test only in Heroku cloud), you need to define the value
of the DATABASE_URL environment variable, which defines where the database is located
and its authentication details.
o Define environment variable before deploying your application locally:
 in Windows CMD command line: set
DATABASE_URL=postgres://yrt...
 In windows PowerShell: $env:DATABASE_URL="postgres://yr..."
 in Linux: export DATABASE_URL=postgres://yr...
 Make screenshots of the heroku addons and heroku config commands after
completing the above steps successfully. (2 points)
o NB! The DATABASE_URL value includes not only the DB address but also your
username and password (the part between postgres:// and @ ). MAKE SURE
THAT YOU DO NOT SHARE THIS WITH US AS A GOOD PRACTICE OF NOT FREELY
DISTRIBUTING CREDENTIALS. You can just draw over that part of the screenshot.
o NB! The entire output of those commands and your app-name must be clearly
visible in the screenshots!
 Experiment 2: Repeat the same test as earlier: post a few messages on the Heroku deployed
version (not local), then leave the app idle for at least 30 min, then re-visit the application.
Answer these questions
o How is the behavior different compared to the text-based version? Why do you
think this is happening? (1 point)

4. Improving the Heroku application

For this task, you have to design & implement some improvements to the application we just
created. You should modify the application to ask for the name of the user and to limit how
many messages are shown on the main page.

1. Modify the Messageboard database class to add a "Author" (name) field to the
database table. (2 points)
 o Study the existing code and check SQLAlchemy documentation and other
online resources how to add extra fields to the database table.
o You need to modify the HTML form, home() method and the Messageboard
class.
o The application should display the author of the message next to the contents
on the main page
o Modifying SQL table structure is somewhat complex in SQLAlchemy. But it
is simple to delete the table (NB! and all data!!) and recreate it with new
structure (After modifying the Messageboard class) by running the drop
table messages SQL command.

db.engine.execute("drop table messages")

db.session.commit()

o NB! remember to remove this line once you are done with the database
change!
o You can also run it from the Python console instead, to be safer:

import app
app.db.engine.execute("drop table messages")
app.db.session.commit()

2. Modify the application to limit the display to only 10 latest messages. (2 points)
o Check SQLAlchemy documentation how to limit the number of entries
returned by database queries (Messageboard.query.all()) and how to order
them.
o The limiting should be done on the Database/SQLAlchemy level, when
querying, NOT with something like Python list manipulation operations or
Jinja template operations.
o The final output looks something like this (your design can vary):
  Make a screenshot of the browser showing your deployed application
o NB! Your app-name must be clearly visible on the screenshot!

Follow these guidelines while submitting the exercises

1. Put a pdf file with your answers and screenshots for the questions along with all the source
code in a zip file, and name it as your studentid_QE3
2. The screenshots are meant to show that the work is done and by you. So, have enough
proofs in your answer sheet.
3. Note: Do not blindly copy the commands. Try to understand them. The provided commands
are to guide you with the initial process.
4. Do not copy code from internet. If you copy, please leave a comment with the URL on that
code/answer sheet
5. Plagiarism will not be tolerated and dealt through the formal channels of the university