Professional Documents
Culture Documents
menu
Table of Contents
Table of Contents
Platform as a Service?
Is it right for me?
I like it, but I have this server
Dokku
Work On A Project
Starting point
PaaS project setup in Dokku
How to build: .buildpacks
How to run: Procfile
Ready!
Moar! Moar! Moar!
First, evolve the code
Then, add a new service
Then, learn about scaling!
Then, we think were ready but
It will not take long now!
Backing Services
Encryption Anyone?
Summing Up
Updates
Platform as a Service?
Wikipedia describes Platform as a Service (abbreviated PaaS) like this:
One of the most widely used PaaS platforms that I know of is Heroku.
Once you set up your project on Heroku (which takes very little energy
with a few commands and configuration files), the promise is that your
development/deployment workflow will be this:
So yes, the promise is that all your deployment effort is just the git
push command. Nifty, uh?
Dokku
Dokku is a tiny PaaS platform that you can install on your server. Note
that I used the singular form: its meant for very restricted environments
where all you have and need is one single server; nothing important or
production that you care too much, so.
It is described as:
and rightly so: its a smart integration of other tools that can keep the
project itself up in a few (well, a couple thousand) lines of Bash code.
Its easy to get started so I will not repeat the official steps here.
If you want to give it a try, you can also head to Digital Ocean and
use one of their one-click apps. I dont completely like this approach
because their deployment is Ubuntu (I tend to like Debian more) and
their Dokku version is 0.6.5 (theres 0.7.2 around) right now.
You can find a small project dokku-boot on GitHub that will help you
set up 0.7.2 on a new Debian instance (it should work just fine on
Ubuntu too). Its more or less a simple wrapper around the main
installation instructions, but it also installs a couple of handy extensions
for Dokku, like support for Lets Encrypt (for free and hassle-free SSL
certificates), Redis and Postgres.
From the README :
apt-get update
apt-get install -y curl perl
curl -LO https://github.com/polettix/dokku-boot/raw/master/dokku-boot.pl
perl dokku-boot.pl
After all of this, you can just submit the form in the page and you will be
all set (you will be redirected to Dokkus documentation page after this
step).
In the case you actually own a domain and you have control over its
DNS settings (as in our example), you should also set up a wildcard
resolution towards your Dokku installation, with a A record like this:
*.example.com IN A $DOKKU_IP
Work On A Project
Work On A Project
Now you have your personal PaaS instance up and running, waiting for
your code to come. Im a Perl enthusiast, so my example will be in Perl.
In the following sub-sections, well see:
Starting point
Our starting point is a project that tracked with git. In this example, we
will assume that we want to work on a simple Hello World! application
(what else) built with Mojolicious, and we will see a couple of directions
for evolving to a more complex application. You can find the whole
example in this repository in GitHub if you want to skip its construction.
The application itself (in file app.pl ) is quite straightforward:
#!/usr/bin/env perl
use Mojolicious::Lite;
get '/' => sub {
my $c = shift;
$c->res->headers->content_type('text/plain');
$c->render(text => "Hello, World!\n");
};
app->start;
We can install the module locally and check that it works actually:
$ cpanm -L local --notest --quiet --installdeps .
Successfully installed IO-Socket-IP-0.38
Successfully installed Mojolicious-7.08
2 distributions installed
# https://hackernoon.com/lesser-known-git-commands-151a1918a60
# see hint about `git it`
$ git init . && git commit -m Root --allow-empty
The following steps are needed whatever application you want to deploy
via
Are we ready for the first push? Well not yet. We still have to fix a
couple things:
how will Dokku figure out how to build our project? Like installing
the right modules, etc.?
how will it know what to run?
Heroku and Dokku dont support Perl applications out of the box. I
know, its a shame, but its easy to fix this via so-called custom
buildpacks (which are the same as Herokus) and many different clever
persons did this in slightly different ways.
The philosophy under heroku-buildpack-perl-procfile is to make no
assumption on your application, apart that its written in Perl and will
need modules from either [CPAN][] or from some project-local directory.
If you mostly stick with web projects and you want to run them via
[Starman][], you will probably want to look into some other package. I
wanted to keep control, so evolved the awesome original heroku-
buildpack-perl-procfile by kazeburo to suit my needs.
To tell Dokku that you want to use a buildpack you have a couple
options (see custom buildpacks for some alternatives), in our case we
will just add a .buildpacks file in our projects root directory:
$ git add .buildpacks && git commit -m 'Add .buildpacks for Dokku builds'
We will start with one single web component, so our Procfile is quite
simple:
As you can see, when Dokku starts a new web instance, it also passes the
PORT environment variable so that your application can bind to the
correct port. How this parameter is used is dependent on the specific
application and/or framework that the application is using.
As for the .buildpacks file, we have to add Procfile to our git
project and commit:
$ git add Procfile && git commit -m 'Add Procfile for Dokku runs'
Ready!
At this point, we are ready for our first push to our Dokku instance, YAY!
In our first push we will also set the --set-upstream so that our
following push es of master will be automatically sent to the remote
dokku :
To dokku@example.com:sample-mojo
* [new branch] master -> master
Branch master set up to track remote branch master from dokku.
Now we are ready to send the very first HTTP request towards our new
service. As you can see, theres a couple of lines that tells us where the
service lives:
$ curl http://sample-mojo.foobar.example.com
Hello, World!
Seems to work!
At this point you can start hacking on your application and follow
the pattern we described in the beginning:
#!/usr/bin/env perl
use Mojolicious::Lite;
# Slow task
app->minion->add_task(poke_mojo => sub {
my $job = shift;
$job->app->ua->get('mojolicious.org');
$job->app->log->debug('We have poked mojolicious.org for a visitor');
});
app->start;
Of course we will need to make sure that the needed plugins are
correctly installed, so our cpanfile becomes:
Again, we can install them locally and make sure that everything works
fine:
Now, lets make a little local test: we first generate a get to the regular
web application, then we start the minion (we might make it in two
different shells but it will work anyway because Minion is decoupled
through the database):
$ git push
Counting objects: 9, done.
#... some building happens here...
-----> Discovering process types
Procfile declares types -> web, minion
#... something else happens...
=====> web=1
-----> Attempting...
So, it seems that Dokku knows about our minion service type, but it
starts none. Now you understand what we meant before when we said
that web is special because Dokku starts one by default!
A quick look to the current scaling setup is worth the time as a
double check:
$ curl http://sample-mojo.example.com
We will poke mojolicious.org for you soon.
# now we wait a few seconds to give the worker process the time get hold of the
# new task. The number of seconds might be reduced but let's play safely
$ sleep 5
Uhm the worker seems still unaware of the request from the
frontend whats happening?
It turns out that its a database issue here. When we run the example
in our development box, both the web service and the minion worker
were running in the same directory of the same box. On the other hand,
in this case the two are running inside two separate Linux containers
created via Docker, so its like they are running in separate hosts.
Fact is that SQLite is a single file database: for our system to work,
both the web and the minion services MUST operate on the same file!
Now you understand why dokku-boot includes Postgres and Redis
plugins, dont you?
$ git diff
diff --git a/app.pl b/app.pl
index 70b7418..25183d8 100755
--- a/app.pl
+++ b/app.pl @@ -1,7 +1,7 @@ #!/usr/bin/env perl use Mojolicious::Lite;
# Slow task
app->minion->add_task(poke_mojo => sub {
$ git push
#... wait for it...
$ curl http://sample-mojo.example.com/
We will poke mojolicious.org for you soon.
# now we wait a few seconds to give the worker process the time get hold of the
# new task. The number of seconds might be reduced but let's play safely
$ sleep 5
It works! As you can see, there are two logs line from the minion.1 app
component that say:
[...] [debug] Performing job "1" with task "poke_mojo" in process 143
[...] [debug] We have poked mojolicious.org for a visitor
which tell us that the minion received the task and executed it.
Its instructive at this point to take a look at the directory that was
created as /var/lib/dokku/data/storage/sample-mojo ; we will need to
impersonate root on the Dokku node this time:
Whats this thing with user and group ids 32767 ? Simple: when Docker
is instructed to run containers by Dokku, it is told to start them as these
user and group id. They are different enough from what you have in the
machine (the highest user id is dokku s at 1000 , the highest group id is
still dokku s at 1000 except for a few service group ids that are at
65534 , so still very distant from 32767 ) so that you can be reasonably
sure there will be no clash or overlapping by chance.
Backing Services
As we saw in a previous section, each service instance runs inside its
own container and is quite isolated from the other ones, even though
they actually run on the same node. Thats what makes Dokku powerful:
it allows you to run multiple services, which might each have their own
quirk about filesystem, configurations, running processes, library
versions and so on, but still guarantee that they will play nicely in the
same host.
One of the consequences is that you have to take explicit actions to
have them share files/directories: in our case, we wanted the web and
minion services to share a common directory where they would be able
to operate on the same SQLite file. We were lucky that there is the
storage management to help us with this.
What if we want to connect services differently? For example, we
might need to use a different database technology, and choose
PostgreSQL instead. Heroku and Dokku can get you covered for a lot of
technologies, and lucky for us theres the Postgres plugin that will help
us right out of the box.
The initial Dokku tutorial help us here, so we will just take the
relevant command without too many comments. First, we take care to
see whether the plugin is really installed or not:
$ dokku help
Usage: dokku [--quiet|...
Commands:
apps L...
certs M...
...
Fine, we do have plugins for Lets Encrypt, Redis and Postgres, listed in
the Community plugin commands section.
These services provided out of the box are called backing services;
there are quite a number of them (e.g. see the official ones) and they
should get you covered in most situations.
Lets define one backing service for our application then, by first
creating it and then linking to our application:
$ dokku postgres:create sample-mojo-pg
Waiting for container to be ready
Creating container database
Securing connection to database
=====> Postgres container created: sample-mojo-db
=====> Container Information
Config dir: /var/lib/dokku/services/postgres/sample-mojo-db/config
Data dir: /var/lib/dokku/services/postgres/sample-mojo-db/data
Dsn: postgres://postgres:1b8c1fb63db2cbee3c407c8fd815152a@dokku-p
Exposed ports: -
Id: ecaf7be1f12cc146da63c2da1b4f9c737e26ccce6f3aef27d7af302feb1d
Internal ip: 172.17.0.5
Links: -
Service root: /var/lib/dokku/services/postgres/sample-mojo-db
Status: running
Version: postgres:9.5.4
As we can see, the plugin took care to set the environment variable
DATABASE_URL in our application to the right value for consumption by
the application itself. We can now use it in our code then, so the app.pl
file is modified as follows:
#!/usr/bin/env perl
use Mojolicious::Lite;
# Slow task
app->minion->add_task(poke_mojo => sub {
my $job = shift;
$job->app->ua->get('mojolicious.org');
$job->app->log->debug('We have poked mojolicious.org for a visitor');
});
app->start;
This will make sure that we can continue to use SQLite locally and
PostgreSQL remotely (even though you should strive to have perfect
alignment across all your deployment environments!).
We just have to make sure that our Perl application will be able to
use PostgreSQL now, so the cpanfile becomes as follows:
$ git push
Counting objects: 7, done.
Compressing objects: 100% (4/4), done.
Writing objects: 100% (4/4), 578 bytes, done.
Total 4 (delta 2), reused 0 (delta 0)
-----> Cleaning up...
-----> Building sample-mojo from herokuish...
-----> Adding BUILD_ENV to build environment...
-----> Multipack app detected
=====> Downloading Buildpack: https://github.com/polettix/heroku-buildpack-perl-procfile
=====> Detected Framework: Perl/Procfile
-----> Installing dependencies
Successfully installed DBD-Pg-3.5.3
Successfully installed Mojo-Pg-2.30
2 distributions installed
... all good up to now...
remote: App container failed to start!!
=====> sample-mojo web container output:
Invalid PostgreSQL connection string "postgres://postgres:1b8c1fb63db2...
... last message repeated a few times...
! [remote rejected] master -> master (pre-receive hook declined) error:
failed to push some refs to ...
Ouch! Well, I think Mark Jason Dominus got it right at this point:
#!/usr/bin/env perl
use Mojolicious::Lite;
# Slow task
app->minion->add_task(poke_mojo => sub {
my $job = shift;
$job->app->ua->get('mojolicious.org');
$job->app->log->debug('We have poked mojolicious.org for a visitor');
});
app->start;
# check logs
$ dokku logs sample-mojo
... app[minion.1]:... Worker 8 started
... app[minion.1]:... Checking worker registry and job queue
... app[minion.1]:... Performing job "1" with task "poke_mojo" in process 145
... app[minion.1]:... We have poked mojolicious.org for a visitor
...
The fact that its performing job 1 again is another hint that it is
actually using PostgreSQL, because the jobs numbering was restarted.
Anyway, we dont need the persistent storage any more at this time, so
we can double check by just removing it:
$ curl http://sample-mojo.example.com
We will poke mojolicious.org for you soon.
# check logs
$ dokku logs sample-mojo
... app[minion.1]:... Worker 7 started
... app[minion.1]:... Checking worker registry and job queue
... app[minion.1]:... Performing job "2" with task "poke_mojo" in process 145
... app[minion.1]:... We have poked mojolicious.org for a visitor
...
Encryption Anyone?
Up to this point, all our traffic has been left into the wilderness of plain
text. If you think that encrypting communications can be a good thing,
though, you probably already started wondering how to add TLS to the
lot.
Dokku provides you means for managing certificates via the certs
group of commands:
$ dokku certs:help
Usage: dokku certs:COMMAND
Additional commands:
certs:add <app> CRT KEY Add an ssl endpoint to an app. Can also import f
certs:chain CRT [CRT ...] [NOT IMPLEMENTED] Print the ordered and complete
certs:generate <app> DOMAIN Generate a key and certificate signing request (
certs:info <app> Show certificate information for an ssl endpoint
certs:key <app> CRT KEY [KEY ...] [NOT IMPLEMENTED] Print the correct key for the
certs Manage Dokku apps SSL (TLS) certs
certs:remove <app> Remove an SSL Endpoint from an app
certs:rollback <app> [NOT IMPLEMENTED] Rollback an SSL Endpoint for a
certs:update <app> CRT KEY Update an SSL Endpoint on an app. Can also impor
This is definitely the way to go if you are using a domain where you cant
define a wildcard resolution or its not immediately below a public suffix,
or more simply you want to use your homebrewn self-signed certificates
(which is perfectly secure if you have full control on all clients, of
course!).
If you instead:
then you can enjoy the services of the Lets Encrypt plugin. Lets see
how.
The only real setup you have to do is define an email address; this is
used by Lets Encrypt to notify you about certificates that are about to
expire (this will be our safenet, because we can setup the plugin to renew
the certificates automatically). Then you just have to activate the plugin.
If youre lazy like me, you can set the email address up as a global
variable, so that every project will use the same. You can still override
this on a per-project basis, anyway.
Then, you just have to activate the plugin for your app:
So, after a bunch of back and forth with Lets Encrypt, the plugin took
care to reconfigure the reverse proxy to:
$ curl -v http://sample-mojo.example.com/
* About to connect() to sample-mojo.example.com port 80 (#0)
...
> GET / HTTP/1.1
> User-Agent: curl/7.26.0
> Host: sample-mojo.example.com
> Accept: */*
>
...
< HTTP/1.1 301 Moved Permanently
< Server: nginx
< Date: Sun, 09 Oct 2016 07:09:55 GMT
< Content-Type: text/html
< Content-Length: 178
< Connection: keep-alive
< Location: https://sample-mojo.example.com:443/
...
Lets follow the redirection now, note that curl does not complain at
all about the certificate:
$ curl -L http://sample-mojo.example.com/
We will poke mojolicious.org for you soon.
Done! Dont worry about the last line, it appears if our Dokku instance is
brand new and no previous crontab setting was present for user
dokku .
Summing Up
After this (admittedly) long journey, we got to this point:
This is probably something that should get you more than started! For
going beyond, you should definitely check the excellent documentation
in Dokkus website.
Have fun!
Updates
2016-10-09 a change for supporting the postgres:// url scheme
in addition to postgresql:// is on its way (see this commit). This
makes using environment variable DATABASE_URL a breeze (see
Backing Services) as it will be immediately consumable by Mojo::Pg
and the associated Minion backend, YAY! (We will have to wait for
the new release of Mojo::Pg though!).
2016-10-16 most of the stuff in this article has been compressed
in a cheatsheet available in this wiki. Nice thing about GitLab is
that its possible to also whip up a few snippets.
2016-10-16 added a Table of Contents for better navigation of the
document.
2016-12-23 added note on Debian 8 in Vultr.