FooBar

Release 0.0.98+b84900a

October 26, 2015

 Contents

1 Bootstrapping 3
1.1 Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Team Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 Developing the backend 5

2.1 Development Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Restful Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 API Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

3 Developing the frontend 11

3.1 Frontend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
3.2 Debugging the frontend build . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4 Reference 13
4.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

5 Documentation 15
5.1 Building the documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Python Module Index 17

i
ii
 FooBar, Release 0.0.98+b84900a

Author pyfidelity UG
Version 0.0.98+b84900a

Contents 1
FooBar, Release 0.0.98+b84900a

2 Contents
 CHAPTER 1

Bootstrapping

Start here to join the team and get a local development intance up and running.

1.1 Installation

1.1.1 Development Requirements

The application is based on the Pyramid framework and requires Python 2.7, PostgreSQL and a basic development
setup.

Mac OS X

On Mac OS it’s recommended to first install the dependencies using brew.

If you want to build the backend:
$ brew install postgres python

If you want to build the frontend:

$ brew install node libpng ruby

Note: Even though Mac OS X ships with Python2.7 it is highly recommended to use the version provided by brew to
avoid changing the system’s Python setup.

To finish the build of the frontend you need to install the following npm modules. While in theory this could be done
locally for this project by its Makefile, in practice it’s much(!) easier to simply install them globally like so and be
done with it:
$ npm install -g yeoman bower grunt-cli generator-angular

1.1.2 Bootstrapping

First clone the repository and change into it:

$ cd foobar

To set up a development environment simply run:

3
FooBar, Release 0.0.98+b84900a

$ make

This assumes a PostgreSQL installation on localhost:5432.

1.1.3 Starting up

For a full stack you need a running instance of the backend plus the frontend.

Backend

To run the application during development use:

$ backend/bin/pserve backend/development.ini
Starting server in PID 76230.
serving on http://0.0.0.0:6543

You can now visit http://localhost:6543.

Frontend

The frontend has its own development webserver which is the recommended way to access the stack during develop-
ment.
For one, it performs automatic reloads in the browser when files change and secondly it proxies requests to the backend
to the pyramid instance.

1.2 Team Resources

1.2.1 Git Access

The git repository is hosted at GitHub. We have a ‘master’ repository located at github.com/snakeoil/foobar. This
master repository only contains the main branches (master, demo, staging, release etc.)
If you don’t already have a GitHub account, create a new one and add your ssh public key (you won’t get access via
HTTP).

1.2.2 Issues

Development is managed using GitHub’s issues.

1.2.3 IRC

When working on the project, developers are expected to be online on IRC at #foobar on irc.freenode.net.

4 Chapter 1. Bootstrapping
 CHAPTER 2

Developing the backend

Continue here, if you develop the application

2.1 Development Conventions

Coding, testing and documentation conventions.

2.1.1 Basic conventions

• All python code must pass pyflakes and pep8 (we allow linelengths of 132, though)
• the master branch must have 100% code coverage at all times.
• all tests must pass on master at all times.

2.1.2 Testing

Tests are written and run using the the pytest framework. To run them simply use:
$ cd backend/
$ bin/tox

To run an individual test, use the -k parameter via bin/tox – -k ... (see the pytest documentation).
To create a coverage report, run:
$ bin/tox -- --cov=backrest

This generates a report on the console, as well as a pretty report in htmlcov/index.html where you can browse the code
and see which lines are not covered:
$ open htmlcov/index.html

2.1.3 Documentation

Documentation is maintained with sphinx. When writing documentation, please see the sections reStructuredText
Primer and especially Inline markup of the sphinx documentation for hints about how to format sections etc.
See PEP 257 for docstring conventions.

5
FooBar, Release 0.0.98+b84900a

The majority of the documentation should be in the code! The .rst files should be only used for meta topics – such
as this, regarding documentation ;-) – or deployment etc.
Doc strings should start with a short sentence describing the purpose and/or reason of the method, followed by the
same for each parameter (see sphinx autoclass documentation and the sphinx Python domain on how to format those.)

Building the documentation

Before merging work you need to make sure the documentation you added doesn’t break anything. Do this by gener-
ating it locally by running:
$ make docs

Make sure that doesn’t result in erros or warnings and then run:
$ open docs/htdocs/index.html

to open it in your browser to see what it looks like.

2.2 Restful Services

The backend is entirely RESTful and exposes its resources using the cornice library along with the colander library
for schemas and validation.

2.2.1 Schemas and validation

For each resource we define a colander schema, which is then attached to the cornice resource. Any action against
the resource will then automatically be validated against the schema by cornice. If it succeeds, the validated (and
perhaps sanitized) values of the data are available on the request object as request.validated. Consumers
should always only access data from here. While the original data is still available on the request it may be in an
unsafe state or contain additional values that are not part of the schema.
As an example:
from colander import MappingSchema, SchemaNode, SequenceSchema
from colander import Integer
from .schemas import ContentSchema, FileSchema, MissingOrRequiredNode

class Foo(ContentSchema):

size = SchemaNode(Integer())

For defining schemas and validators refer to the colander documentation.

If validation fails, cornice returns a 400 response for us along with a JSON body that contains the actual errors (there
may be more than one, afterall). This behavior is documented here.

2.2.2 Declaring resources

rest-base contains the main components with which we define resources: rest_resource (a decorator),
Content a base class, and Resource a generic resource implementation on top of cornice.
Let’s look at the Foo model as an example:

6 Chapter 2. Developing the backend

 FooBar, Release 0.0.98+b84900a

from .models import Content

class Foo(Content):

Next, we define its schema:

schema = schemas.Foo

Each content is identified by a global namespace id (which is already part of the Content schema, so we didn’t have
to include it above):
id = Column(Integer, ForeignKey(Content.id), primary_key=True)

Then we have the attributes of a foo:

size = Column(Integer(), nullable=False, default=0)

Each content must provide a update method that receives key-value data and persists it. Note, that we do not perform
any validation here! The assumption is that we receive already validated data, as it has passed the cornice validation.
This means, that the update method can usually be called directly with with **request.validated:
def update(self, size=23, **data):
[...]

Finally, each content must be able to render itself as valid JSON. To this end it must provide a __json__ method
that gets a request and must return a dictionary:
def __json__(self, request):
return dict(super(Foo, self).__json__(request),
size=self.size)

2.2.3 Exposing resources

Once the model and schema have been defined, the resource can be exposed to the nasty, scare interweb. This is done
with the @rest_resource view decorator. Back to our foo example:
from cornice.resource import view
from sqlalchemy.sql.expression import desc

from .views import rest_resource, Content, Resource

from . import _, models, schemas

@rest_resource(model=models.Foo)

The decorator ties it with model and also defines default paths (routes) via convention. It does so by adding the API
root (by default /-/ with the lower caseed name of the class and the same in plural form for the collection variant of
the model, i.e. in this case /-/foo and /-/foos. This behavior is also partiall defined in the Content resource
which we subclass:
class Foo(Content):
model = models.Foo

In following method we expose /-/foos for GET and return a list of all foos in reverse chronological order:
@view(renderer='json', permission='view')
def collection_get(self):
return self.model.query.order_by(desc(self.model.creation_date)).all()

2.2. Restful Services 7

FooBar, Release 0.0.98+b84900a

Notice, that the view only needs to return instances of our (SQLAlchemy) models. Their __json__ along with
pyramid’s built-in JSON support take care of everything else.

2.3 API Documentation

2.3.1 Public API

The public API is exposed as a RESTful JSON interface using cornice and consists of the following services.

Note: For frontend developers, this is the only API documentation you should need.

Signup

New users can register an account by POSTing to /signup:

>>> browser.post_json('http://example.com/-/signup', {
... "email": "alice@example.com",
... "password": "hurz"
... }).json['status']
u'success'

Login

Existing users can log in using a PUT request at /login:

>>> browser.put_json('http://example.com/-/login', {
... "login": "alice@foo.com",
... "password": "alice"
... }).json['status']
u'success'

While logged in the frontend may request authentication info via GET:
>>> info = browser.get_json('http://example.com/-/login').json
>>> info['authenticated']
True
>>> info['firstname'], info['lastname'], info['email']
(u'Alice', u'Kingsleigh', u'alice@foo.com')

Logging out again is handled via a PUT at /logout:

>>> browser.put_json('http://example.com/-/logout', {}).json['status']
u'success'

Password reset

Existing users can request a password reset by POSTing to /reset/:

>>> browser.post_json('http://example.com/-/reset/', {
... "email": "alice@foo.com"
... }).json['status']
u'success'

8 Chapter 2. Developing the backend

 FooBar, Release 0.0.98+b84900a

Password change

Existing users can change their password by PUTing to /password:

>>> browser.put_json('http://example.com/-/password', {
... "current": "alice",
... "password": "foo!"
... }).json['status']
u'success'

Email change

Existing users can request to change their email address by PUTing to /-/email/:
>>> browser.put_json('http://example.com/-/email/', {
... "email": "alice@bar.com",
... "password": "alice"
... }).json['status']
u'success'

User profile

Users can change their profile data by PUTing to /userprofile:

>>> data = browser.get_json('http://example.com/-/userprofile').json
>>> data['firstname']
u'Alice'
>>> data['lastname']
u'Kingsleigh'

>>> browser.put_json('http://example.com/-/userprofile', {
... "firstname": "Alien",
... "lastname": "Species",
... }).json['status']
u'success'

2.3.2 Base API Classes

The above JSON services are based on a few helper classes.

backrest.views.id_factory(model)
Returns a factory for the given model, which in turn will return the instance of that model with the id taken
from the request’s matchdict. If no item with the id in question exists, NotFound is raised.
class backrest.views.Content(context, request)
A REST resource collection for content objects.
Every GET returns a JSON representation of the given resource, designed to be passed back into a PUT or POST
call.
Every POST or PUT returns a (possibly updated) representation of the object that has been modified back to the
calling client.
A client that wishes to perform an update should always post back the smallest subtree possible.
Technically, we use pyramid’s JSON renderer to generate the response, so in the service methods we simply
return instances of our models, which in turn have a __json__ method, which is called by the renderer.

2.3. API Documentation 9

FooBar, Release 0.0.98+b84900a

When posting or putting a resource we call its update method passing in the data. That method is expected
to only process the keys that the model provides and to silently ignore any that are not. This behavior can be
used to pass (read-only) meta data to the client, which by convention lives in a singly entry with a key named
__meta__.
Ideally the client would never post data that contains this key, but even if so, it will be ignored.

2.3.3 Base Models

The below models try to provide a basis for content-like and file-like types. The latter deal with binary data using
repoze.filesafe in order to be transaction-safe.

10 Chapter 2. Developing the backend

 CHAPTER 3

Developing the frontend

Continue here, if you develop the frontend application

3.1 Frontend

The frontend is a self-contained HTML5 application built on top of the AngularJS framework. It lives inside
frontend and all commands here assume you are in that directory.
It has been bootstrapped using the generator-angular for yeoman.

3.2 Debugging the frontend build

The frontend is built using grunt which minifies JS, CSS etc and creates the site as a self-contained folder inside
frontend/dist. To test it, use grunt server:dist.

11
FooBar, Release 0.0.98+b84900a

12 Chapter 3. Developing the frontend

 CHAPTER 4

Reference

Keep these under your pillow :)

4.1 Architecture

The project is designed as a client/server application. The server is written in Python using the Pyramid framework
and (mostly) serves and processes JSON data.
The client is a web application written in Javascript using the AngularJS framework.
On the server this model is implemented (and persisted) using SQLAlchemy. On the client side using AngularJS’s
controller concept.
The JSON representation used to communicate between server and client simply consists of nested dictionaries.
Give the general securrity restrictions of browsers, both the client application and the ReST service must be hosted on
the same machine. The frontend is configured at /, the entry point for the JSON API is at /-.

4.1.1 URL authority

Given that the web client only ever loads the initial seed HTML from the server and that all subsequent requests are
handled as XHR, we essentially have two URL namespaces. One for the JSON API (mapping services and resources
to URLs) and the other for the AngularJS application which does its own (# based) routing ($routeProvider).
This means, that the client must only call verbatim service urls it receives from the server (i.e. never compute a
resource URL from an id or another URL etc.) and in turn it is the duty of the service to provide all required URLs as
fully qualified, absolute URLS to the client.
OTOH the server must never return angular application URLs, only service URLS.

13
FooBar, Release 0.0.98+b84900a

14 Chapter 4. Reference
 CHAPTER 5

Documentation

(This) documentation is maintained with sphinx. When writing documentation, please see the sections reStructured-
Text Primer and especially Inline markup of the sphinx documentation for hints about how to format sections etc.
See PEP 257 for docstring conventions.
The majority of the documentation should be in the code! The .rst files should be only used for meta topics – such
as this, regarding documentation ;-) – or deployment etc.
Doc strings should start with a short sentence describing the purpose and/or reason of the method, followed by the
same for each parameter (see sphinx autoclass documentation and the sphinx Python domain on how to format those.)

5.1 Building the documentation

Before merging work you need to make sure the documentation you added doesn’t break anything. Do this by gener-
ating it locally by running:
$ make docs

Make sure that doesn’t result in erros or warnings and then run:
$ open docs/htdocs/index.html

to open it in your browser to see what it looks like.

15
FooBar, Release 0.0.98+b84900a

16 Chapter 5. Documentation
 Python Module Index

b
backrest, 8
backrest.models, 10
backrest.views, 9
backrest.views.change_email, 9
backrest.views.change_password, 9
backrest.views.login, 8
backrest.views.reset_password, 8
backrest.views.signup, 8
backrest.views.user_profile, 9

f
foobar, 1

17
FooBar, Release 0.0.98+b84900a

18 Python Module Index

 Index

B
backrest (module), 8
backrest.models (module), 10
backrest.views (module), 9
backrest.views.change_email (module), 9
backrest.views.change_password (module), 9
backrest.views.login (module), 8
backrest.views.reset_password (module), 8
backrest.views.signup (module), 8
backrest.views.user_profile (module), 9

C
Content (class in backrest.views), 9

F
foobar (module), 1

I
id_factory() (in module backrest.views), 9

19