PostgreSQL-Audit Documentation

Release 0.10.0

Konsta Vesterinen

Jul 23, 2018

 Contents

1 Installation 3
1.1 Supported platforms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Installing an official release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.3 Installing the development version . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.4 Checking the installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

2 SQLAlchemy integration 5
2.1 Excluding columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.2 Versioning many-to-many tables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.3 Tracking inserts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.4 Tracking updates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.5 Tracking deletes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.6 Finding history of specific record . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
2.7 Temporarily disabling inserts to the activity table . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3 Flask extension 9
3.1 Overriding activity values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
3.2 Recording IP address behind proxy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

4 Migrations 13
4.1 Changing column name . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
4.2 Alter column type . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.3 Removing columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
4.4 Adding columns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
4.5 Rename table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Python Module Index 17

i
ii
 PostgreSQL-Audit Documentation, Release 0.10.0

Contents:

Contents 1
PostgreSQL-Audit Documentation, Release 0.10.0

2 Contents
 CHAPTER 1

Installation

This part of the documentation covers the installation of PostgreSQL-Audit.

1.1 Supported platforms

PostgreSQL-Audit has been tested against the following Python platforms.

• cPython 2.7
• cPython 3.3
• cPython 3.4
• cPython 3.5

1.2 Installing an official release

You can install the most recent official PostgreSQL-Audit version using pip:

pip install postgresql-audit

1.3 Installing the development version

To install the latest version of PostgreSQL-Audit, you need first obtain a copy of the source. You can do that by
cloning the git repository:

git clone git://github.com/kvesteri/postgresql-audit.git

Then you can install the source distribution using the setup.py script:

3
PostgreSQL-Audit Documentation, Release 0.10.0

cd postgresql-audit
python setup.py install

1.4 Checking the installation

To check that PostgreSQL-Audit has been properly installed, type python from your shell. Then at the Python
prompt, try to import PostgreSQL-Audit, and check the installed version:
>>> import postgresql_audit
>>> postgresql_audit.__version__
0.10.0

4 Chapter 1. Installation
 CHAPTER 2

SQLAlchemy integration

SQLAlchemy integration offers easy way of using PostgreSQL-Audit with SQLAlchemy ORM. It has the following
features:
• Automatically marks all declarative classes which have __versioned__ class property defined as versioned.
• Attaches after_create DDL listeners that create versioning triggers for all versioned tables.
• Provides Activity model for easy ORM level access of activities
from postgresql_audit import versioning_manager

versioning_manager.init(Base)

class Article(Base):
__tablename__ = 'article'
__versioned__ = {}
id = Column(Integer, primary_key=True)
name = Column(String)

article = Article(name='Some article')

session.add(article)
session.commit()

2.1 Excluding columns

You can easily exclude columns from being versioned by adding them as a list to ‘exclude’ key of __versioned__ dict.
class Article(Base):
__tablename__ = 'article'
__versioned__ = {'exclude': 'created_at'}
(continues on next page)

5
PostgreSQL-Audit Documentation, Release 0.10.0

(continued from previous page)

id = Column(Integer, primary_key=True)
name = Column(String)
created_at = Column(DateTime)

2.2 Versioning many-to-many tables

Versioning Table objects is easy. Just call audit_table method with the desired table.

class User(Base):
__tablename__ = 'user'
__versioned__ = {}
id = Column(Integer, primary_key=True)
name = Column(String)

class Group(Base):
__tablename__ = 'group'
__versioned__ = {}
id = Column(Integer, primary_key=True)
name = Column(String)

group_user = Table(
'group_user',
Base.metadata,
Column(
'user_id',
Integer,
ForeignKey('user.id', ondelete='CASCADE'),
nullable=False,
primary_key=True
),
Column(
'group_id',
Integer,
ForeignKey('place.id', ondelete='CASCADE'),
nullable=False,
primary_key=True
)

versioning_manager.audit_table(group_user)

2.3 Tracking inserts

Now we can check the newly created activity.

Activity = versioning_manager.activity_cls

activity = Activity.query.first()
activity.id # 1
activity.table_name # 'article'
(continues on next page)

6 Chapter 2. SQLAlchemy integration

 PostgreSQL-Audit Documentation, Release 0.10.0

(continued from previous page)

activity.verb # 'insert'
activity.old_data # {}
activity.changed_data # {'id': '1', 'name': 'Some article'}

2.4 Tracking updates

article.name = 'Some other article'

session.commit()

activity = Activity.query.order_by(db.desc(Activity.id)).first()
activity.id # 2
activity.table_name # 'article'
activity.verb # 'update'
activity.old_data # {'id': '1', 'name': 'Some article'}
activity.changed_data # {'name': 'Some other article'}

2.5 Tracking deletes

session.delete(article)
session.commit()

activity = Activity.query.order_by(db.desc(Activity.id)).first()
activity.id # 3
activity.table_name # 'article'
activity.verb # 'delete'
activity.old_data # {'id': '1', 'name': 'Some other article'}
activity.changed_data # {}

2.6 Finding history of specific record

In this example we want to find all changes made to article with id=3. The query is a bit complex since we have to
check old_data and changed_data separately. Luckily the Activity model has a hybrid_property_ called ‘data which
is a combination of these two. Hence you can get the desired activities as follows:

activities = session.query(Activity).filter(
Activity.table_name == 'article',
Activity.data['id'].astext.cast(db.Integer) == 3
)

2.7 Temporarily disabling inserts to the activity table

There are cases where you might not want to track changes to your data, such as when doing big changes to a table.
In those cases you can use the VersioningManager.disable context manager.

2.4. Tracking updates 7

PostgreSQL-Audit Documentation, Release 0.10.0

with versioning_manager.disable(session):
for i in range(1, 10000):
db.session.add(db.Product(name='Product %s' % i))
db.session.commit()

8 Chapter 2. SQLAlchemy integration

 CHAPTER 3

Flask extension

Flask extension provides means for easy integration with PostgreSQL-Audit, Flask-Login and Flask-SQLAlchemy. It
provides all the goodies that SQLAlchemy integration provides along with:
• By default the Flask extensions tries to get the current user from Flask-Login and assigns the id of this object
as the actor_id of all activities in given transaction. It also assigns the current user ip address to all present
activities.
• Easy overriding of current activity values using activity_values context manager

from postgresql_audit.flask import versioning_manager

from my_app.extensions import db

versioning_manager.init(db.Model)

class Article(db.Model):
__tablename__ = 'article'
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String)

article = Article(name='Some article')

db.session.add(article)
db.session.commit()

3.1 Overriding activity values

In some situations you may want to override the current activity values. One scenario is where you want to track
changes to associated objects and mark those changes with target_id property of Activity model.

9
PostgreSQL-Audit Documentation, Release 0.10.0

Consider for example the following model structure with Articles and Tags. Let’s say we want to show the changelog
of an article that contains all changes to this article and its tags.
from postgresql_audit.flask import versioning_manager

from my_app.extensions import db

versioning_manager.init(db.Model)

class Article(db.Model):
__tablename__ = 'article'
__versioned__ = {}
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String)

class Tag(db.Model):
__tablename__ = 'tag'
__versioned__ = {}
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String)
article_id = db.Column(
db.Integer,
db.ForeignKey(Article.id, ondelete='CASCADE')
)
article = db.relationship(Article, backref='tags')

When tracking the changes to article we don’t need any changes.

article = Article(name='Some article')
db.session.add(article)
db.session.commit()

When adding tags we need to make the generated activities use the article id as the target_id, so that we can track them
later on.
from postgresql_audit.flask import activity_values

with activity_values(target_id=str(article.id)):
article.tags = [Tag(name='Some tag')]
db.session.commit()

Now we can find all activities for given article with the following query.
Activity = versioning_manager.activity_cls

activities = Activity.query.filter(
db.or_(
db.and_(
Activity.target_id == str(article.id),
Activity.target_table_name == 'article'
),
db.and_(
db.or_(
Activity.row_data['id'] == article.id,
(continues on next page)

10 Chapter 3. Flask extension

 PostgreSQL-Audit Documentation, Release 0.10.0

(continued from previous page)

Activity.changed_fields['id'] == article.id
),
Activity.table_name == 'article'
)
)
).order_by(Activity.issued_at)

3.2 Recording IP address behind proxy

By default PostgreSQL-Audit stores the client address as found in the request and does not attempt to make assump-
tions on server proxy configuration. Thus, in case the flask app runs after an http server (e.g nginx), and depending on
configuration, flask may receive no IP. To overcome this, it is advised to follow flask documentation on proxy setups.

3.2. Recording IP address behind proxy 11

PostgreSQL-Audit Documentation, Release 0.10.0

12 Chapter 3. Flask extension

 CHAPTER 4

Migrations

Usually your schema changes over time. The schema of PostgreSQL-Audit is very flexible, since it stores the data in
JSONB columns. Your schema can change without the need of changing the version history JSONB data columns.
However in case you want to show the version history on the application side you may want to reflect the changes
you make to your schema to old_data and changed_data columns of activity table. The other solution is to make your
application code aware of all the schema changes that have happened over time. This can get a bit tedious if your
schema is quickly evolving.

4.1 Changing column name

postgresql_audit.migrations.change_column_name(conn, table, old_column_name,

new_column_name, schema=None)
Changes given activity jsonb data column key. This function is useful when you want to reflect column name
changes to activity table.

from alembic import op

from postgresql_audit import change_column_name

def upgrade():
op.alter_column(
'my_table',
'my_column',
new_column_name='some_column'
)

change_column_name(op, 'my_table', 'my_column', 'some_column')

Parameters
• conn – An object that is able to execute SQL (either SQLAlchemy Connection, Engine or
Alembic Operations object)

13
PostgreSQL-Audit Documentation, Release 0.10.0

• table – The table to run the column name changes against

• old_column_name – Name of the column to change
• new_column_name – New colum name
• schema – Optional name of schema to use.

4.2 Alter column type

postgresql_audit.migrations.alter_column(conn, table, column_name, func, schema=None)

Run given callable against given table and given column in activity table jsonb data columns. This function is
useful when you want to reflect type changes in your schema to activity table.
In the following example we change the data type of User’s age column from string to integer.

from alembic import op

from postgresql_audit import alter_column

def upgrade():
op.alter_column(
'user',
'age',
type_=sa.Integer
)

alter_column(
op,
'user',
'age',
lambda value, activity_table: sa.cast(value, sa.Integer)
)

Parameters
• conn – An object that is able to execute SQL (either SQLAlchemy Connection, Engine or
Alembic Operations object)
• table – The table to run the column name changes against
• column_name – Name of the column to run callable against
• func – A callable to run against specific column in activity table jsonb data columns. The
callable should take two parameters the jsonb value corresponding to given column_name
and activity table object.
• schema – Optional name of schema to use.

4.3 Removing columns

postgresql_audit.migrations.remove_column(conn, table, column_name, schema=None)

Removes given activity jsonb data column key. This function is useful when you are doing schema changes that
require removing a column.
Let’s say you’ve been using PostgreSQL-Audit for a while for a table called article. Now you want to remove
one audited column called ‘created_at’ from this table.

14 Chapter 4. Migrations
 PostgreSQL-Audit Documentation, Release 0.10.0

from alembic import op

from postgresql_audit import remove_column

def upgrade():
op.remove_column('article', 'created_at')
remove_column(op, 'article', 'created_at')

Parameters
• conn – An object that is able to execute SQL (either SQLAlchemy Connection, Engine or
Alembic Operations object)
• table – The table to remove the column from
• column_name – Name of the column to remove
• schema – Optional name of schema to use.

4.4 Adding columns

postgresql_audit.migrations.add_column(conn, table, column_name, default_value=None,

schema=None)
Adds given column to activity table jsonb data columns.
In the following example we reflect the changes made to our schema to activity table.

import sqlalchemy as sa
from alembic import op
from postgresql_audit import add_column

def upgrade():
op.add_column('article', sa.Column('created_at', sa.DateTime()))
add_column(op, 'article', 'created_at')

Parameters
• conn – An object that is able to execute SQL (either SQLAlchemy Connection, Engine or
Alembic Operations object)
• table – The table to remove the column from
• column_name – Name of the column to add
• default_value – The default value of the column
• schema – Optional name of schema to use.

class postgresql_audit.migrations.jsonb_change_key_name(*clauses, **kwargs)

Provides jsonb_change_key_name as a SQLAlchemy FunctionElement.

import sqlalchemy as sa
from postgresql_audit import jsonb_change_key_name

data = {'key1': 1, 'key3': 4}

(continues on next page)

4.4. Adding columns 15

PostgreSQL-Audit Documentation, Release 0.10.0

(continued from previous page)

query = sa.select([jsonb_merge(data, 'key1', 'key2')])
session.execute(query).scalar() # {'key2': 1, 'key3': 4}

4.5 Rename table

postgresql_audit.migrations.rename_table(conn, old_table_name, new_table_name,

schema=None)
Renames given table in activity table. You should remember to call this function whenever you rename a
versioned table.

from alembic import op

from postgresql_audit import rename_table

def upgrade():
op.rename_table('article', 'article_v2')
rename_table(op, 'article', 'article_v2')

Parameters
• conn – An object that is able to execute SQL (either SQLAlchemy Connection, Engine or
Alembic Operations object)
• old_table_name – The name of table to rename
• new_table_name – New name of the renamed table
• schema – Optional name of schema to use.

16 Chapter 4. Migrations
 Python Module Index

p
postgresql_audit.migrations, 13

17
PostgreSQL-Audit Documentation, Release 0.10.0

18 Python Module Index

 Index

A
add_column() (in module postgresql_audit.migrations),
15
alter_column() (in module postgresql_audit.migrations),
14

C
change_column_name() (in module post-
gresql_audit.migrations), 13

J
jsonb_change_key_name (class in post-
gresql_audit.migrations), 15

P
postgresql_audit.migrations (module), 13

R
remove_column() (in module post-
gresql_audit.migrations), 14
rename_table() (in module postgresql_audit.migrations),
16

19