Data delivery approaches

The intention of this document is to demonstrate a pagination strategy for data delivering to
the new UI.

Prerequisites

Currently the existing application usually delivers all needed data to the client side.
Sometimes data can be hundreds or thousands of rows which make delivering to the client to
slow down. Having in mind that the new UI is going to be a cloud hybrid based such
approach is not recommended. In similar cases the best practices say data should be
paginated.

Pagination

Data which has to be delivered is contained in a specific table or view. In most cases it is
going to be a view. According to the needs this data can be filtered before its pagination. In
both cases - after filtering or without filtering - you can look at it such as a dataset. This data
set should be sorted by required criteria, so your algorithm recognizes the records for the
demanded page.

In common case next points should be considered:

● If a filter is there, the columns which participated in the filter are recommended to
participate in an index, especially the most selective column to be the first column in
the key.
● Pagination also is tied to a sort operation, so if the sort columns are indexed it will
bring cheaper execution.
● Pages should contain a small amount of rows in order to be extracted faster, to
generate small traffic and to be delivered in the fastest way possible.

There are two methods for pagination described below:

Pagination method Pros Cons

Sequentially It is going to be faster, It requires a unique key for

because it uses a stop key the paginated dataset which
which will prevent unneeded should be used for start
sorts. position for the records of
the next page.

Specific page It is flexible. It not requires More expensive cost. It

additional information where should sort the entire
you should start from, dataset to find the desired
page.
Materialized view

When a given view works slowly and its result reduces data, it makes sense to consider its
materialization.

For example, in the test database there are several slow views, one of the reasons is the
amount of data. There is a view which returns 1405 rows for about 30 seconds. It works over
40 million records aggregating them. After creating a materialized view and using it, the time
is under 500 milliseconds.

Materialized views should be refreshed periodically, so the question here is how often it
should be refreshed.

Conclusion

Data pagination combined with materialized views if it is needed will reduce extraction time to
hundreds even to tens of milliseconds. Above recommendations are overall, so for each
specific case you have to consider the approach which you are going to use. Considering
existing or new indexes can bring additional benefits.

Examples

Specific page

WITH SORTED AS (
SELECT <list of columns>
,ROW_NUMBER() OVER(ORDER BY <list of sort columns>) AS
ROW_SEQUENCE
FROM <view/table>
WHERE if it is needed
)
SELECT *
FROM SORTED
WHERE ROW_SEQUENCE BETWEEN <the sequence number of first row of the page>
AND <the sequence number of last row of the page>

Next page

WITH SORTED AS (
SELECT <list of columns>
FROM <view/table>
WHERE <the filter conditions if it is needed>
AND <a unique key column> > <the key value of the last row of the
previous page, for the first page it is zero>
ORDER BY <list of sort columns>
)
SELECT *
FROM SORTED
WHERE ROWNUM <= <the row count of page>
Previous page

WITH SORTED AS (
SELECT <list of columns>
FROM <view/table>
WHERE <the filter conditions if it is needed>
AND <a unique key column> < <the key value of the first row of the
previous page, for the latest page it is should a big enough>
ORDER BY <list of sort columns descending>
)
SELECT *
FROM SORTED
WHERE ROWNUM <= <the row count of page>