Scribd Logo
Upload

Welcome to Scribd!

  • 14 Perf Counters

    Uploaded by

    angelcarballobasto
    2 views35 pages
    This document discusses Firebird performance counters and metrics for analyzing bottlenecks. It describes page-level metrics like reads, fetches, writes, and marks. It also covers record-level metrics like sequential reads, indexed reads, inserts, updates, deletes, backouts, purges, and expunges. Plans are discussed to expand metrics for page writes, add index and temporary file counters, and capture time-based statistics. Integrating the collection and reporting of these metrics with tools like StatsD, Graphite, Nagios and Zabbix is also mentioned.

    Original Description:

    Copyright

    © © All Rights Reserved

    Available Formats

    PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    This document discusses Firebird performance counters and metrics for analyzing bottlenecks. It describes page-level metrics like reads, fetches, writes, and marks. It also covers record-level metrics like sequential reads, indexed reads, inserts, updates, deletes, backouts, purges, and expunges. Plans are discussed to expand metrics for page writes, add index and temporary file counters, and capture time-based statistics. Integrating the collection and reporting of these metrics with tools like StatsD, Graphite, Nagios and Zabbix is also mentioned.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    2 views35 pages

    14 Perf Counters

    Uploaded by

    angelcarballobasto
    This document discusses Firebird performance counters and metrics for analyzing bottlenecks. It describes page-level metrics like reads, fetches, writes, and marks. It also covers record-level metrics like sequential reads, indexed reads, inserts, updates, deletes, backouts, purges, and expunges. Plans are discussed to expand metrics for page writes, add index and temporary file counters, and capture time-based statistics. Integrating the collection and reporting of these metrics with tools like StatsD, Graphite, Nagios and Zabbix is also mentioned.

    Copyright:

    © All Rights Reserved
    Download as PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 35

    Firebird performance counters

    in details

    Dmitry Yemanov
    mailto:dimitr@firebirdsql.org

    Firebird Project
    http://www.firebirdsql.org/
    Thank you

    FIREBIRD INTERNATIONAL CONFERENCE '2014 2


    Analysing bottlenecks
    Disk
    Database
    Temporary files

    FIREBIRD INTERNATIONAL CONFERENCE '2014 3


    Analysing bottlenecks
    Disk
    Database
    Temporary files

    Memory
    Static (cache)
    Dynamic (pools)
    Shared

    FIREBIRD INTERNATIONAL CONFERENCE '2014 4


    Analysing bottlenecks
    CPU, waits, etc
    Real execution time
    User/kernel time
    Was time spent for work or waiting?
    What operations were performed?

    FIREBIRD INTERNATIONAL CONFERENCE '2014 5


    Legacy performance counters
    Page level
    Fetches, reads, marks, writes
    Shared in SS, per connection in CS
    Available via API

    FIREBIRD INTERNATIONAL CONFERENCE '2014 6


    Legacy performance counters
    Page level
    Fetches, reads, marks, writes
    Shared in SS, per connection in CS
    Available via API

    Record level
    Selected (seq/idx), inserted, updated, deleted etc
    Per connection and per table
    Available via API

    FIREBIRD INTERNATIONAL CONFERENCE '2014 7


    Improvements in Firebird 2.x
    Multi-level aggregated counters
    Database (SS only)
    Connection
    Transaction
    Statement
    Nested PSQL call (procedure / trigger)

    FIREBIRD INTERNATIONAL CONFERENCE '2014 8


    Improvements in Firebird 2.x
    Multi-level aggregated counters

    Database

    Connection 1 Connection 2

    Transaction 1 Transaction 2

    Statement 1 Statement 1

    FIREBIRD INTERNATIONAL CONFERENCE '2014 9


    Improvements in Firebird 2.x
    New interfaces
    Monitoring tables
    Trace/audit

    FIREBIRD INTERNATIONAL CONFERENCE '2014 10


    Page level counters
    Page reads
    From disk to the page cache (physical reads)
    Usually means a cache miss

    FIREBIRD INTERNATIONAL CONFERENCE '2014 11


    Page level counters
    Page reads
    From disk to the page cache (physical reads)
    Usually means a cache miss

    Page fetches
    Access page in cache (logical reads)
    Includes both cache hits and cache misses
    Corresponds to a shared page lock / latch

    FIREBIRD INTERNATIONAL CONFERENCE '2014 12


    Page level counters
    Page reads
    From disk to the page cache (physical reads)
    Usually means a cache miss

    Page fetches
    Access page in cache (logical reads)
    Includes both cache hits and cache misses
    Corresponds to a shared page lock / latch
    Cache hit ratio = 1 - reads / fetches ???

    FIREBIRD INTERNATIONAL CONFERENCE '2014 13


    Page level counters
    Page writes
    From the page cache to disk (physical writes)
    At transaction commit/rollback
    Cache is full of dirty pages
    Asynchronous notification is received (CS/SC)
    Immediately after modification

    FIREBIRD INTERNATIONAL CONFERENCE '2014 14


    Page level counters
    Page writes
    From the page cache to disk (physical writes)
    At transaction commit/rollback
    Cache is full of dirty pages
    Asynchronous notification is received (CS/SC)
    Immediately after modification

    Page marks
    Access page in cache (logical writes)
    Corresponds to an exclusive page lock / latch
    FIREBIRD INTERNATIONAL CONFERENCE '2014 15
    Record level counters
    Sequential record reads
    Records retrieved through a full table scan
    Includes sweep

    Indexed record reads


    Records retrieved positionally
    Includes bitmap index scans, index navigational
    walks, DBKEY based retrievals

    FIREBIRD INTERNATIONAL CONFERENCE '2014 16


    Record level counters
    Record inserts, updates, deletes
    Pretty obvious, huh?

    FIREBIRD INTERNATIONAL CONFERENCE '2014 17


    Record level counters
    Record inserts, updates, deletes
    Pretty obvious, huh?

    Record backouts
    Latest (uncommitted) version removed
    Happens after savepoint rollback, explicit or implicit
    May happen after transaction rollback

    FIREBIRD INTERNATIONAL CONFERENCE '2014 18


    Record level counters
    Record purges
    Old versions removed while keeping the primary
    version in place
    Outdated versions found while chasing

    FIREBIRD INTERNATIONAL CONFERENCE '2014 19


    Record level counters
    Record purges
    Old versions removed while keeping the primary
    version in place
    Outdated versions found while chasing

    Record expunges
    Whole version chains removed, along with the
    primary version
    Record is deleted and nobody is interested

    FIREBIRD INTERNATIONAL CONFERENCE '2014 20


    New record level counters in v3.x
    Record repeated reads
    Record is retrieved multiple times
    BEFORE triggers
    Sort-based updates/deletes

    FIREBIRD INTERNATIONAL CONFERENCE '2014 21


    New record level counters in v3.x
    Record repeated reads
    Record is retrieved multiple times
    BEFORE triggers
    Sort-based updates/deletes

    Record locks
    Record is selected usng WITH LOCK clause

    FIREBIRD INTERNATIONAL CONFERENCE '2014 22


    New record level counters in v3.x
    Record waits
    Attempts to update/delete/lock record
    owned by a concurrent active transaction
    Transaction is in the WAIT mode

    FIREBIRD INTERNATIONAL CONFERENCE '2014 23


    New record level counters in v3.x
    Record waits
    Attempts to update/delete/lock record
    owned by a concurrent active transaction
    Transaction is in the WAIT mode

    Record conflicts
    Unsuccessful attempts to update/delete/lock record
    owned by a concurrent active transaction
    UPDATE CONFLICT is reported

    FIREBIRD INTERNATIONAL CONFERENCE '2014 24


    New record level counters in v3.x
    Backversion reads
    Versions chased while finding a visible one
    Means old snapshots

    FIREBIRD INTERNATIONAL CONFERENCE '2014 25


    New record level counters in v3.x
    Backversion reads
    Versions chased while finding a visible one
    Means old snapshots

    Fragment reads
    Fragmented records
    Means extra page fetches/reads

    FIREBIRD INTERNATIONAL CONFERENCE '2014 26


    New record level counters in v3.x
    MON$TABLE_STATS MON$RECORD_STATS

    MON$STATS_ID
    MON$STATS_ID
    MON$STATS_GROUP
    MON$STATS_GROUP
    MON$RECORD_SEQ_READS
    MON$TABLE_NAME
    MON$RECORD_IDX_READS
    MON$RECORD_STATS_ID
    etc

    FIREBIRD INTERNATIONAL CONFERENCE '2014 27


    TODO: page level counters
    Page writes (expanded)
    Regular writes (immediate / commit / rollback)
    Overflow writes
    Asynchronous writes

    FIREBIRD INTERNATIONAL CONFERENCE '2014 28


    TODO: page level counters
    Page writes (expanded)
    Regular writes (immediate / commit / rollback)
    Overflow writes
    Asynchronous writes

    Page waits
    How many times page requests waited
    Shows contention inside the page cache

    FIREBIRD INTERNATIONAL CONFERENCE '2014 29


    TODO: index counters
    Possible metrics
    Index scans
    Node inserts / deletes
    Bucket splits / merges
    Keys scanned / compared while searching

    Reported per index


    MON$INDEX_STATS

    FIREBIRD INTERNATIONAL CONFERENCE '2014 30


    TODO: temporary space counters
    Operation metrics
    Reads / writes resolved through the cache
    Reads / writes redirected to temp files

    I/O amount metrics


    Bytes read / written through the cache
    Bytes read / written from/to temp files

    FIREBIRD INTERNATIONAL CONFERENCE '2014 31


    TODO: time statistics
    Elapsed time
    Total inside the engine
    Spent in the user space
    Spent in the system / kernel space

    FIREBIRD INTERNATIONAL CONFERENCE '2014 32


    TODO: time statistics
    Elapsed time
    Total inside the engine
    Spent in the user space
    Spent in the system / kernel space

    Wait time
    Database (and maybe temp files) I/O
    Page cache waits
    Transaction waits

    FIREBIRD INTERNATIONAL CONFERENCE '2014 33


    TODO: integration
    Collecting the statistics
    StatsD, CollectD

    Reporting / analyzing
    Graphite / Graphene

    Other tools
    Nagios, Cacti, Zabbix

    FIREBIRD INTERNATIONAL CONFERENCE '2014 34


    Questions?
    mailto:dimitr@firebirdsql.org

    FIREBIRD INTERNATIONAL CONFERENCE '2014 35

    You might also like