Stored Routines Tips, Tricks and Solutions

Alex Nozdrin
Software Developer, Server team Alexander.Nozdrin@mysql.com

George Trujillo
Senior Instructor George.Trujillo@mysql.com
Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 1

Agenda
• Stored Objects Overview
– – – – – – – – – Stored Procedures vs Stored Functions Features of triggers User and stored routine variables SQL SECURITY INVOKER / DEFINER When and why?

5 min

• Technical Details
CREATE-time context Storing stored objects Current limitations Non-documented features

10 min

• Dynamic SQL
– SQL Injections – Dealing with PS limitations

10 min 25 min
The World’s Most Popular Open Source Database 2

• Real-life Examples
Copyright 2007 MySQL AB

Stored Objects Overview
• Stored Objects Overview
– – – – – Stored Procedures vs Stored Functions Features of triggers User and stored routine variables SQL SECURITY INVOKER / DEFINER When and why?

• • • •

Technical Details Dynamic SQL DBA Examples Q&A

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

3

Stored Objects Overview
Definition

• Stored Objects
– Stored Routines
• Stored Procedures • Stored Functions

– Triggers – Events*

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

4

Stored Objects Overview
Stored Procedures vs Stored Functions
Stored Procedures Return or Generate
[MySQL extension]

Stored Functions a single value

result set(s)

> CALL sp1(); +---------------+ | Hello, world! | +---------------+ | Hello, world! | +---------------+ 1 row in set (0.01 sec) +----------------------+ | Again: Hello, world! | +----------------------+ | Again: Hello, world! | +----------------------+ 1 row in set (0.02 sec)

> SELECT sf1(); +------+ | sf1() | +------+ | 1 | +------+ 1 row in set (0.00 sec)

How to invoke

CALL
CALL sp1('Hello, world')

within an expression
SELECT sf(c1), c2 FROM t1

Parameters Prepared statements

IN; OUT; INOUT yes

IN-only no

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

5

Stored Objects Overview
• • • • • • Triggers Activation Time: BEFORE, AFTER Events: INSERT, UPDATE, DELETE Special variables: NEW, OLD FOR EACH ROW only One trigger / table for {activation time, event} No result set
– No CALL, which leads to generating result set – Use OUT-parameters to pass data from stored procedures

• Can not begin or end transactions
– Explicitly or implicitly – No DDL

• No temporary tables or views • No prepared statements
Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 6

Stored Objects Overview
User and Stored Routine Variables
Scope Declaration Type system Examples

User session NO Weakly-typed
SET @a = 1; SET @b = '2'; SELECT @a + @b; --> 3 SELECT CONCAT(@a, @b); --> 12

Stored routine execution Mandatory Strongly-typed
DECLARE a INT DEFAULT 1; DECLARE b CHAR(10) DEFAULT '2'; DECLARE c INT; SET c = a + b; --> 3 SET c = CONCAT(a, b); --> 12, warning

• User variable – an item in expression • Store routine variable – a column in a temporary table

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

7

Stored Objects Overview
SQL SECURITY: Definition

• sql security DEFINER
– SUID object – Executes under the “definer” – the user who created the object

• sql security INVOKER • DEFINER clause
DEFINER = username@hostname

– Requires SUPER privilege – Specifies the user which privileges will be used when executing the object

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

8

Stored Objects Overview
SQL SECURITY: Example

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

9

Stored Objects Overview
Stored Routines: Why?

1. Security
• •

No need to grant privileges on the tables Code “security” – application developer can not easily modify SQL code Cached on the server Can be designed to reduce network traffic No “copy & paste” Separation of interface and implementation Implementation can change No need to change clients during upgrade

2. Performance
• •

3. Consistency
• •

4. Maintenance
• •

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

10

Stored Objects Overview
Triggers: Why?

1. Log 2. Audit 3. Change data of an operation (INSERT, UPDATE) 4. Prevent some operations
CREATE PROCEDURE throw_error() ROLLBACK; -- Any stmt causing COMMIT or ROLLBACK CREATE TABLE t1(c INT) ENGINE=InnoDB; CREATE TRIGGER t1_bi BEFORE INSERT ON t1 FOR EACH ROW BEGIN IF NEW.c = 2 THEN CALL throw_error(); END IF;
Copyright 2007 MySQL AB END The World’s Most Popular Open Source Database 11

Stored Objects Overview
What is BAD?

• Processing close to data – scale-up • MySQL's way – scale-out • Current limitations

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

12

Technical Details
• Stored Objects • Technical Details
– – – – – – CREATE-time context Storing stored objects SHOW CREATE and INFORMATION_SCHEMA Current limitations SHOW CODE SHOW PROFILE

• Dynamic SQL • Real-life Examples • Q&A

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

13

Technical Details
CREATE-time context: Execution Overview

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

14

Technical Details
CREATE-time context

• CREATE-time Context – a list of attributes that are fixed at CREATE time
– sql_mode
• The “main switch” of MySQL behavior.

– character_set_client
• Defines character set of the CREATE-statement.

– collation_connection
• Defines character set / collation rules for the execution.

– collation_database
• Owner database collation

– current database – time_zone (events)

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

15

Technical Details
• • • • Storing stored objects Stored routines – mysql.proc table Triggers – TRG, TRN files mysql – a system database Stored objects are stored in the original form
– Original character set – Character set introducers untouched – ...
CREATE PROCEDURE p1() SELECT 'text1', _cp1257 'text2';

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

16

Technical Details
SHOW CREATE vs INFORMATION_SCHEMA • SHOW CREATE
– Returns original query in the original character set – Query can be “binary” (consists of several charsets)
CREATE PROCEDURE p1() SELECT 'text1', _cp1257 'text2';

• INFORMATION_SCHEMA
– Returns “normalized” query in UTF8 – “Normalized” == no character set introducers
CREATE PROCEDURE p1() SELECT 'text1-in-utf8', 'text2-in-utf8';

• Confused?
– Use SHOW CREATE for
• Dump / backup – mysqldump uses SHOW CREATE
Copyright 2007 MySQL AB

– Use INFORMATION_SCHEMA for

The World’s Most Popular Open Source Database

17

Technical Details
Limitations

• Common • Prepared Statements • Stored Routines

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

18

Technical Details
Limitations: Common

• Meta-data is fixed at the compilation time
– i.e. Don't change meta-data on the fly – i.e. NO ALTER

• BAD Examples:
– Prepare stmt & ALTER meta-data – Execute a SR & ALTER meta-data – Create a trigger & ALTER meta-data

• Why?
– The server does not track dependencies on the objects in cache (compiled objects)

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

19

Technical Details
Limitations: Prepared Statements

• • • •

No support for OUT-parameters No query plan cache No arrays Placeholders are not supported everywhere
– Table names – Column names – Dynamic search condition

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

20

Technical Details
Limitations: Stored Routines

• Cache
– Per connection – No limit – Almost any DDL flushes all caches

• • • •

No COLLATE for stored routine variable No SIGNAL / RESIGNAL SR-variable shadows column Prepared statements
– PREPARE uses only user variables (no SR vars) – EXECUTE uses only user variables (no SR vars, no constants)

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

21

Technical Details
Non-documented features
• SHOW PROCEDURE | FUNCTION CODE • SHOW PROFILE (5.0, Community patch)
CREATE PROCEDURE p1(x INT) BEGIN IF x < 0 THEN INSERT INTO TError VALUES (CONCAT('Negative value: ', x)); ELSEIF x > 100 THEN INSERT INTO TError VALUES (CONCAT('Too large value: ', x)); ELSE INSERT INTO Tvalues VALUES (x); END IF; END

SHOW PROCEDURE CODE
| Pos | Instruction |

p1;

+-----+---------------------------------------------------+ | | (CONCAT('..." | The World’s Most Popular Open Source Database
22

0 | jump_if_not 3(7) (x@0 < 0) "INSERT INTO TError VALUES

| 1 MySQL AB Copyright 2007| stmt 5

Dynamic SQL
• Stored Objects • Technical Details • Dynamic SQL
– SQL Injections – Dealing with PS limitations
• Dynamic table / column names • Dynamic WHERE • Arrays

• Real-life Examples • Q&A

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

23

Dynamic SQL
Kinds of Dynamic SQL

• Dynamic queries
– Pros
• Good for prototyping • Universal approach • The only way to do some things

• Prepared statements
– Pros
• A query is parsed only once • No SQL injections

– Cons
• Extra job if query is executed only once • Not all SQL statements supported • Arrays are not supported • Placeholders can not be used everywhere

– Cons
• Cluttering the client code, copy & paste • Query has to be parsed each time • SQL Injections

– HINT: close PS after use

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

24

Dynamic SQL
SQL Injections (1)

• Dynamic query
my $name = ...; # read from HTML form my $s = “SELECT id FROM t1 WHERE name = '$name'”; mysql_query($mysql, $s);

– Arbitrary SQL query can be executed
$name: “'; DELETE FROM t1 WHERE 'a' = 'a” $s: SELECT id FROM t1 WHERE name = ''; DELETE FROM t1 WHERE 'a' = 'a';

– SQL query can be compromised
Copyright 2007 MySQL AB

$name: “' OR 'a' = 'a”

The World’s Most Popular Open Source Database

25

Dynamic SQL
SQL Injections (2)

• Prepared statements
– Stored procedures – just for convenience here
CREATE PROCEDURE check_user(name VARCHAR(255)) PREPARE stmt FROM 'SELECT id FROM t1 WHERE name = ?'; EXECUTE stmt USING name; CALL check_user("valid_user_name"); --> id --> 1 CALL check_user("'; DELETE * FROM t1 WHERE 'a' = 'a"); --> Nothing CALL check_user("' OR 'a' = 'a");
Copyright 2007 MySQL AB

--> Nothing

The World’s Most Popular Open Source Database

26

Dynamic SQL
SQL Injections (3)

• C API (other instruments):
– mysql_real_escape_string()
• • • • Quotes CR / LF NUL Slash / backslash

– mysql_real_query()
• One query at a time

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

27

Dynamic SQL
Dynamic Table / Column names

• Avoid that
– Tables are types – Columns are attributes

• Workaround
– Use prepared statement to pass parameters – Check column or table names

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

28

Dynamic SQL
Dynamic table name: Example
create function check_table_name(table_name char(10)) returns bool return strcmp(lcase(table_name), 't1') = 0 or strcmp(lcase(table_name), 't2') = 0; create function check_table_name(table_name char(10)) returns bool return instr(table_name, "'") = 0 and ... instr(table_name, '(') = 0; create procedure p1(table_name char(10), value char(255)) l1: begin if check_table_name(table_name) != 1 then leave l1; end if; SET @s = CONCAT('SELECT name FROM ', table_name, ' WHERE World’s Most Popular Open Source Database id = ?'); The

Copyright 2007 MySQL AB

29

Dynamic SQL
Dynamic WHERE

• SELECT * FROM table_name WHERE ?
– Must not be used in a general case – Can be replaced by a static SQL in common cases

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

30

Dynamic SQL
Dynamic WHERE: Example
CREATE TABLE t1(c1 INT, c2 INT, c3 TEXT); INSERT INTO t1 VALUES (1, 1, 'hello'), (1, 2, 'world'), (2, 1, 'foo'); CREATE PROCEDURE p1(v1 INT, v2 INT, v3 VARCHAR(255)) BEGIN SELECT c1, c2, c3 FROM t1 WHERE (v1 IS NULL OR c1 = v1) AND (v2 IS NULL OR c2 = v2) AND (v3 IS NULL OR STRCMP(c3, v3) = 0); END
Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 31

Dynamic SQL
Arrays

• The problem
SELECT * FROM table WHERE id IN (?)

• Solution
– CSV-list: “123; 456.789;10;hello world;
• • • • • • Proper escaping (delimiters) Extra spaces Malicious (illegal) input Empty items Slow Legacy or imported data

another list item”

– Fixed-width list: “ red blue green whiteorange” (6 symbols)
• More memory • Not always possible • Usually faster than CSV-list

– Temporary table
• Probably, the best
Copyright 2007 MySQL AB The World’s Most Popular Open Source Database 32

Dynamic SQL
MySQL Forge

• http://forge.mysql.com
– Split a Delimited String in SQL – MySQL General Purpose Stored Routines Library
• • • • Arrays and basic data structures "FOR EACH" loops Named parameters Syntax helpers

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

33

Real-life Examples
Senior Instructor George.Trujillo@mysql.com

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

34

Questions ?

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

35

Thanks!

Alexander.Nozdrin@mysql.com George.Trujillo@mysql.com

Copyright 2007 MySQL AB

The World’s Most Popular Open Source Database

36