The curse and blessings of dynamic SQL

Page 1 of 36

The Curse and Blessings of Dynamic SQL
An SQL text by Erland Sommarskog, SQL Server MVP. Latest revision: 2011-06-23. An earlier version of this article is also available in German. Translations provided by SQL Server MVP Frank Kalis.

If you follow the various newsgroups on Microsoft SQL Server, you often see people asking why they can't do:
SELECT * FROM @tablename SELECT @colname FROM tbl SELECT * FROM tbl WHERE x IN (@list)

For all three examples you can expect someone to answer Use dynamic SQL and give a quick example on how to do it. Unfortunately, for all three examples above, dynamic SQL is a poor solution. On the other hand, there are situations where dynamic SQL is the best or only way to go. In this article I will discuss the use of dynamic SQL in stored procedures and to a minor extent from client languages. To set the scene, I start with a very quick overview on application architecture for data access. I then proceed to describe the feature dynamic SQL as such, with a quick introduction followed by the gory syntax details. Next, I continue with a discussion on SQL injection, a security issue that it is essential to have good understanding of when you work with dynamic SQL. This is followed by a section where I discuss why we use stored procedures, and how that is affected by the use of dynamic SQL. I carry on with a section on good practices and tips for writing dynamic SQL. I conclude by reviewing a number of situations where you could use dynamic SQL and whether it is a good or bad idea to do it. The article covers all versions of SQL Server from SQL 6.5 to SQL 2008, with emphasis on SQL 2000 and later versions. Contents: Accessing Data from an Application Introducing Dynamic SQL
A First Encounter sp_executesql EXEC() SQL Injection – a Serious Security Issue Dynamic SQL and Stored Procedures The Permission System Caching Query Plans Reducing Network Traffic Encapsulating Logic Keeping Track of what Is Used Ease of Writing SQL Code Addressing Bugs and Problems

Good Coding Practices and Tips for Dynamic SQL
Use Debug Prints! Nested Strings Spacing and Formatting Dealing with Dynamic Table and Column Names Quotename, Nested Strings and Quotestring QUOTED_IDENTIFIER sp_executesql and Long SQL Strings in SQL 2000 Dynamic SQL in User-Defined Functions Cursors and Dynamic SQL


The curse and blessings of dynamic SQL

Page 2 of 36

EXEC() at Linked Server

Common Cases when to (Not) Use Dynamic SQL
SELECT * FROM @tablename SELECT * FROM sales + @yymm UPDATE tbl SET @colname = @value WHERE keycol = @keyval SELECT col AS @myname SELECT * FROM @dbname + '..tbl' SELECT * FROM tbl WHERE col IN (@list) SELECT * FROM tbl WHERE @condition Dynamic Search Conditions Dynamic Crosstab SELECT * FROM tbl ORDER BY @col SELECT TOP @n FROM tbl CREATE TABLE @tbl CREATE TABLE with Unknown Columns Linked Servers OPENQUERY Dynamic Column Widths Dynamic SQL and Maintenance Tasks

Acknowledgements and Feedback Revision History Note: many of the code samples in this text works against the pubs and Northwind databases that ship with SQL 2000 and SQL 7, but not with SQL 2005 and later. You can download these databases from Microsoft's web site.

Accessing Data from an Application
Before I describe dynamic SQL, I like to briefly discuss the various ways you can access data from an application to give an overview of what I'll be talking about in this article. (Note: all through this text I will refer to client as anything that accesses SQL Server from the outside. In the overall application architecture that may in fact be a middle tier or a business layer, but as that is of little interest to this article, I use client in the sake of brevity.) There are two main roads to go, and then there are forks and sub-forks. 1. Send SQL statements from the client to SQL Server. a. Rely on SQL generated by the client API, using options like CommandType.TableDirect and methods like .Update. LINQ falls into this group as well. b. Compose the SQL strings in the client code. i. Build the entire SQL string with parameter values expanded. ii. Use parameterised queries. 2. Perform access through stored procedures. a. Stored procedures in T-SQL i. Use static SQL only. ii. Use dynamic SQL together with static SQL. b. Stored procedures in a CLR language such as C# or VB .Net. (SQL 2005 and later.) Fork 1-a may be good for simple tasks, but you are likely to find that you outgrow it as the complexity of your application increases. In any case, this approach falls entirely outside the scope of this article. Many applications are built along the principles of fork 1-b, and as long as you take the sub-fork 1-b-ii, it does not have to be bad. (Why 1-b-i is bad, is something I will come back to. Here I will just drop two keywords: SQL Injection and Query-Plan Reuse.) Nonetheless, in many shops the mandate is that you should use stored procedures. When you use stored procedures with only static SQL, users do not need


The curse and blessings of dynamic SQL

Page 3 of 36

direct permissions to access the tables, only permissions to execute the stored procedures, and thus you can use the stored procedure to control what users may and may not do. The main focus for this text is sub-fork 2-a-ii. When used appropriately, dynamic SQL in stored procedures can be a powerful addition to static SQL. But some of the questions on the newsgroups leads to dynamic SQL in stored procedures that are so meaningless, that these people would be better off with fork 1-b instead. Finally, fork 2-b, stored procedures in the CLR, is in many regards very similar to fork 1-b, since all data access from CLR procedures is through generated SQL strings, parameterised or unparameterised. If you have settled on SQL procedures for your application, there is little point in rewriting them into the CLR. However, CLR code can be a valuable supplement for tasks that are difficult to perform in T-SQL, but you yet want to perform server-side.

Introducing Dynamic SQL
In this chapter I will first look at some quick examples of dynamic SQL and point out some very important implications of using dynamic SQL. I will then describe sp_executesql and EXEC() in detail, the two commands you can use to invoke dynamic SQL from T-SQL.

A First Encounter
Understanding dynamic SQL itself is not difficult. Au contraire, it's rather too easy to use. Understanding the fine details, though, takes a little longer time. If you start out using dynamic SQL casually, you are bound to face accidents when things do not work as you have anticipated. One of the problems listed in the introduction was how to write a stored procedure that takes a table name as its input. Here are two examples, based on the two ways to do dynamic SQL in Transact-SQL:
CREATE PROCEDURE general_select1 @tblname sysname, @key varchar(10) AS DECLARE @sql nvarchar(4000) SELECT @sql = ' SELECT col1, col2, col3 ' + ' FROM dbo.' + quotename(@tblname) + ' WHERE keycol = @key' EXEC sp_executesql @sql, N'@key varchar(10)', @key CREATE PROCEDURE general_select2 @tblname nvarchar(127), @key varchar(10) AS EXEC('SELECT col1, col2, col3 FROM ' + @tblname + ' WHERE keycol = ''' + @key + '''')

Before I say anything else, permit me to point out that these are examples of bad usage of dynamic SQL. Passing a table name as a parameter is not how you should write stored procedures, and one aim of this article is to explain this in detail. Also, the two examples are not equivalent. While both examples are bad, the second example has several problems that the first does not have. What these problems are will be apparent as you read this text. Whereas the above looks very simple and easy, there are some very important things to observe. The first thing is permissions. You may know that when you use stored procedures, users do not need permissions to access the tables accessed by the stored procedure. This does not apply when you use dynamic SQL! For the procedures above to execute successfully, the users must have SELECT permission on the table in @tblname. In SQL 2000 and earlier this is an absolute rule with no way around it. Starting with SQL 2005, there are alternatives, something I will come back to in the section The Permission System.


which holds the time a row last was updated. and nvarchar(MAX) in SQL 2005 and later. and you pass them as you pass parameters to a stored procedure. sp_executesql sp_executesql is a built-in stored procedure that takes two pre-defined parameters and any number of user-defined parameters. sp_executesql was added in SQL 7. but also comes to the rescue in SQL 2000 and SQL 7 when the SQL string exceeds 4000 characters. You can provide the parameter names for them. Temp tables created in the dynamic SQL will not be accessible from the calling procedure since they are dropped when the dynamic SQL exits. The syntax of @params is exactly the same as for the parameter list of a stored procedure. Not all parameters you declare must actually appear in the SQL string. Any USE statement in the dynamic SQL will not affect the calling stored procedure. You want to be able to find out how many rows in each table that were modified at least once during a period. EXEC() is mainly useful for quick throw-away things and DBA tasks. In application code. Invoking a block of dynamic SQL is akin to call a nameless stored procedure created ad-hoc. many tables have a column LastUpdated.The curse and blessings of dynamic SQL Page 4 of 36 Next thing to observe is that the dynamic SQL is not part of the stored procedure. in SQL 6. just like when you call a stored procedure. Beware that you must pass an nvarchar/ntext value (that is. either positional or named. http://www. The second parameter @params is optional. The first parameter @stmt is mandatory. but you will use it 90% of the time. (Compare to how temp tables created in a stored procedure go away when you exit the procedure. but something you run as a DBA from time to time. For now I will only give two keywords: SQL Injection and Query-Plan Reuse. the data type of @params is ntext SQL 2000 and earlier and nvarchar(MAX) since SQL 2005. The parameters can have default values and they can have the OUTPUT marker. To get a value back from your output parameter. Note that the first two parameters. Let's look at an example. sp_executesql should be your choice 95% of the time for reasons that will prevail. This has a number of consequences: Within the block of dynamic SQL. This is not something you run as part of the application. whereas EXEC() has been around since SQL 6. As you've seen there are two ways to invoke dynamic SQL. you cannot access local variables (including table variables) or parameters of the calling stored procedure. a Unicode value). EXEC() is the sole choice. Here is what it could look like: DECLARE @tbl @sql sysname. obviously. so you just keep it as a script that you have a around. If you issue a SET command in the dynamic SQL. And.5. @stmt and @params. sp_executesql and EXEC(). The rest of the parameters are simply the parameters that you declared in @params. you must specify OUTPUT with the parameter. The block of dynamic SQL has a query plan of its own.html 26-1-2012 . In the next two sections we will look at these two commands in detail. Say that in your database. (Whereas all variables that appear in the SQL string must be declared. The query plan for the stored procedure does not include the dynamic SQL. nvarchar(4000).) The block of dynamic SQL can however access temp tables created by the calling procedure.sommarskog. @params declares the parameters that you refer to in @stmt. but constitutes its own scope.) Just like @stmt. or in @params. must be specified positionally. but these names are blissfully ignored. and contains a batch of one or more SQL statements. the effect of the SET command lasts for the duration of the block of dynamic SQL only and does not affect the caller. A varchar value won't do. either with a DECLARE inside @stmt. The data type of @stmt is ntext in SQL 7 and SQL But you can pass parameters – in and out – to a block of dynamic SQL if you use sp_executesql.

Note also the appearance of '' around the date literal – the rule in T-SQL is that to include the string delimiter in a string. However. you must specify the N. but I wanted to stress that the @cnt in the dynamic SQL is only visible within the dynamic SQL.' END DEALLOCATE tblcur I've put the lines that pertain directly to the dynamic SQL in bold face. Since sp_executesql is a built-in stored procedure. the dynamic SQL has three parameters: one mandatory input parameter. The answer is that you can't. but @count in the surrounding script. you may want to make it a routine to declare @sql as nvarchar(MAX).The curse and blessings of dynamic SQL Page 5 of 36 @params nvarchar(4000). Overall. @count int DECLARE tblcur CURSOR STATIC LOCAL FOR SELECT object_name(id) FROM syscolumns WHERE name = 'LastUpdated' ORDER BY 1 OPEN tblcur WHILE 1 = 1 BEGIN FETCH tblcur INTO @tbl IF @@fetch_status <> 0 BREAK SELECT @sql = N' SELECT @cnt = COUNT(*) FROM dbo. technically this is not necessary (as long as you stick to your 8bit character set). whereas @count is not visible there. I also prefix the table name with "dbo. Since I left out one variable. as we will see when we look at dynamic SQL and query plans. I must specify the last. When I assign the @sql variable. ' + N'@cnt int OUTPUT' EXEC sp_executesql In SQL 2005 and later. N'@x int'. '20060101'. you must double it. and I leave in spaces to avoid that two concatenated parts are glued together without space in between. Dynamic SQL is just like any other SQL. that's the http://www. In this example. which is why I've left out @todate in the call to sp_executesql. which is a good habit. @x = 2 If you remove any of the Ns. I put the table name in quotename() in case a table name has any special characters in it. You may note that I've prefix the string literals with N to denote that they are Unicode strings. You may wonder why I do not pass @tbl as a parameter as well. which could cause a syntax error. You can see that I have declared the @sql and @params variables to be of the maximum length for nvarchar variables in SQL 2000. As @sql and @params are declared as nvarchar.' + quotename(@tbl) + N' WHERE LastUpdated BETWEEN @fromdate AND ' + N' coalesce(@todate. one optional input parameter. Note also that the variable is called @cnt in the dynamic SQL. @params. @cnt by name – the same rules as when you call a stored procedure. You can't specify a table name through a variable in T-SQL. you might want to use the same name. you will get an error message.html 26-1-2012 . there is no implicit conversion from varchar. and one output parameter. ' + N'@todate datetime = NULL.sommarskog. @cnt = @count OUTPUT PRINT @tbl + ': ' + convert(varchar(10). I've assumed that this time the DBA wanted to see all changes made after 2006-01-01. I am careful to format the statement so that it is easy to read. @count) + ' modified rows.". Normally. when you provide any of the strings directly in the call to sp_executesql. ''99991231'')' SELECT @params = N'@fromdate datetime. I will cover this sort of good practices more in detail later in the text. as in this fairly silly example: EXEC sp_executesql N'SELECT @x'. more about this just below.

Thus. In SQL 2005 and later. there are situations where this is an enormous blessing. @sql2 and @sql3 can be 4000 characters long – or even 8000 characters as EXEC() permits you to use varchar. this is not an issue. however. in SQL 7 and SQL 2000.The curse and blessings of dynamic SQL Page 6 of 36 whole story. which can be convenient at times. but it is not uncommon to exceed that limit.sommarskog. this is not legal: EXEC('UPDATE STATISTICS ' + quotename(@tbl) + ' WITH FULLSCAN') Best practice is to always use a variable to hold the SQL statement. since you can say: EXEC(@sql1 + @sql2 + @sql3) Where all of I used quotename(). While the parameter is ntext. you must interpolate them into the string. In this case. For very simple cases. you cannot as easily get values out from EXEC() as you can with sp_executesql. Since EXEC only permits string literals and string variables to be concatenated and not arbitrary expressions. For instance. EXEC does not have this limitation. but without the many restrictions of ntext. described just below. but here I've let it suffice with adding brackets. there is a limitation with sp_executesql when it comes to the length of the SQL string. you cannot use this data type for local variables. EXEC() EXEC() takes one parameter which is an SQL statement to execute. when you need to specify things like table names.html 26-1-2012 . so the example would better read: FETCH tblcur INTO @tbl IF @@fetch_status <> 0 BREAK SELECT @sql = 'UPDATE STATISTICS ' + quotename(@tbl) + ' WITH FULLSCAN' EXEC(@sql) The fact that you can concatenate strings within EXEC() permits you to make very quick things. EXEC() is less hassle than sp_executesql. The parameter can be a concatenation of string variables and string literals. you can in practice only use 4000 characters in your SQL string with sp_executesql. If you are on SQL 2000 or SQL 7. but cannot include calls to functions or other operators. In many cases this will do fine. However. EXEC() permits impersonation so that you can say: EXEC(@sql) AS USER = 'mitchell' EXEC(@sql) AS LOGIN = 'CORDOBA\Miguel' This is mainly a syntactical shortcut that saves you from embedding the invocation of dynamic SQL in http://www. I will show you an example later on. in case there is a table named Order Details (which there is in the Northwind database). you will need to use EXEC(). say that you want to run UPDATE STATISTICS WITH FULLSCAN on some selected tables. Here you can use the new data type nvarchar(MAX) which can hold as much data as ntext. Thus. use INSERT-EXEC to insert the result set from EXEC() into a table. Since SQL 2005. column names etc dynamically. you will have to stick to nvarchar(4000). As I mentioned. It could look like this: FETCH tblcur INTO @tbl IF @@fetch_status <> 0 BREAK EXEC('UPDATE STATISTICS [' + @tbl + '] WITH FULLSCAN') In the example with sp_executesql. You can. Since you cannot use parameters. but it can lead to poor habits in application code. when I also show you how you can use EXEC() to pass longer strings than 4000 characters to sp_executesql.

So this is the starting point. he will get a syntax error. including the text in red. consider this input for @shipname: ' DROP TABLE Orders -- The resulting SQL becomes: SELECT * FROM dbo. @shipname nvarchar(40) = NULL AS DECLARE @sql nvarchar(4000) SELECT @sql = ' SELECT OrderID. And if this web app logs into SQL Server with sysadmin or db_owner privileges. A plain user who runs a Windows application and who logs into SQL Server with his own login. you need to learn about SQL injection and how you protect your application against it.) SQL 2005 adds a valuable extension to EXEC(): you can use it to execute strings on linked servers. it is open for SQL injection: CREATE PROCEDURE search_orders @custid nchar(5) = NULL. (Which is disabled by default on SQL 2005 and later.) http://www. This is not a line of attack that is unique to MS SQL Server. (This is. A real-life example of such a procedure would have many more parameters.) As the procedure is written. but all RDBMS are open to it.Orders WHERE 1 = 1 ' IF @custid IS NOT NULL SELECT @sql = @sql + ' AND CustomerID LIKE ''' + @custid + '''' IF @shipname IS NOT NULL SELECT @sql = @sql + ' AND ShipName LIKE ''' + @shipname + '''' EXEC(@sql) Before we look at a real attack. but I've cut it down to two to be brief. with sysadmin rights. (I discuss these statements more in detail in my article Granting Permissions Through Stored Procedures. ShipName ' + ' FROM dbo. so that they can search for Brian O'Brien and Samuel Eto'o. SQL injection is a technique whereby an intruder enters data that causes your application to execute SQL statements you did not intend it to. dynamic SQL generated in T-SQL stored procedures. Do you see what will happen? Because @shipname includes a single this attack may or may not succeed. So even if you think that SQL injection is no issue to you. OrderDate. by the way. because you trust your users. be that SQL statements sent from the client. Mind you. the attacker has access into your network far beyond SQL Server through xp_cmdshell.The curse and blessings of dynamic SQL Page 7 of 36 EXECUTE AS and REVERT. A delimiter. CustomerID. But it is not uncommon for web applications to have a general login that runs SQL queries on behalf of the users. the attacker can add users and logins as he pleases.html 26-1-2012 . SQL injection is possible as soon there is dynamic SQL which is handled carelessly. And if the service account for SQL Server has admin privileges in Windows. let's just discuss this from the point of view of user-friendliness. Assume that the input for the parameters @custid and @shipname comes directly from the user and a naïve and innocent user wants to look for orders where ShipName is Let's Stop N Shop. or SQL batches executed from CLR stored procedures. usually a single quote. you still need to read this section. Here is an example. For instance. so he enters Let's.Orders WHERE 1 = 1 AND ShipName LIKE '' DROP TABLE orders --' This is a perfectly legal batch of T-SQL. he can change that. Since there is something called permissions in SQL Server. the attack succeeds. and a malicious user can take benefit of this. but if the attacker has achieved sysadmin rights on the server.sommarskog. is not likely to have permissions to drop a table. affects your dynamic SQL. a problem for which dynamic SQL is a very good solution. The purpose of the procedure below is to permit users to search for orders by various conditions. SQL Injection – a Serious Security Issue Before you start to use dynamic SQL all over town. I will cover this form of EXEC() in a separate section later in this text.

Keep in mind that user input comes from more places than just input fields on a form. One approach I seen mentioned from time to time. Never let the web site log in as sa! For web applications: never expose error messages from SQL Server to the end user. The second point makes the task for the attacker more difficult as he cannot get feedback from his attempts. @key varchar(10) AS EXEC('SELECT col1. The first point is mainly a safeguard. A web site that logs into a database should not have any elevated privileges. in a T-SQL procedure use sp_executesql. col2. is to validate input data in some way. col3 FROM ' + @tblname + ' WHERE keycol = ''' + @key + '''') and assume that @tblname comes from a URL. If this yields a syntax error. preferably only EXEC and (maybe) SELECT permissions. the attacker knows that there is a vulnerability. and then he can add his own SQL statement. Users that log into an application with their own login should normally only have EXEC permissions on stored procedures. The procedure search_orders above should be coded as: CREATE PROCEDURE search_orders @custid nchar(5) = NULL. @shipname nvarchar(40) = NULL AS DECLARE @sql nvarchar(4000) SELECT @sql = ' SELECT in a T-SQL procedure where the value is passed as an int parameter there is no risk. the intruder will not be able to do that much harm. so that if there is a injection hole. there are more options an attacker could succeed with. CustomerID. You may think that it takes not only skill. but also luck for someone to find and exploit a hole for SQL injection. And don't overlook numeric values: they can very well be used for SQL injection. and you must take precautions to protect your applications against it. The most commonly used area for injection attacks on the Internet is probably parameters in URLs and cookies. Here are are the three steadfast principles you need to follow: Never run with more privileges than necessary. but if you have dynamic table and column names. SQL injection is a serious security issue. it should be confined to reading operations so that users only need SELECT permissions. That is. Thus. not EXEC(). But it is the third point that is the actual protection.The curse and blessings of dynamic SQL Page 8 of 36 Typically. There are quite some options that an attacker could use to take benefit of this hole. Of course. there is a huge potential for SQL injection. ShipName ' + ' FROM dbo. He then finds out if he needs any extra tokens to terminate the query. and that we will look a little closer at. Single quote is the most common character to reveal openings for SQL injection. Take this dreadful version of general_select: CREATE PROCEDURE general_select2 @tblname nvarchar(127). but in my opinion that is not the right way to go. OrderDate. But remember that there are too many hackers out there with too much time on their hands. Finally he adds a comment character to kill the rest of the SQL string to avoid syntax errors. but if a supposedly numeric value is directly interpolated into an SQL string in client code. be very careful how you handle anything that comes into your application from the outside. If you use dynamic SQL. Always used parameterised statements.Orders WHERE 1 = 1 ' IF @custid IS NOT NULL SELECT @sql = @sql + ' AND CustomerID LIKE @custid ' IF @shipname IS NOT NULL SELECT @sql = @sql + ' AND ShipName LIKE @shipname ' http://www.html 26-1-2012 . an attacker first tests what happens if he enters a single quote (') in an input field or a URL.sommarskog.

Dynamic SQL and Stored Procedures http://www. By specifying adCmdStoredProc. Similar measures apply to other client APIs. Since this is so important. note that since we can include parameters in the parameter list. custid) End If If shipname <> "" Then cmd. @shipname Since the SQL string does not include any user input. you will find that it invokes – sp_executesql. If you compose an EXEC command into which you interpolate the input values. this is true. we can always pass all input parameters to the SQL string. you call the stored procedure through RPC. we don't need any complicated logic to build the parameter list. you need to call your procedure with the command type adCmdStoredProc and use . @shipname nvarchar(40)'. In this case you must enclose all such object names in quotename().se/dynamic_sql. @custid. Remote Procedure Call. you will get a completely incomprehensible error message. you are back on square one and you are as open to SQL injection as ever.Orders WHERE 1 = 1 " If custid <> "" Then cmd. adParamInput. In ADO you use ? as a parameter marker. but it is also more efficient.Net. here is an example of coding the above in VB6 and ADO: Set cmd = CreateObject("ADODB.CommandText = cmd. Yes.CommandText = " SELECT OrderID. 40.sommarskog. In the same vein. OrderDate. CustomerID. there is no opening for SQL injection. adWChar.Append cmd. 5.Parameters.) If you use the SQL Profiler to see what ADO sends to SQL Server. but can keep it static.ActiveConnection = cnn cmd. all APIs I know of supply a way to call a stored procedure through RPC. In the section Caching Query Plans.CommandText = cmd. ShipName " & _ " FROM dbo. N'@custid nchar(5). I will explain this example only briefly. Protection against SQL injection is not the only advantage of using parameterised queries. It's as simple as that. adParamInput. we will look more in detail on parameterised queries and at a second very important reason to use them. _ adVarWChar. which not only protects you against SQL injection. and you can only pass parameters that actually appear in the SQL string.CommandText & " AND ShipName LIKE ? " cmd. even if they don't actually appear in the SQL string.Command") Set cmd. The same advice applies to SQL generated in client code or in a CLR stored procedure. By the way. you cannot pass everything as parameters to dynamic SQL.CreateParameter to specify the parameters. In ADO.CommandType = adCmdText cmd.CreateParameter("@custid".Parameters. As you may recall.CommandText & " AND CustomerID LIKE ? " cmd. but! It depends on how you call your stored procedures from the client.html 26-1-2012 .The curse and blessings of dynamic SQL Page 9 of 36 EXEC sp_executesql @sql.Execute Since the main focus of this text is dynamic SQL in T-SQL procedures. (If you specify too many parameters. that I will return to in the section Good Coding Practices and Tips for Dynamic SQL. The example above was for dynamic SQL in a T-SQL stored procedure. for instance table and column names. You may think that an even better protection against SQL injection is to use stored procedures with static SQL only. shipname) End If Set rs = cmd. This section also includes an example of composing and sending a parameterised SQL statement for SqlClient in VB .CreateParameter("@shipname".Append cmd.

Instead. Giving Permissions through Stored Procedures. Instead. For this reason. A second method is to use the EXECUTE AS clause to impersonate a user that has been granted the necessary permissions. you cannot use dynamic SQL. but has side effects that can have unacceptable consequences for auditing. The reason for this is very simple: the block of dynamic SQL is not a procedure and does not have any owner. the application authenticates the users outside SQL Server and logs into SQL http://www. and I also take a closer look on ownership chaining. and grant that user (which is a user that cannot log in) the rights needed for the dynamic SQL to execute successfully. I presented various strategies for data-access for an application. application and they cannot perform updates that violate business rules. Users log into SQL Server but have no permissions on their own beyond the database access. the same is true for them. so it is nothing you introduce at whim. Any use of dynamic SQL requires that the users have direct permissions on the accessed tables. There are however. would take up too much space here.sommarskog. UPDATE and DELETE rights on tables only to permit dynamic SQL in some occasional procedure. You associate the certificate with a user. In a locked-down database. "application proxies" and Terminal Server. Ownership chaining never applies since all data access in a CLR procedure is through dynamic SQL. using stored procedures has been the way to give users access to data. ownership chaining does not work when you use dynamic SQL. I will also look at what happens when you use dynamic SQL in a stored procedure. Application roles were introduced in SQL 7. All and all. Depending on the sensitivity of the data in the application. But you can use certificates or impersonation to avoid having to give users direct permissions on the tables. the application performs all access through stored procedures that retrieve and update data in a controlled way. and I said that in many shops all data access is through stored procedures. It is that plain and simple. some ways to arrange so that users only have access to the data through the application.The curse and blessings of dynamic SQL Page 10 of 36 In the introduction. SQL 2005 and later In SQL 2005 and later versions of SQL Server. Instead I've written a separate article about them. and this application role has the permissions needed. where I discusses both certificates and impersonation in detail. whereas other are unaffected. This works as long as the procedure and the tables have the same owner. SQL 2000 and earlier On SQL 2000 there is no way to combine dynamic SQL with the encapsulation of permissions that you can get through stored procedures. If your security scheme precludes giving users permissions to access tables directly. With "application proxies". users do not have permissions to access tables directly. Describing these methods more closely. it may be acceptable to give the users SELECT permissions on the tables (or on some tables) to permit the use of dynamic SQL. my strong recommendation is to use certificates. there are three alternatives. Thus the chain is broken. so that users only get to see data they have access to. In this section. through a mechanism known as ownership chaining. I strongly recommend against granting users INSERT. row-level security schemes and system monitoring. As I have already mentioned. typically dbo (the database owner). All require you to change the application architecture or infrastructure. this can be addressed by signing a procedure that uses dynamic SQL with a certificate. and show that you lose some of the advantages with stored procedures. the application activates the application role by sending a password somehow embedded into it. The Permission System Historically. I will look a little closer at the advantages with using stored procedures over sending SQL statements from the client. This method is easier to use. If you write CLR procedures that perform data access.html 26-1-2012 .

and next time you run the = OD. or it is invalidated for some reason. SQL Server will put the plan into the cache.html 26-1-2012 . it does not matter much if the query is recompiled for an extra second each time. or execute it with EXEC(): SELECT FROM JOIN WHERE AND O. On the other hand. that included dynamic SQL as well. and do not grant more permissions than needed. The final possibility is to put the application on Terminal Server. All this changes. More importantly. or a different period. If a query needs to run for four minutes. SQL Server saves the plan in cache.sommarskog.ProductID = 76) BY O. the use of dynamic SQL nullified the benefit with stored procedures in this regard. if you add a single space somewhere.OrderID. SQL Server builds a query plan for it – or as the terminology goes – it compiles the query. Furthermore. Loose batches of SQL were compiled each time.. and next time you run this query.UnitPrice * OD.UnitPrice * OD. keep in mind about SQL injection. This proxy login impersonates the users in SQL Server. they cannot log into SQL Server outside the application.OrderID AND OD2.[Order Details] OD ON O.Quantity) Orders O [Order Details] OD ON O.OrderDate BETWEEN @from AND @to AND EXISTS (SELECT * http://www. SUM(OD. particularly if the query is executed over and over again..The curse and blessings of dynamic SQL Page 11 of 36 Server on their behalf with a proxy login. I discuss these two methods a little further.OrderID WHERE O. And since the query plan for dynamic SQL is not part of the stored procedure. SUM(OD. Thus in SQL 6. Thus. Caching Query Plans Every query you run in SQL Server requires a query plan. Thus. The query plan stays in cache until it's aged out because it has not been used for a while. since the users do not have any login on their own. Since the cache lookup is by a hash value computed from the query text. it is not unlikely that next time you want to run the query for a different product. When you run a query the first time.OrderID = OD2.Orders O JOIN dbo. But only if it is exactly the same query.) The reuse of cached query plans is very important for the performance of queries where the compilation time is in par with the execution time or exceeds it.Quantity) FROM dbo.OrderID O. the application is their only way to the data. Up to SQL 6. However.OrderID. the plan will be reused. Starting with SQL 7. if you instead use sp_executesql to run your query with parameters: DECLARE @sql nvarchar(2000) SELECT @sql = 'SELECT O. the network is configured so that they cannot access SQL Server from their regular computers. there is a huge gain with the cached plan.OrderID GROUP The query returns the total order amount for the orders in February 1998 that contained the product Lakkalikööri.. the cache is space.5 the only plans there were put into the cache were plans for stored procedures.5. if the execution time of the query is 40 ms but it takes one second to compile the query. SQL Server also caches the plans for bare statements sent from a client or generated through dynamic SQL. the plan is not reused. Users log into the terminal server which is set up so that all they can do is to run this application. the plan is reused. In Giving Permissions.OrderDate BETWEEN '19980201' AND '19980228' EXISTS (SELECT * FROM [Order Details] OD2 WHERE O. (Why this happens falls outside the scope of this article. and thus their permissions apply. For all these methods.and case-sensitive.OrderID = OD. Say that you send this query from the client.

in SQL 2005. Recall also that the cache is space. @prodid int'. but even if you don't. so if you generate the same query in several places.and space-sensitive fashion. can lead to serious performance degradation. and the cache lookup alone could take several seconds.ProductID = @prodid) GROUP BY O. SUM(OD.Orders. or they failed to prefix tables with dbo.sommarskog. still in a case. The same rules apply: unparameterised statements are cached but with little probability for reuse. I would assume that this is somewhat more expensive than looking up a stored procedure. before it looks for dbo. or SQL statements generated in CLR procedures. But since the parameter values are not part of the query text. In the section on SQL Injection. And this is not restricted to the SQL statement.OrderID" & _ " AND OD2. Do you see that I've prefixed all tables in the query with dbo? There is a very important reason for this.[Order Details] OD ON O.Orders. and up to SQL 2000.. In fact. Here is an example with VB .OrderID = OD. whereas parameterised queries can be as efficient as stored procedures if you remember to always prefix the tables with dbo. Users can have different default schema. the applications in question either did not use parameterised queries at all.CommandType.html 26-1-2012 .ProductID = @prodid)" & _ http://www. all users had a default schema equal to their username. '19980228'. (And still with the caveat that the cache lookup is space.OrderID' EXEC sp_executesql @sql.Quantity)" & _ " FROM dbo. It's easy to forget that dbo. I included an example on how to do parameterised queries with ADO and VB6. '19980201'.OrderID AND OD2. From what I have said here.) Most client APIs implement parameterised queries by calling sp_executesql under the covers. you will get as many entries in the cache for the query as there are users running it. and this users runs a query that goes "SELECT .and case-sensitive. but it seems to be a bad idea to rely on it. But in this regard there is very little difference to SQL statements sent from the client. under extreme circumstances. it follows that if you use dynamic SQL with EXEC() you lose an important benefit of stored procedures whereas with sp_executesql you don't. If you instead use stored procedures.OrderID = OD2.The curse and blessings of dynamic SQL Page 12 of 36 FROM dbo. user1 cannot share cache entry with a user different default = _ " SELECT O. that there have been hash collisions galore. @to datetime.and case-sensitive. and if you leave it out in just a single place in the query.Orders could appear on the scene at any time.OrderID" & _ " WHERE O. if default schema for user1 is user1.Net and SqlClient: cmd. you may inadvertently have different spacing or inconsistent use of case.OrderID. Yes. it is not equally important to prefix tables with dbo. Thus.UnitPrice * OD.CommandType = System. So far.Text cmd. Presumably.Data. To make this really efficient there is one more thing you need to observe. Microsoft still recommends that you do. FROM Orders". SQL Server must first check if there is a table user1. At least in theory. the parameter list is as much part of the cache entry. N'@from datetime.[Order Details] OD2" & _ " WHERE O.Orders O " & _ " JOIN dbo. since the cache lookup is by a hash value computed from the query text..OrderDate BETWEEN @from AND @to" & _ " AND EXISTS (SELECT *" & _ " FROM dbo. Since user1.[Order Details] OD2 WHERE O. 76 The principle for cache lookup is the same as for a non-parameterised query: SQL Server hashes the query text and looks up the hash value in the cache. it is perfectly possible that all users have dbo as their default schema.OrderID = OD2. Some of my MVP colleagues have observed systems with lots of memory (> 20 GB) when the plan cache has been so filled with plans for SQL statements. I've only talked about dynamic SQL in stored procedures. heavy use of dynamic SQL. Furthermore. the same plan can be reused even when the input changes. users with different default schema can share the same query plan.

But if the request is for the entire year.Parameters.Value = "1998-02-28" cmd. there is a huge gain. Yes.OrderID" cmd. the same index would be a disaster. that if the query is going to run for four minutes. SqlDbType..html 26-1-2012 .Int) cmd.Value = "1998-02-01" cmd. there is a good non-clustered index that can be used for a sub-second response time. and thereby degrading the performance of their application. . SqlDbType. Here is a demo that you can try on yourself.Parameters("@to"). and a table scan is better. and worst of all opening their database to SQL injection.Add("@prodid".Add("@from".Datetime) Forced parameterisation is. SQL Server parameterises all queries that comes its way (with some exceptions documented in Books Online). For your own development you should not rely on any form of auto-parameterisation. With forced parameterisation. but not identical. the query plan will be reused. mainly a setting to cover up for poorly designed third-party application that uses unparameterised dynamic SQL.Parameters. Whatever client API you are using. SqlClient uses names with @ for parameters. OrderDate FROM dbo. And if that recompilation slashes the execution time from forty minutes to four.The curse and blessings of dynamic SQL Page 13 of 36 " GROUP BY O. so I will not go into details on how the Parameters collection works. If you submit: SELECT OrderID. OrderDate FROM dbo. On SQL 2000 and earlier. I need to turn this upside down. Simple is the default and is the only option on SQL 2000 and earlier. Say that you have a query where the user can ask for data for some time span.Add("@to". With simple parameterisation. and just when you thought you were safe. http://www. it seems.Parameters("@from").Parameters("@prodid"). (But in the situation you really a want a new query plan each time.Orders WHERE CustomerID = N'ALFKI' SQL Server may recast this as SELECT OrderID. and thus you can still use sp_executesql to get the best protection against SQL injection.Orders WHERE CustomerID = @P1 so if next time you submit BERGS instead of ALFKI. with some inconsistency. if you have SQL 2005. there is a tone of desperation in my voice. SqlDbType. Autoparameterisation comes in two flavours: simple and forced. Instead. First create this database: CREATE DATABASE many_sps go USE many_sps go DECLARE @sql nvarchar(4000). auto-parameterisation happens only with very simple queries. I should mention that SQL Server is able to auto-parameterise queries.) They say seeing is believing. Recall what I said in the beginning of this section. Most queries benefit from cached parameterised plans. The syntax for defining parameters is similar to ADO. one second extra for compilation is not a big deal. but not all do. This article is long enough.Parameters. please learn how to use parameterised commands with it. I refer you to MSDN where both SqlClient and ADO are documented in detail.. For the sake of completeness.Datetime) cmd. in my opinion.Value = 76 In contrast to ADO. I don't know how many posts I've seen on the newsgroups over the years where people build their SQL strings by interpolating the values from input fields into the SQL string. it may in fact be better to interpolate critical parameters into the query string when you need to force recompilation each time. Starting with SQL 2005 you can force a query to be recompiled each time it is executed by adding OPTION (RECOMPILE) to the end of the query. and. you may have to verify that it doesn't happen when you don't want to. If the user asks for a summary for a single day.

On my machine scripting now completed in five seconds!. C. issue this command: ALTER DATABASE many_sps SET PARAMETERIZATION FORCED and redo the operation.OrderID. cnt = COUNT(*).OrderID = O. Once scripting is complete.Orders O JOIN Northwind. That is. Encapsulating Logic This is not a question of security or performance. How long time this takes depends on your hardware. Mgmt Studio emits a couple of queries with the procedure name embedded. but they have nothing to do with the topic of this article.) In this case.[Order Details] GROUP BY OrderID) AS OD ON OD.sommarskog.html 26-1-2012 . O.The curse and blessings of dynamic SQL Page 14 of 36 @x int SELECT @x = 200 WHILE @x > 0 BEGIN SELECT @sql = 'CREATE PROCEDURE abc_' + ltrim(str(@x)) + '_sp @orderid int AS SELECT O. although it may require some careful use of your client API. Reducing Network Traffic Another advantage with stored procedures over SQL sent from the client is that less bytes travel the network. http://www. With stored procedures you can use temp tables to hold intermediate results. If all logic is outside the database.CustomerID. nor does it matter if it is a CLR procedure. If the stored procedures use static SQL only. (You can use temp tables from outer layers as well. no parameterised FROM Northwind. Then from the context menu select to script them as CREATE TO to a new query window. you can stop and restart SQL Server before the two scripting operations. you can see this from one more angle. This gets more significant if the computation requires several queries. and in this case there is no dispute that you should use stored procedures to encapsulate your logic.CompanyName. total = SUM(Quantity * UnitPrice) FROM Northwind. This particular issue have been addressed in SQL Server Management Studio 2008.CustomerID = C. You still get the gains of reduced network traffic. Myself. (And that Microsoft can not use their own products properly. SSMS 2008 has its own scripting issues.OrderDate. By using stored procedures. or invoke dynamic SQL does not matter. you don't have to bog down your client code with the construction of SQL statements. you will see that for each procedure. possibly with logic in between. Totalsum = OD. Select all procedures.1 END Then in SQL Server Management Studio 2005.CustomerID JOIN (SELECT OrderID. press F7 navigate down to the list of stored procedures. but one of good programming practice and modularising your code.OrderID = @orderid' EXEC(@sql) SELECT @x = @x . it depends a little on what you put into those stored procedure..) If you run SQL Server on your local machine. I am of the school that the business logic should be where the data this could mean that data has to travel up to the client.Customers C ON O. Rather than sending a 50-line query over the network. the dividing line goes between sending SQL from the client or running stored procedures. Prodcnt = OD. If you use the Profiler to see what Mgmt Studio is up to. Then again. you only need to pass the name of a stored procedure and a few parameters. This demonstrates that the difference between parameterised and unparameterised can be dramatic. That difference lies entirely in the plan cache. and then use Task Manager to see how much physical memory SQL Server uses in the two cases. but on my machine it took 90 seconds and at the same time SQL Server grabbed over 250 MB of memory. only to travel back in the next moment.OrderID WHERE O.cnt. O.

You could just as well employ careful programming practices in your client language and send SQL strings. Because of deferred name resolution. With dynamic SQL.sql_expression_dependencies. In any case. the arguments for using stored procedures for encapsulation may not be equally compelling. While the main dividing line here is between static SQL and any form of dynamic SQL. You can even search the column sys. Available since SQL 2005. The alternative is to employ brute-force search. In this case. It has to be admitted that the strength of this argument is somewhat reduced by the fact that T-SQL is not too industrious on reporting semantic errors. this is less reliable. Ease of Writing SQL Code One distinct advantage of writing stored T-SQL procedures is that you get a syntax check directly.sql_dependencies in SQL 2005 and later. If you drop and recreate a table. sometimes there is no other application than Query Analyzer or SQL Server Management Studio. because it is very difficult to maintain complete dependency information in SQL Server. Nothing of this changes if you use dynamic SQL in your stored procedures. Then again. What I do myself is to regularly build an empty database from our version-control system. be that misspellings or temp tables created within the procedure. dynamic SQL in T-SQL procedures.sommarskog. the only container of logic available is stored procedures. and how it looks on the inside does not matter. an occasional stored procedure that uses dynamic SQL is not likely cause the Armageddon I pictured above. If not. and it's immaterial whether they use dynamic SQL or not. and if the construction of dynamic SQL is confined to some well-defined set of modules.) In this case. for this to be a very important reason to use stored procedures. there may be some query. and since our build tool loads all tables before any stored procedure or trigger. then you are probably better off in this regard to do it all in client code. where one or more tables are missing. as there is less code to search. but as the procedure text there is chopped into 4000-char slices. and which eventually becomes unbearable complex and inefficient because of all the legacy baggage it's carrying around. you embed the SQL code into strings. I'm here assuming that most of your procedures use static SQL only. all dependency information for the table is lost. you may end up with a database where no one ever dares to drop or change a column or a 26-1-2012 . Your SQL code is a string delimited by single quotes('). But it is a good argument for being restrictive with dynamic SQL in any form. If all access to tables is from static SQL in stored procedures. which makes programming far more complex. (sysdepends in SQL 2000. or SQL generated by CLR stored procedures . In SQL 2008 there is also sys.definition using SQL. In SQL 2000 you can search syscomments. I know that I can trust the dependency information in that database. The stored procedure is still a container for some piece of logic. dynamic SQL in T-SQL stored procedures is probably the least harmful. http://www. Even if you test your code carefully. (Typically this would be tasks that are run by an admin. a trivial syntax error may not show up until run time. because you are considering changing or dropping it. Another side of this coin is that when you write dynamic SQL. and who prefer to have the business logic elsewhere.The curse and blessings of dynamic SQL Page 15 of 36 But there are also people who like to see the database as a unintelligent container of lose this opportunity. you may be able find all references by using the system stored procedure sp_depends or query a system table directly. SQL Server will not examine queries in stored procedures. If you throw dynamic SQL into the mix – be that SQL sent from client.sql_modules.) I say may. Keeping Track of what Is Used In a complex system with hundreds of tables. you may need to know where a certain table or column is referenced. Nevertheless. that is only run in odd cases and not covered in your test suite. If all your stored procedures generate dynamic SQL. SQL Server does report sufficiently many errors. or some variation of a query. sys. this may work.

This feature has been further enhanced in SQL 2008. with an application that generates SQL in the client.) The most commonly used client languages with T-SQL . To fix it. or if it also uses dynamic SQL. and I refer to Books Online for details. so dynamic SQL in client code or CLR stored procedures is less prone to that particular problem. It sure does not serve to make coding easier. it does not matter whether the stored procedure uses static SQL only.html 26-1-2012 . even less to change it. (If you encrypt your procedures. This difference is even more emphasised. Use Debug Prints! When you write a stored procedure that generates dynamic SQL. If your application uses stored procedures. (I should add that SQL 2005 offers a new feature that permits the DBA to change the plan for a query without altering the code.) Good Coding Practices and Tips for Dynamic SQL Writing dynamic SQL is a task that requires discipline to avoid losing control over your code. and the combination of dynamic SQL and user-defined functions. if the problem is in a stored procedure. You may also prefer to ship your procedures WITH ENCRYPTION to protect your intellectual property. If you just go ahead. In this section. Or that it simply performs unbearably slow. which is likely to contain other code that also has changed since the module was shipped. you should always include a @debug parameter: http://www. in VB you don't have multi-line (In the section Good Coding Practices and Tips for Dynamic SQL. but this is best controlled through license agreements. by adding a plan guide. This is quite an advanced feature. we will look a little closer on how to deal with this problem.The curse and blessings of dynamic SQL Page 16 of 36 and this string may include strings itself. an ampersand and an underscore for continuation. as an ISV you may not want your customers to poke around in your code. as long as he is able to find a way to decrypt them. For CLR procedures it depends on many objects you have in your assemblies. he may be able to fix that by adding an index hint. troubleshoot and maintain. you may be able to deploy a fix into production within an hour after the problem was reported. I will also discuss some special cases: how you can use sp_executesql for input longer than 4000 chars in SQL 2000. VBScript – all use the double quote (") as their string delimiter. Then again. This means that before the fix can be put into production. the DBA can still change them. Addressing Bugs and Problems Somewhat surprisingly.) In this case. your code can become very messy. and to include a single quote into the string you need to double it. installing a new version of a CLR procedure is as simple as replacing a T-SQL procedure. we will look at how to avoid this. and how to use dynamic SQL with cursors. and the fix is trivial. one of the strongest arguments for stored procedures today may be that they permit you to quickly address bugs and performance problems in the application. C++. Of course. a DBA may be able to address problems directly without opening a support case. You are relieved from all this hassle. Which any DBA that knows how to use Google can do. and that there is an error in it. the module will have to go through QA and testing. C#. If you have one assembly per object. In contrast.Visual Basic. Say that you generate SQL statements in your application. On the other hand. his hands will be tied. if you are an ISV and you ship a product that the customer is supposed administer himself. You can easily get lost in a maze of quotes if you don't watch out.sommarskog. if you use stored procedures with static SQL only. and be difficult to read. For instance. so at the end of each line you have to have a double quote. you need to build a new executable or DLL. if a procedure runs unacceptably slow.

you get an error for that. http://www. Spacing and Formatting Another thing to be careful with is the spacing as you concatenate the parts of a query. .. in this case you can easily escape the mess by using sp_executesql instead – yet another reason to use parameterised statements. col3 FROMfoo WHERE keycol = 'abc' This is not a syntax error. earlier in this article. yes. you will be told that the columns keycol. Here it is again: CREATE PROCEDURE general_select2 @tblname nvarchar(127). I like to emphasise that this sort of procedure is poor use of dynamic SQL. it can be very difficult to spot the error only by looking at the code that constructs the SQL. there are situations when you need to deal with nested quotes even with sp_executesql. it can be very confusing. and you may not even discern where it comes from. And since you know that the table you passed to the procedure has these columns you will be mighty confused. If you work with dynamic SQL. col1. the error is much more likely to be apparent to you. For instance. I had this code: N' WHERE LastUpdated BETWEEN @fromdate AND ' N' coalesce(@todate. So always include a @debug parameter and a PRINT! Nested Strings As I've already mentioned. even if there is no FROM clause. one problem with dynamic SQL is that you often need to deal with nested string delimiters. col3 FROM ' + @tblname + ' WHERE keycol = ''' + @key + '''') (Again.The curse and blessings of dynamic SQL Page 17 of 36 CREATE PROCEDURE dynsql_sp @par1 int. Here is an example where it goes wrong: EXEC('SELECT col1. This is a fairly simple example. col2. it's legal to use a WHERE clause. But since the columns cannot exist out of the blue.sommarskog. Once the SQL code is slapped in your face.) SQL is one of those language where the method to include a string delimiter itself in a string literal is to double it.. I showed you the procedure general_select2. However. ''99991231'')' We will look at some tips of dealing with nested strings later in this section. So those four consecutive single quotes ('''') is a string literal with the value of a one single quote ('). it can get a lot worse.html 26-1-2012 . For instance. in the beginning of this article. @debug bit = 0 AS . because FROMfoo is a column alias to col3.. but when you run it. And. you must learn to master nested strings.. col3 are missing. col2. assuming the parameters foo and abc: SELECT col1. But this is the actual code generated. IF @debug = 1 PRINT @sql When you get a syntax error from the dynamic SQL. @key varchar(10) AS EXEC('SELECT col1. And even when you do. col3 FROM' + @tblname + ' WHERE keycol = ''' + @key + '''') See that there is a space missing after FROM? When you compile the stored procedure you will get no error. Obviously. col2.

@key varchar(10). you cannot pass a table or a column name as a parameter to sp_executesql. As I've said. had we included the dbo prefix.) While general_select still is a poor idea as a stored procedure. so in contrast to VB. b. Still you should protect it against SQL injection. (Although you could use the built-in function parsename() to split up a @tblname parameter in parts. (It can make perfectly sense for admin tasks). and the second is a pair of delimiters to wrap the string in. @debug bit = 0 AS DECLARE @sql nvarchar(4000) SET @sql = 'SELECT col1. quotename('Orders') returns [Orders]. If the code looks like this: SELECT @sql =' SELECT col1. you should use the built-in function quotename() (added in SQL 7). as a matter of routine. so if you have a really crazy table name like Left]Bracket. It could be that bad it comes from user input. To this end. col2.Orders') returns [dbo. An advantage of this is that your debug PRINT is easier to read.The curse and blessings of dynamic SQL Page 18 of 36 This is also a good example why you should use debug prints. col3 FROM' + @tblname + ' WHERE keycol = ''' + @key + '''' IF @debug = 1 PRINT @sql EXEC(@sql) It would be much easier to find the error by running the procedure with @debug = 1. quotename() takes two parameters: the first is a this error could not occur at all. best practice is to add dbo in the dynamic SQL and only pass the table name. I have a space after the opening single quote on each line to avoid syntax problems due to missing spaces. col2. o and a dot. Dealing with Dynamic Table and Column Names Passing table and column names as parameters to a procedure with dynamic SQL is rarely a good idea for application code. Thus. though. here is nevertheless a version that summarises some good coding virtues for dynamic SQL: CREATE PROCEDURE general_select @tblname nvarchar(128). col3 ' + ' FROM ' + @tblname + ' WHERE keycol = ''' + @key + '''') As you see. quotename() will return [Left]]Bracket].' + quotename(@tblname) + ' WHERE keycol = @key' IF @debug = 1 PRINT @sql http://www. and then add the string delimiters outside of that. quotename('dbo. A tip in such case is to do something like this: EXEC(' SELECT col1. Try to write the query as you would have written it in static SQL. col2. to have a string terminator on each line. but you must interpolate it into the SQL string. The default for the second parameter is [].Orders]. T-SQL permits you to embed newlines in string literals (as testified by the example above).sommarskog.) Overall. If you work with different schemas. You may prefer. each component should be quoted separately. good formatting is essential when working with dynamic SQL. you don't need a string delimiter on each line. (Obviously. pass the schema as a separate parameter. and in the case of a syntax error the line number in the error message may guide you. quotename() takes care of nested delimiters. col3 FROM dbo.html 26-1-2012 . Note that when you work with names with several components. As long as you only work with the dbo schema. but that is a table in an unknown schema of which the first four characters are d.

including single quotes. @sq + @sq) RETURN(@sq + @ret + @sq) END This version is for SQL 2000. SQL 6. (I should add that I got the suggestion to use quotename() or a user-defined function from SQL Server MVP Steve Kass. A remedy is this user-defined function: CREATE FUNCTION quotestring(@str nvarchar(1998)) RETURNS nvarchar(4000) AS BEGIN DECLARE @ret nvarchar(4000). http://www. This part of the dynamic SQL becomes: AND custname = 'D''Artagnan' There is a limitation with quotename(): its input parameter is nvarchar(128). you can use quotename() to protect yourself against SQL injection by help of this function. you are interpolating user input into the SQL string. To wit. N'@key varchar(10)'. if you for some reason prefer to use EXEC(). The default for this setting depends on context. So with quotename() and quotestring(). which means that any single quote in the input is doubled. On SQL 2005 and later. you would have to implement quotestring as a stored procedure. I don't know of any way to inject SQL that slips through quotename() or quotestring(). But you can specify other delimiters as well. @key = @key I'm using sp_executesql rather than EXEC(). you can also use double quotes (") as a string delimiter. is make use of the fact that T-SQL actually has two string delimiters. On SQL 7. There is a @debug parameter. replace 1998 and 4000 with MAX.) QUOTED_IDENTIFIER Another alternative to escape the mess of nested quotes. @sq. '''') Say that @custname has the value D'Artagnan. so you are a bit out of luck there. Thus. if the setting QUOTED_IDENTIFIER is OFF. whereas with parameterised commands. @sq char(1) SELECT @sq = '''' SELECT @ret = replace(@str.5 does not have replace(). Here is an example of using this function: IF @custname IS NOT NULL SELECT @sql = @sql + ' AND custname = ' + dbo. so it does not handle long strings. I'm prefixing the table name with dbo. Nested Strings and Quotestring The main purpose of quotename() is to quote object names. but the preferred setting is ON.The curse and blessings of dynamic SQL Page 19 of 36 EXEC sp_executesql @sql. do we have as good protection against SQL injection as we have with parameterised commands? Maybe. to make it work for any string length. The result is the same as above.sommarskog. you don't. Here is an example. IF @custname IS NOT NULL SELECT @sql = @sql + ' AND custname = ' + quotename(@custname.html 26-1-2012 . I'm wrapping @tblname in quotename(). Nevertheless. which is why the default for the second parameter is brackets.

col2. there is actually a WHERE state = @state' EXEC('EXEC sp_executesql N''' + @sql1 + @sql2 + '''.authors WHERE state = @state' INSERT #result (cnt) EXEC('DECLARE @cnt int EXEC sp_executesql N''' + @sql1 + @sql2 + '''. @key key_type. To wit. this is an inferior method to both sp_executesql and quotestring(). Thus. @sql2 nvarchar(4000). sp_executesql and Long SQL Strings in SQL 2000 There is a limitation with sp_executesql on SQL 2000 and SQL 7. you should use nvarchar(MAX) to avoid this problem. You can even use output parameters by using INSERT-EXEC. the code is much easier to read. since you cannot use longer SQL strings than 4000 characters. this is not a first-rate alternative. @cnt int OUTPUT''. @state char(2).) If you want to use sp_executesql when your query string exceeds this limit to make use of parameterised query plans. N''@state char(2)''.The curse and blessings of dynamic SQL Page 20 of 36 and it must be ON in order to use XQuery. All and all. you can do this: CREATE PROCEDURE general_select @tblname nvarchar(127). @debug bit = 0 AS DECLARE @sql nvarchar(4000) SET @sql = 'SET QUOTED_IDENTIFIER OFF SELECT col1. @state = ''' + @state + '''. and it may be the best way to go on SQL 6. (On SQL 2005 and later. as in this example: CREATE TABLE #result (cnt int NOT NULL) DECLARE @sql1 nvarchar(4000). it does not have any limitation in size. because the @stmt parameter to sp_executesql is ntext. @cnt = @cnt OUTPUT SELECT @cnt') http://www. @state = ''' + @state + '''') This works.html 26-1-2012 . you can wrap sp_executesql in EXEC(): DECLARE @sql1 nvarchar(4000). col3 FROM dbo. so by itself. since you are not protected against SQL injection (what if @key includes a double quote?). @state char(2) SELECT @state = 'CA' SELECT @sql1 = N'SELECT COUNT(*)' SELECT @sql2 = N'FROM dbo.5. The single quotes are for the SQL string and the double quotes are for the embedded string literals. but if you are aware of the caveats. N''@state char(2).sommarskog. indexed views and indexes on computed columns. @sql2 nvarchar(4000). But it would be OK to do for some sysadmin task (where SQL injection is not likely to be an issue). @mycnt int SELECT @state = 'CA' SELECT @sql1 = N'SELECT @cnt = COUNT(*)' SELECT @sql2 = N'FROM dbo.' + quotename(@tblname) + ' WHERE keycol = "' + @key + '"' IF @debug = 1 PRINT @sql EXEC(@sql) Since there are two different quote characters.

including updates. back out and redo your design. so that if you perform an update operation from your function.) A word of warning though: data access from scalar UDFs can often give performance problems. col2. FROM tbl WHERE dbo. and. you will get caught. as you know by There is however a way to use locally-scoped cursors with dynamic SQL.) EXEC() at Linked Server http://www. Since you can do anything from dynamic SQL. so that you don't exit the procedure without deallocating the cursor. You must be extra careful with error-handling though.) Once you have declared the cursor in this way. col3 FROM ' + @table EXEC sp_executesql @sql You may be used to using the LOCAL keyword with your cursors. (Because. as you see from the example. But if you want to use dynamic SQL in a UDF. (You are safe-guarded. @my_cur OUTPUT FETCH NEXT FROM @my_cur You refer to a cursor variable.sommarskog.sysobjects. However. as a local cursor will disappear when the dynamic SQL exits.html 26-1-2012 . you can pass them as a parameters. it is obvious why dynamic SQL is not permitted. N'@my_cur cursor OUTPUT'. you have to put the entire DECLARE CURSOR statement in dynamic SQL: SELECT @sql = 'DECLARE my_cur INSENSITIVE CURSOR FOR ' + 'SELECT col1. Dynamic SQL in User-Defined Functions This very simple: you cannot use dynamic SQL from used-defined functions written in T-SQL.The curse and blessings of dynamic SQL Page 21 of 36 SELECT @mycnt = cnt FROM #result You have my understanding if you think this is too messy to be worth it. OPEN @my_cur'. I've seen more than one post on the newsgroups where people have been banging their head against this.MyUdf(somecol) = @value and MyUdf performs data access. (I have to confess I have never seen any use for cursor variables until Anthony Faull was kind to send me this example. but there is an @ in front. Recall that all data access from the CLR is dynamic SQL. the dynamic SQL is its own scope. In SQL 2005 and later. Cursors and Dynamic SQL Not that cursors are something you should use very frequently. so I give an example for the sake of completeness. If you say SELECT .. but people often ask about using dynamic SQL with cursors. Anthony Faull pointed out to me that you can achieve this with cursor variables. you can use the cursor in a normal fashion. as in this example: DECLARE @my_cur CURSOR EXEC sp_executesql N'SET @my_cur = CURSOR STATIC FOR SELECT name FROM dbo. and in SQL 2000 there is no way out. you could implement your function as a CLR function. you have more or less created a hidden cursor. You have hit a roadblock. it is important to understand that you must use a global cursor.. This is because you are not permitted do anything in a UDF that could change the database state (as the UDF may be invoked as part of a query). just like named cursors. You cannot say DECLARE CURSOR EXEC().

dbo. Active directory or whatever. You can run this: DECLARE @cnt int EXEC('SELECT ? = COUNT(*) FROM Northwind. And while many of these questions taken by the letter have no other answer than dynamic SQL.) As with regular EXEC(). but also that in many cases that it is an outright bad idea. OLE DB translates that parameter marker as is appropriate for the data source on the other end.The curse and blessings of dynamic SQL Page 22 of 36 A special feature added in SQL 2005 is that you can use EXEC() to run pass-through queries on a linked server. and OLE DB uses ? as the parameter marker. but I did some quick experimenting. You may ask why the inconsistency with a different parameter marker from sp_executesql? Recall that linked servers in SQL Server are always accessed through an OLE DB provider. instead you use question marks (?) as parameter holders.Orders WHERE CustomerID = ?'.se/dynamic_sql. far too many examples use EXEC() without any thought of query plans. (The login to use on the remote server can be defined with sp_addlinkedsrvlogin. you can specify AS USER/LOGIN to use impersonation: EXEC('SELECT COUNT(*) FROM ' + @db + '. Say that you are on an SQL 2005 box. as seen by this example: EXEC('SELECT COUNT(*) FROM ' + @db + '. in this section I will explore some situations where you could use dynamic SQL. and you are dying to know how many orders VINET had in the Northwind database.dbo. there is almost every day someone who asks a question that is answered with use dynamic SQL with a quick example to illustrate.sysobjects') AT SQL2K SQL2K is here a linked server that has been defined with sp_addlinkedserver. but ever so often the person answering forgets to tell about the implications on permissions or SQL injection. there is often a real business problem which has a completely different solution without dynamic SQL – and a much better one. The confuse matters. You will see that sometimes dynamic SQL is a good choice.) Common Cases when to (Not) Use Dynamic SQL When you read the various newsgroups on SQL Server. There is one thing that you can do with EXEC() at a linked server. an Access database. but it could also be an Oracle server. (Not all RDBMS use @ for variables. This could be another instance of SQL Server. a convention inherited from ODBC. N'VINET') AT SQL2K SELECT @cnt Note here that the parameter values must appear in the order the parameter markers appear in the query. @cnt OUTPUT. On top of that. So. you don't use parameters with names starting with @. SQL 2005 does not ship with Northwind. both for input and output. The SQL could be a single query or a sequence of statements. you can either specify a constant value or a variable. and found that what you are impersonating is a local user or login.sysobjects') AS USER = 'davidson' AT SQL2K This begs the question: is davidson here a local user or a remote user at SQL2K? Books Online is not very clear about this. not a login on the remote server. Unfortunately.sommarskog.dbo. The syntax is simple.html 26-1-2012 . When passing a parameter. but you have a linked server set up to an instance of SQL 2000 with Northwind. and could it be composed dynamically or be entirely static. SELECT * FROM @tablename A common question is why the following does not work: CREATE PROCEDURE my_proc @tablename sysname AS http://www. that you cannot do with EXEC() on a local server: you can use parameters.

when it comes to building a query plan. Not the least. and future requirements may make the tables more dissimilar. even if they are similar to each other. as far as SQL Server is concerned. OK: 1) if the SQL statement is very complex. Nevertheless. You would still have one set of procedures per table. and the name includes some partitioning component. But they do describe different entities. should be the first column of the primary key in the united sales table. you could consider using a preprocessor like the one in C/C++. Partitioned Tables Partitioned tables were added in SQL 2005. The table may be huge (say over 10 GB in size). typically year and sometimes also month. or you want to be able age to out old data quickly. Finally. You can divide a table in up to 999 partition according to a partition function. Furthermore. 2) As we have seen. but have experience from other languages such as C++. When you start to pass table and column names as parameters. (Or at least it should!) Of course. where there is a suite of tables that actually do describe the same entity. (If your SQL statements are complex.html 26-1-2012 . These partitions can be split up over different filegroups to spread out the load.The curse and blessings of dynamic SQL Page 23 of 36 SELECT * FROM @tablename As we have seen. it is important to get a grip of what's being used. to save some typing. But in such case you should do partitioning properly.sommarskog. In the following. this is a bad idea. All tables have the same columns. the old truth does not hold. each table has its set of statistics and presumptions that are by no means interchangeable. Now. New tables are created as a new year/month begins. in a complex data model. there are situations where partitioning makes sense. So. In this case. And. as it describes a unique entity. despite different tables being used. You could just as well send the dynamic SQL from the client. It is much better to write ten or twenty stored procedures. but the code would be in one single include file. In a proper database design.) SELECT * FROM sales + @yymm This is a variation of the previous case. Parameterising the table name to achieve generic code and to increase maintainability seems like good programmer virtue. so that there actually is a considerable gain in maintainability to only have them in one place. http://www. VB etc where parameterisation is a good thing. starting with SQL 2005 there are methods to deal with permissions. and their semblance should be regarded as mere chance. you definitely lose control. admittedly. You should not have one sales table per month. There seems to be several reasons why people want to parameterise the table One camp appears to be people who are new to SQL programming. But you may be stuck with a legacy application where you cannot easily change the table design. let's make this very clear: this is a flawed table design. so even with one procedure per table you would still need a dynamic dispatcher. a name column and some auditing columns. it is not uncommon to end up with a dozen or more look-up tables that all have an id. you save some network traffic and you do encapsulation. but it should also be clear that we gain none of the advantages with generating that dynamic SQL in a stored procedure. because the user may want to specify a date range for a search. So if you want to do the above (save the fact that SELECT * should not be used in production code). I will look at three approaches to deal with partitioning without using dynamic SQL. each table is unique. you should have one single sales table. you are on the wrong path. and the month that appear in the table name. we can make this procedure work with help of dynamic SQL. But it is just that when it comes to database objects. writing one stored procedure per table is not really feasible.

it's easy to add new tables to the view or remove old tables as the data is aged out.. I'm not going into further details. as the query will access all tables in the view. CustomerID... not in Standard. say. so you can insert data into it. but refer you to Books Online.. But with a few more steps. because for queries that include the partitioning column in the WHERE clause. And such a view is updatable..sommarskog.Orders WHERE year(OrderDate) = 1996 ALTER TABLE Orders96 ALTER COLUMN OrderID int NOT NULL SELECT OrderID + 0 AS OrderID.. . you can now say: SELECT . EmployeeID INTO Orders96 FROM Northwind.. Also. OrderDate.. Views and Partitioned Views If you have an old application. . EmployeeID INTO Orders97 FROM Northwind.. OrderDate. and the data will end up in the right table.. Here is how it looks for Orders96: ALTER TABLE Orders96 ADD Year char(4) NOT NULL CONSTRAINT def96 DEFAULT '1996' CONSTRAINT check96 CHECK (Year = '1996') This column must be the first column in the primary key. CustomerID.The curse and blessings of dynamic SQL Page 24 of 36 Another important benefit of partitioned tables is that deleting a partition is a pure meta-data operation. you could make it into what SQL Server knows as a partitioned view. col2.html 26-1-2012 .. For this reason. FROM dbo. Table partitioning is only available in Enterprise and Developer Edition.Orders WHERE year(OrderDate) = 1997 ALTER TABLE Orders97 ALTER COLUMN OrderID int NOT NULL SELECT OrderID + 0 AS OrderID. the view is not updateable. this view is not terribly efficient. 12 months old. These columns need a default (so that processes that insert directly into these tables are unaffected) and a CHECK constraint. FROM sales WHERE year = '2006' AND . you can do this with the wink of an eye. Furthermore. which means that if you want to throw away all orders that are more than. where you cannot easily merge the umpteen sales tables into one. OrderDate. col1. EmployeeID INTO Orders98 FROM Northwind.sales2006 UNION ALL SELECT year = '2005'.sales2005 UNION ALL . a simple approach is to define a view like this: CREATE VIEW sales AS SELECT year = '2006'.Orders WHERE year(OrderDate) = 1998 ALTER TABLE Orders98 ALTER COLUMN OrderID int NOT NULL go ALTER TABLE Orders97 ADD CONSTRAINT pk97 PRIMARY KEY (OrderID) ALTER TABLE Orders96 ADD CONSTRAINT pk96 PRIMARY KEY (OrderID) ALTER TABLE Orders98 ADD CONSTRAINT pk98 PRIMARY KEY (OrderID) First step is to a add Year column to each table. FROM dbo. Assume that as legacy of a poor design we have these three tables: SELECT OrderID + 0 AS OrderID. Here is a quick example/demo on how to properly set up a partitioned view. Instead of composing the table name dynamically.. A true partitioned view can be very efficient. Unfortunately. col2. because it would break other parts of the application. a feature added in SQL 2000 (and available in all editions of SQL Server). CustomerID.. so we need to drop the current primary key and recreate it: http://www. SQL Server will only access the relevant table(s).

col3 http://www. it may seem that all three tables are accessed. CustomerID. This will not be a proper partitioned view. OrderDate. but when you define your real view. and the view will not be updatable. Compatibility Views If you have very many tables. you should probably set up a job for this. '19970101'. There is a risk that columns could come in different order in the tables. but replace them with views: CREATE VIEW sales200612 AS SELECT col1. That would be an example of good use of dynamic SQL. EmployeeID) VALUES ('1997'. You first create that new table. You should verify that it works for your situation. but with some luck SQL Server may still apply startup expressions. OrderID) Again. You now have a proper partitioned view that you can perform inserts and updates through. If you look at the query plan casually. when I tested this. when I ran a quick test. but if you check the Filter operators you will find something called STARTUP EXPR. OrderID. the start-up expression was not included for some reason I have not been able to understand. Good reading is also Stefan Delmarco's detailed article SQL Server 2000 Partitioned Views. 12000. For instance you can run: INSERT Orders(Year.Orders98 Note: I have here use SELECT * to save some space in the article. Then you drop the old tables. 2) And if you run a query like: SELECT FROM WHERE AND OrderID.sommarskog. You can find the full rules for partitioned views under the topic for CREATE VIEW in Books Online.Orders96 UNION ALL SELECT * FROM dbo. EmployeeID Orders Year = @year CustomerID = N'BERGS' SQL Server will at run-time only access the OrdersNN table that maps to @year. but it requires running a cursor over sysobjects to compose a CREATE VIEW statement that you then execute with sp_executesql or EXEC(). this must be performed for all three tables.html 26-1-2012 . In this case you could add a UNIQUE constraint with the partitioning column + the real primary key.) For your real-world case you may find it prohibitive to change the primary key. 'BERGS'.Orders97 UNION ALL SELECT * FROM dbo. I leave out example code. with all the data in it. This was a concentrated introduction to partitioned views. On SQL 2000. the view needs to be redefined. and access only one of the base tables. for instance by each month. This means that SQL Server determines at run-time whether to access the table or not. you should list the colunms explicitly. Henrik Staun Poulsen suggested an alternate solution that evades this restriction. At least I got it to work. Finally. I only got this result on SQL 2005 and SQL 2008. there is a risk that you will hit a roadblock with a partitioned view: SQL Server only permits 256 tables in a query.The curse and blessings of dynamic SQL Page 25 of 36 ALTER TABLE Orders96 DROP CONSTRAINT pk96 ALTER TABLE Orders96 ADD CONSTRAINT pk96 PRIMARY KEY (Year. When a new table is added with a new year. (In fact. OrderDate. If this happens frequently. you can create the view: CREATE VIEW Orders AS SELECT * FROM dbo.

sales_1 money NULL. this is simple to do without any dynamic SQL on SQL 2005 and later: DECLARE @mycolalias sysname SELECT @mycolalias = 'This week''s alias' CREATE TABLE #temp (a int NOT NULL.. one would wonder why people want to do this.The curse and blessings of dynamic SQL Page 26 of 36 FROM WHERE sales yearmonth = '200612' Old functions that uses dynamic SQL or whatever they do. but what happens is simply that the variable @colname is assigned the value in @value for each affected row in the table. this is kind of difficult. . If you don't know about the CASE expression. UPDATE tbl SET @colname = @value WHERE keycol = @keyval In this case people want to update a column which they select at run time. can continue to do so. please look it up in Books Online. PRIMARY KEY (prodid)) It could make more sense to move these sales_n columns to a second table: CREATE TABLE product_sales (prodid prodid_type NOT NULL. Maybe it's because their tables look like this: CREATE TABLE products (prodid prodid_type NOT NULL..html 26-1-2012 .. b) SELECT 12. month tinyint NOT NULL. sales money NOT NULL. but you don't have to write it by hand. sales_2 money NULL. this solution requires you to produce a lot of code.sommarskog. So there is all reason to avoid it. . col2 = CASE @colname WHEN 'col2' THEN @value ELSE col2 END. month)) SELECT col AS @myname The request here is to determine the name for a column in a result set at run-time.. Here is a fairly simple workaround: UPDATE tbl SET col1 = CASE @colname WHEN 'col1' THEN @value ELSE col1 END. PRIMARY KEY ( you can easily write a program in the language of your choice to generate the views and triggers.. something not to take lightly.. b int NOT NULL) INSERT #temp(a. The above is actually legal in T-SQL.sp_rename '#temp. My gut reaction. prodname name_type NOT NULL.b'. is that this should be handled client-side. In any case. If they perform INSERT. sales_12 money NULL. Obviously. 'COLUMN' http://www. Then again. But if your client is a query window is Management Studio or similar. UPDATE or DELETE operations. In this case dynamic SQL would call for the user to have UPDATE permissions on the table. 17 EXEC tempdb. @mycolalias. you need to implement INSTEAD OF triggers to support this.. . It's a very powerful SQL feature.

(You need to qualify sp_rename with tempdb to have it to operate in that database. @par2.... SELECT @sp = @dbname + '. This can be dynamic. SELECT @dbname = quotename(dbname) FROM .. If there is a need for a second set of databases.. SELECT * FROM @dbname + '. There is yet one thing to be aware of on SQL 2000: you cannot use sp_rename in a stored procedure that is to be run by plain users. although not entirely without dynamic SQL: you need put the SELECT from the temp table in EXEC(): EXEC('SELECT * FROM #temp') This is because on SQL 2000.. even if this is only a temp table. if you are in db1 and need to get data from db2. look at my article How to Share Data between Stored Procedures for suggestions.sommarskog.. That is. Get Data from another Database If you for some reason have your application spread over two databases. so if the the SELECT is in the same batch.) You will get an informational message Caution: Changing any part of an object name could break scripts and stored procedures. and depending on your underlying reason. you call a stored procedure in db2. as sp_rename thinks you need to be a member of the db_owner or db_ddladmin database roles.dbo. the statement fails with Invalid column name 'b'. FROM This issue has been addressed in SQL 2005. This trick works on SQL 2000 too. and then you use sp_rename to rename the column along your needs. you only have to update the synonyms. There seems to be several reasons why people want to do this..tbl' In this case the table is in another database which is somehow determined dynamically. If you want to get result sets back from db2. because EXEC permits you to specify a variable that holds the name of the procedure to execute.some_sp' EXEC @ret = @sp @par1.tbl as just otherdbtbl.. There may still be cases you may find that dynamic SQL is the only feasible situation.. and there is no need to use dynamic SQL. sp_rename apparently does not trigger a recompile. Yet a way to avoid dynamic SQL is to use stored procedures for all inter-database communication. you first get the data into a temp table..html 26-1-2012 . because if someone asks for a second environment on the same server. http://www. added in SQL 2005: CREATE SYNONYM otherdbtbl FOR otherdb. what you absolutely not should do is to have code that says: SELECT .dbo. The best solution for this particular problem is to use synonyms.tbl You can then refer to otherdb. but you may be able to live with that. you have a lot of code to change.tbl JOIN . the solution is different. The most obvious is: SELECT @dbname = quotename(dbname) FROM . This can be done in two ways.dbo. This is bad..The curse and blessings of dynamic SQL Page 27 of 36 SELECT * FROM #temp That is.

Personally. COUNT(*) FROM sysobjects' As you might guess..localtbl .. A "Master" Database The scenario here is that you have a suite of databases with identical schema. It will give you a much bigger maintenance problem.html 26-1-2012 . you are better off skipping stored procedures altogether and do all access from client code instead. and employ a strict row-level security scheme. As above. the dynamic SQL in this example runs in @dbname. I don't give you the details on how to do it and I strongly recommend that you don't go there.. if the query is complex.. Whereas queries only would return the data they should.. and for sysadmin tasks dynamic SQL is usually a fair game. demonstrated by this example: sp_MSforeachdb 'SELECT ''?''.The curse and blessings of dynamic SQL Page 28 of 36 SELECT @sql = ' SELECT . I should hasten to add that sp_MSforeachdb is not documented in Books Online. and each customer can access his database (but of course no one else's). What else can you do? Some people might suggest that you should collapse the databases into one. Other customers may be prepared to pay for extra functions that only they have access to. the day you want to update the table definitions. ' EXEC sp_executesql @sql. because your code will entirely littered with dynamic SQL. Some people see a problem with the same stored procedures in fifty databases. Nevertheless there is an kind of alternative: sp_MSforeachdb. is that every database serves a different customer. The trick here is that when you specify a system stored procedure in three-part notation with the database name. Another wild approach is to use SQL Server's own master database and install the application procedures as system procedures. FROM dbo.othertbl ' + ' JOIN ' + quotename(db_name()) + '.. I make use of that you can specify the procedure name dynamically with EXEC. But. In a complex application.. What then is the real solution? Install the stored procedures in each database and develop rollout routines for your SQL objects. FROM ' + @dbname + ' . query plans and error messages may indirectly disclose information to users who are not authorised to see it. In such case there is only one place you need to specify the database: the connection string. ' SELECT @dbname = quotename(dbname) FROM . and believe that they face a maintenance nightmare.sp_executesql' EXEC @sp_executesql @sql.dbo. So while I mention the possibility. SELECT @sp_executesql = @dbname + '.otherdbtbl ' + ' JOIN dbo. . @params. so what you win is that you don't have to write the control loop yourself.localtbl .. you can do that. In fact. not the current database... Thus. if you feel that this is the only alternative.. the procedure executes in the context of that database. this is entirely unsupported. which also means that use of it is not supported by Microsoft and it could be changed or withdrawn from SQL Server without notice. row-level security cannot be implemented entirely waterproof in SQL Server.. sp_MSforeachdb uses dynamic SQL internally. it http://www. This also permits you to have some flexibility. because neither caching nor permissions are issues. I would never accept such a solution as a potential customer. Do Something in Every Database This sounds to me like some sysadmin venture.. @params. Even more importantly. but I am told that it still works in SQL 2008. The typical reason they are different databases and not one.dbo. bugs can easily lead to that information is exposed to people who should not see it. . and most of the tables are in the remote database you can also do: SELECT @sql = ' SELECT . Yes. In any case. I have not played with this for a long time. Some customers may prefer to skip an upgrade. So they get the idea that they should put the procedures in a "master" database. You need this anyway...

I have not been able to think of anything useful. I describe a whole range of ways to do this. The correct method is to unpack the list into a table with a user-defined function or a stored procedure. but require his own instance. a security-aware customer would not even accept to share the same instance of SQL Server... but it is an extremely poor solution. If you are doing this. and then are puzzled why the query above does not return any rows. You cannot pass the list as a parameter to sp_executesql. I would not accept to share database with other customers.3.2. you have not completed the transition to use stored procedure and you are still assembling your SQL code in the client.. In fact.html 26-1-2012 . so you would have to use EXEC() and be open to SQL injection. SELECT * FROM tbl WHERE @condition If you are considering to write the procedure CREATE PROCEDURE search_sp @condition varchar(8000) AS SELECT * FROM tbl WHERE @condition Just forget it. On top of that.' SELECT @sp_executesql = quotename(@dbname) + '. but there are jump-start links in the beginning of the article. please drop me a line.4'. it took SQL Server 15 seconds to build the query plan for a list with 10000 elements. and Use dynamic SQL is a far too common answer. But this example lapses into Dynamic Search Conditions A not too uncommon case is that the users should be able to select data from a broad set of parameters. I also present performance data for the various These two conditions are the same: col IN (@list) col = @list IN does not mean "parse whatever data there is at runtime as a comma-separated list". you will get a match.The curse and blessings of dynamic SQL Page 29 of 36 permits you to easily scale out and move some databases to a second server. we have already seen how to do this: SELECT @sql = 'CREATE VIEW .. This is a very common question on the newsgroups. but if you find out something. http://www. if there is a row where col has the value '1. (You may ask whether not synonyms could be used to implement the "master" database. Yes. In my article.) Creating an Object in Another Database This question sometimes comes up. It's a compile-time shortcut for col = @a OR col = @b OR . Most often people have problems with the USE command. make use of that you can set the database context by calling sp_executesql with three-part notation.2. The procedure search_orders in the section on SQL injection is a very simple example of this.4' in @list. you can do this with dynamic SQL. Arrays and Lists in SQL Server. for long lists.3. Well.sommarskog. SELECT * FROM tbl WHERE col IN (@list) It is fascinating how may people who put '1. IN has extremely poor performance – in some tests I did.sp_executesql' EXEC @sp_executesql @sql That is. depending on which version of SQL Server you are using. I mentioned that as a customer. (Dynamic SQL is at the bottom of that list!) This is a long article. In fact. The correct solution is to avoid USE altogether in this case.

[1996] = SUM(CASE Year(OrderDate) WHEN '1996' THEN 1 ELSE 0 END).sommarskog. This article exists in two versions. not to say impossible to make the procedure fool-proof since it accepts query text as parameter. but the only option for good performance is to use dynamic SQL. For instance. this call will succeed as ownership chaining applies. but you should not make a procedure like this one accessible to anyone. you use the technique I described in the section SELECT col AS @myname. it's simple to write a single static query with conditions like: AND (CustomerID = @custid OR @custid IS NULL) But in SQL 2005 and earlier if is not possible to get good performance from such a query. A SELECT statement returns a table. so Microsoft reverted on that change for a while. The general technique is a two-step operation: 1) Get the unique values to pivot on. I like to stress one thing though: The way pivot_sp is written.EmployeeID = E. because the original implementation had a serious bug. but leave it to you to explore it on your own. or even how many there will be. One approach is to set an upper limit of how many output columns you support and use dummy names for the columns. For instance. and which aggregation operation you want. Dynamic Crosstab Another common request is to make a dynamic crosstab query. The file for pivot_sp includes an example to demonstrate http://www. say that you want to display the number of orders handled by each employee in Northwind with one column per year. generate a query like the one above. both with dynamic SQL and static SQL. in this example. Rather than going into details here. and one for earlier versions. As this article is already long enough.The curse and blessings of dynamic SQL Page 30 of 36 Any programmer that tackles this realises that writing a static solution with a tailor-made query for each combination of input parameters is impossible. However that is a bit complicated.LastName But in many situations you don't exactly which columns you will have in the data. [1998] = SUM(CASE Year(OrderDate) WHEN '1998' THEN 1 ELSE 0 END) FROM Orders O JOIN Employees E ON O. you will want to employ dynamic SQL for this. [1997] = SUM(CASE Year(OrderDate) WHEN '1997' THEN 1 ELSE 0 END). one for SQL 2008 and later. Rather I recommend that you do DENY EXECUTE ON pivot_sp TO public To make sure that plain users cannot run it. I don't go into details to try to explain how it works. It most cases. we may not know beforehand for which years there are orders. This is no problem as long as you use it as your own utility procedure and have full control over the input.LastName. in the very most cases. 2) with those values. And there is not really any alternative. You can make a shortcut with the stored procedure pivot_sp. If you make a call to pivot_sp in a stored procedure. and it is very difficult. so there is no way you can write a static SELECT statement to achieve this. While it's a short GROUP BY E. where I discuss this type of searches in more detail and where I present several methods.html 26-1-2012 . This changed in SQL 2008. it takes some time to get everything in place. The procedure takes a number of parameters permitting you to specify the query. where you transform rows into columns. provided that you use the RECOMPILE hint. and a table has a known number of columns with known names. the rows to group by and to pivot by. I refer you to my article. something I have adapted from a procedure originally written by SQL Server MVP Itzik Ben-Gan. it is wideopen to SQL injection. Once you have run the query. Dynamic Search Conditions. This query works well: SELECT E. However.

the optimizer may avoid the sort if there is a suitable index. CASE @col1 WHEN 'col2' THEN col2 ELSE NULL END. CASE @col1 WHEN 'col3' THEN col3 ELSE NULL END If you also want to make it dynamic whether the order should be ascending or descending. If you add an ORDER BY clause in dynamic SQL. which is a third-party tool. col3 FROM dbo. review the CASE expression in Books Online. col2.tbl ORDER BY CASE @sortorder WHEN 'ASC' THEN CASE @col1 WHEN 'col1' THEN col1 WHEN 'col2' THEN col2 WHEN 'col3' THEN col3 END ELSE NULL END ASC. SELECT TOP @n FROM tbl http://www. Instead. but I have heard several good comments about it. col2. Another option for dynamic crosstab is CASE @sortorder WHEN 'DESC' THEN CASE @col1 WHEN 'col1' THEN col1 WHEN 'col2' THEN col2 WHEN 'col3' THEN col3 END ELSE NULL END Or use the form in the second example to deal with different data types. SELECT * FROM tbl ORDER BY @col This can easily be handled without dynamic SQL in this way: SELECT col1.tbl ORDER BY CASE @col1 WHEN 'col1' THEN col1 ELSE NULL END. It should be added that these solutions has the disadvantage that they will always cause a sort which for a large data set could be expensive. I have never used it myself. you can do this: SELECT col1. col3 FROM dbo. SQL Server MVP Itzik Ben-Gan had a good article on this topic in the March 2001 issue of SQL Server Magazine. col2. Note that if the columns have different data types you cannot lump them into the same CASE expression. where he offers other suggestions.tbl ORDER BY CASE @col1 WHEN 'col1' THEN col1 WHEN 'col2' THEN col2 WHEN 'col3' THEN col3 END Again. as the data type of a CASE expression is always one and the same.html 26-1-2012 . add one more CASE: SELECT col1.The curse and blessings of dynamic SQL Page 31 of 36 this. if you are not acquainted with it. col3 FROM dbo.sommarskog.

and http://www. because they create a temp table from dynamic SQL. the set of tables and columns are supposed to be constant. it appears that they want to construct unique names for temporary tables. If you want to create a permanent table which is unique to a user. CREATE TABLE with Unknown Columns Sometimes I see persons on the newsgroups that are unhappy.) I guess a common reason for wanting to do this is to implement paging in web applications. CREATE TABLE @tbl The desire here is to create a table of which the name is determined at run-time. They may change with the installation of new versions. for instance ##temp. as long as you can accept the security implications. since SQL 2005 added new syntax that permits a variable: SELECT TOP(@n) col1. Sometimes when people are doing this. the user who runs the procedure must have permissions to create tables. so you need to use dynamic SQL to use TOP. but not during run-time. When told that they have to create the table outside the dynamic SQL. Nevertheless: Why? Why would you want to do this? If you are creating tables on the fly in your application. and then they can't access it. you have missed some fundamentals about database design. because it disappeared when the dynamic SQL exited. Etc. one with two # in the name.The curse and blessings of dynamic SQL Page 32 of 36 This is no longer an issue. If you say: CREATE TABLE #nisse (a int NOT NULL) then the actual name behind the scenes will be something much longer. but you don't want to stay connected and therefore cannot use temp tables. and no other connections will be able to see this instance of #nisse. If we just look at the arguments against using dynamic SQL in stored procedures. Such a table is visible to all processes (so you may have to take precautions to make the name unique). In a relational database. it may be better to create one table that all clients can share.html 26-1-2012 . SQL Server MVP Aaron Bertrand has an article which is the standard reference on this topic. but where the first column is a key which is private to the client. But there is an alternative: CREATE PROCEDURE get_first_n @n int AS SET ROWCOUNT @n SELECT au_id. few of them are really applicable here. One solution is to create a global temp table. (But it's not worth to change the permissions only for this. because they don't know the structure of the table until run-time.sommarskog. If a stored procedure has a static CREATE TABLE in it. au_fname FROM authors ORDER BY au_id SET ROWCOUNT 0 It can be disputed whether SET ROWCOUNT @n is really a better solution than running a dynamic SQL statement with TOP. TOP does not accept variables. as this is a built-in feature in SQL Server. A dynamic TOP is probably a better choice. Plan caching obviously has nothing to do with it. This is completely unnecessary. they respond that they can't. so dynamic SQL will not change anything. col2 FROM tbl On SQL 2000. I discuss this method a little more closely in my article How to Share Data between Stored Procedures.

. The above is only possible under certain conditions: http://www.dbo.dbo. and you don't know the structure of your data until run-time.. it exists until your process exits. I first create a dummy entry for MYSRV. it must also have the database and tables that you access. although it's only usable in some situations. VB or Perl.remotetbl If you can confine the access to the linked server to a stored procedure call. as demonstrated by this snippet: EXEC sp_addlinkedserver MYSRV.sommarskog.html 26-1-2012 . you can build the SP name dynamically: SET @sp = @server + 'db.. You can use sp_addlinkedserver to define the linked server at run-time. dynamic SQL is probably the best way if you are on SQL 2000.sysservers WHERE srvname = 'MYSRV') EXEC sp_dropserver MYSRV EXEC sp_addlinkedserver MYSRV. but in this case we want to access a linked server of which the name is determined at run-time. There exists however an alternative. @par2.master. @provider='SQLOLEDB'. @srvproduct='Any'. But the real question is: what are these guys up to? If you are working with a relational database. As the linked server must exist when the procedure is created. and then at run-time defines MYSRV to point to @server. the best solution is probably to use synonyms: CREATE SYNONYM myremotetbl FOR Server. Because. determined in the flux of the moment. all access to that table would have to be through dynamic SQL.sysdatabases go EXEC sp_dropserver MYSRV go CREATE PROCEDURE linksrv_demo @server sysname AS IF EXISTS (SELECT * FROM master. If you want to join a local table with a remote table on some remote server. Two of the solutions for dynamic database names apply here as well: On SQL 2005 and later. then there is something fundamentally wrong. I can't really provide any alternatives. @datasrc=@@SERVERNAME go CREATE PROCEDURE linksrv_demo_inner WITH RECOMPILE AS SELECT * FROM MYSRV. @srvproduct='Any'. As I have never been able to fully understand what the underlying business requirements are. be that C#. linksrv_demo_inner is the procedure where we actually access the linked server.some_sp' EXEC @ret = @sp @par1. you should seriously consider to run your SQL from a client program.) linksrv_demo is the outside interface which takes a server name as a parameter. Linked Servers This is similar to parameterising the database name. which I subsequently drop once the procedure has been created.db. (Not only must the linked server But I would suggest that if you need to go this road. @provider='SQLOLEDB'. @datasrc=@server EXEC linksrv_demo_inner EXEC sp_dropserver MYSRV go EXEC linksrv_demo 'Server1' EXEC linksrv_demo 'Server2' There are two procedures.The curse and blessings of dynamic SQL Page 33 of 36 unless you explicitly drop it. and composing dynamic SQL strings is easier in languages with better string capabilities.

so that only rows of interest are brought across the wire.quotestring(@state) SELECT @localsql = 'SELECT * FROM OPENQUERY(MYSRV. The function quotestring() that I showed you earlier can be of great help: DECLARE @remotesql nvarchar(4000). I don't think this is really necessary. plain users do not apply. as SQL Server should sense the changed definition. normally only the roles sysadmin and setupadmin have these permissions. Strict discipline is absolutely necessary when working with dynamic SQL for OPENQUERY. Dynamic Column Widths Say that you write a stored procedure that is to present some data. you may have to deal with up to three levels of nested quotes. ' + dbo. @localsql nvarchar(4000). If you don't watch out. Thus. in which case there are no permission issues. Typically you would use a temp table to hold the data. but as they say. (It goes without saying. (This is because the optimizer builds a plan for the distributed query when the procedure is compiled. you cannot have several instances of the procedure running. but neither do you want any extraneous spaces. that you should use the alias in this procedure only. Since EXEC() at linked servers can take parameters. as the SQL statement easily can exceed the limit of 129 characters for the input parameter to quotename(). you can use EXEC() to run an SQL statement on a linked server. Their second argument is an SQL string.Orders ' + 'WHERE CustomerID = N''''VINET'''''')' PRINT @sql EXEC(@sql) to try to find out if you might you have one ' too many or too few. ' + '''SELECT * FROM Northwind. OPENQUERY The rowset functions OPENQUERY and OPENROWSET often calls for dynamic SQL. Since the remote SQL string can include string literals. @state char(2) SELECT @state = 'CA' SELECT @remotesql = 'SELECT * FROM pubs. I've added WITH RECOMPILE to linksrv_demo_inner. you want the column width to be so wide that no data is truncated. On SQL 2005 and later.html 26-1-2012 .The curse and blessings of dynamic SQL Page 34 of 36 The procedure must be run by someone who has privileges to set up linked servers. and they do no accept variables. to prevent that a cached plan does not access a different server. This is something you can achieve with dynamic SQL. In fact. and the GUI it is to be run from is Query Analyzer or SQL Server Management Studio (presumably because it is a sysadmin procedure). you can spend a full day looking at things like: DECLARE @sql varchar(8000) SELECT @sql = 'SELECT * FROM OPENQUERY(MYSRV.quotestring(@remotesql) + ')' PRINT @localsql EXEC (@localsql) The built-in function quotename() is usually not useful here. This you cannot do with EXEC().) As you can see in the example. better safe than sorry. To make the output easy to you may not even have to split the code over two procedures.) So any single parameter you want to pass to the SQL statement for that remote server requires you to use dynamic SQL. Since you change a server-wide definition. this can make things considerably easier. you can join OPENQUERY with local tables.dbo.sommarskog.authors WHERE state = ' + dbo. This is a safety precaution. Then again. http://www.

If the columns come in different order in the underlying tables. Just keep those letters and cards coming! If you have suggestions for improvements or corrections on contents. poor query-plan reuse. I encourage you to post to any of the newsgroups microsoft. code that runs once a night or once a week or even less frequently. Dynamic SQL and Maintenance Tasks I've written this text with a main focus on application code. I also like to thank Frank Kalis for providing the German translation. Steve Kass. and result in code that is difficult to read and maintain. A user could create a table with a name that injects an SQL command into your maintenance script when you run it. Keith Kratochvil. 2009-09-12 – Gennadi Gretchkosiy pointed out using that SELECT * in a partitioned view is not OK.public.sommarskog. this is a risk that you definitely should not neglect. for this sort of code. Hal Berenson and Aaron Bertrand. If you write a job that performs operations on tables in every database. bad usage of dynamic SQL can cause serious harm by opening for SQL injection. If you are the DBA at a hosting curse and blessings of dynamic SQL Page 35 of 36 Rather than giving an example. Revision History 2011-06-23 – Updated the text in the section on Dynamic Search Conditions to mention the RECOMPILE hint. If you are a DBA that writes some stored procedure to be run by junior operators that do not have sysadmin privilege themselves. 2009-02-14 .programming or comp. two points about SQL injection I like to make. Henrik Staun Poulsen. you will get a mess. Acknowledgements and Feedback I like to thank the following persons who have provided valuable suggestions and input for this article: SQL Server MVPs Tibor Karaszi. The same applies to code that does not require permissions outside the database. Marcus Hansfeldt. Query plans are rarely an issue. If you have technical questions that any knowledgeable person could answer. so that they don't outsmart you. that Microsoft is now reverting on because of a serious bug. language or formatting. as well as Pankul Verma. Jeremy Lubich and Simon Hayes. This is particularly important if there are nonsysadmin users that own databases. Karl Jones. and is to be run by users with db_owner privileges. Generally.html 26-1-2012 . 1. Not the least I like to thank all you people who have pointed out typos and spelling errors. 2.Removed text in the section on Dynamic Search Conditions that referred to the new behaviour of OPTION (RECOMPILE) in SQL 2008. Anthony Faull. because it is mainly in application tasks. dynamic SQL is almost always a fair be careful to use quotename() when you build the SQL strings.. Gennadi Gretchkosiy. Umachandar Jaychandran. You can find the code by entering exec master. there are no permissions issues. And if the code is to be run by users with sysadmin privileges.sp_helptext I refer you to the source code for the popular (but undocumented) system procedure sp_who2. There are however. since the bug mentioned in the entry from 2009-02-14 has been fixed. you must of course take precaution against SQL injection. http://www. Here. please mail me at esquel@sommarskog. I like to briefly discuss code is for maintenance jobs.

and many sections rearranged. The examples of cases where (not) to use dynamic SQL have had an overhaul as well.html 26-1-2012 . Added a section on dynamic crosstab. 2004-02-08 – German translation now available. 2006-07-25 – Corrected syntax in example with cursor variable after comment from Anthony Faull. I also stress the importance of using parameterised statements for query-plan reuse. http://www. 2008-12-02 – Updated the article for SQL 2008. lots of old text dropped. Back to my home page. Added use of nvarchar(max) on SQL 2005 for quotestring and elsewhere. and I note that prefixing with dbo is essential for query-plan reuse. Minor language corrections. I'm now more strongly favouring sp_executesql over EXEC(). added a note on forced parameterisation and a demo of the performance penalty for failing to use parameterised Rewrote the section on Sales + @yymm tables and added one more approach. 2008-03-18 – Added a section on how to handle a dynamic column alias. 2006-12-27 – In the section Caching Query Plans. if not equally drastic. Modified description of first parameter to sp_executesql.Net. The article now also includes snippets for parameterised commands from VB6 and VB . resulting in lots of new text. 2005-04-17 – Added example of EXEC + sp_executesql with OUTPUT parameter. I'm now giving a very quick example of partitioned views for the sales + @yymm case. 2006-04-23 – Thoroughly reworked the article to cover SQL 2005 in full. 2008-06-06 – Added an example on how to deal with dynamic sort order in the section on ORDER BY. and I put more stress on SQL injection.The curse and blessings of dynamic SQL Page 36 of 36 2008-12-06 – Modified the section on dynamic crosstab to stress that pivot_sp is open to SQL injection. Various other minor changes. 2003-12-02 – Added example of using cursor variable with dynamic SQL.sommarskog.

Sign up to vote on this title
UsefulNot useful