This action might not be possible to undo. Are you sure you want to continue?
SAP HANA SQLScript Basics & Troubleshooting
Rich Heilman, HANA Product Management, SAP Labs LLC. Werner Steyn, Customer Solution Adoption, SAP Labs LLC. October, 2012
This presentation outlines our general product direction and should not be relied on in making a purchase decision. This presentation is not subject to your license agreement or any other agreement with SAP. SAP has no obligation to pursue any course of business outlined in this presentation or to develop or release any functionality mentioned in this presentation. This presentation and SAP's strategy and possible future developments are subject to change and may be changed by SAP at any time for any reason without notice. This document is provided without a warranty of any kind, either express or implied, including but not limited to, the implied warranties of merchantability, fitness for a particular purpose, or non-infringement. SAP assumes no responsibility for errors or omissions in this document, except if such damages were caused by SAP intentionally or grossly negligent.
© 2012 SAP AG. All rights reserved.
Introduction to SQLScript
Overview Exercise 1 - 3 R Language Integration Troubleshooting Tracing & Debug Information Exercise 4 - 6
Tips, Tricks & Best Practices
© 2012 SAP AG. All rights reserved.
Introduction to SQLScript
When it comes to data mining and statistic analysis. These extensions are keys to avoiding massive data copies to the application server and to leverage sophisticated parallel execution strategies of the database. aggregation. All rights reserved. like join. which provides enhanced control flow capabilities and is positioned to be more suitable for pushing complex parts of application logic to the database. SQLScript supports stored procedures.SQLScript Motivation & goals The set of SQL extensions for the SAP HANA database which allow developers to push data intensive logic into the database is called SQLScript. etc. SQLScript is not suitable for implementing complex algorithms. 5 . It can meet some simple requirement for reporting. © 2012 SAP AG.
All rights reserved. imperative logic Container for special-purpose operator logic • R script (Released in SPS4) © 2012 SAP AG. Built-In CalcEngine functions • Orchestration Logic including DDL. DML. assignment. 6 .SQLScript Motivation & goals Improve readability and structure of data-intensive logic • Pass results of one SQL statement to another • Break complex SQL into smaller chunks Bring data-intensive application logic close to database • Declarative Logic including SELECT queries.
7 . All rights reserved. If applications fetch data as sets of rows for processing on application level they will not benefit from these features © 2012 SAP AG. query optimization and parallel execution.SQLScript Why SQLScript? The main goal of SQLScript is to allow the execution of data intensive calculations inside SAP HANA There are two reasons why this is required to achieve the best performance: Moving calculations to the database layer eliminates the need to transfer large amounts of data from the database to the application Calculations need to be executed in the database layer to get the maximum benefit from SAP HANA features such as fast column operations.
SQL views have no parameters SQLScript supports local variables for intermediate results with implicitly defined types.SQLScript SQLScript advantages Compared to plain SQL queries. while a SQL query returns only one result set Complex functions can be decomposed into smaller functions. reuse and a better understandability by functional abstraction. it would be required to define globally visible views even for intermediate steps SQLScript has control logic such as if/else that is not available in SQL Increased performance via parallel processing © 2012 SAP AG. With standard SQL. However. This enables modular programming. standard SQL only allows the definition of SQL views. For structuring complex queries. 8 . SQLScript has the following advantages: Functions can return multiple results. All rights reserved.
All rights reserved. 9 . new model SQLScript is a collection of SQL extensions to push data-intensive logic into the DB layer Traditional Model: “Data to Code” Application Layer New Model: “Code to Data” Application Layer Code Massive data copies Bottle neck ! DB Layer Only transfer RESULTS DB Layer Code © 2012 SAP AG.SQLScript Traditional model vs.
Delete.SQLScript Procedure Logic Orchestration Logic (Read-Write procedures) Insert. Client Orchestration Logic Imperative Extension Declaritive Logic Functional Extension 10 . Update. DDL. DML statements IF/Else. loops and conditions Row based processing iterate through result sets Imperative logic is executed sequentially Not suitable for HIGH performance queries Can call Functional procedures Declarative Logic (Read-Only procedures) Utilized exclusively in Calculation Views Side effect free. Cursors. All rights reserved. no transactional support Suitable for mass amounts of data and performance Encouraged for Analytical programming High potential of parallelism and optimization Recommended to use CE Functions instead of SQL Unable to call Imperative procedures © 2012 SAP AG.
All rights reserved. books WHERE pub_id = pid AND pub_id = publisher AND crcy = :currency.Query Q4 FROM :big_pub_books GROUP BY year.SQLScript Code example BEGIN big_pub_ids = SELECT publisher FROM books -. SqlScript Compiler © 2012 SAP AG.Query Q2 FROM :big_pub_ids.Query Q3 FROM :big_pub_books GROUP BY publisher.Query Q1 GROUP BY publisher HAVING COUNT (isbn) > :cnt. price -. big_pub_books = SELECT title. output_pubs = SELECT SUM(price) -. END. publishers. output_year = SELECT SUM(price) -. 11 .
<TAB2>.SQLScript Parallel processing Select statements are executed in parallel unless: Local scalar parameters and variables are used in the procedure Read/Write procedures or DML/DDL operations are executed Imperative Logic is used within the procedure SQL statements are used that are not assigned to a variable The following 3 queries are processed in parallel: BEGIN – p_out1 = select * from <schema>. END. 12 .<TAB3>. – p_out2 = select * from <schema>.<TAB1>. – p_out3 = select * from <schema>. All rights reserved. – – © 2012 SAP AG.
C from "ATTRIBUTE_VIEW" out = SELECT A. [A. col_tab1 = CE_COLUMN_TABLE("COLUMN_TABLE1".D]). CE_CALC('midstr("D". C]). col_tab= CE_COLUMN_TABLE("COLUMN_TABLE").B. KEY2]. [KEY1. B.2. [A. B. B. "B". B. col_tab2 = SELECT A. 13 . D FROM "COLUMN_TABLE2". out = CE_AGGREGATION( (col_tab. SUM(D) from "COLTAB1" INNER JOIN "COLTAB2" WHERE "COLTAB1". ["A". [A. B. [A. D]) LEFT OUTER JOIN out = CE_LEFT_OUTER_JOIN("COLTAB1". [A.:col_tab2). Y."KEY1" AND "COLTAB1". [KEY1. C."KEY2" = "COLTAB2". D FROM "COLUMN_TABLE1". B.[A. "C". B. col_tab= CE_COLUMN_TABLE("COLUMN_TABLE"). col_tab2 = CE_COLUMN_TABLE("COLUMN_TABLE2". out = CE_JOIN("COLTAB1"."COLTAB2". Y.[A. B.5)'. ' "B" = ''value'' AND "C" = ''value'' ').C.5) FROM "COLUMN_TABLE" col_tab1 = SELECT A.2. C.D]). C]). B. C]."KEY2" out = SELECT A. B. C out = SELECT A. SUM(D) from "COLTAB1" LEFT OUTER JOIN "COLTAB2" WHERE "COLTAB1". D]) SQL Expressions UNION ALL proj_tab = CE_COLUMN_TABLE("COLUMN_TABLE")."KEY2" out = SELECT A."KEY1" AND "COLTAB1". SUBSTRING(D. Y. Y. B. GROUP BY INNER JOIN out = SELECT A. out = CE_UNION_ALL(:col_tab1. C]) out = CE_JOIN_VIEW("ATTRIBUTE_VIEW". All rights reserved. C]) out = CE_OLAP_VIEW("ANALYTIC_VIEW". out = CE_PROJECTION(col_tab. C from "COLUMN_TABLE" out = SELECT A. string) ]). out = CE_PROJECTION( :proj_tab. © 2012 SAP AG."KEY1" = "COLTAB2". out = SELECT * FROM :col_tab1 UNION ALL SELECT * FROM :col_tab2. C. C. B."KEY2" = "COLTAB2". B. [A. [A. SUM(D).C. B. B.SQLScript Built in functions SQL SELECT on column table SELECT on attribute view SELECT on olap view WHERE HAVING out = SELECT A. SUM(D) FROM"COLUMN_TABLE" GROUP BY A."KEY1" = "COLTAB2". SUM(D) from "ANALYTIC_VIEW" WHERE B = 'value' AND C = 'value' CE-Built In Function out = CE_COLUMN_TABLE("COLUMN_TABLE".B. KEY2]. C out = SELECT A. B. B. C. B."COLTAB2". SUM(D) from "ANALYTIC_VIEW" GROUP BY A. B. C.
14 . All rights reserved.SQLScript Built in function code example Built in functions should be used exclusively where possible Calculation Engine functions should not be mixed with standard SQL statements Client queries can be well optimized and parallelized by the engine Usually much better performance results than calculation view via SQL Preferred © 2012 SAP AG.
15 . All rights reserved.SQLScript Table types Allows for the definition of new Table Types Similar to a database table but do not have an instance Used to define procedure parameters Created when the procedure is activated © 2012 SAP AG.
SQLScript Integration with Information Modeler Both Read-Only and Read/Write procedures can be created Following will apply for the procedures created in the information modeler IN (Input) parameters can be of scalar or table type OUT (Output) parameters must have table type Tables types required for the signature are generated automatically Activated procedure can be called by other procedures Name of procedure: “_SYS_BIC”. 16 . All rights reserved.”<package-name>/<proc>” © 2012 SAP AG.
All rights reserved.SQLScript Integration with Calculation Views SQL or SQLScript required to create script-based Calculation Views Write SQL Select statements against existing raw tables or Column Stores (preferred) Define output structure. 17 . activation creates table type based on output parameter © 2012 SAP AG.
Exercise 1 .3 .
R Language Integration .
S2: RA Becker. TJ Hastie 1998 .S3: JM Chambers. Homepage: http://cm. © 2012 SAP AG. A Wilks 1992 . Seattle WA. of Statistics of U of Auckland. 20 .R Language The history: R. SPIN. etc. Implemented with C and Fortran. Product name: S-plus. JM Chambers. S and S-plus S Language: an interactive environment for data analysis developed at Bell Laboratories since 1976 1988 .com/cm/ms/departments/sia/S/history. Java.S4: JM Chambers Won ACM System Award’98 (along with Unix.bell-labs.) Exclusively licensed by AT&T/Lucent to Insightful Corporation. All rights reserved. New Zealand during 1990s.html R Language: Open Source S under GNU GPL Initially written by Ross Ihaka and Robert Gentleman at Dep. 15 people with access to common CVS archive. WWW. Since 1997: international R-core team of ca.
.org/ The packages cover wide range topics Cluster Analysis & Finite Mixture Models Probability Distributions Computational Econometrics Empirical Finance Statistical Genetics Graphic Displays.r-project. http://www. All rights reserved.R Language What is open source R An open source software language and environment for statistical computing and graphics with over 3000 add-on packages. Graphic Devices & Visualisation Machine Learning & Statistical Learning Medical Image Analysis Multivariate Statistics Natural Language Processing Statistics for the Social Sciences Time Series Analysis 21 © 2012 SAP AG. Dynamic Graphics.
subroutines. All rights reserved.R Language What R supports Data handling and storage: numeric and textual Matrix algebra Hash tables and regular expressions High-level data analytic and statistical functions graphics Programming language: loops. OO © 2012 SAP AG. Branching. 22 .
R Language Code examples © 2012 SAP AG. 23 . All rights reserved.
Parallelism.) 2 Run the R scripts RClient 1 Send data and R script SAP RHANA Package 3 Get back the result from R to SAP HANA R Integration Open Source R NewDB © 2012 SAP AG. statistics. 24 . All rights reserved. etc. Engine Join OP ROP OLAP OP OpenSource R Space R External Packages (Forecasting.R Language Integration with HANA database Sample Codes in SAP HANA SQLScript NewDB Space Calc.
26 .Tracing Configuration All trace configuration is under the “Trace Configuration” tab in the Administration perspective. © 2012 SAP AG. All rights reserved.
INFO or DEBUG. This is relevant for all aspect of SQLScript the relate to L For each of the components one can set the trace level to NONE. © 2012 SAP AG. FATAL. ERROR. 27 . All rights reserved.Tracing Global database trace levels Administration Perspective Trace Configuration Global Database Trace sqlscript: information related to the SQLScript compiler ceinstantiate: information collected during instantiation of a calculation model calcengine: information collected by the calculation engine at runtime. WARNING. llang: information collected by the L component.
All rights reserved. 28 .Tracing SQL trace SQL-Trace: Administration Perspective Trace Configuration SQL Trace Trace level: ALL or ALL WITH RESULT Trace File: configure trace file name User Filter: filter on database or application user © 2012 SAP AG.
All rights reserved.Tracing Performance tracing Administration Perspective Trace Configuration Performance Trace Start/Stop Trace file name Filter by database user © 2012 SAP AG. 29 .
30 .Tracing Intermediate results TRACE Operator Syntax: y = TRACE(:x). Analyze Traces using table SQLSCRIPT_TRACE select * from sqlscript_trace. select * from <temp_table>. All rights reserved. call truncate_sqlscript_trace. Trace intermediate result bound to x and pass unchanged via variable y Certain optimizations will not be performed due to the TRACE operator lt_tab = trace(:lt_tab). © 2012 SAP AG.
procedure_name = ‘GETOUTPUT‘ order by line. Understand dataflow structure and variable renaming: select * from SYS.PROCEDURE_MAPPING as m where df. 31 . All rights reserved.column. SYS.Debug Information in SQLScript Tracing for SQLScript compiler Compile with debug information: ALTER PROCEDURE getOutput RECOMPILE WITH PLAN.dataflow_name = m. © 2012 SAP AG.PROCEDURE_DATAFLOWS as df.dataflow_name and df.
All rights reserved. 'EUR'. Trace Mapping of SQLScript plan to Calc-Models: select * from SYS.M_CE_DEBUG_NODE_MAPPING. NULL) WITH OVERVIEW IN DEBUG MODE. NULL.Debug Information in SQLScript Tracing SQLScript invocation Execute in Debug-Mode to generate debug information CALL getOutput(0. 32 . © 2012 SAP AG.
Use a tools like http://jsonviewer.stack.hu/ to analyze CalcModels © 2012 SAP AG.Debug Information in SQLScript Tracing SQLScript invocation Trace Calc-Models generated during instantiation and statements executed select * from SYS.M_CE_DEBUG_JSONS. All rights reserved. 33 .
Exercise 4 .6 .
Tips. Tricks & Best Practices .
refrain from row-level processing © 2012 SAP AG. All rights reserved. 36 . tricks & best practices Column engine is best optimized for OLAP queries.Tips. avoid operations that are not natively supported Push down queries by exploiting operations and calculations supported by the column engine Reduce complexity of procedures by breaking into smaller statements Row-based ABAP programs re-written in SQLScript will not necessarily run faster without re-design Do calculations on the smallest subset of data possible.
tricks & best practices Avoid mixing Calculation Engine(CE) functions and SQL statements within a single procedure Avoid using cursors.Tips. 37 . Replace by (a flow of) SQL statements for better optimization Refrain from writing SQL statements without filters such a WHERE clauses and GROUP BY clauses Refrain from returning large (millions) record sets out of stored procedures and or SQL statements Model business logic within Analytical & Calculation views instead of writing procedures when possible © 2012 SAP AG. All rights reserved.
38 . All rights reserved. avoid dependent paths so execution can be in parallel © 2012 SAP AG.Tips. tricks & best practices Query runtime object (Analytical & Calculation) views first before querying RAW column tables Refrain from joining Modeled views without supplying WHERE clauses for each input source Avoid dynamic SQL when high query performance is a requirement Employ the EXPLAIN PLAN facility to investigate the performance impact of different SQL queries SQLScript is translated into a dataflow graph.
ABAP application expects client specific result Solution: Encapsulation via views – Create a view per master data table – Access views instead of tables Constants Problem: You don’t want hard coded constants in coding – Local variables can be used. 39 . tricks & best practices Client Handling Problem: SQL-based table access is client agnostic – however.Tips. All rights reserved. but still hard coded in the procedure Solution: Create special table that holds constants – | constant’s key | int value | double value | … – Use in coding © 2012 SAP AG.
40 . ReadWrite -> up in call stack ReadWrite procedures are executed sequentially (very negative impact on performance) Problem: If Proc A calls Proc B and Proc B is a ReadWrite procedure. then Proc A must be executed sequentially as well. tricks & best practices ReadOnly vs.Tips. Solution: Watch out to have ReadWrite procedures high in the call stack UNION ALL SQL UNION ALL currently is executed on row store (very negative impact on performance) Unfortunately very frequent operation when coding business logic (ends branching in data flows) Workaround: use CE_UNION_ALL © 2012 SAP AG. All rights reserved.
analytics. Combining SAP HANA expertise with deep application & industry knowledge. Accelerate existing applications Extend HANA-based solutions Your Advantage Business Success Combining the speed of SAP HANA with your unique processes to build differentiating solutions that help you to stay ahead of the competition © 2012 SAP AG. All rights reserved. and leveraging SAP development methodologies 41 .Customer-Specific Solutions Powered by SAP HANA In-Memory Computing with SAP Custom Development Our Offerings Our Experience Development of customer-specific solutions Based on SAP HANA Integrated into your process landscape For transactions. Fast Time to Value Delivering your solutions fast with SAP developers who know SAP and latest technologies best Protect Investment Ensuring your solution runs smoothly over time by offering maintenance. optimization Global network of experienced developers. aligning with SAP future product and technology roadmap. prediction.
Future Enhancements .
SQLScript Future enhancements New Editor File based editor. keywords and error checking Set break-points © 2012 SAP AG. team provider approach Code hints Syntax highlighting. All rights reserved. 43 .
All rights reserved.SQLScript Future enhancements New Debugger Run SQLScript in debug mode Breakpoint management Break on break-points Basic step debugging Variable evaluation © 2012 SAP AG. 44 .
45 .com/community/hana-in-memory https://www.com/ SAP Education and Certification Opportunities https://training. All rights reserved.sap.experiencesaphana.com/online © 2012 SAP AG.com/us/en/search?query=SAP+HANA Watch SAP TechEd Online www.sap.sapteched.Further Information SAP Public Web http://scn.
Thanks for attending this SAP TechEd session.Feedback Please complete your session evaluation for CD166. .
iPad. SAP NetWeaver. Google Updater. Gmail. Google Mail. Intelligent Miner. No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP AG. and Reader are trademarks or registered trademarks of Adobe Systems Incorporated in the United States and other countries. BlackBerry Bold. and BlackBerry App World are trademarks or registered trademarks of Research in Motion Limited. iAnywhere. BlackBerry Pearl. POWER6. Some software products marketed by SAP AG and its distributors contain proprietary software components of other software vendors. iPhone. iTunes. Data contained in this document serves informational purposes only. Crossgate. Wi-Fi is a registered trademark of Wi-Fi Alliance. World Wide Web Consortium. OS/390. Web Intelligence. System p. PowerPoint. X/Open. YouTube. HACMP. POWER7. RACF. Citrix. and Smarter Planet are trademarks or registered trademarks of IBM Corporation. Google Voice. SAP. copied. SAP BusinessObjects Explorer. and other Business Objects products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Business Objects Software Ltd. R/3. BBM. Google Data API. Motorola is a registered trademark of Motorola Trademark Holdings LLC. POWER6+. Google Mobile Updater. BlackBerry. SQL Anywhere. Storwize. B2B 360°. XML. and B2B 360° Services are registered trademarks of Crossgate AG in Germany and other countries. System p5. XIV. Google Checkout. Power Systems. SAP HANA. Dalvik and Android are trademarks or registered trademarks of Google Inc. DB2. DB2 Connect. System i5. Linux is the registered trademark of Linus Torvalds in the United States and other countries. PowerVM. POWER. Power Architecture. Google Store. Adobe. Sybase and Adaptive Server. 47 . VideoFrame. Informix. Sybase is an SAP company. Retina. Tivoli. Apple. iPhoto. RETAIN. WinFrame. Computop is a registered trademark of Computop Wirtschaftsinformatik GmbH. BlackBerry PlayBook. and W3C are trademarks or registered trademarks of W3C®. Business Objects and the Business Objects logo. pureScale. BlackBerry Storm. and MultiWin are trademarks or registered trademarks of Citrix Systems Inc. IOS is a registered trademark of Cisco Systems Inc. or transmitted in any form or for any purpose without the express prior written permission of SAP AG. Xcelsius. System i. Windows. Redbooks. Crystal Reports. BusinessObjects. Objective-C. BlackBerry Curve. PowerPC. the Adobe logo. Acrobat. Bluetooth is a registered trademark of Bluetooth SIG Inc. WebSphere. z/OS. Google App Engine. INTERMEC is a registered trademark of Intermec Technologies Corporation. Massachusetts Institute of Technology. All rights reserved. z10. Sybase 365. XHTML. The information contained herein may be changed without prior notice. Crossgate is an SAP company. m@gic EDDY. MetaFrame. ICA. and other Sybase products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of Sybase Inc. ByDesign. PartnerEdge. and Xcode are trademarks or registered trademarks of Apple Inc. PowerHA. System z10. Duet. The information in this document is proprietary to SAP. Excel. PostScript. iPod. IBM. GPFS. System Storage. OS/2. No part of this document may be reproduced. Multi-Touch. App Store. Business Objects is an SAP company. Silverlight. AIX. All other product and service names mentioned are the trademarks of their respective companies. National product specifications may vary. Google Mobile Ads. System z. OSF/1. Program Neighborhood. Microsoft. Siri. BlackBerry Torch. System x. Outlook. RIM. Google Sync. and Visual Studio are registered trademarks of Microsoft Corporation. StreamWork. Crystal Decisions. HTML. Oracle and Java are registered trademarks of Oracle and its affiliates. Safari. DB2 Universal Database. All rights reserved. and Motif are registered trademarks of the Open Group.© 2012 SAP AG. Google Apps. and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP AG in Germany and other countries. © 2012 SAP AG. iBooks. z/VM. BlackBerry Storm2. UNIX. BladeCenter. Google Maps. zEnterprise. Google Mobile.