You are on page 1of 5

NESUG 2010

Coders' Corner

LOG CHECKING: What to check and why? Sridhar R Dodlapati, i3 Statprobe, Basking Ridge, NJ Kiran Kumar Karidi, Novartis, Florham Park, NJ Mahipal R Vanam, EMD Serono, Rockland, MA
ABSTRACT
Not all the messages in the log that pose potential problems get enough emphasis from all the programmers. Every SAS programmer need to know the significance of some of the important messages in the log to avoid potential danger of ignoring them. After executing our SAS® programs, it is obvious that we all check the corresponding log files to see if it has “ERROR” and/or “WARNING” messages listed in the log, as these are considered serious problems. However there are many other important messages that are listed in “NOTE” section that could pose potential issues such as “Uninitialized”, “Merge statement has more than one dataset with repeats of BY values” … etc, but many programmers ignore them, not realizing the magnitude of the problems that can be caused, if these messages were not taken care off.

INTRODUCTION
The various messages in the log are shown under ERROR, WARNING and NOTE sections produced during compilation or execution time. They could be related to syntax, improper usage of SAS elements, data issues and processing. Most often the ERROR and WARNING messages get enough attention by a SAS programmer, but some of the most important messages that are listed as NOTE messages are ignored. The probable reason is that there are some hundreds of different types of NOTE messages, and most of them are just informative and not considered as important, and programmer gets overwhelmed to check all of them. But there are certain NOTE messages that ought to be checked as they are as equally important as ERROR / WARNING messages and indicate potential issues in the program code or logic or data. In this paper we are listing some of the very important messages in the log that need to be checked thoroughly. Also the importance of checking the last modified date and time of the SAS program and the corresponding log file is explained. Upon looking for the issues that one need to check in the log, there are many (in hundreds), but at least the ones that are mentioned in this paper are worth checking, every time a SAS program is run. Apart from letting the user know the syntax mistakes in the program, the log file also provides vital information about the correctness of the logic / algorithm used in the SAS program. It is important that while updating the programs based on the information provided in the log files, one has to start working from the top of the program (top to bottom approach).

WHAT TO CHECK AND WHY?
1. LAST MODIFIED DATE & TIME OF SAS FILE AND CORRESPONDING LOG FILE: One of the most important points that need to be checked, even before checking the contents of a log file is to check the last modified date and time of the SAS and corresponding LOG file. I.e. the log file creation date time should be always greater than or equal to the corresponding SAS program last modified date time stamp. If any SAS program is updated for any reason, it has to be rerun to recreate the corresponding deliverable or whatever the program was supposed to do. For this, we need to get the information from the operating system, but not from the log file itself. In WINDOWS environment irrespective of how old the files are we can always get the last modified date time stamp of the files by using the operating system commands, directly (calling the operating system commands) or indirectly (using SAS functions which in turn will call the operating system commands). Calling the operating system commands directly is more efficient. Below is the command for WINDOWS operating system: %let file=c:\logcheck\test.sas; filename foo pipe "dir &file /t:w /a:-d"; /t:w indicates you want a time field 'T', of type Last Modified 'W' /a:-d return information for files only, not directories

1

• Macro-related: Occurs when the macro facility is used incorrectly during macro compilation or execution time. With out the proper resolution of the macro or macro variable. ERROR: These are MUST BE FIXED serious messages that have to be taken care off in order to make the program run. then the time (hour and minute) will be replaced by the year. causing the program to abort. SAS issues a message saying that it is not resolved. but the element might not be valid for a particular usage during compilation time. Then SAS creates the variable. If the files are older than six months. print " -. • Semantic: Occurs when the language element is correct. this message is as serious as an error message.. NOT RESOLVED: When a macro or macro variable does not get resolved. } 2. correct results are not assured. • Execution-time: Occurs when SAS attempts to execute a program and execution fails during execution time. as the name itself indicates. in UNIX environment if the files are older than six months. they are considered as MAY or MAY NOT serious messages implying that they may or may not influence the program to run correctly. or makes sure that the program runs correctly even if they are present. DATA or PROC step compilation or execution time. NOTE: Most of the time the NOTE messages are just informative and not important. is uninitialized" message. SAS prints the "variable . otherwise the program will abort. 3.sas". • Data-related: Occurs when data values are invalid during execution time. With out fixing these messages. 6. In such cases we need to use the PERL or other script language to extract the complete date time stamp. 4. There are five types of errors as follows: • Syntax: Occurs when programming statements do not conform to the rules of the SAS language during compilation time. program does not even get executed completely.NESUG 2010 Coders' Corner Below is the command for UNIX operating system: filename foo pipe "ls -g -o ~/test. print "\n" . The following belong to this category: 5. it is imperative that the programmer must put every effort to clear it from the log. It’s nice that SAS runs the DATA step. Hence when this message appears in the log. printf scalar localtime $sb->mtime . but not always. foreach $ab (@ARGV) { $sb = stat($ab) . but some of the NOTE messages should be considered as serious as ERROR / WARNING messages because of the potential problems they can cause. but you probably don’t want the variable to have missing values for all observations.$ab". Because of this uncertain nature. FATAL: There are many different scenarios that can cause this message in the log and. SAS prints the variable-not-found message and does not run the procedure at all. A more serious problem ensues when SAS is unable to find a variable in a PROC step. then we can not get the complete last modified date time stamp of the files by using the operating system commands directly or indirectly. these are also serious messages most of the time. sets its values to missing for all observations.. and runs the DATA step. ls is the UNIX equivalent of DIR -g specifies not to print the file's owner -o specifies not to print the file's group However. UNINITIALIZED: When SAS is unable to find a variable in a DATA step. Below is the PERL script to get the last modified date & time: #!/usr/bin/perl use File::stat. 2 . WARNING: Similar to ERROR messages.

one way is. this message will be generated in the log. when SAS expects none of the scenarios/events and encounters one or more. 14. MISSING VALUES WERE GENERATED: The missing-values-were-generated note tells you that SAS was unable to compute the value of a new variable because of existing missing values in the data. INSUFFICIENT: Occur when SAS encounter an out-of-resources condition. User should be wary of using the merge statement in data step for many to many merge. because when a dataset is created we don't expect it to have zero observations. A very complicated method for getting the correct results using the merge statement in data step for a many to many merge is available. "Illegal Instruction In Task" etc. When these conditions occur. NO OBSERVATIONS: Similar to the previous message. This may not indicate a problem. SAS sets the problematic variable to missing for that observation and then prints a detailed message. INVALID ARGUMENT TO FUNCTION: Correct arguments with appropriate attributes need to be passed to any function for their normal functionality.e. Here the SAS uses the default informat for the conversion. 16. "illegal mathematical operations" such as taking the logarithm of zero or trying to take the square root of a negative number. SAS will convert the data from one type to the other. DATA SET DOES NOT EXIST: When the specified data set does not exist. 8. so there are no surprises. 12. 10. it can come back to haunt you at a later time when the variable that you think is numeric is now character or vice versa. INVALID DATA: Whenever SAS encounters invalid data while reading with an INPUT statement. ILLEGAL: There are many scenarios that can lead to this note in the log. 17. or insufficient memory for a SAS procedure to complete. which could be undesirable some times. SAS attempts to find resources for current use. or to free the memory in which macro variables are stored. 15. CHARACTER VALUES HAVE BEEN CONVERTED TO NUMERIC: For the same reason mentioned above. It’s nice that SAS tries to fix the problem. such as a full disk. AT LEAST: There are two scenarios that can lead to this note in the log. If a variable needs to be converted. but when the input dataset that is read has zero observations. SAS issues this note to alert us and avoid any unintended results. then SAS issues this note in the log to alert us. 13. this messages will appear in the log as a cautious note to warn us about the potential incorrect results. HAS 0 OBSERVATIONS: SAS gives this note in the log whenever there are zero observations in the data set that is newly created. I. DIVISION BY ZERO: 3 . run the program anyway. the user should do it explicitly. SAS uses the default format for the conversion. 18. instead a PROC SQL can be used to achieve it. For example.NESUG 2010 Coders' Corner 7. but this doesn’t mean that it can be ignored. SAS may ask the user for permission to delete temporary data sets that might no longer be needed. 19. MERGE STATEMENT HAS MORE THAN ONE DATA SET WITH REPEATS OF BY VALUES: When merge statement is used in data step to merge datasets for MANY to MANY merge. Another way is. The missing-values-were-generated note tells that SAS automatically assigned missing values for us. NUMERIC VALUES HAVE BEEN CONVERTED TO CHARACTER: If user accidentally mixes numeric and character variables. Some of them are "illegal arguments to functions". one would not want to see this message in the log. If you let SAS convert your variables. Occasionally programmers get invalid-data messages because they are trying to read non-printable characters such as carriage returns. and print a note stating that the "values have been converted". when SAS expects more than one scenarios/events and encounters none. otherwise the simple straight forward way of using the merge statement almost all the time yields incorrect results. but it warrants an investigation. which could be undesirable some times. 11. 9.

• • • Keeping them in mind. • Impractical because. or correct mathematical operations but SAS does not have the capability of performing them. OUTSIDE THE AXIS RANGE: When the data points / values that need to be plotted in the graph are outside the specified axis range. making the log checking an automated process is always preferred over manual checking. only if our intention is to read the data that flows over into several lines of input. Inefficient as it consumes lot of resources both time and effort. they will not appear in the graph and this note will be printed to the log. otherwise this will result in incorrect data reading. 21. then one has to look into the log for this message. Specifying the correct overall Format width and appropriate number of decimal places is essential to having SAS present your data the way you want them to appear. As this is not sufficient. This paper does not contain a complete list of messages and the user / programmer will need to add additional messages depending on their requirement. We all know that a perfectly normal log file does not mean that the program has worked correctly.D FORMAT WAS TOO SMALL: The width of the specified format is not wide enough to contain the complete value for one or more values. For example. if the log file contains any of the messages. Log checking will not guarantee the accuracy of the results. we can delete the duplicate records with the NODUPKEY option. each SAS program is usually re-run multiple times in its lifetime forcing us to check the same log file again and again. this note will be printed in the log immediately after that error message. 4 . as the program can hinder logical mistakes. but increases the accuracy of the validation process and assures that the log file has been checked for the specified messages. No proof or hard evidence is left to indicate that one has checked a log file manually. Notice that when decimal specifications are supplied that are smaller than the internal decimal value. W. OBSERVATIONS WITH DUPLICATE KEY VALUES WERE DELETED: By using the proc sort. SAS WENT TO A NEW LINE WHEN INPUT STATEMENT REACHED PAST THE END OF A LINE: When the INPUT statement tries to read past the end of the current input data record. unless otherwise the intention is to do a blind merge which does not require a by statement. 24. Error prone as there is always a possibility of oversight of some of the messages. 22. and one should avoid this in the programming. or even if they are expected but interested in knowing how many are deleted.NESUG 2010 Coders' Corner Division by ZERO results in infinite. it then moves the input pointer to column 1 of the next record to read the remaining values. as computers do not have the capabilities to do such calculations. 25. MATHEMATICAL OPERATIONS COULD NOT BE PERFORMED: When the SAS encounters incorrect/impossible mathematical operations. When the duplicate records are not expected and still one wants to make sure that they don't exist. then this message will appear. each log file could be huge and there could be many log files. The opposite can be proved. But this is okay. AUTOMATION Checking the log file for various messages manually is not a good idea for the following reasons. 20. the Format “rounds up” the output to the nearest specified decimal place. Some times the log files seem to be clear with out any messages but the program/results can still be defective. but almost all the times this yields erroneous results. 23. if a by statement is missed after merge statement the log does not show any messages indicating this. THE SAS SYSTEM STOPPED PROCESSING THIS STEP BECAUSE OF ERRORS: When the SAS encounters a fatal error.

"A Utility Program for Checking SAS® Log Files" (SUGI 27. SAS programmers must pay attention to these messages / information in the log and always must check them keeping in mind the consequences of ignoring them. Please contact the authors at: Sridhar R Dodlapati i3 Statprobe 131 Morristown Road Basking Ridge.Check your .V@gmail. one would not want to see them in their log file.Dodlapati@i3statprobe. CONTACT INFORMATION We appreciate your valuable comments and suggestions. NJ 07920 Sridhar. however there are certain messages and information that must be checked to make sure that there are no / minimal issues in the program or data.LOG and . MA Mahipal. REFERENCES SAS online documentation Smoak Carey G. product or service names are registered trademarks or trademarks of SAS Institute Inc. After knowing the trouble they can cause.com 5 . ® indicates USA registration. in the USA and other countries.com Kiran Karidi Novartis 180 Park Avenue Florham Park.Karidi@novartis. NJ 07932 Kiran.LST dates from Within SAS® using Operating System Commands" ACKNOWLEDGMENTS SAS and all other SAS Institute Inc. considering the huge volume of the log files.SAS.NESUG 2010 Coders' Corner CONCLUSION It is not practical to check all the messages in SAS log files every time they were run. Other brand and product names are registered trademarks or trademarks of their respective companies. "You be Your own QC Cop . . Paper 96-27) Chakravarthy Venky.com Mahipal Vanam EMD Serono Rackland.