You are on page 1of 271


txt Page 1 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

++ Basic UNIX Tutorials
Quick Unix Tutorial (
UNIX Tutorial for Beginners (
Basic Introduction to UNIX/linux

++ Intermediate UNIX Tutorials
UNIXhelp for Users (
The CERN Unix User Guide (

++ Advanced UNIX Tutorials
Advanced UNIX Tutorial ( (150 KB PDF
Unix System Administration Independent Learning (
Unix System Administration (
Administrating Peripherals (

++ C/C++ Programming Tutorials
Introduction to C Programming (
Programming in C/UNIX System Calls and Subroutines using C
C++ tutorial for C users (
Compiling "C" And "C++" Programs On Unix Systems - gcc/g++
Building And Using Static And Shared "C" Libraries
Debugging "C" And "C++" Programs Using "gdb"
How do I use gdb? (")
RMS's gdb Tutorial (
Automating Program Compilation - Writing Makefiles
ANSI C for Programmers on UNIX Systems
Debugging with GDB (

++ Advanced Programming Tutorials
UNIX Systems Programming I & II (
Network programming under Unix systems
Beej's Guide to Network Programming (
Getting Started With Posix Threads (
Multi-Threaded Programming With POSIX Threads
Serial Programming Guide for POSIX Operating Systems
Guide to Realtime Programming
Which is better, static or dynamic linking?
Building And Using Static And Shared "C" Libraries

++ Very Advanced Programming Tutorials
Linux Assembly HOWTO (
Linux Assembly Tutorial Step-by-Step Guide (
adb Tutorial (
Writing Device Drivers: Tutorial
(Digital Unix)

Unix_tricks.txt Page 2 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

Device Driver Tutorial ( (SunOS)
A Whirlwind Tutorial on Creating Really Teensy ELF Executables for Linux

++ Shell Scripting Tutorials
Sh - the Bourne Shell (
Csh (
Regular Expressions (
Shell Programming (
Linux Shell Scripting Tutorial v1.05r3 (
The UNIX Bourne Shell (
Bash Reference Manual (
Korn Shell (ksh) Programming (

++ Advanced Shell Scripting Tutorials
UNIX shell scripting with sh/ksh (
Linux I/O Redirection (
Advanced Bash-Scripting Guide (
Common Desktop Environment: Desktop KornShell User's Guide

++ Perl Tutorials
Perl in 20 pages -- A guide to Perl 5 for C/C++, awk, and shell programmers
Perl Tutorial ( (Perl 4)
Introduction to Perl (
The Perl You Need to Know (
(Perl 4, aimed towards web/cgi)
introduction to Perl 5 for web developers (
An Introduction to Perl's XML::XSLT module

++ Version Control Systems
Small tutorial on SCCS (
SCCS Tutorial (
Very Quick RCS Tutorial (
Revision Control System (
Sys Admin File Revision Control with RCS (
GNU RCS Tutorial (
Concurrent Versions System (
CVS Tutorial (
CVS Tutorial ( (nearly identical to
above link)
Open Source Development with CVS, 3rd Edition ( (An
on-line book)

++ Individual Command Tutorials
Mastering the VI editor (
List of Commands for vi - An Unix Editor (
Using find to locate files
Find (
Using grep, fgrep, and egrep to search for strings of words
Getting started with awk (
An Awk Tutorial (
Awk (
sed tutorial (
Sed - An Introduction (
A collection of sed tutorials (

Unix_tricks.txt Page 3 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

How to set up sar (

++ X Windows Tutorials
Introduction To The X-Window System (
(102 KB PDF file)
X Windows/Motif Programming (

++ Networking Tutorials
An Overview of TCP/IP Protocols and the Internet (
Networking Tutorial (
Using Samba (
HTML Code Tutorial (
An Introduction To HTML (

++ Other Tutorials
Computer Numbering Formats (
Using Java, 2nd Edition (
SQL Tutorial (
Celeste's Tutorial on Solaris 2.x Modems and Terminals
( -- Website design, promotion, programming and revenue making
ProgrammingTutorials (
Tutorial Maniacs ( (Mostly Windows tutorials)
Programming Texts and Tutorials ( (The Mother

locate a file

To quickly find a specific file, or group of files, use the 'locate' command.

Find all files on your Mac called 'my_letter', or which have 'my_letter' as a part of their name:

locate my_letter

Find all files with an extension of '.txt':

locate .txt

'locate' does not search the file system, but uses a database of file names so the search is very

update locate database

The 'locate' command searches a file name database that is updated at 4:30 am every Saturday
morning (if your Mac is switched on!). Files created more recently than the last update will not
be found. Deleted files will be erroneously found.

To manually update the data base issue the following command:

sudo /usr/libexec/locate.updatedb

giving your administrator password.

find a file

Unix_tricks.txt Page 4 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

To search for a specific file use the 'find' command. 'find' does a live search on the file
system, unlike 'locate' which searches a database.

Find all files in your home directory called 'my_letter':

find ~ -name my_letter

To search the current directory instead, replace '~' with '.'. To search the entire file system
use '/'.

find a group of files

To search for a group of files with a common prefix or ending, use 'find' with wildcards in the
filename. Enclose the name in quotes.

Find all files with extension text in the current directory:

find . -name "*.txt"

Find all files starting with 'temp' searching the whole file system:

find / -name "temp*"

find big and small files

The 'find' command can be used to search for files equal to, greater than, or less than a given
size. The size is given as the number of 512 byte blocks. If preceded with '+' this means greater
than the given size, '-' means less than.

To find all files in your home directory bigger than 1 Meg use:

find ~ -size +2048

The size 2048 is obtained from: 1 Meg divided by the block size of 512.

To find all empty files in the current directory use:

find . -size -1

man basics

If you are not sure how to use a Unix command (such as 'ls') read its manual page using:

man ls

And to find out how to use 'man' itself...

man man

The manual contains other useful information besides commands - try these:

man hier
man ascii

search by keyword

Unix_tricks.txt Page 5 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

If you need to do something but cannot think of a Unix command that might help, locate a suitable
command by keyword lookup. For example, commands that may change the owner of a file can be
located with either:

man -k owner
apropos owner

the 8 man sections

The Unix manual is divided into 8 sections:

* (1) user commands
* (8) system commands
* (5) configuration Files

less useful:

* (6) terminal games - empty :-(
* (7) miscellaneous
* (2,3,4) library calls for programmers

Find out about each section with:

man <section number> intro

Unfortunately, 'intro' is lacking detail in OS X. Instead, list section commands with (for
section 1):

man -k "(1)"

'files' and 'see also'

When reading a manual page, look at the 'FILES' and 'SEE ALSO' sections at the end. 'FILES' lists
configuration files associated with the command. 'SEE ALSO' lists similar commands and is a good
way to learn more Unix commands.

Often, information about the format of a file listed in 'FILES' can be got by:

man name-of-file
man 5 name-of-file

the man pager

The environment variable 'PAGER' tells 'man' the command it should use to display its output. The
default is 'more'. Read the manual page for 'more' in order to find out how to better control the
output from 'man'.

If you wish to use a different pager, such as 'less', set the environment variable with:

setenv PAGER less

To get 'man' to write its output to a file, and also remove the formatting, type:

man the-command | col -b > a-file

and 'bug'.txt' The character '?' tells the shell to match any one character. as in: ls [mt]*day[0-9] Match filenames that start with 'm' or 't'.Unix_tricks. use: find . [a-z0-9] matches any lowercase letter or digit. -name \*. -name "*. A leading '. but not 'bfg' or 'bags'. Use either quotes or backslash. 'testa'. you must prevent the shell from globbing them first by escaping those characters.*'.html . and 'test$'. It will not remove 'test' or 'test12'. 2008 3:20:00 PM star and query So what is globbing? Globbing is where you specify an ambiguous filename and the shell finds all those files (in the given directory) that match it. [0-9] matches any digit. -name '*. rm * will remove ALL files in the current directory. glob patterns may be combined.txt will remove all files with the extension '. and end in 'day' followed by one digit. separate a pair of characters with '-'. The globbing character '*' tells the shell to match 0 or more characters . escaping If you wish to pass any of the characters * ? [ ] to a command. 'bog'. rm test? will remove 'test1'.html' find . 'beg'. rm *. so filenames that begin with dot must be removed with 'rm . [a-z] matches any lowercase letter.html" find . ls b[aeiou]g will list 'bag'. that's not quite true. The shell processes matched filenames in alphanumeric order. May 27. To pass '*' to the 'find' command.txt Page 6 of 271 Printed: Tuesday. ranges To match any one of a range of characters. 'big'.' must be matched explicitly. Actually. have any number of intervening characters. selected characters Enclose characters in '[ ]' to tell the shell to match any one of a specific list of characters.any character will match.

use: cat -v odd-file. or the scroll bar. Press: return to advance one line d to advance one half page space to advance a full page nf to move forward n pages nb to move back n pages (miss out 'n' for a single page) /pattern to skip through the file until 'pattern' is found q to quit 'more' less . Use the page up and page down keys. You can increase the size of the scroll-back buffer in Terminal preferences.txt' is big.txt Page 7 of 271 Printed: Tuesday. Paging like this makes use of the Terminal's scroll-back buffer.txt more If the file you wish to view is bigger than a Terminal screen-full. more todo. the upper portion will scroll off the top of the Terminal window. use 'more' instead of 'cat'.txt The last line of output shows some inverse text giving the filename and how far you are through viewing the file.txt If 'todo. If the file contains control and other non-displayable characters.into your home directory: ls ~ is equivalent to ls /Users/your-short-name and for other users: ls ~janice is equivalent to ls /Users/janice cat Use your feline friend to display the contents of a file to a Terminal window. pico "my file" pico 'my file' pico my\ file tilde The tilde character is also 'globbed' . 2008 3:20:00 PM Escaping is also used to reference a filename that includes spaces. cat todo. to view the whole of the file. May 27.Unix_tricks.

May 27. Most of the navigation commands from the text editor 'vi' will work too. then monitor the file printing out additional lines as and when they are added. Press 'control-c' to quit 'tail'.txt tail -n 15 todo. Press: control-g to display file status information ng to go to line n G to go to the end of the file n% to position n% into the file /pattern to search forward for 'pattern' ?pattern to search backwards for 'pattern' n to repeat the last search All occurrences of a match pattern are highlighted.txt chase your tail To view a file live (as it is updating) use option '-f' with 'tail'.txt Page 8 of 271 Printed: Tuesday.txt'. Place bookmarks and return to them at any time (exactly as for 'vi') with: ma to mz to place a bookmark 'a to 'z to return to a bookmark heads or tails To view the first few lines of a file use 'head'. plus many others.txt To view the last few lines of a file use 'tail': tail todo. To view a file with less: less todo. Use the cursor (arrow) keys to move back and forth through a file. and to view a wide file. tail -f todo. directory sizes . head todo. 2008 3:20:00 PM Better than 'more' is 'less'.txt To view the first or last 'n' lines of a file: head -n 20 todo.log will watch the console log.txt The commands are as for 'more'. The application Console in Utilities is simply a GUI front-end to this command.txt will print the last few lines of 'todo. The command: tail -f /var/tmp/console.Unix_tricks. This is useful to monitor progress and log files.

tar To expand an archive and reconstruct the original directory: tar -xvf Documents. Specifying option '-x' tells 'du' to avoid mounted file systems. use the 'df' commnd: df -k Option '-k' causes sizes to be displayed in k-bytes. To query your home directory for example: du ~ This will display the size of every directory from your home directory down. but without displaying that information for every sub-directory.similar to an Aladdin 'stuffit' file or a disc image. The size of a directory includes the size of all directories within it. To compress the archive we created on Thursday: gzip Documents.tar is created and is a single file that contains a copy of all the directories and files in Documents. This is the GNU implementation of the Lempel-Ziv coding used in the 'zip' family of commands. tar balls A 'tar ball' is Unix-speak for a single file into which many other files have been bundled. The size is reported in file-system blocks. To list the contents of an archive: tar -tf Documents. Documents. or archived .tar . The last line displayed will be the total size of your home directory. use the '-s' or summary option. To archive 'Documents' use: tar cvf Documents. 2008 3:20:00 PM These tricks apply equally to 'disks' and 'discs' :-) To report on how much disc space a directory occupies. du -sk ~ Will print a single line displaying the total size of your home directory.txt Page 9 of 271 Printed: Tuesday.tar Documents 'Documents' remains untouched. summary of directory sizes To report on how much disc space a directory occupies. summary of disc usage To display how full each disc and partition is. use the disc usage command 'du'.tar zipping up files To compress a file use 'gzip'. Take the answer you get and divide by 2 to get the size in k-bytes. Each block is 512 bytes or 1/2k.Unix_tricks. May 27. and in k-bytes too (requested by the '-k' option).

Use the 'z' option to get 'tar' to zip and unzip.txt Page 10 of 271 Printed: Tuesday.gz This will overwrite Documents. tar cvfz test.tar.. 2^64 18446744073709551616 365*24*60*60 31536000 42*(6+9) 630 scale=5 1/3 . The results from 'bc' are shown in italic in the following examples.gz is very much like an Aladdin stuffit archive. 'tar' can archive and compress.. May 27.gz with Documents. A single compressed file that can be decompressed and expanded.tgz Unix Calendar To display a calendar for any month or year use the 'cal' command.gz.tar. or decompress and expand.tar. 2008 3:20:00 PM This will overwrite Documents... Documents.tgz test tar xvfz test.' In this case try using 'unzip'. Sometimes gunzip will give an error message something like: '.Unix_tricks. This month: cal This year: cal -y April 1900: cal 4 1900 Year 2112: cal 2112 Unix Calculator To perform complex calculations use the calculator 'bc'.tar.more than one file.tar.33333 scale=100 22/7 3.14285714285714285714285714285714\ .tar with Documents. in one go. To decompress use: gunzip Documents.

Create Banners Create a short banner with the Unix command 'banner'. 2008 3:20:00 PM 2857142857142857142857142857142857\ 1428571428571428571428571428571428 a=20 b=60 a*b 1200 ++a*++b 1281 And this is just the tip of the 'bc' iceberg. May 27.3048 / 3.77161792 control-c to exit.2808399 You have: kilo-litres You want: gallons * 264.0037854118 You have: zetta-ergs You want: joules * 1e+14 / 1e-14 You have: scruple You want: grams * 1.17205 / 0. units You have: feet You want: metres * 0. banner hello # # ###### # # #### # # # # # # # ###### ##### # # # # # # # # # # # # # # # # # # # # ###### ###### ###### #### . and so I type 'feet' then 'metres' (or 'meters'). In the first example I wish to convert between feet and metres (meters). Unit Conversions To find out how to convert from anything to anything use the Unix command 'units'.2959782 / 0.Unix_tricks.txt Page 11 of 271 Printed: Tuesday.

2008 3:20:00 PM Quiz Time Today. copies.txt Page 12 of 271 Printed: Tuesday. then . Drag and Drop Drag and Drop onto the command line. May 27. How many days were there in September 1752? And why? How many days into 2002 are we today? How many pico-gallons in a bushel? Copy to the Clipboard Copy the output of a Unix command to the GUI clipboard (that to which one cuts. On the command line type (for example): ls -al | pbcopy or cal | pbcopy In a GUI app such as TextEdit select menu Edit->Paste. Copy a page from TextEdit using command-c. or command-v. Change your current working directory to a folder you can see in the Finder without typing the full pathname. Now paste it to the Unix command 'wc' ('wc' counts the number of words in its input). and pastes).Unix_tricks. Simply type: cd <space> Now select the destination folder and drag and drop in anywhere on the Terminal window. pbpaste | wc -w 348 Note that Mac GUI programs and Unix programs use a different end of line character. a quiz. to paste the output copied from the Unix command. August 2002 S M Tu W Th F S 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 Paste from the Clipboard Copy from the GUI clipboard into a Unix command. so the clipboard will appear to contain one long line.

for example all text files in ~/Documents. or open a new Terminal window. To display a friendly and personalised greeting on startup. echo Hello $user Use an editor such as pico to create this file.txt test2. which automatically runs the script. Remember that the command must be executable. as long as the 'ssh' user and the user logged into GUI on the remote machine are the same.command'.txt Page 13 of 271 Printed: Tuesday. To launch an application use 'open -a'. May 27. Use: chmod +x script-name. 2008 3:20:00 PM press return.txt open *. This works for files as well as folders.tcshrc' in your home directory. type: cd ~/Documents ls *. add the following line to '.probably TextEdit. the new shell executes commands from (among others) ~/. open -a /Applications/ This can also be done remotely via ssh. Now double-click it and a Terminal window will open. or use the following command: echo 'echo Hello $user' >> ~/.tcshrc Also. and give it the extension '. have a look at /etc/motd.txt' . Simple as that. from the command line. Guaranteed to surprise the user of the remote machine :-) To open documents. Customised Prompt Change the Terminal prompt.command Terminal Startup When you start the Terminal. or open a document. Create a shell script as usual.txt Both files will open in the default application for '. Launch GUI Applications Launch a GUI application.Unix_tricks. set prompt="%B%c4%b %d %D %w %t %n@%M\n%# " This gives a two-line prompt: ~/SitesTutorials Wed 21 Aug 5:37pm % . Double Click a Shell Script Run a shell script by double-clicking it in the GUI.txt test.

txt Page 14 of 271 Printed: Tuesday.).tcshrc' . Possible values for # are: Attributes: 0 reset. Alternatively. standard prompt Add your custom prompt to '. This can also be done on the command line. 1 bold. %d %D %w %t is day. 37 white 90-97 are lighter variants of the above colours Background: 40-47.tcshrc' . one can use printf as in: printf "\033]0.tcshrc Use ANSI Colours Change the colour of your prompt. display current directory up to 4 levels..Hello ${user}@${host}\n" Add your custom prompt to '. month.Unix_tricks. It can also do this for filenames as in the following example.. set prompt="%{\033[31m%}%c3%{\033[0m%} %M %# " This sets a prompt like: ~/Sites/Tutorials melkor % The sequence %{\033[#.#. 35 magenta. date. Issue the command: echo "^[]0. which offers more flexibility.Hello ${user}@${host}" to change the Title Bar to 'Hello user-name@host-name' . Use an editor or issue the command: echo 'set prompt="%B%c4%b %d %D %w %t %n@%M\n%# "' >> ~/. .tcshrc Customised Title Bar Terminal preferences allow one to change what is written to the Title Bar of the Terminal window.see Monday's trick.#. %B %c4 %b is bold on. 4 underline 5 flashing 7 inverse Text: 30 black. May 27. 36 cyan.for example: Hello janice@melkor Note that ^[ is obtained by pressing control-v followed by the 'esc' or escape key. new line. host name. 31 red.see Monday's trick. bold off. 32 green 33 yellow 34 blue. time %n %M \n %# is user name. Hello ${user}@${host}\n"' >> ~/. 2008 3:20:00 PM Pick and mix these. Use an editor or issue the command: echo 'printf "\033]0. colours as for text 100-107 are lighter variants Enable Filename Correction The shell auto-corrects mistyped command names.#m%{ sets the ANSI colour or attribute specified by # (or several attributes can be specified by #.

but not 'day' as in 'today' or 'Wednesday'. and all directories within. Let's do the grep Sherlock allows one to search the contents of files for a specific string. rather than text that is contained within a word. Let's do the Recursive grep One can search a whole directory structure for a given string using recursive 'grep' Use 'grep' to find all files in you home directory. grep -nis the-string-you-wish-to-search-for * The option 'n' prints a line number. grep = Get Regular ExPression. To search for Saturday or Sunday but no other days use: egrep -nis "(Satur|Sun)day" * Note that we must use 'egrep'. grep -nisr the-string-you-wish-to-search-for * Searching for Words 'grep' can be made to search for a whole word. by specifying option '-r'. 2008 3:20:00 PM ls LearningCanter ls: LearningCanter: No such file or directory set correct=all ls LearningCanter OK? ls LearningCenter? yes AdvancedUnix UnixTutorials . (e)grep can do all sorts of fancy pattern matching. and 's' suppresses warnings. If you speak 'vi' then you can use (mostly) the same regular expressions in . using what are termed 'regular expressions' These are quite powerful and beyond the scope of this Trick. which is an extended version of grep. May 27. For example: grep -nisw day * will find matches for 'day' as a whole word. One can do this on the command line too. Searching for Patterns Use grep to search files for text that matches a pattern rather than a fixed string.. 'i' ignores case.txt Page 15 of 271 Printed: Tuesday. If in doubt use egrep all the time. Use 'grep' to find all files in you home directory containing a given string. by specifying option '-w'. but will be covered in depth in the Learning Center.Unix_tricks.. for this particular example.

Unix_tricks. By specifying option '-v' 'grep' will print all lines that don't match the specified expression. To write the processed file back to the original file use: sed 's/colour/color/g' file. 138 words.html' use: sed 's/colour/color/g' old.html' and write the output to ' Use option '-w' to match 'rem' as a whole word.html sed = Stream EDitor Strip with sed Use 'sed' to strip out unwanted lines in files. To replace the occurrence of 'colour' with 'color' from 'old. Use: .txt This letter has 16 lines. May 27. not just the first. To remove lines containing the text DELETE-ME: sed '/DELETE-ME/d' x > x. wc letter.html 's' stands for substitute. mv tmp Strip with grep Use 'grep' to strip lines from files. you must translate all occurrences of <CR> to the Unix end of line <NL>.html > To remove all blank lines: sed '/^$/d' x > x. 2008 3:20:00 PM (e)grep.txt Page 16 of 271 Printed: Tuesday.txt 16 138 882 letter. The document should be plain text. The 'g' at the end of the substitute pattern says to substitute all occurrences of colour on each line. Word Counting Count the number of words (and characters and lines) in a document. Therefore to strip all lines that contain 'rem' use: grep -vw "rem" x > x.html > new. Translate with tr If you have a file with Mac style end of line character <CR> that you want to look sensible in a Unix editor. Substitute with sed If you wish to replace a particular string with another throughout a file use 'sed'. and 882 characters.

2008 3:20:00 PM tr \\r \\n <x >x.txt Hardware Profile Get a hardware profile of your Mac with the command line capturing a man page with: man grep > grep. Strip with col Use the 'col' command to strip backspace characters from files. For example. Type: uname machine Network Id Query the hostname of your Mac with: hostname And set it with: hostname my-new-host-name Display the domain name with: domainname (which may be to translate x and write the output to x.) Display interface configuration with: ifconfig -a Keepy-Uppy How long has your Mac been up and running? Find out with: uptime Let's Date To query the current date and time use: . Type: AppleSystemProfiler What on What? Obtain the Unix variant and processor name. Use: man grep | col -b > grep.Unix_tricks.txt will produce odd-looking output.txt Page 17 of 271 Printed: Tuesday. May 27.

xps OmniWeb melkor 965 0.tcshrc' in your home directory. A shell alias is shorthand for a command.0 7. make sure it does not have the same name as an existing command with: which your-new-name Useful Aliases When you start the Terminal. the new shell executes commands from (among others) ~/.tcshrc. make an alias of it. it times execution of the command that follows it. or open a new Terminal window.50 OmniWeb For an easy grep: alias xgrep "egrep -ilrs \!:1 *" .txt Page 18 of 271 Printed: Tuesday.0 157072 59540 ?? S 79:23. alias cdut cd ~/Sites/Tutorials/LearningCenter/UnixTutorials Now type: cdut to change directory to ~/Sites/Tutorials/LearningCenter/UnixTutorials View Aliases To view the aliases currently set type: alias To view a particular alias type: alias alias-name-here To remove an alias type: unalias alias-name-here When you create an alias. To have aliases set automatically add them to '. If you have a directory to which you often refer. The command 'time' does not display or set the You may omit as much of the leading section as you wish. Teleporting Shell aliases are not the same as Finder aliases. May 27.08 2002 August 22 23hrs 55mins 8 seconds This command must be run as superuser and has the format: yyyymmddhhmm. Here are a couple of examples: alias xps "ps auxc | grep -i \!:1" 'xps' will now print the process id (PID) for a given command.Unix_tricks. 2008 3:20:00 PM date 'date' can also be used to set the date and time with: sudo date 200208222355.

To find all HTML and WebSiphon (. The alias 'period' is run every 'tperiod' minutes.html" \) . Periodic Aliases Run any command periodically.FBCIndex Library/Addresses/Address" -or -name "*.. use: find ~ -mtime 1 Find This or That Find has a very powerful syntax and can combine criteria. 2008 3:20:00 PM 'xgrep' will search all files in the directory hierarchy for the given string.Unix_tricks. printing a list of files that match the string. case insensitive. for example to do a daily backup.slt/cookies. xgrep osxfaq ~ melkor % xgrep osxfaq Development/. Recently Modified Files Find is able to search files dependant on their time-stamp. Shell Special Aliases Special aliases are executed by the shell at specific times. 'postcmd' runs before each command gets executed..txt . `pwd` $host@$user xxx\007"' will update the Terminal window title bar with the current directory and the host and user name each time you issue a command (in particular 'cd'). 'shell' specifies the interpreter for executable scripts which do not themselves specify an interpreter. 'precmd' runs just before each prompt is printed.addressbook Library/Application Support/AOL Instant Messenger (SM)/Buddies Cache Library/Application Support/Chimera/Profiles/default/eb0lqzsx. May files in ~/Sites use: find ~/Sites \( -name "*. To find all files in your home directory modified within the last 24 hours. For example: alias precmd 'printf "\033]0. For example: alias periodic date set tperiod=30 will execute the 'date' command every 30 minutes.txt Library/Application Support/Chimera/Profiles/default/eb0lqzsx.slt/cookperm.txt Page 19 of 271 Printed: Tuesday.

Unix_tricks.tcshrc ~/. Search Make an alias of the find command to do common searches. find . For example to search for strings in text files: alias ftxt 'find . Restore the 10. you can restore a few of the more useful shell features missing in 10. If you notice that your Terminal (or more correctly the shell) does not behave as it used to.tcshrc'.) Some Useful Settings If you do not wish to follow Monday's tip. For example.logout (Note that ~ or tilde is shorthand for your home directory. and you would like the behaviour restored. 2008 3:20:00 PM Find and Exec Use the power of find to root out the files you are interested in.0 and 10.htm" -or -name "*. \( -name "*. use the -exec option to invoke grep. type: cat /usr/share/tcsh/examples/README and follow the instructions. the default shell initialization that came with Mac OS X 10. to search all HTML files in your Sites directory. set autolist # List autocomplete alternatives when tab pressed set correct = cmd # Correct misspelled commands set savehist = 100 # Save command line history when Terminal closed As an example of the first two settings: cd ~ ls D <hit the tab key> Desktop/ Development/ Documents/ .2 by adding these commands to the tcsh initialization file '~/.txt files containing hello.txt" -exec grep -il \!:1 {} \.1 is included but not activated in 10.' Then type: ftxt hello to find all . May 27. and then apply a given command to each file found. -name "*.login ~/.2.html" \) -exec grep -il warning {} \.1 Shell Defaults For whatever reason. It is not necessary to follow this part: mkdir ~/Library/init/tcsh as you can add your own customisations to the files you have just created: ~/.txt Page 20 of 271 Printed: Tuesday. Ensure that the aliases you create do not have the same name as existing commands.

To make it non case sensitive add the following to your tcsh initialization file '~/. If this does not work for you try: set cdpath = (~) You can add more directories to this search path with: set cdpath = (~ ~/Sites) If 'cd' cannot find the given directory in the current directory. By way of an example: cd ~ ls d <hit the tab key> Desktop/ Development/ Documents/ The Path Home You may not realise this. it will search your home directory (~). history is merged in with previously saved history. command and filename completion by the tab key is case sensitive.. then 'Sites' in your home directory. Notice the merge option. the size of your command line history is 100 or 150 lines.tcshrc': set complete = enhance # Complete case insensitive. 'cd' automatically searches your home directory if it cannot find the specified directory. It does this because (or if) the 'cdpath' variable is set. Enhance also considers '-' and '_' to be equivalent. cd / cd data-base . but typing. Type: set to see all the shell's variables. You can increase this with the commands: set history = 2000 # History remembered is 2000 commands. 2008 3:20:00 PM mun ls CORRECT>man ls (y|n|e|a)? .txt Page 21 of 271 Printed: Tuesday. This saves the last 2000 commands.Unix_tricks. So Insensitive By default. set savehist = (2000 merge) # Save and merge with existing saved commands. this is very useful if you have more than one Terminal window open at a time. Change History Be default.. These should be added to your tcsh initialization file '~/. May 27. for example: cd / cd Desktop will find the directory Desktop even though it is not in the current directory.tcshrc'. Instead of the last Terminal session overwriting the history saved by the others.

jpg=36:*. Auto Complete in Color The 'tcsh' shell has a built-in 'ls' that it uses for auto-completion. add this.tar=31:*. set the autolist variable: set autolist List in Color Following on from the previous tip.htm=35:*.txt Page 22 of 271 Printed: Tuesday. the second and third file extensions versus colours.html=35:*.Unix_tricks. to your ~/.gif=36:*. You can customise this as you wish.tgz=31:*. May 27. Type (with no spaces): ls-F You may want to make an alias to avoid typing the '-F' part: alias ls ls-F and put this line in your ~/. as one long line with no spaces. Changing Color If you wish to change the colors that 'ls' and auto-completion use.tcshrc file. If tab completion does not work for you. set the shell environment variable LS_COLORS. setenv LS_COLORS 'no=00:fi=00:di=00.33:ln=04:ex=32: *. 2008 3:20:00 PM goes to ~/Sites/data-base Note that autocompletion does not follow the 'cd' search path. As an example. Remember to set the shell variable 'color'. one can use the 'tsch' built-in colour 'ls' instead of the regular 'ls'.jpeg=36:*.sh=32:*.sit=31: *. File types are: no Normal (non-filename) text fi Regular file di Directory ln Symbolic link pi Named pipe (FIFO) so Socket bd Block device cd Character device .tcshrc file.css=35:' The first line defines file type versus colours.psd=36:*. This can be made to list in colour by setting the shell variable 'color' set color Now type: ls <now hit control-d or tab if you have autolist enabled> (You must type a space after the 'ls') The auto completion will now be shown in colour.gz=31:*.

It can do a recursive copy of an entire directory structure. 2008 3:20:00 PM ex Executable file mi Missing file or Orphaned symbolic link Colours are: 0 default Terminal colours 1 brighter colour 4 underline 5 flashing 30 black foreground 31 red foreground 32 green foreground 33 dark yellow foreground 34 blue foreground 35 magenta foreground 36 cyan foreground 37 dark white (grey) foreground 40 black background 41 red background 42 green background 43 dark yellow background 44 blue background 45 magenta background 46 cyan background 47 dark white (grey) background Shut Up! If you use auto-completion frequently then the shell's constant beeping may annoy you. timestamps.) ditto You may be aware that Unix commands do not understand Mac resource forks.tcshrc file to make the change permanent. It does this as a 'carbon copy' of the directory. An alternative is to set it to beep only when no match can be found: set matchbeep=nomatch Really Shut Up! To tell the shell never to beep issue the command: set nobeep (This is not the same as the Terminal Window Settings --> Emulation .no beep option. and so commands like 'cp' cannot be used to copy files that have them. etc. preserving owners. which stops all programs run within the Terminal from beeping. The command 'ditto' does. especially if you have the sound turned up while listening to iTunes. Set the shell variable matchbeep to stop this.Unix_tricks.txt Page 23 of 271 Printed: Tuesday. It's an ideal way to backup the /Users directory. set matchbeep=never Add this to your ~/. The command: . permissions. May 27.

but the Finder will hide it. so don't expect the file to vanish instantly. This command is part of the Developer Tools so it won't be available unless you have installed them. May 27. The Finder will not update its information on the file immediately. Type: man SetFile for full details.print to an AT printer . copying the /Users directory to a disc or partition called 'Backup' If you have the developer tools installed. and type of a file on an HFS+ formatted 'ping' but for AT rather than TCP/IP atlookup .give status information for At devices like printers at_cho_prn . The native Unix file system is called UFS . Note that it is still visible to 'ls'. and run the command as root or with 'sudo'. appleping . To set the start up disc to OS 9: sudo bless -folder9 '/Volumes/Mac OS 9/System Folder' -setOF And for OS X: sudo bless -folder '/System/Library/CoreServices' -setOF or to boot from another disc: sudo bless -folder '/Volumes/volume-name-here/System/Library/CoreServices' -setOF AppleTalk Here are a few useful AppleTalk command line utilities. The command: SetFile will set the attributes. there is also a command called CpMac that does a similar thing. For example: SetFile -a V file-name will make a file called 'file-name' invisible in the Finder.txt Page 24 of 271 Printed: Tuesday. Bless You! To Change start up discs from the command line use 'bless'. (HFS+ is the default Mac file system. SetFile -a v file-name will make it visible a printer to send to via AT atprint .10. 2008 3:20:00 PM sudo ditto -rsrc /Users/ /Volumes/Backup/2002.lists visible devices registered with AT atstatus . Setting HFS+ File Attributes Unix commands do not understand all the attributes of HFS+ files.16/Users will do just this. timestamps. creator prizes for guessing what that stands for).

Use: id uid=501(saruman) gid=20(staff) groups=20(staff). .Unix_tricks. followed by all the groups to which that user belongs. This will kill a process by name. May 27. but not specific to OS X. 1493 ?? S 0:00.85 ATSServer 182 ?? Ss 19:37.. User ID and Group ID To display a user's name and primary group. Admin users created in 10.36 Window Manager . 2008 3:20:00 PM appletalk .2. use 'id'.2 do not belong to group wheel Use: groups staff wheel admin to list just the names of groups to which a user belongs.05 tcsh kill 1493 Remember that Unix is case-sensitive.configure AT Kill All! 'killall' is new in OS X 10.49 Clock 813 std S 0:01. 80(admin) for the currently logged in user. This displays both the numeric ID and textual name.2 will belong to groups as above.. For example: killall Clock does the same job as: ps -cx PID TT STAT TIME COMMAND 170 ?? Ss 2:50. Use: id root uid=0(root) gid=0(wheel) groups=0(wheel) 1(daemon) 2(kmem) 3(sys) 4(tty) 5(operator) 20(staff) 31(guest) 80(admin) for a given user.txt Page 25 of 271 Printed: Tuesday. Note that admin users pre 10. 0(wheel). Or: groups root wheel daemon kmem sys tty operator staff guest admin for a given user. so 'killall clock' will not work. The good old 'kill' command requires the PID (Process ID). eg root.

Unix_tricks.txt Page 26 of 271 Printed: Tuesday. or users started with 'su'. whoami saruman su janice Password: whoami janice To get a history of logins since quite a long time ago (the last time your Mac tidied its log files) use: last . 1. You can also use the command 'w'. 13Oct02 14days - saruman p2 . May 27. not each user. not ftp users. users saruman evil-cracker You will only see genuine logins. user 'users'. and twice in the Terminal (ttyp#). w 12:01AM up 14 days. Who is Logged in? Use 'who' to determine who is logged into your Mac. Who Am I? Severe case of amnesia? whoami saruman Useful if you 'su' to other users.49. 1. A user connected to three Terminal sessions will be listed three times by 'who'. 2 users. Just to check no one is sneaking onto your Mac. load averages: 1. 2008 3:20:00 PM List Current Users To list the names of all users currently logged in. This differs from 'users' in that it lists each Terminal connection. Hopefully you will be the only user listed.19. 8:26.03 USER TTY FROM LOGIN@ IDLE WHAT saruman co . Sun11PM 0 - It also gives the information provided by 'uptime'. which saves a whole two characters of typing and gives a little extra information. but only once by 'users'. who saruman console Oct 13 16:36 saruman ttyp1 Oct 27 21:38 saruman ttyp2 Oct 27 23:33 saruman is connected 3 times. once as the original login when the machine boots up (console). telling how long your machine has been up and running between re-boots.

...txt Page 27 of 271 Printed: Tuesday.cancels interactive screen shot Apple System Profiler The Apple System Profiler in /Applications/Utilities can be called from the command line to write a profile report to the Terminal. For example: AppleSystemProfiler > ~/profile.pdf to capture the whole screen to the given file.19:54 (00:00) saruman ttyp1 Thu Oct 24 17:17 .17:17 (1+19:37) saruman ttyp1 Tue Oct 22 21:39 ..pdf will start in interactive mode where: control key . The option '-c' is available to force the screen capture to go to the clipboard instead of a file.13:34 (00:00) saruman ttyp2 Wed Oct 16 23:02 .21:39 (2+05:48) saruman ttyp1 Sun Oct 20 15:51 .15:26 still logged in saruman ttyp1 Sat Oct 26 19:54 .17:17 (00:00) saruman ttyp1 Tue Oct 22 21:39 ..16:14 (02:39) saruman ttyp1 Sat Oct 19 13:34 .txt will write a profile report to the given file. in pdf format.15:26 (20:32) saruman ttyp1 Sat Oct 26 19:54 . 2008 3:20:00 PM saruman ttyp1 Sun Oct 27 15:26 .15:51 (00:00) saruman ttyp1 Sat Oct 19 16:14 .15:38 (10+17:35) .16:14 (00:00) saruman ttyp1 Sat Oct 19 13:34 .Unix_tricks. May 27.15:51 (23:37) saruman ttyp1 Sat Oct 19 16:14 . Alternatively: screencapture -i ~/Pictures/sc2.toggle between mouse and window selection modes escape key .21:39 (00:00) saruman ttyp1 Sun Oct 20 15:51 . Screen Capture The Mac OS X Screen Capture program 'Grab' can be called from the command line with: screencapture ~/Pictures/sc1.19:54 (2+02:36) saruman ttyp1 Thu Oct 24 17:17 .causes screen shot to go to clipboard space key .

and sleep time for all (battery and charger) settings.2.the screen dim time (dim) .restart required .plist.g. e. 4350K .2. 9330K .txt Page 28 of 271 Printed: Tuesday. 19620K . 2008 3:20:00 PM Note: this worked well in 10.3359 QuickTime (6. The 'womp 1' parameter says to 'Wake on Magic Packet' and is the equivalent of the 'Wake for network administrator access' check box in the Energy Saver preference pane. For example: pmset -a dim 5 spin 5 sleep 20 womp 1 will set the screen dim.the disc spin-down time (spin) .0.the sleep time (sleep) including the individual battery (-b) and charger (-c) settings of an iBook or PowerBook.. For example. Software Update The Software Update utility can be called from the command line. Software Update found the following new or updated software: .Unix_tricks. These are recorded in ~/Library/Preferences/pref-file-name. Use pmset (power management set) to control: .1).3283 Internet Explorer 5.2).restart required To install an update.3280 AirPort Software (2. disc spin-down. It will list the updates available and allow one to select an update for installation. Inc. For example: softwareupdate Software Update Tool Copyright 2002 Apple Computer. run this tool with the item name as an argument. Applications present preference panes in which the user can set preferences.1..2 Security Update (5. These preferences can be viewed and set from the command line using 'defaults'. or all (-a).1 but seems to be flaky in 10.' Energy Saver The settings controlled by the Energy Saver system preference pane can also be set from the command line. Preferences Preference can be set from the command line. May 27. 'softwareupdate <item> . to view the Terminal settings: .2).

. but only in 10.log' and the process will run in the background to free up the command line (putting '&' at the end of a command causes it to run in the background).terminal TerminalOpaqueness 0..telnet.. CleanCommands = "rlogin. It displays a list of running processes and the resources they are currently using. May 27. Autowrap = YES. The command: defaults read will display all settings for all programs. etc Setting can be changed with: defaults write TerminalOpaqueness = 1. . Top of the Hogs The 'top' command displays system usage statistics including cpu and memory usage.terminal { AppleSavePanelExpanded = NO. For example: top -s60 -l120 > top. Note that such options may not work correctly so experiment carefully.2 did it appear in the preference AutoFocus = NO. Top Logging Top can be used to periodically log usage statistics over a prolonged period using the '-l' and '-s' options. Backwrap = YES.ssh.Unix_tricks. 2008 3:20:00 PM defaults read com.5 You will often find many more settings in these files than appear in the preference panes. .txt Page 29 of 271 Printed: Tuesday. CursorShape = 0. For example the Terminal Opaqueness setting has always been available. BlinkCursor = YES. For example: top -u lists processes in order of cpu usage (the top cpu hogs). and: top -e lists event information.log & will log every 60 seconds for 2 hours (120 logs) The output will be written to file 'top.slogin". Bell = YES. Columns = 80.

30 Clock Kill(all) To (force)quit an errant process use the kill command. 3587 ?? S 0:00.) . This takes the process id (or PID) as a parameter For example. by default just those of the current user that are running within a Terminal window. killall Clock (This is case sensitive.Unix_tricks. A Useful Alias for 'ps' Create an alias to 'ps' a given application by name. To get information for 'Clock' type: xps clock saruman 3677 0.2 includes the 'killall' command which can kill processes by name. ps xc will display a list of all your processes. and the option 'ww' avoids truncation of long lines..tcshrc to add the alias to your shell startup script.. May 27. Type this: echo "alias xps 'ps aucx | grep -i \\!:1'" >> ~/. then start a new shell or Terminal window. OS X 10.5 72676 3552 ?? S 7:05PM 0:03.0 0.41 ATSServer .58 Clock 3359 p4 S 0:00.txt Page 30 of 271 Printed: Tuesday. The options 'l' and 'u' display more information for each process. 2008 3:20:00 PM Running Processes The 'ps' command lists running processes.60 tcsh 3377 std S ps cx PID TT STAT TIME COMMAND 178 ?? Ss 7:19.52 tcsh kill 3587 If the Clock refuses to lay down and die: kill -KILL 3587 '-KILL' says to force quit the application instead requesting it to quit. and: ps axc will display a list of all processes for all users. to kill the clock application: open /Applications/Clock.

as illustrated for 'cd' above.specify the draw style desired for quadrics mount_cd9660(8) . May 27. 2008 3:20:00 PM What is Whatis? The command 'whatis' displays a brief one-line summary of the given command..util(8) . It searches the same database.mount an ISO-9660 filesystem mount_cddafs(8) . Now try this: ~ % which which which: shell built-in command.mount an Audio CD . Note that 'man -k' is equivalent to 'apropos'. probe. and also what the alias will be expanded to. I also have an alias called 'ls'.helps mount. Most likely this will be /usr/local/share/man for commands you have added yourself. I have two versions of 'ls'. It displays any lines that contain the search term as a complete word. % which ls ls: aliased to ls --color=tty tells me that the alias will be run. To update this database after you have added new commands do this: sudo /usr/libexec/makewhatis man-path where man-path is the directory containing the man pages. This tells me which 'which' I executed ..and it was in fact the shell's built-in 'which' . % apropos cd cd9660.Unix_tricks. Escaping From Aliases . For example.txt Page 31 of 271 Printed: Tuesday. Which Which? 'which' informs you which alternative of a given command would be executed if it were typed at the command line. and unmount cd9660 filesystems gluQuadricDrawStyle(3) . Is Apropos Appropriate? The command 'apropos' gives a brief one-line summary for a given command. % whatis ls ls(1) . one in /usr/bin and one in /usr/local/bin.list directory contents 'whatis' actually searches a database containing a one-line summary of all commands taken from the man pages.hence its knowledge of aliases. This is the same as Monday's 'whatis' except that the search term does not have to be a complete word.

. .Unix_tricks. . Or scroll down for the answer. From this weeks tips it should be possible to guess at the commands. . . either 1) ~ % which \ls /usr/local/bin/ls . To tell us about non-built-in commands we can do one of two things.. . .. . 'which' is also built-in. 2008 3:20:00 PM This is a handy trick if you wish to run a command .. May 27. In order to run the original 'ls' I could give the full pathname (if I knew it): % /bin/ls Alternatively I can escape the command with '\': % \ls Which Which is Which? Following on from Wednesday's and Thursday's tips. . .. which is why it considers aliases. Wednesday's tip told us that 'ls' was aliased. . .txt Page 32 of 271 Printed: Tuesday. I can't simply type: % which ls ls: aliased to ls --color=tt because this just tells me about the alias 'ls'. . I now wish to know which 'ls' I am running when I type: % \ls (ie the non-aliased 'ls'). . . . . Remember that from: ~ % which which which: shell built-in command. For example. .and you want to run the actual executable - but it has been aliased or the shell has a built-in version.

Unix_tricks. You may set either. May 27. % echo $PATH /usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin % setenv PATH /Users/saruman/bin:$PATH % echo $PATH /Users/saruman/bin:usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin (You could instead use ~/bin. The path is described by the environment variable PATH. paths and PATHs Two Paths? % echo $path /Users/saruman/bin /usr/local/bin /usr/bin /bin /usr/local/sbin /usr/sbin /sbin % echo $PATH /Users/saruman/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin What is the difference between 'path' and 'PATH'? 'path' is a shell variable and 'PATH' is an environment variable.login to have the path set automatically when you login.txt Page 33 of 271 Printed: Tuesday. You can check the path with: echo $PATH /usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin A Longer Path If you wish to put commands in a new bin directory such as: /Users/you/bin add this directory to your path so commands will be found automatically. and whichever is changed the shell will synchronise the other. 2008 3:20:00 PM or 2) % \which ls /usr/local/bin/ls And to determine which 'which' is which: % \which which /usr/bin/which or: % which \which /usr/bin/which I hope you followed that! :-) The Path to Execution When you execute a command the shell first looks for that command by following a path from one 'bin' directory to another (after first resolving aliases and checking if the command is built in to the shell itself).) Add the 'setenv' command to ~/. without typing the full pathname. .

/dev/disk0 is a disk /dev/disk0s9 is a partition (or slice) of disk0. simply typing: % my_script my_script: Command not found.txt Page 34 of 271 Printed: Tuesday./my_script script runs.. Well. it is blind to all exectuables except those on the PATH. mount. Why? Is the shell blind? Yes. rehash will cause the shell to recreate its cache. not disk or volume names.' to the search path: % echo $PATH /usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin % setenv PATH "${PATH}:.' to your path has security implications. Type: disktool -l . it follows PATH and remembers (caches) all the commands in each directory. it doesn't really. rename.' to the path as in Thursday's tip. Consequently when you add a new command or script it will not be found. Warning: adding '. 2008 3:20:00 PM Dotty Paths If you write a shell script and wish to execute it. About disktool 'disktool' will unmount. previously I said that the shell searches PATH to find commands. or eject drives and partitions. It does not have a manual page. will not work even though my_script is in the current directory. You must supply device names to 'disktool'..Unix_tricks. which is much quicker than searching PATH. When the shell first starts. the shell must treat it as a special case. When you execute a command the shell simply looks in its cache. or when the PATH is changed." % echo $PATH /usr/local/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:. To avoid doing this add the current directory '. disktool will list the options. If you think about it: adding '. May 27. By giving a pathname the shell does not need to use the search path. Note that the 'bash' shell rehashes automatically. You must type % .. Rehash the Cache Ok.

. Use: disktool -r to refresh the Finder's view. fsType = 'hfs'. disktool -l will tell you which disk represents the CD. To eject 'Return to Zork' (disk1) use: disktool -e disk1 disk1 device will attempt to be ejected .Unix_tricks.disk1s1s1 ***Responding yes to eject . volName = 'Return to Zork') .disk1s1s2 ***Disk Unmounted('disk1s1s2') ***Responding yes to eject .vol (read-only) /dev/disk0s10 on /Volumes/Grima (local) .txt Page 35 of 271 Printed: Tuesday.disk1s1 ***Disk Ejected('disk1') Unmount with disktool You can use 'disktool' to unmount whole disks or individual partitons... Refresh Disk Icons with disktool If you mount a server or drive using the command line 'mount_afp' or 'mount_hfs'. First use 'mount' or 'disktool -l' to determine the device: mount /dev/disk0s9 on / (local) devfs on /dev (local) fdesc on /dev (union) <volfs> on /. 2008 3:20:00 PM to determine which volume names (also called mount points) correspond to which /dev/.. I wish to unmount the partition /Volumes/Backup.Mountpoint = '/Volumes/Return to Zork'. the Finder may not show the icon in 'Computers'.disk1 ***Responding yes to eject . device names.. Eject with disktool Use 'disktools' to eject a CD that seems stuck. For example.. disktool -l *** Unrecognized disk appeared on disk1 *** *** Unrecognized disk appeared on disk1s1s1 *** ***Disk Appeared ('disk1s1s2'. May 27.disk1s1s2 ***Responding yes to eject . ***Notifications Complete for type 1 ***Disk Unmounted('disk1') ***Disk Unmounted('disk1s1s1') ***Disk Unmounted('disk1s1') ***Responding yes to unmount .

5G) 12: Apple_Free 0+@ 78140152 What's the Code for London? Q. ***Notifications Complete for type 1 ***Responding yes to unmount .. May 27. mount /dev/disk0s9 on / (local) /dev/disk0s10 on /Volumes/Grima (local) /dev/disk0s11 on /Users (local) Numbered 9.. To find out what all the other partitions (0 to 8) are.1G) 10: Apple_HFS Grima 5732288 @ 21120784 ( 2. use: 'pdisk' sudo pdisk Password: Top level command (? for help): l /dev/disk0 /dev/disk0 map block size=512 #: type name length base ( size ) 1: Apple_partition_map Apple 63 @ 1 2: Apple_Driver43*Macintosh 56 @ 64 3: Apple_Driver43*Macintosh 56 @ 120 4: Apple_Driver_ATA*Macintosh 56 @ 176 5: Apple_Driver_ATA*Macintosh 56 @ 232 6: Apple_FWDriver Macintosh 512 @ 288 7: Apple_Driver_IOKit Macintosh 512 @ 800 8: Apple_Patches Patch Partition 512 @ 1312 9: Apple_HFS Curunir 21118960 @ 1824 ( 10.7G) 11: Apple_HFS Isengard 51287080 @ 26853072 ( 24.txt Page 36 of 271 Printed: Tuesday. 2008 3:20:00 PM /dev/disk0s11 on /Users (local) automount -fstab [400] on /Network/Servers (automounted) automount -static [400] on /automount (automounted) /dev/disk2s2 on /Volumes/Font Tools (local. read-only) /dev/disk3s11 on /Volumes/Saruman (local) /dev/disk3s10 on /Volumes/Sauron (local) /dev/disk3s9 on /Volumes/Melkor (local) /dev/disk3s12 on /Volumes/Backup (local) Then unmount using the '-p' option of disktool: disktool -p disk3s12 disk3s12 partition will attempt to be unmounted . For example I have three partitions on my startup disk. nodev. grep -i London /usr/share/misc/inter. as shown by 'mount'.. 10. 11. What is the international dialling code for London? What's the ASCII Code For. nosuid.? .disk3s12 ***Disk Unmounted('disk3s12') pdisk You might wonder why your disk partitions are named as they are.

'ls -l' does not work as it lists the directory contents. not the letter ell).. 12 Across : t _ _ _ k _ _ _r A. Alternatively use: ls | cat Directory Details To list the details of a file one would use: ls -l letter.. cat /usr/share/calendar/* | grep Apple | grep founded (In case you are wondering.) Wanna Cheat at Crosswords? Q.16 saruman staff 544 Nov 22 13:43 Documents List Long /etc ...txt Page 37 of 271 Printed: Tuesday..r' (You didn't get this from me. (-1 is the number one. cat /usr/share/dict/words/ | grep -wi 't. When was Apple Computer founded? A. ls -ld Documents/ drwx-----. What is the Unix directory /var used for? A. To list details of the directory itself use the '-d' option.txt If you wish to do the equivalent for a directory. man hier When was Apple Founded? Q. man ascii What's with /var? Q.k. 2008 3:20:00 PM Q. using cat and grep gives a cleaner output than grep alone when one passes a wildcard to cati. May 27.. right!) One Column Output To list a directory one file per line (without a long listing using ls -l) use: ls -1 Desktop Development Documents Library .Unix_tricks. What is the ASCII code for 'A'? A.1 saruman staff 0 Nov 22 17:32 letter.txt -rw-r--r-.

To list the contents use either: ls -l /private/etc/ or better. rather than back-deleting a character at a time.Unix_tricks.en1 ..nvram. (This tip assumes you have 'autolist' set. then a simple: ls will not give any indication of what a file is .. etc.. type control-u to delete the whole line.tcshrc file. May 27. executable. This is because /etc is a link (to /private/etc/).1 root wheel 6 Nov 22 15:22 appletalk. If you have started to type a command line and wish to abandon it.) Distinguish File Types If you don't use colour 'ls'..cfg -rw-r--r-. If not enable it by adding the line set autolist to your ~/.1 root wheel 753 Jul 15 04:03 6to4.cfg -rw-r--r-. 2008 3:20:00 PM If you've ever tried to do a long list (ls -l) on the directory /etc: ls -l /etc lrwxrwxr-t 1 root admin 11 Nov 22 14:03 /etc -> private/etc you will have found that it does not list the directory contents. Simply hit the tab key on an empty command line.. The Quickest way to List a Directory Q. directory. . link.1 root wheel 256 Nov 22 15:23 appletalk. Until then have a good: grep 12/25 /usr/share/calendar/* and happy birthday Sir Isaac Newton! Kill A Command Line I hope Santa delivered that new Mac you asked for :-). Use option '-F' ls -F directory/ file1 file2 link1@ my_script* * @ = link * / = directory * * = executable The next tip will be on Monday 30th.plain file.txt Page 38 of 271 Printed: Tuesday. use the '-L' option: ls -lL /etc total 1256 -rw-r--r-.1 root wheel 515 Jul 14 07:20 afpovertcp..conf -rw-r--r-. How can you list the current directory with one keystroke (ls takes 3 including the return)? A.

To jump straight back to a command. Another esp-p will jump back further to a previous match. Have a happy: grep 01/01 /usr/share/calendar/* and happy birthday the Internet. May 27. to list an expanded globbing rather than expanding it to the command line. hit control-x followed by G (upper case G). ls fil?* (hit control-x followed by G) file1 file2 ls fil?* Which Command? As in a previous tip. 20 years old! Glob Now If I type: rm fil?* how can I check exactly which files will be removed before I hit return? Either precede the command with: ls fil?* or expand the globbing ('*' '?' or '[]' wildcard characters). the command: . List Globbing I hope the new year's hangover has subsided and you are no longer seeing double! Similar to Wednesday's tip.Unix_tricks. use control-k will delete from the cursor to the end of the line. otherwise hit control-u to delete the command line (instead of the files!). Typing control-x followed by '*' will expand 'fil?*' onto the command line: rm fil?* (hit control-x followed by *) You will end up with an expanded command line like: rm file1 file2 Hit return if this is ok. to list an expanded globbing rather than expanding it to the command line.txt Page 39 of 271 Printed: Tuesday.) Deja Vu We all know that pressing the up arrow key moves back through the command history. ls fil?* (hit control-x followed by G) file1 file2 ls fil?* Similar to Wednesday's tip. type the first part of it then hit esc-p (the escape or 'esc' key followed by 'p'). But this is tedious if the desired command is a long way back in the history sequence. 2008 3:20:00 PM (Also. hit control-x followed by G (upper case G).

txt.txt 0a1 >This line not in file one 2c3 < This line has File One --- > This line has File Two 4. May 27. diff -bBq f1.5c5 < This Line not in File Two < Completely different in each file --- > Not at all the same as in file one The "< and "> characters indicate which file.txt is an addition compared to f1. What's a Little White Space Between Friends Sometimes files may differ in insignificant ways.txt f2. or have been deleted.txt f2.txt This line in both This line has File One This line in both This Line not in File Two Completely different in each file cat f2. Imagine each as an arrow pointing to the position of the file name on the "diff" command line.txt . The "2c3" indicates line 2 of f1. "diff" will report on all lines that differ.Unix_tricks.txt This line not in file one This line in both This line has File Two This line in both Not at all the same as in file one diff f1.txt The '-q' option can also be useful. "0a1" means line 1 of f2.txt f2. and is line 3 because of the additional line as above. To ignore only differences in white space (spaces and tabs) use option '-b'. Following a command with escape then '?' will also do this: ls (hit escape followed by ?) /usr/local/bin/ls --color=tty ls What's the difference? Use the "diff" command to compare two files.txt Page 40 of 271 Printed: Tuesday.txt has changed in f2. diff -bB f1. have been added. cat f1.txt. Diff assumes the first file is the original and the second file is the newer. To ignore differences due to blank lines use option '-B'. 2008 3:20:00 PM which ls ls: aliased to ls --color=tty says which variant of 'ls' will be executed.

txt > diff. 2008 3:20:00 PM Files f1.txt It is easier to direct output to a file and view it with an editor .Unix_tricks.txt f3.txt f2. This line in both This line in both This line has File One | This line has File Two This line in both This line in both This Line not in File Two < (This may not format correctly in your email.txt and f2. The report treats file2 as the oldest file.txt f3.txt This line in both This line has File One This line in both This Line not in File Two This line in both This line has File Two This line in both This line in both This line has File Three This line in both This Line not in File Two diff3 f1.txt f2.txt ==== 1:2c This line has the words File One 2:2c This line has the words File Two 3:2c This line has the words File Three ====2 1:4c 3:4c .preferable in a wide window. and shows how the updates to file1 differ from those to file3.txt This line in both This line has File Two This line in both diff -y f1.txt f2. cat f1.txt Page 41 of 271 Printed: Tuesday.txt differ Side by Side Diff can compare two files by arranging them side by side via the '-y' opton: cat f1.) diff diff diff 'diff3' can compare three files at once.txt This line in both This line has File One This line in both This Line not in File Two cat f2. May 27. It will compare the difference between file1 and file3 relative to file2.

showing hidden files too. Otherwise directories are ignored.tcshrc or navigate to the directory containing the file and type its name into the Go To box: . Hit 'View Source' in OmniWeb's toolbar and you can edit a text file too.txt Page 42 of 271 Printed: Tuesday. hidden Unix files (those starting with a dot) are not shown. Double click a text file to open it. May 27. I can do this with the command: echo "set autolist" >> ~/. OmniWeb can be used as a file browser. and each directory has a self-named .. Editing Hidden Files in GUI Apps When you choose 'File -> Open.txt 3d2 < This line in both Only in test: test.txt in test2 has had a line Here. If you wish to edit one either: type the full name into the Go To box at the bottom of the open dialog window: ~/.. and reporting files that are in one directory but not the other. f1. The recursive option '-r' tells diff to look into and compare like-named nested directories too.tcshrc . Double click directories to navigate them. For example to active the shell's autolist feature I need to add: set autolist to my ~/. Edit With 'echo' If you need to add a line to a Unix text file. 2008 3:20:00 PM This Line not in File Two 2:3a diff Directories 'diff' can compare whole directories. comparing like-named files.Unix_tricks. an easy way is to use echo.txt test2/f1. Editing Hidden Files in OmniWeb Enter the location: file:/// into the address file Only in test2: test2. diff test test2 diff test/f1.tcshrc file.' in an editor such as TextEdit.tcshrc TextEdit understands Unix end of line characters.

For example: cat to-sort.Unix_tricks. The double redirect appends to a file.html <pre> <tt>file1</tt> <tt>file2</tt> <tt>file3</tt> <tt>file4</tt> <tt>file5</tt> </pre> 'sort' Command 'sort' will sort the lines of a text file alphabetically (or alphanumerically).txt ls 3 list files cd 2 change directory pwd 4 print working directory grep 1 search files for regular expression sort to-sort. Edit With 'cat' Use the 'cat' command to create a new file.. May 27.txt type here.txt Type your text here until you have finished then type control-d To add to the end of an existing file use double redirect: cat >> newfile. or add to the end of an existing file. printf "<tt>%s</tt>\n" `ls` . cat > newfile. ls file1 file2 file3 file4 file5 ( printf "<pre>\n" . 2008 3:20:00 PM NOTE: Use a >> not a single >..txt cd 2 change directory .txt Page 43 of 271 Printed: Tuesday. the single redirect will start a new file overwriting any existing file of that name.html cat ~/list. Edit with 'printf' Here is a neat trick using printf. printf "</pre>\n" ) > ~/list.

/ls">list files <a href=". as in: cat to-sort./grep">search files for regular expression ./cd">change directory <a href=".html <a href="./pwd">print working directory <a href=". Sort on Another Field Command 'sort' will sort on any field. and 'sort' assumed that fields were separated by white space. May 27.. For example. to make the second field the sort key use: cat to-sort.. We can specify a custom separator with option '-t'.txt ls 3 list files cd 2 change directory pwd 4 print working directory grep 1 search files for regular expression sort -k2 to-sort. 2008 3:20:00 PM grep 1 search files for regular expression ls 3 list files pwd 4 print working directory The sorted output is written to the Terminal. What if fields are separated by a character other than white space? A.Unix_tricks.. Mac OS X uses GNU 'sort'. where fields are separated by white space. DON'T write the output back to the input file or you will end up with an empty file. Q.txt Page 44 of 271 Printed: Tuesday. Changing the Field Separator Tuesday's example sorted a file using the second field of each line.txt grep 1 search files for regular expression cd 2 change directory ls 3 list files pwd 4 print working directory In this example the files were separated by tab characters.. Use redirection or option '-o' to write to a file.

/ls">list files <a href=".txt grep 1 search files for regular expression cd 2 change directory ls 3 list files pwd 4 print working directory Sorting and Merging Files Command 'sort' can be used to sort and merge two or more files./pwd">print working directory <a href=". (Note that '<' is escaped to prevent the shell from interpreting it as a redirection character - see Tutorial 7.. the sequence number. This example attempts to sort by field 2.txt cd 2 change directory ls 3 list files pwd 4 print working directory grep 1 search files for regular expression Use option '-b' to ignore leading blanks when finding sort keys: sort -b -k2 to-sort. rather than tabs as in Tuesday's example. 2008 3:20:00 PM sort -t\> -k2 to-sort./grep">search files for regular expression Now the file is sorted according to the description rather than the URL.) Space Separated Fields You may have a file where the fields are separated by multiple spaces. Using 'sort' results in rather odd behaviour..Unix_tricks. May 27. .txt ls 3 list files cd 2 change directory pwd 4 print working directory grep 1 search files for regular expression sort -k2 to-sort./cd">change directory <a href=".. cat to-sort.html <a href=".txt Page 45 of 271 Printed: Tuesday..

2008 3:20:00 PM cat s1.txt Page 46 of 271 Printed: Tuesday. the last issued 'printenv' call: !printenv Recall a Command by Number Issuing the command 'history' will list a numbered history of commands. You can recall any command with '!n' where 'n' is the command number given by the history 99 23:13 history 100 23:13 ls ..txt 2 3 8 6 5 sort s1.txt 9 7 1 4 cat s2. history 1 09:31 cd ~ . For example: !pri will recall.Unix_tricks. May 27.txt s2. then type '!' followed by enough of the command to make the recall unambiguous. for me currently.txt 1 2 3 4 5 6 7 8 9 Recall a Command by Name If you wish to repeat a previously issued command. 95 23:04 printenv 96 23:07 echo "Hello" 97 23:07 ls -al 98 23:07 vim index.

For example: history .txt 2048 23:27 cat index. May 27. 2044 23:26 ls -al 2045 23:26 vim log. The last issued command that matches the string will be 2049 23:27 history !?.txt Page 47 of 271 Printed: Tuesday.txt <-. if you want to be sure of the command to be executed by: !pri hit the tab key and tcsh will complete the command and wait for you to hit return to execute it. Recall and Tabbed Completion Following on from Monday's tip. 2008 3:20:00 PM !96 echo "Hello" Hello The last command is recalled with: !! and the ones before that with: !-2 !-3 etc.this command will be found and issued . or control-u to dismiss it. type history n to display just the last n commands.Unix_tricks.txt 2046 23:26 ls 2047 23:26 cat log. If your history is long.txt? cat log. Search and Recall A command may be recalled by giving a search string.

2008 3:20:00 PM Recalling Command Arguments The individual arguments of a previously issued command may be recalled.txt Page 48 of 271 Printed: Tuesday. when you issue 'cd -' the current and old working directories are effectively swapped. It is also possible to specify arguments of other commands with !n:n and !-n:n (see Tuesday's tip). and so on..this command will be issued !!:0 is the command itself. This makes it possible to jump back and forth between two directories. Now jump back to the original directory: cd - pwd /Users/saruman/Documents/My Setup The Old Working Directory The shell keeps a record of the current and old working directories in variables 'cdw' and 'owd'.Unix_tricks. !!:2 the second. May 27. and !!:1 is the first argument.lproj/ pwd /System/Library/Displays/Overrides/Contents/Resources/English. cd ~ cd Documents/My\ Setup/ echo $cwd /Users/saruman/Documents/My Setup echo $owd /Users/saruman Monday's trick is equivalent to: cd $owd Back and Forth Following on from Monday's tip. and then wish to jump back to the original directory. cd ~ . For example: less a-very-long-filename-to-type-again vi !!:1 vi a-very-long-filename-to-type-again <-.. To save retyping the original directory name one can make use of the shell's special 'old working directory' shortcut '-'. Start here: pwd /Users/saruman/Documents/My Setup Change directory temporarily: cd /System/Library/Displays/Overrides/Contents/Resources/English. The 'cd' Dash Suppose you change directory to issue a few commands.lproj .

Unix_tricks. and then ~/Sites So Insensitive (See also Weeks 1 and 13) As Mac OS X is case insensitive.html pwd /Users/saruman/Sites The command 'cd Sites' should not work. . This behaviour is controlled by the shell variable 'cdpath' set cdpath= ( ~ . it makes sense to search for files in a case insensitive way. cdpath How does the 'cd Sites' command below work? cd /System ls Library cd Sites ls Tips Tutorials archive areas css frames images index.txt Page 49 of 271 Printed: Tuesday. the command is executed in preference to changing directory. but it has managed to change directory to 'Sites' in my home directory. set implicitcd Sites pwd /Users/saruman/Sites Naturally. if a command of the same name exists. Instead of using 'find -name' use 'find -iname'. 2008 3:20:00 PM cd Documents/My\ Setup/ pwd /Users/saruman/Documents/My Setup cd - pwd /Users/saruman cd - pwd /Users/saruman/Documents/My Setup cd - pwd /Users/saruman Implicit CD Setting the shell variable 'implicitcd' causes the shell to treat a directory name typed without 'cd' as a request to change to that directory. ~/Sites ) Tells 'cd' that if it can't find the requested directory in the current directory. cd ~ ls Desktop Documents Movies Pictures Sites bin osxfaq Development Library Music Public Temporary Items Sites Sites: Permission denied. search my home directory (~). May 27.

now it's safe to actually delete: find .ws /Users/saruman/Sites/Tips/unix-tricks/week1/thursday. Use this with /Users/saruman/osxfaq/templates/tips-week-template/tuesday.tmp" -delete find . For example.tmp . -iname "*./Sites/Tips/unix-tricks/schedule. to search my home directory for all the weekly tips.Unix_tricks./. running a test find first. though you may have to 'sudo' it. ./Library/Preferences/BBEdit Preferences/File Filters Deleting Files (See also Weeks 1 and 13) 'find' has a built-in delete option that can be used to delete all found files.tmp ./Documents/test/FILE2 .TMP .vim/filetype.tmp .DS_Store' files. -iname "file*" ./Documents/test/file1 .tmp" . to delete all ./Documents/Letters/ /Users/saruman/osxfaq/templates/tips-week-template/monday./Sites/Tips/unix-tricks/ /Users/saruman/osxfaq/templates/tips-week-template/thursday./osxfaq/templates/template. I can use: find ~ -path "*week*" /Users/saruman/osxfaq/templates/tips-week-template /Users/saruman/osxfaq/templates/tips-week-template/friday.vim/filetype.txt Page 50 of 271 Printed: Tuesday. 2008 3:20:00 PM find .tmp" This command can be used to remove all '.ws /Users/saruman/Sites/Tips/unix-tricks/week1/wednesday. -iname "*.ws /Users/saruman/Sites/Tips/unix-tricks/week1/monday./Documents/test/File3 .vim ..vim ./Documents/Miscellaneous/list.tmp Check the list of files . For example. Selective Paths (See also Weeks 1 and 13) 'find' can search a whole directory structure and return just those files that are in a given range of /Users/saruman/Sites/Tips/unix-tricks/week1 /Users/saruman/Sites/Tips/unix-tricks/week1/friday./Documents/test/file1 find .ws ../. May /Users/saruman/osxfaq/templates/tips-week-template/wednesday. The option '-ipath' is also available.tmp files from the home directory use: cd ~ find . -iname "*. -name "file*" .ws /Users/saruman/Sites/Tips/unix-tricks/week1/tuesday.

the files will be deleted and then the directory. To change directory use: pushd To return use: popd Each command echoes the current directory /Users/saruman/Sites/Tips/unix-tricks/week10/monday. Locate All Empty Files and Directories (See also Weeks 1 and 13) find . which has suddenly become empty. -size 0 will not detect empty directories as they have a non-zero size.Unix_tricks. suppose I wish to find the weekly tips for each /Users/saruman/Sites/Tips/unix-tricks/week1/monday. use the shell's directory stack. If a directory contains just empty files. -empty -delete The delete option will remove files and directories. To remove them use: find .txt Page 51 of 271 Printed: Tuesday. Note that: find . For example: pwd /Users/saruman pushd /System/Library/Extensions/ /System/Library/Extensions ~ pushd /Library/StartupItems/Tablet/" /Users/saruman/osxfaq/templates/tips-week-template/monday. May 27. Pushing and Popping If you issue 'cd' many times and then wish to retrace your steps back to the original directory. All of the files that pass the '-path' filter are filtered again by the '-name" filter. Only files that pass both filters will be displayed. I can apply a second filter to 'find' as in: find ~ -path "*week*" -name " /Users/saruman/Sites/Tips/unix-tricks/week11/ . -empty will do just that for the current directory. 2008 3:20:00 PM Multiple Filters (See also Weeks 1 and 13) Following on from Wednesday's tip. will also be deleted.

txt Page 52 of 271 Printed: Tuesday. set pushdsilent Viewing the Push/Pop Stack If you wish to view the current directory stack. if you don't want pushd and popd to echo the current directory stack. set the shell variable 'pushdsilent'.Unix_tricks. issue the command: dirs /Library/PreferencePanes /System/Library /Developer/Applications ~ If the stack is deep. May 27. 2008 3:20:00 PM /Library/StartupItems/Tablet /System/Library/Extensions ~ pushd /etc /etc /Library/StartupItems/Tablet /System/Library/Extensions ~ popd /Library/StartupItems/Tablet /System/Library/Extensions ~ pwd /Library/StartupItems/Tablet popd /System/Library/Extensions ~ pwd /System/Library/Extensions popd ~ pwd /Users/saruman Silencing pushd and popd Following on from Monday's tip. use: dirs -v 0 /Library/PreferencePanes 1 /System/Library 2 /Developer/Applications 3 ~ Saving the Directory Stack The directory stack may be saved between login sessions by setting the shell variable 'savedirs' set savedirs It may also be saved and loaded manually with the commands: dirs -S and dirs -L For example: dirs /Library/PreferencePanes /System/Library /Developer/Applications ~ dirs -S dirs -c dirs . useful if 'pushd' and 'popd' are silenced.

Unix_tricks. May 27. For example: dirs -v 0 ~/Sites/Tips/unix-tricks 1 /Library/PreferencePanes 2 /System/Library 3 /Developer/Applications 4 ~ pwd /Users/saruman/Sites/Tips/unix-tricks pushd +3 pwd /Developer/Applications dirs -v 0 /Developer/Applications 1 ~/Sites/Tips/unix-tricks 2 /Library/PreferencePanes 3 /System/Library 4 ~ System Cron The Unix cron daemon runs continually looking at /etc/crontab (the cron table). Add your own scheduled commands to this file (which must be edited as the super-user). Jumping to a Stacked Directory 'pushd' will jump to any directory in the stack by specifying '+n' where 'n' is the position in the stack. This file lists commands against the time at which they should be run. 2008 3:20:00 PM /Library/PreferencePanes dirs -L dirs /Library/PreferencePanes /System/Library /Developer/Applications ~ Nore that 'dirs -c' was used to clear the directory stack.txt Page 53 of 271 Printed: Tuesday. .

Unix_tricks. User Crontabs Create a user-specific crontab using the command 'crontab' Create a file with each line of the form: minute hour month-date month day-of-week command and argumants Issue the command: crontab to enter the commands into your crontab. @hourly once an hour == "0 * * * *" @daily once a day == "0 0 * * *" @weekly once a week == "0 0 * * 0" . Stopping 'cron' From Spamming 'cron' will mail you the results of running you crontab commands.txt Page 54 of 271 Printed: Tuesday. To list your crontab: crontab -l and to remove it: crontab -r Note that the format of user crontabs is the same as that for the system crontab except that the 'who' field is omitted. crontab Tricks The time fields in your crontab can be more complex than simple numbers. To prevent this (especially if you do not have sendmail running) add: MAILTO=" to the top of your crontab file. May 27. 2008 3:20:00 PM The '*' means 'on every'. Commands are run as the user who created the crontab file. Each of the fields are as commented in /etc/crontab. use: 0 11. and 8pm. Remove the old crontab and enter the new one (see Tuesday's tip).20 * * * command And to run every 10 minutes: */10 * * * * command Time Shortcuts Instead of the five number fields.16. crontab can take special strings: @reboot run at startup. 4pm. To run 'command' at 11am.

txt Page 55 of 271 Printed: Tuesday.. Roll Your Own The shell allows one to change or add key bindings. issue the command: bindkey -l Emulating Your Favourite Editor The command line editing functions of the shell can be bound to keystrokes that emulate the vi editor.. if you're of a different religion: bindkey -e for GNU Emacs. Binding to Strings 'bindkey' normally binds a key sequence to one of the built-in functions. From the following extract. bindkey -s ^xa "Adrian Mayo" Whenever I type the key sequence 'control-x a'.. shown by 'bindkey -l'..Unix_tricks. it is replaced with my name. .. For example: bindkey ^[a list-glob causes the key sequence 'escape-a' to list globbing.. . 2008 3:20:00 PM @monthly once a month == "0 0 1 * *" @yearly once a year == "0 0 1 1 *" List the Current Bindings The command: bindkey will list all the shell's key bindings. To bind keys to an arbitrary string use the '-s' option. Binding to Commands 'bindkey' can bind a key to a command with the '-c' option. To find out what functions can be bound to keys. we can determine that 'escape-p' ('^[' signifies the escape key) searches back in the command line history. "^[p" -> history-search-backward "^X*" -> expand-glob "^XG" -> list-glob . May 27. and 'control-X *' expands globbing (see Week 25).. bindkey -v Or.

txt Page 56 of 271 Printed: Tuesday. For example: cat f1 f2 A line B line D line F line A line B line C line E line . May 27. The files must be sorted lexically which can be achieved with 'sort' (see Week 28). For example. The -c option reports the repeat count. Filtering Repeated Lines The command 'uniq' will report or filter out repeated lines in a file. Note that it is not necessary to press return after the key sequence. 2008 3:20:00 PM The command: bindkey -c ^xe "ls -alL /etc" will bind the key sequence 'control-x e' to the ls command specified. It reads and writes the standard streams. so file redirection is required to read and write files. to filter repeated lines in f3 cat f3 Line 1 Line 1 Line 2 Line 3 Line 4 Line 4 Line 4 Line 4 Line 4 Line 4 Line 5 Line 1 Line 1 Line 1 uniq -c 2 Line 1 1 Line 2 1 Line 3 6 Line 4 1 Line 5 3 Line 1 1 Comparing Files The command 'comm' will report on lines that are common to two files.Unix_tricks. 'uniq' is useful for trimming log files where the same message is repeated many times. Multiple blank lines are also filtered.

and '-3'. Merge Files with Paste Paste will merge two files. use opton '-23' or '-13' Splitting a Large File Use the command 'split' to break a large file into multiple smaller ones. The options specify which columns to cut out of the report. For the difference lines. split -b5k big-file will split big-file into multiple 5k byte files. May 27.txt Page 57 of 271 Printed: Tuesday. Continuing from Tuesday's example.Unix_tricks. Extracting Common and Difference Lines Using command 'comm' with options '-1'. And: split -l200 big-file split_ will split big-file into multiple 200-line files. '-2'. As an example: cat f1 f2 Line 1 Line 2 Line 3 Line 4 rest of line 1 rest of line 2 rest of line 3 . and the generated filenames will be prefixed with 'split_'. to extract the common lines between f1 and f2: comm -12 f1 f2 A line B line F line or: comm -12 f1 f2 >f3 to write back to a new file. 2008 3:20:00 PM F line comm f1 f2 A line B line C line D line E line F line The command 'diff' also compares files. one can extract common and difference lines between two files. line by line.

'tar' was originally used to archive to magnetic tape (tape archiver).tar This extracts the original files into a directory called 'Sites'. May 27. in this case sites. tar -tf sites.tar.tar .aka zip. The option 'x' says to extract from existing archive.tar bin will add all the files in directory 'bin' to an existing archive called 'sites'tar'. tar czf sites. use: cd ~ tar cvf sites.verbose. Simply add the option 'z' when compressing and extracting.archive to a named file. which is created if it does not already exist. not zip.tgz Don't use capital Z as this compresses using 'compress'.tar The original directory is untouched.tgz Sites and tar xzf sites. the Unix 'tar' command allows one to create an archive (often called a tar ball). To decompress the archive use the command: tar xvf sites.txt Page 58 of 271 Printed: Tuesday. name each file as it is added to the archive f . adding to and listing archives One can add files to an existing tar ball using option 'r' (replace?) instead of 'c'. but now a days it is more often used to archive to a named file.create a new archive v . To examine an archive use option 't' (table of contents).sit' file. 2008 3:20:00 PM rest of line 4 paste f1 f2 Line 1 rest of line 1 Line 2 rest of line 2 Line 3 rest of line 3 Line 4 rest of line 4 creating and extracting tar balls Just as Aladdin Systems' StuffIt allows one to 'stuff' many files into a single '.tar Sites The options 'cvf' do the following: c . The command: tar rf sites. To archive the whole of the ~/Sites directory into a single file called sites. zipping tar balls 'tar' is able to compress and decompress archives using Lempel-Ziv coding (LZ77) .Unix_tricks.

tar Sites differs from: tar cvf sites. 9 (--best) is the highest level of compression but slowest algorithm. but strips the leading /. absolute paths The command: tar cvfP sites.tar /Users/saruman/Sites The first variant creates a tar file that extracts into 'Sites' in the current directory.gz . The option -'P' prevents the leading '/' from being stripped when the archive is made. independent of the current directory. so as you know where the extracted files are going to be placed. use: gzip filename This will replace the file with a compressed version with the extension '.tar before actually extracting.gz' To unzip: gunzip filename or gzip -d filename The default level of compression (which is 6) can be changed.txt Page 59 of 271 Printed: Tuesday. May 27. Compare this with Thursday's tip. 2008 3:20:00 PM path names The command: tar cvf sites. it will still be stripped when the archive is extracted unless the extract command also uses option '-P'. and creates a tar file that extracts into Users/saruman/Sites in the current directory. OS X comes with the GNU version. However. 1 (--fast) is the lowest level of compression but the fastest algorithm. The second variant uses the whole path name. gzip --best filename Zipping Several files To compress more than one file into a single archive. use the '-c' option.tar Zipping I guess the zip compression utility needs no introduction. gzip -c filename1 filename2 > two-files.Unix_tricks. It's a good idea to always try: tar -tf sites. To compress (zip) a file. gzip.tar /Users/saruman/Sites creates an archive that can be extract into /Users/saruman/Sites. tar xvfP sites.

2008 3:20:00 PM This technique can be used to avoid replacing the original file: gzip -c filename > filename. to peek at old system log files: zcat /var/log/system. it probably means that the original files was compressed with 'zip'. plus connectives and proper nouns. or use zcat. The supplemental 'web2a' list contains hyphenated terms as well as assorted noun and adverbial phrases. bzip2 compresses better that gzip. gunzip will decompress '. May 27.Z' files. but part of the fink base package) zless. 'compress' is an older variant. complaining about more than one member. bzgrep. bzless. mail. To decompress such a file use 'unzip' (without the 'g'). A number of 'bz' commands are also available. Too Many Members If gzip gives you an error whilst trying to decompress a file. and bzip2recover for damaged compressions. For example. The wordlist makes a dandy 'grep' victim. and usage is similar to gzip. netinfo. including the system log.txt Page 60 of 271 Printed: Tuesday. man bzip2 will tell you all. and (not standard OS X. Displaying Without Unzipping If you wish to display a compressed file without the hassle of decompressing it and then having to compressing it again. Log Files Visit: /var/log for most of the log files kept by OS X. The Dictionary Visit: /usr/share/dict This has a couple of dictionaries.0. such as bzcat. and uses the extension '. The 1934 copyright has elapsed. use the '-c' option with gunzip. according to the supplier. lookupd.gz There are many more 'z' variants such as zmore.log. ftp. From the README: Welcome to web2 (Webster's Second International) all 234.Z'. and apache. . zdif.936 words worth. Other Compression bzip2 is the new black.gz The '-c' option is actually sending the compressed file to the terminal.Unix_tricks.

Learn About Samba Issue the command: % open /usr/share/swat/using_samba/index.Unix_tricks. The full pathname: % set fn="/Users/saruman/Documents/Essentials/todo.0" 200 809 This will update each time an entry is added to the end of the log.175 .44.ida?XXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX%u9090%u6858%ucbd 3%u7801%u9090%u6858%ucbd3%u7801%u9090%u6858%ucbd3%u7801%u9090 %u9090%u8190%u00c3%u0003%u8b00%u531b%u53ff%u0078%u0000%u00=a HTTP/1.57.rtf Extract the extension: % echo $fn:e rtf .155.207.. Control-c to stop..0" 200 809 217.ida?XXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX XXXXXXXXXXXXXXXXXXXXXXXXXXX%u9090%u6858%ucbd3%u7801 %u9090%u6858%ucbd3%u7801%u9090%u6858%ucbd3%u7801%u9090%u9090 %u8190%u00c3%u0003%u8b00%u531b%u53ff%u0078%u0000%u00=a HTTP/1.html Learn About Cups Issue the command: % open /usr/share/doc/cups/documentation. In this case we see attacks by a Code Red variant.220 .[31/Mar/2003:15:01:18 +0100] "GET /default.txt Page 61 of 271 Printed: Tuesday.html Separating Filename Components I Use the following technique in tcsh scripts to extract components of a full pathname from a variable or parameter. May 27. which is back with us at the moment.rtf" % echo $fn /Users/saruman/Documents/Essentials/todo. running apache we are immune to this attack.[31/Mar/2003:14:22:15 +0100] "GET /default.rtf Extract the pathname (head): % echo $fn:h /Users/saruman/Documents/Essentials Extract the filename (tail): % echo $fn:t todo. 2008 3:20:00 PM Keep an eye on log files with (for example the apache access log): % tail -f /var/log/httpd/access_log 217. Of course.

txt/ todo.html rename . In particular they can be combined with the tips from Week 29.Unix_tricks. -name "*.htm" -exec rename {} html \.htm % find . May 27./c.rtf/.html c.htm c.rtf % echo $fn:t:s/.html b.html c.htm ./a. 2008 3:20:00 PM Remove the extension: % echo $fn:r /Users/saruman/Documents/Essentials/todo Separating Filename Components II Following on from Monday's tip. Friday: % ls *.htm . the ':' modifiers can be combined. % cat rename #!/bin/tcsh set fn_new={$1:r}.htm b./a.html a.html rename .html Extraction on the Command Line The techniques from Monday and Tuesday can be applied to tcsh commands issued on the command line.rtf % echo $fn:t:r todo You can change a filename using substitution: % echo $fn /Users/saruman/Documents/Essentials/todo.txt Page 62 of 271 Printed: Tuesday. To extract just the filename without the extension combine t and r: % echo $fn /Users/saruman/Documents/Essentials/todo.htm ls: No match./c.html % ls !*:r ls * a. . Use the techniques from Monday and Tuesday to write a script to rename.html % ls a.$2 echo rename $1 $fn_new mv $1 $fn_new Combine with 'find' from weeks 1 and 13 to batch rename: % ls a.html b.html c.html b./b./b. rename .txt Batch Rename Mac OS X does not have a rename command capable of renaming multiple files.html rename rename2 and % ls *.htm .

% cat unix2mac . 2008 3:20:00 PM % ls !*:r. -name "$1" -exec $2 {} \.html" will list all matching files in the current directory and subdirectories.html I'll leave you to figure out what is happening.html b. basename and dirname The commands basename and dirname extract the respective parts from a full pathname. Mac to Unix Here is a script to change a file with traditional Mac end of line characters to have Unix end of line characters. % echo $fn /Users/saruman/Documents/Essentials/todo. These are commands and can therefore be used in any script.rtf % echo `basename $fn` todo. rather than /bin/tcsh. and using 'find' is a good approach.html a. fi This script is written in /bin/sh. % cat xfind #!/bin/sh if [ "$1" = "" ]. unlike the tricks at the beginning of the week that are specific to tcsh.html" less will display them.txt Page 63 of 271 Printed: Tuesday. May 27. then find . -name "$1" else find .Unix_tricks. You can write the following script to make this simpler. % xfind "*.html c.rtf % echo `dirname $fn` /Users/saruman/Documents/Essentials Extended 'find' Sometimes one needs to apply the same command to many files. then echo "Usage: `basename $0` filetype [command to -exec]" exit fi if [ "$2" = "" ].html ls *. Don't forget to make the script file executable. % xfind "*.

May 27. here's the reverse: % cat mac2unix #!/bin/sh if [ "$1" = "" ].$1 do mv $fn `basename $fn $1`$2 done The command: % rename txt rtf will change *. 2008 3:20:00 PM #!/bin/sh if [ "$1" = "" ].txt to *. for fn in *. then echo "Usage: `basename $0` filename" exit fi tr \\n \\r < $1 > $1. Use Backquotes .rtf in the current directory.tmp .html" unix2mac Unix to Mac To complement Tuesday's script. It can be adapted to perform other functions. mv $1.Unix_tricks.. then echo "Usage: `basename $0` filename" exit fi tr \\r \\n < $1 > $1.txt Page 64 of 271 Printed: Tuesday. % cat rename #!/bin/sh #I'll leave the error checking to you. mv $1.tmp $1 This script can be combined with the xfind script from Monday's tip: % xfind "*...tmp .tmp $1 Batch Rename Here is a script to rename files in the current directory.

) Enabling 'at' The Unix command 'at' can be used to execute a given command at a given time. For example. bbedit. Uncomment (remove the # from) this line in /etc/crontab: #*/5 * * * * root /usr/libexec/atrun to check the queued 'at' jobs every 5 minutes. 'atq' and 'atrm' One can examine the list of jobs on the current user's 'at' queue with: % atq Date Owner Queue Job# 21:26:00 04/20/03 saruman a a010b400a. The commands to run can be supplied either directly on the command line.deny (Or create 'at.html"` (or vim.. (But not on a given date in OS X). Create an empty 'deny' file to allow all users to run 'at' sudo touch /var/at/at. Create the following directory if it does not already exist: sudo mkdir /var/at/spool 3. However. The following example schedules the Clock application to be launched at 21:30 today.txt Page 65 of 271 Printed: Tuesday. 1.) Using 'at' Use the 'at' command to schedule a (sequence of) command(s) to be executed later. 2. 2008 3:20:00 PM Remember that enclosing a command in backquotes means we can pass its output as arguments to another command.Unix_tricks.. pico. % cat at-file open /Applications/Clock. May 27.000 will be executed using /bin/sh At 21:30 the Os X Clock application will be launched.allow' and add a list of the allowed users. to edit all html files in a directory and its subdirectories (using Monday's 'xfind' script): % vi `xfind "*.000 passing the rather ugly job number.000 and delete jobs with: % atrm % at -f at-file 21:30 Job a010b400a.) (See week 33 for more scheduling commands. or read from a file with the '-f' option. These must be done as root. emacs. . a few tweaks must be made to OS X before it will work.

2008 3:20:00 PM More on 'at' When the list of commands given by 'at' are executed. May 27. If commands need to refer to your home directory use: $HOME Times can be specified relative to now with: at -f at-file now + 10 minutes at -f at-file now + 1 hour The Unix Calendar Use the 'calendar' command to query appointments and events.Unix_tricks. date. The environment variables and the current directory are re-created to match those from when the 'at' command was issued. you can miss out the date specification: % calendar 20 April Sunday's job 04/20 Sunday's job If the file is other than 'calendar' in the current directory use the '-f' option: % calendar -f /path/name/file-name -d 0429 04/29 Dentist Unix has some pre-written calendar files in: /usr/share/calendar For example: % calendar -f /usr/share/calendar/ -d 0421 04/21 San Jacinto Day in Texas 04/22 Arbor Day in Nebraska & Delaware 04/22 Oklahoma Day in Oklahoma 04/21 Tiradentes in Brazil ping . and event on each line. Create a plain text file (default name 'calendar') containing a month. % cat calendar 10 April Thursday's job 20 April Sunday's job 26 April Saturday's job 04/01 April Fool 04/20 Sunday's job 04/29 Dentist Print the events for a given date (specified as MMDD) with the following commands: % calendar -d 0410 10 April Thursday's job % calendar -d 0429 04/29 Dentist For today (20 April in this example).txt Page 66 of 271 Printed: Tuesday. they are done so by the /bin/sh shell.

150) traceroute to (62.2) 19.255 See if your router responds to: % ping 0 and who responds to: % ping 255.wcg.78) PING apple. eg: % ping 10.922 ms 64 bytes from 17.zen.3.0.819 ms 0.353 ms 2 gauss-dsl1.063 ms 99.251 Ping the broadcast address of your local subnet to discover all the hosts on (195.wh.292 ms (17.255 You can ping an AppleTalk host using 'appleping'.254.994 ms 7 nycmny2wcx2-oc12. traceroute The traceroute utility shows the path or route to remote hosts by reporting on each 'hop' a packet makes on its way through the ( ( (The name 'ping' comes from the sound sonar makes.859 ms 169.229 ms 0.check if your network connection is in good order .1.233 ms 6 lndnuk1icx1.255 or % ping (62.254.168.check if a host (eg apple.843 ms 169.83.149) 100.zen ( ms 4 deleuze-ge-0-2-0.37) 169. % traceroute (62.wh. For (64. 40 byte packets 1 valinor.893 ms .com) is alive .183: icmp_seq=1 ttl=45 time=165.find the IP address of a named host For example: % ping apple.122 ms 22.zen.105) 29.78 ms (64.946 ms 22.049 ms 29.155. to ping Apple once a minute (specified as 60 seconds) for 30 minutes: ping -i 60 -c 30 apple.183): 56 data bytes 64 bytes from 17.183: icmp_seq=0 ttl=45 time=165.te.hq.wcg.) ping tricks Try pinging all the hosts on your local network to see which are running Rendezvous: % ping May 23. 2008 3:20:00 PM We all know of the venerable 'ping' utility.936 ms 9 chcgil1wcx3-oc48.224.txt Page 67 of 271 Printed: Tuesday. 30 hops max.1.wcg.212 ms ms 5 suarez-so-0-0-0.254.78) 22.222 ms 8 nycmny2wcx3-oc48.541 ms The '-i' option can be used to change the interval between ping transmissions.84.3.62) 29.17 ms 20.266 ms on the general responsiveness of all things en route to a particular host (the 'time=' field) .wcg. and the '-c' option used to send a specified number of pings.589 ms 29. It's a long way to OSXFAQ.3..294 ms 169.521 ms 29.385 ms 21.3.Unix_tricks.30) (64.999 ms 3 chrysippus-ve-131. . ms 22.

154 ms (64.1002 udp4 0 0 localhost.242 ms 168.197 ms 167.472 ms 170.visualware.55038 osxfaq.1.758 ms 171.ntp *.http ESTABLISHED tcp4 0 0 saruman. try the '-p' option with a big port number.785 ms 177.171 ms 11 snfcca1wcx3-pos11-0.zen.ntp *.63 ms 171.1002 udp4 0 0 localhost.977 ms visual traceroute Try this cool web site for a more graphical version of traceroute: http://visualroute.m-l.116 ms 172.http CLOSE_WAIT tcp4 0 0 localhost.1006 localhost. May (64.200.053 ms If traceroute fails.1019 ESTABLISHED tcp4 0 0 localhost.http ESTABLISHED tcp4 0 284 saruman.1019 localhost. Some examples: To list current connections.128 ms 170.301 ms 14 is10-0-0m..200.64 ms 18 ms 17 ife0-0m.94) 170.84.951 ms 170.. eg: % traceroute -v -p 35353 osxfaq.1033 ESTABLISHED tcp4 0 0 localhost.1033 *.198.160.253) 173. .scotthaneda.1033 localhost.496 ms 173.* LISTEN udp4 0 0 saruman.403 ms 12 snfcca1wce1-pos3-0.978 ms 15 if0-0-0m.029 ms 182.55039 osxfaq.nacio.238) 169..154 ms 174.239.177) 173..240.* udp4 0 0 localhost.1033 localhost.30) 179.1033 *.200. routing (64.821 ms 19 osxfaq.1033 ESTABLISHED tcp4 0 0 localhost. To list your machine's routing tables: % netstat -r Routing tables Internet: ..804 ms 178. .txt Page 68 of 271 Printed: ( (64.* Active LOCAL (UNIX) domain sockets Address Type Recv-Q Send-Q Inode Conn Refs Nextref Addr 25b2540 stream 4782 0 0 0 0 0 25b23b8 stream 0 0 0 25b24d0 0.wcg..84.wcg.nov55c-75-002.1006 ESTABLISHED tcp4 0 0 localhost.. .49155 localhost.822 ms ( 172.194) (167.239. or just to probe around.18) 168...06 ms (64.414 ms 173. and such like.675 ms 13 snfcca1wce1-naciosystems-se.Unix_tricks.199.37.409 ms 16 ( (64.195) 172.265 ms 171.zen. 2008 3:20:00 PM 10 dnvrco1wcx2-pos10-0.* udp4 0 0 netstat netstat gives you lots of vital statistics on your network osxfaq.sfob-02.nacio.66) 169.49154 localhost.84.634 ms 172.nacio. including hostnames and local port numbers: % netstat Active Internet connections Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp4 0 286 saruman. It is useful to debug you connection.242.nacio. mail is handled (pri=10) by mail. your router name/IP address and its MAC address.15 mayo-family.155. mayo-family. To do the reverse. plus the various names and addresses assigned to your own machine. % host -a mayo-family.zen UGSc 36 433 en0 localhost localhost UH 8 26887 lo0 169.183 183.) You should see the default router. simply give the IP address: host 17.' (as in the above example) to ensure that it is not interpreted relative to your own 'host' Tricks 'host' has a whole host of options.zen 0:50:7f:6:82:34 UHLW 36 201 en0 1072 Internet6: Destination Gateway Flags Netif Expire UH lo0 fe80::%lo0 Uc lo0 link#1 UHL lo0 fe80::%en0 link#4 UC en0 0:3:93:b2:d6:4 UHL lo0 fe80::%en1 link#5 UC en1 0:30:65:6:59:b2 UHL lo0 ff01:: U lo0 ff02::%lo0 UC lo0 ff02::%en0 link#4 UC en0 ff02::%en1 link#5 UC en1 (Use option '-n' to display IP addresses instead of hostnames. 2008 3:20:00 PM Destination Gateway Flags Refs Use Netif Expire default This command also returns the mail server for the domain. i.txt Page 69 of 271 Printed: Tuesday.84.37. The command: arp -a shows you all the hosts on the local subnet for which your machine has a MAC to IP address link#4 UCS 1 0 en0 saruman.3. Query Host Information with 'host' To find the IP address of a host use: % host Trying null domain rcode = 0 (Success).168. When connected to a router you will probably just see the details of your router. The most useful is '-a' to grab all records and display in verbose mode. May 27. lookup a hostname from an IP has address 64. ancount=5 .zen localhost UHS 0 96 lo0 valinor.17. It is advisable to end the domain name with a '.254 link#4 UCS 0 0 en0 217.e.ARPA domain name pointer apple.

240. class = IN .euro. got answer: 883 IN A 79546 IN MX 10 mail.. status: NOERROR.118 Note that: * A = IP Address * MX = mail server * NS = name server 'dig' and 'nslookup' The commands 'dig' and 'nslookup' are alternatives to 'host' and may be better suited to your needs. QUERY SECTION: . id: 2 .com.144. apple.72. 1d23h43m9s IN NS nserver.opcode: QUERY.37. apple.3 <> 79546 IN NS 79546 IN NS 79546 IN NS dns2.intermag. ANSWER SECTION: ADDITIONAL: 4 . 4d22h14m31s IN A 17. AUTHORITY SECTION: 65327 IN A 882 IN A 216..254. 2008 3:20:00 PM The following answer is not authoritative: The following answer is not verified as authentic by the server: mayo-family.84. 23h13m18s IN A 1d8h48m29s IN A 203. ANSWER: 1.183 .com.112.112. QUERY: mayo-family. see: mayo-family.. 1d23h43m9s IN NS nserver3. 26m56s IN A Additional information: flags: qr rd ra.intermag. % dig apple.120. 1d23h43m9s IN NS nserver.txt Page 70 of 271 Printed: 79546 IN NS .6 ns1. ADDITIONAL SECTION: 79546 IN A . AUTHORITY: 79546 IN NS ns1. mayo-family.144.64 79546 IN NS dns2. 1d23h43m9s IN NS nserver4. 1d23h43m9s IN NS nserver.5 nserver. ->HEADER<. For authoritative answers. 4d22h14m31s IN A 17. res options: init recurs defnam dnsrch . type = .15 mayo-family. 1d23h43m9s IN NS 80083 IN A mayo-family.104 dns3. May 27. <> DiG 8.3..intermag. ..

.COM Name Server: DNS3. and name servers for a Name Server: DNS2. England BA15 1BL UK . WHEN: Sun May 4 20:47:50 2003 .zen. Total query time: 47 msec .. % whois mayo-family. owner.txt Page 71 of 271 Printed: Tuesday.Unix_tricks. 'whois' The 'whois' command queries whois.212.mayo-family.COM. If the domain was not registered by internic.1 Non-authoritative answer: Name: mayo-family..COM Status: ACTIVE Updated Date: 27-feb-2003 Creation Date: 19-jul-2001 Expiration Date: 19-jul-2003 .INTERMAG... Referral URL: http://www.bulkregister.8. use the 'man' command to discover more.INTERMAG. a second whois is Address: Adrian Mayo 54 Greenland Mills Bradford-on-Avon. MSG SIZE sent: 27 rcvd: 252 % nslookup apple.internic..bulkregister. In this case a query to the registrar's whois server is required using option '-h' Whois Server Version 1.HOSTWIZARD.254. FROM: saruman.. Domain Name: MAYO-FAMILY. Whois Server: whois. % whois -h .com.1 . 2008 3:20:00 PM .bulkregister. Server: ns0.183 As ever.COM Registrar: Address: 212.3. May to SERVER: default to discover the registrar.COM Name Server: NS1.

COM Administrative Contact: Carlton Thomas domreg@gifford.nic. Stoke Gifford Registrant: British Broadcasting Corporation Registrant's Agent: The British Broadcasting Corporation t/a BBC Technology [Tag = BBC] Relevant Dates: Last updated: 13-Mar-2001 Domain Name: bbc.COM' Gifford Internet Services 14 Tyrrel Way. 2008 3:20:00 PM Domain Name: MAYO-FAMILY. % whois -h whois.200.104 DNS3.Unix_tricks.218.INTERMAG.118 NS1.COM bbc. one has to use another 'whois' service. Stoke Gifford Bristol. May 27.COM England BS34 8UY UK Phone: +44 117 9397722 Fax: +44 117 9397733 Record updated on 2001-07-19 11:28:07 Record created on 2001-07-19 Record expires on 2003-07-19 Database last updated on 2003-05-04 15:58:47 EST Domain servers in listed order: DNS2.14 'whois' for non TLD To do a 'whois' query on a non Top Level Domain. England BS34 8UY UK Phone: +44 117 9397722 Fax: +44 117 9397733 Technical Contact: Carlton Thomas for example ' Gifford Internet Services 14 Tyrrel Way.txt Page 72 of 271 Printed:

uk 132. pwd . 2008 3:20:00 PM Name servers listed in order: ns. pwd).Unix_tricks.err Redirection 3 The 'tcsh' does not provide an obvious way to redirect standard error. To execute several commands on one line. pwd / / Redirection This applies to the 'tcsh' 212.132. To redirect standard output use '>'.co. in a new shell. May 27. For WHOIS database last updated at 21:20:01 04-May-2003 And if anyone knows of a reliable way to determine the whois address for the various TLD and country domains. Sub Shells To execute a command.. pwd / /Users/saruman % cd / .txt Page 73 of 271 Printed: compare the following two commands that make use of both of the 132. % (cd /.bbc.150. Use the following trick to do so: (ls -al filename > /dev/tty) >& bracket the commands. but not standard out. To redirect both standard output and standard error use '>&".185.132.224. or commands.21 Eg: ls -al kfhsdhdjhsjkhf >& list Redirection 2 The 'tcsh' shell does not provide an obvious way to redirect standard output and error to different files.58. let me know.21 ns1.thny. Use the following trick to do so: (ls -al filename > list ) >& list. separate them with a 38. Eg: ls -al * > list Output and error messages write to different streams.err ns1.

txt Page 74 of 271 Printed: Tuesday.65 /Applications/Clock. Many Ways to Kill The 'kill' and 'killall' commands send a 'TERM' signal to the process. % ls >! list % Notice the use of '!' to temporarily override 'noclobber'.list list.Unix_tricks.. For example: % ps -axww | grep -i clock 15048 ?? S 0:00. % ls > list list: File exists. A PID of -1 broadcasts the signal to all process for which you have the necessary permissions to kill. The '-m' option of 'killall' takes a regular expression as the process name.. Other signals are available... May 27. we can use: killall Clock which kills by (case sensitive) name. set the 'tcsh' 'noclobber' variable.. For example: % set noclobber % ls .00 grep -i clock and then issue the 'kill' command: % kill 15048 Note the use of option 'ww' which ensures that the output from 'ps' is not truncated losing the name we are grepping for. 2008 3:20:00 PM No Clobber If you wish to avoid accidentally overwriting files with redirection. This always kills the process. including -KILL .err . -HUP .app/Contents/MacOS/Clock 15054 std R+ 0:00. which is a request to that process to terminate. Be careful as this command kills all process that match the given name.tell the process to restart.. we first need to find its Process ID or PID.make the process an offer it can't refuse.. Alternatively. This is useful to restart demons (like configd and lookupd). . and kills all those processes that match it. Killing Time To kill an application (or process) from the command line. as they will re-read their configuration files on receipt of a -hup signal.

conf utmp These should be independent of the name of the ndc pppconfd syslog configd. For proxy ntpd. www servers should report their pid at ' niconfig_local.txt Page 75 of 271 Printed:'.pid' else echo "restarting demon $1" sudo kill -HUP `cat /var/run/$1. You can make a handy alias to restart a | sed -e 's/\. such as: % alias nfs-mount 'sudo kill -HUP `cat /var/run/ sudo autodiskmount. and: % man sigaction gives more` fi It will list the demons: . The Hanging Script Here is a handy script to restart a demon: % cat hanger #!/bin/sh if [ "$1" = "" ]. then echo "List of demons:" cd /var/run ls -1 *.Unix_tricks.drift StartupItems davlocks`' % alias nfs-export 'sudo kill -HUP `cat /var/run/mountd. Find Your Victim Many Unix demons kindly leave their address. Check out: % ls /var/run/ AppleFileServer. The Killing Script Here is a handy script to kill processes by name: % cat killer #!/bin/sh kill $(ps auxww | grep -i $1 | grep -v $1 | awk '{print $2}') % killer clock *gone!* Remember to make the script`' which will restart the NFS client and server respectively after new shares and connections have been mDNSResponder. 2008 3:20:00 PM The command: % kill -l lists all the available signals. May 27.xml resolv.

Apache on Mac Os X servers from /Library/WebServer/Documents. like ~/Sites. you must restart Apache to cause it to re-read them. 2008 3:20:00 PM ~ % hanger List of demons: AppleFileServer autodiskmount automount configd cron httpd inetd lookupd mDNSResponder named netinfo_local ntpd sshd syslog xinetd and kill them: % hanger lookupd restarting demon lookupd Remember to make the script executable. May 27. Start and Stop Apache To start and stop apache from the command line use the Apache control utility: % sudo apachectl start /usr/sbin/apachectl start: httpd started % sudo apachectl stop /usr/sbin/apachectl stop: httpd stopped To cause Apache to be started when your Mac boots.conf Replace this line (line 363?): . If you wish to change this to another directory. and change WEBSERVER=-NO- to WEBSERVER=-YES- Restart Apache If you change Apache's configuration files. edit Apache's configuration file: % sudo pico /etc/httpd/httpd.Unix_tricks. To do this gracefully (without disturbing existing connections on a live server) use: % sudo apachectl graceful /usr/sbin/apachectl graceful: httpd gracefully restarted Change WebServer Root By default.txt Page 76 of 271 Printed: Tuesday. edit the file: /etc/hostconfig as root.

% which which which: shell built-in command. Examine the access log to see who is viewing your site. To list all builtins: % builtins : @ alias alloc bg bindkey break breaksw builtins case cd chdir . % which \which /usr/bin/which .conf Search for: CustomLog "/private/var/log/httpd/access_log" common and replace with: CustomLog "/private/var/log/httpd/access_log" combined List the Shell Built-ins The tcsh shell has many useful built-in commands (commands that are part of the shell and do not live as separate executables in /bin etc). You can change the amount of logging information written to these files. For example. to add the referrer (the site which linked to your site) and the user agent (the browser) to the log file. May 27. 2008 3:20:00 PM DocumentRoot "/Library/WebServer/Documents" with: DocumentRoot "/Users/your-user-name-here/Sites" and this line (line 388?): <Directory "/Library/WebServer/Documents"> with: <Directory "/Users/your-user-name-here/Sites"> The Apache Manual Access the Apache on-line manual with the URL: http://localhost/manual/ The Apache Log Files Apache writes logging information to: /private/var/log/httpd/error_log and: /private/var/log/httpd/access_log Examine the error log if you experience problems in setting up the server. the shell built-in will be executed. do the following. Edit the apache configuration file: % sudo pico /etc/httpd/httpd.Unix_tricks. If the same command exists as both a separate executable and a shell built-in. To override this.txt Page 77 of 271 Printed: Tuesday... precede the command with '\'.

Try this: % hashstat umask The umask built-in command tells the file system the default permissions to apply to newly created files and directories. 1 saruman staff 0 Jun 9 11:22 test-file Give write access to your group: % rm test-file % umask 002 % touch test-file % ls -al test-file -rw-rw-r-. it will search its own built-in commands first. If you add a new command or shell script to one of these directories. 1 saruman staff 0 Jun 9 11:23 test-file For shared files: % rm test-file % umask 000 % touch test-file % ls -al test-file -rw-rw-rw. or applied just to a partiaular Terminal session. To see the path.txt Page 78 of 271 Printed: Tuesday. the shell searches these directories when it first starts.1 saruman staff 0 Jun 9 11:16 test-file The umask is usually set to 022: % umask 022 % touch test-file % ls -al test-file -rw-r--r-. The umask can be set in your tcsh startup file. 2008 3:20:00 PM and try this: % \which which The Command Hash Table When the shell is given a command to execute.Unix_tricks. May 27. 1 saruman staff 0 Jun 9 11:23 test-file . issue either of: % echo $PATH % echo $path In order to speed up this search process. the shell will not recognize it. and caches the information in memory. Without a 'umask' directories are set to: drwxrwxrwx 2 saruman staff 68 Jun 9 11:16 test-dir and files to: -rw-rw-rw. To get the shell to refresh its cache issue the built-in command 'rehash' % rehash No response means all is ok. then all the directories specified in the PATH or path environment/shell variables.

.sh UID PID PPID CPU PRI NI . a command is issued with a nice value of 4: % .Unix_tricks.. 2008 3:20:00 PM For private files: % rm test-file % umask 077 % touch test-file % ls -al test-file -rw------. 0 10142 10130 0 41 -10 . May 27. which adjusts the priority of running processes: % man renice exec The built-in command 'exec' allows one to run a command in place of the shell... 1 saruman staff 0 Jun 9 11:23 test-file Be Nice The 'nice' shell built-in is used to execute a command with a priority greater or less than the default priority.. Check the (not built-in) command 'renice'.. % exec my-daemon Shell Startup . 501 10036 3457 0 11 20 . 501 9997 3457 0 27 4 ... one must be a super-user: % sudo -s # nice -10 % ps -axlhww | grep test.txt Page 79 of 271 Printed: UID PID PPID CPU PRI NI ./ UID PID PPID CPU PRI NI .. This can be useful to start a command that runs as a daemon and does not require its parent % ps -axlhww | grep test. The shell starts the command and exits. A negative nice value ensures that command gets a greater share of the CPU (high prioriry). Normally./test. showing a priority (PRI) of 31-4=27 and a nice (NI) of 4 To issue a low-priority command use: % nice +20 % ps -axlhww | grep test. showing a priority of 31-20=11 (low) and a nice of 20 as specified To increase the priority./test. and a positive value ensures the command see less of it (low priority).

These you are free to change (or create) if you wish to customise your own command line environment. apologies for the lack of tips last week . Login and Interactive Shells What are login and interactive shells? Opening a Terminal window starts an interactive login shell.tcshrc ~/. and the second two are per-user. Therefore.login whereas non-login shells do not.tcshrc'.txt Page 80 of 271 Printed: Tuesday.Unix_tricks.I'm fit and well again for this week. The two are interchangeable. the script file: ~/.cshrc' is no longer executed.LoginShell: True Login-only Commands Optimize the shell startup sequence by using the login-shell-only startup files: /etc/csh.login You may add Unix and 'tcsh' shell commands to these files. never rely on having both of these files executed.login ~/. tcshrc and cshrc For backward compatibility with the 'csh' shell (from which the 'tcsh' shell was derived).login In these files set up aspects of the environment that persist. Non-interactive shells take their commands from a script. Login shells differ in that they read and execute: /etc/csh. The first two files are global to all users.Xdefaults: # xterm XTerm*. Some terminals (such as xterm that runs in X11) by default do not start a login shell. add this to ~/. To get xterm to do so. Typing 'tcsh' at the command line spawns an interactive non-login shell.login ~/.tcshrc'. When the 'tcsh' shell starts up it reads and executes several script files.cshrc' is executed. Interactive shells are attached to a Terminal window so you can type commands into them.cshrc /etc/csh. In the absence of '~/. and stick to it even if you are advised to add lines to the other. 2008 3:20:00 PM First off. They are: /etc/csh. Once you create '~/. Choose one. such as environment variables. and therefore behave more like Apple's Terminal. Executing a shell script starts a non-interactive non-login shell. .cshrc may or may not be executed when the 'tcsh' shell starts up. '~/.login ~/.tcshrc'. '~/. preferably '~/. May 27.

and 31) have dealt with 'find'./week5/thursday.. 2008 3:20:00 PM This applies especially to commands like: setenv PATH ${PATH}:xxx which add to an environment variable.tcshrc if ($?prompt) set interactive Then surround all interactive-only commands with: if ($?interactive) then .ws . .tcshrc are not necessary for non-interactive" -exec grep -il warning {} \.. grep -il warning file1 file2 file3 file4 .ws" | xargs grep -il warning . Optimize the shell startup sequence. endif 'xargs' Previous tips (from weeks .txt Page 81 of 271 Printed: . which will include the output from 'find'. 13./week5/ .../week5/thursday../week21/thursday.. A command like this must not be executed more than once otherwise your $PATH will grow and grow. -iname "*.ws . thus. -iname "*. interactive only commands are here./week13/thursday.. This weeks tips expose more tricks using this powerful command in combination with the 'xargs' command. and the bindkey command./week13/ . Include this statement near the top of ~/ The same effect can be achieved by piping to 'xargs': % find .ws ../week9/monday.. It forms a command from its parameters./week9/monday. Week 13 gave an example using the '-exec' option of 'find' to execute a command on each file found: % find ./week5/friday. Most in fact./week21/thursday. . Interactive-only Commands Following on from Wednesday's tip. such as setting shell variables and aliases. Filter these commands out for non-interactive . Many commands in ~/.. May 'xargs' is a Unix command in its own right and is not specific to 'find'.

ws ./week10/thursday./week9/ .ws . 'xarg' option Monday's example. 2008 3:20:00 PM 'xargs' vs '-exec' Referring back to Monday's tip. -iname "*. In this . Why? Because '-exec' will execute the 'grep' command once for each file found . -iname "*..causes 'xargs' to call 'grep' once it has 100 filenames./week5/friday./week8/friday. 'xargs' on the other hand absorbs all the arguments and passes them to 'grep' in one . ./week8/monday../week9/monday.some may tell you that piping to 'xargs' is less efficient than using the built-in '-exec' ./week9/ ./week21/ . Limit this by using option '-n' which states the maximum number of parameters that 'xargs' will pass to the command (in this example to 'grep') % find .ws" | xargs -n100 grep -il warning .ws ./week10/monday./week10/friday. -iname "*../week1/friday. 'xargs' would call 'grep' four times. If a command takes only one file at a time. and in fact the latter example will run at least ten times as fast as the .ws .ws" | xargs grep -t -il warning grep -il warning .ws" | xargs grep -il warning .. -iname "*.ws .. % find . many files.Unix_tricks.. -iname "*./week5/thursday./week8/thursday./week1/monday./week9/monday./week1/thursday. over 200 files matched "*. versus: % find . ../week8/" | xargs -n1 my-simple-script Option '-t' is useful (trace) .. If 350 filenames were found. the resulting command line formed by 'xargs' may be too and causes 'xargs' to echo to standard output the command it has formed and will .ws Too Many Files? If 'find' finds many.. May . % find ./week8/ .ws ./week1/ . -iname "*.txt Page 82 of 271 Printed: Tuesday./week1/ .ws" | xargs grep -il warning .ws ./".works because 'grep' can take a long list of files to process./week9/" -exec grep -il warning {} \.ws . This is not so. % find ./week9/ ./week13/thursday. % find . use option '-n1'.

and is available in three flavours: pure Terminal.Unix_tricks.txt Page 83 of 271 Printed: Tuesday. -iname "*..back to start of previous line ^ back to start of line b back to start of word w forward to start of word $ forward to end of line + forward to start of next line ) forward to start of next sentence } forward to start of next paragraph G to end of file (plus the obvious cursor keys). and Paste in Vim ./a file name. the 'find | xargs' combination will The Mac downloads page is: http://www. Vim is an improved version of 'vi'. or Carbon GUI. The main vim homepage is: http://www.doc Get Vim Get vim. May 27./b: No such file or directory grep: file: No such file or directory grep: name. followed by an ASCII NUL character (character code 0).vim. Cursor Movement in Vim Learn basic cursor movement keystrokes.doc" | xargs grep -il "Dear Sirs" grep: .ch/software/macosx/welcome.doc % find ..php#mac For the Carbon GUI: http://macvim.swdev. % ls -1 a file name. Mac OS X ships with 'vi'. 'find' will print the pathname of the current file. 2008 3:20:00 PM Filenames With Spaces If filenames have spaces. use option '-0' with 'xargs' so it will use NUL (``\0'') instead of whitespace as the argument separator. Copy. % find .doc: No such file or directory grep: Marc Liyanage maintains command line versions for pure Terminal and X11: http://www. The most useful are: { back to start of paragraph ( back to start of sentence .doc b file name. but some handy tips for those who have got beyond the 'esc' and 'i' keys./a: No such file or directory grep: file: No such file or directory grep: name. Cut. -iname "*.org/download.doc: No such file or directory Use option '-print0' with 'find'. X11. Next.html#vim This week's hints are not a vim tutorial for the absolute beginner.doc" -print0 | xargs -0 grep -il "Dear Sirs" .vim.

Additional: '' (two single quotes) toggles between current and last cursor position 'a moves to the start of the marked line `a moves to the marked character position within the marked line Set bookmarks across different files with the 36 interfile books marks mA (to mZ. no mater where the cursor is within the sentence Shortcuts: dd is short for delete current line yy is short for yank current line Use the 26 buffers in Vim Vim has 26 named buffers (a to z).. and paste: d deletes (cut) y yanks (copy) p pastes (below the current line.Unix_tricks. and P pastes above it) y and d must be followed by a cursor movement key (see Tuesday's tip). copy.txt Page 84 of 271 Printed: Tuesday. Recall as usual with 'A or `A (to 'Z. Examples: d$ deletes to the end of the line d) deletes to the end of the sentence (y) copies the entire sentence. "Ad$ will cut to the end of the line. Continuing from Wednesday's example: "ay) copies to the end of the sentence and places the text into named buffer "a "ap pastes the text that is in buffer "a Note that the unnamed buffer is always written to in addition to any named buffer. Place a bookmark with ma (to mz) and recall it (move the cursor back to it) with 'a (to 'z). . Copy or paste a portion of text with: <move to one end of desired selection> ma <move to other end of selection> "by'a This will additionally copy the section into named buffer "b.Sum File Sizes The tips for this week are culled from my replies to the many emails I receive asking: "How do I do such and such?" . and the command will dump into or from that buffer. May 27. 2008 3:20:00 PM Use cut. and will copy or delete from the cursor to the move target. and append the cut text to any that was already in buffer "a Bookmarks in Vim Vim has 26 named bookmarks (a to z). and m0 to m9).. '9). Precede a command by a buffer name. You can append text to a buffer by naming it with an uppercase letter. in addition to the unnamed general buffer.

May 27.7M .}" 139252 /Users/saruman/Library 267020 /Users/saruman/Pictures/complete 357432 /Users/saruman/Pictures/iPhoto Library/2001 214084 /Users/saruman/Pictures/iPhoto Library/2002 195788 /Users/saruman/Pictures/iPhoto Library/2003/01 124948 /Users/saruman/Pictures/iPhoto Library/2003/05 149976 /Users/saruman/Pictures/iPhoto Library/2003/06 545460 /Users/saruman/Pictures/iPhoto Library/2003 1149224 /Users/saruman/Pictures/iPhoto Library 1548924 /Users/saruman/Pictures 1902804 /Users/saruman This reports on all directories who size in K-bytes is six digits or more (controlled by {6.Delete Blank Lines How do I delete empty lines from a file? Using 'sed': % sed '/^$/d' in-file > out-file Using 'tr': % tr -s \\n < in-file > out-file If the file has blank lines containing spaces: % sed '/^[ ]*$/d' in-file > out-file .. Use -k instead to see file sizes in K-bytes../Pictures/web-site/misc/web5.316M. 2008 3:20:00 PM How do I find the total size of all Photoshop files (*. ./Pictures/web-site/osxfaq/banner1.psd 1.psd 316M total There! . GNU package 'fileutils' contains superior versions of many basic commands.txt Page 85 of 271 Printed: Tuesday. The Mac OS X standard 'du' command does not understand option '-h' (human readable output)./Pictures/complete/ferdi-gala.Seek Out Large Directories How do I report on all Directories who's contents exceed 100 Meg? Previous week's tips have covered a similar question to report on all individual files that exceed a given size. find all files that exceed 20 Meg: % find .psd" -print0 | xargs -0 du -ch 15M .psd 96k .. It is most easily got using Fink./Pictures/web-site/osxfaq/btn-./Pictures/web-site/osxfaq/banner2.Unix_tricks..psd 48k ./Pictures/complete/ferdi-coll3.. 600k ./Pictures/complete/ferdi-cool.. ./Pictures/web-site/osxfaq/end.}).psd 30M . -size +40000 To apply this to directories use: % du -k ~ | egrep "^[0-9]{6. For example.psd 72k .psd 28k .psd) in my home directory? % find ~ -iname "*.psd . including 'du' and a colour version of 'ls'.

and no directory names have spaces. best illustrated by an example./sh2 . it reads and writes files.. to strip spaces use: % tr -d " " < in-file > out-file How can we use a command like 'tr' in a shell script.txt Page 86 of 271 Printed: Tuesday.Unix_tricks. providing no two filenames in a directory differ by only spaces.Remove Spaces from Filenames How can I eliminate spaces in all my filenames? This 'sh' shell script will do the trick. The secret is to stream the value of the input parameter. For example. thus: In tcsh use: set var=`echo $var | tr -d " "` In sh use: var=$( echo $var | tr -d " " ) . to strip spaces from the contents of a variable? The effect we want is as if we could say: var = tr ($var). Like most commands. OS X 'sed' command does not seem to understand '\t' as meaning tab. 2008 3:20:00 PM If the blank lines have tabs too: % sed '/^[ \t]*$/d' in-file > out-file BUT.. -print0 | xargs -0 -n1 .. and capture the output stream. The command 'tr' translates or strips characters from the contents of a file.. then echo mv "$1" "$target_fn" mv "$1" "$target_fn" fi % ls -R1 .: dirname file name file2 name sh sh2 tsh dirname: file2 name file3name % find .Treat a Command as a Function This addresses a generic question. % cat sh2 #!/bin/sh target_fn=$( echo $1 | tr -d " " ) if [ ! "$1" = "$target_fn" ]. or standard IO. May 27. .

txt Page 87 of 271 Printed: Tuesday. which are then passed to the second 'grep'.log.tmp . Grep for Several Strings (AND) How do I use 'grep' to search for files containing string1 AND string2? % grep -l "apple" *. May 27.log .txt | xargs grep -H "microsoft" '-l' (letter ell) in the first 'grep' causes it to list just filenames. They are run via /etc/crontab.: dirname file2name filename sh sh2 tsh dirname: file2name file3name Grep for Several Strings (OR) How do I use grep to search for files containing string1 OR string2? % grep -e "apple" -e "microsoft" *./dirname/file2name mv .log > x./filename mv . Standard 'grep' only understands 'regular' regular expressions.log.txt '-e' introduces a regular expression to 'grep'./dirname/file2 name . Perl and 'awk' will do this too.txt Extended 'grep' understands extended regular expressions. '-H' tells the second 'grep' to print the filename for each match. Is there a more elegant way to do this with 'grep'? Trim Log Files How can I trim my log files to contain just the last 1000 lines? The OS X 'periodic' scripts in /etc/periodic do a good job of taming growing log files.Unix_tricks. This command trims a log file to just the last 1000 lines: % tail -n1000 x.tmp x./file2name % ls -R1 . 2008 3:20:00 PM mv . and I plan to cover them in a Learning Centre tutorial. which searches each file for an occurrence of any of the regular expressions listed. Using extended 'grep' ('egrep' or 'grep -E') one can specify: % egrep "(microsoft|apple)" *./file name . mv x./file2 name .

This doesn't always work file: % sudo cat /var/run/sendmail. Diffing Two Files with Vim The command: % vimdiff file-1 file-2 will start vim with two files. which is useful for comparing two files side by side.. sudo kill -HUP `head -n1 /var/run/$1. or all text files.../sh2: Bourne shell script text .pid` .pid 11024 /usr/sbin/sendmail -L mta -bd -q30m Change the script to use 'head' instead of 'cat'. But what about file contents. Use the 'file' command. :vert new [file-1] will split the screen vertically and optionally read file-1 into the second buffer. combined with 'find' and 'grep': % find .Unix_tricks. which was what was required for this question./tsh: Tenex C shell script text will find all 'sh' and 'tcsh' scripts. Restart a Daemon Week 45. because some cheeky little daemons write additional information to their . Use ctrl-WW (or ctrl-W ctrl-W) to toggle between buffers/files. in my home directory? We all know how to use 'find' to do this based on file extension. The magic is specified by /etc/magic. May 27./sh: Bourne shell script text . and mark all differences between the files. Edit Many Files with vim Use the 'new' command to split the screen to edit many files in the same Terminal window. so as to cut the additional lines: .txt Page 88 of 271 Printed: Tuesday. Friday gave a script to do just this. :new splits the screen in half and creates a new edit buffer.. create a vertical split. . -print0 | xargs -0 file | egrep "(Bourne|Tenex)" . Vertically Split Screen in Vim Vim can split a window vertically too. 2008 3:20:00 PM Find Files of a Given Type How do I locate all Bourne shell scripts. and: :new file-1 splits and reads 'file-1' into the new edit buffer.

May 27. if you are editing file-1 already.'by a<CR> " star c .put buffer a at current cursor location map \P :pu a<CR> " backslash Y .'bd<CR> " backslash P . List the buffers with: :buffers and jump to a specific buffer (eg buffer 2) with: :b2 Vim Shortcuts For those familiar with vim shorcuts.txt Page 89 of 271 Printed: Tuesday. 2008 3:20:00 PM Alternatively.convert to lowercase map *c :'a. use: :diffsplit file-2 or :vert diffsplit file-2 to load file-2 and view the file in list :prev . Submitted by Mike Schienle " typos ab teh the ab pritn print " key maps " these rely on blocks marked as a (begin) and b (end) " \D delete everything between marks a and b map \D :'a.previous file in list :wn or :wprev is necessary to write out changes and then edit the next file.yank a-b into buffer a map \Y :'a. Batch Edit Many Files with Vim The command: % vim f1 f2 f3 f4 f5 will cause vim to read the five named files. here are some that can be added to your .vimrc file.'bs/\<[A-Z]/\l&/g<CR> . if the current file has changed. and start editing f1.Unix_tricks. To move between files use: :n .

May 27. How to set variables and environment variables: tcsh sh/bash set var = value % var=value setenv ENV value % ENV=value.Unix_tricks.bash) both diff files cmd >fn 2>fn2 stdin to append >>fn no clobber >|fn pipe stdout cmd | cmd2 pipe both cmd 2>&1 | cmd2 .'bs/\<[a-z]/\u&/g<CR> Setting Variables and Environment Variables This week's tips compare how some common tasks look in tcsh versus sh/bash. like me. export ENV In bash (additional to sh) variables can be declared and marked as exported and/or read only: declare [-options] var=value -r is read only -x is exported (environment variable) % declare -rx ENV=cant_change Redirection and Pipes For tcsh: stdout cmd >fn stderr (>/dev/tty) >&fn2 both cmd >& fn both diff files (cmd >fn) >&fn2 stdin cmd <fn to append >> no clobber >! pipe stdout cmd | cmd2 pipe both cmd |& cmd2 For sh/bash: stdout cmd >fn stderr cmd 2>fn both same file cmd >fn 2>&1 (cmd &>fn . 2008 3:20:00 PM " star C .txt Page 90 of 271 Printed: Tuesday. you are continually flipping between the two and get the syntaxes mixed up. This is a handy reference if.convert to uppercase map *C :'a.

May 27.tcshrc bash non-login shell: % bash ~/.login ~/. and in the order given. Both login and non-login shell startups are given. then echo "No" else echo "Not sure" .tcshrc ~/.bash_profile tcsh non-login shell: % tcsh /etc/csh. 2008 3:20:00 PM close stdout cmd >&- close stderr cmd 2>&- close stdin cmd <&- open file cmd <>fn Startup Files The following files are executed when the shell starts up in interactive mode.cshrc /etc/csh.bashrc Control Constructs 1 (Choice) The scripting language of tcsh looks very much like the 'c' programming language (hence the 'c' in tcsh) while sh/bash does not. 'if' construct: #!/bin/tcsh if ("$1" == "positive") then echo "Yes" else if ("$1" == "negative") then echo "No" else echo "Not sure" endif #!/bin/sh if [ "$1" = "positive" ]. tcsh login shell: % tcsh -l /etc/csh.cshrc ~/.txt Page 91 of 271 Printed: Tuesday. then echo "Yes" elif [ "$1" = "negative" ].login bash login shell: % bash --login /etc/profile ~/.Unix_tricks.

"negative") echo "No" . esac Control Constructs 2 (Looping) The scripting language of tcsh looks very much like the 'c' programming language (hence the 'c' in tcsh) while sh/bash does not.. 'for' loops: #!/bin/tcsh foreach word (hello goodbye au-revoir) echo $word end #!/bin/sh for word in hello goodbye au-revoir do echo $word done . *) echo "Not sure" ..txt Page 92 of 271 Printed: Tuesday. May 27.. 2008 3:20:00 PM fi 'switch/case' construct: #!/bin/tcsh switch ("$1") case "positive": echo "Yes" breaksw case "negative": echo "No" breaksw default: echo "Not sure" breaksw endsw #!/bin/sh case "$1" in "positive") echo "Yes" .Unix_tricks.

HTML 529632 users 590441 D??#^ . May 27. .txt 563547 page3. % ls -i 590795 Danger 579654 bvt 563546 page2.txt Page 93 of 271 Printed: Tuesday. To do this boot the machine into single user mode by holding down command (apple)-S. DB4#^ Danger If the type-able portion of the file in not unique: % ls d<hit tab key> DB)b^D"o#?b^B. repairing the disk can help make odd ghostly files a little more solid. Use tabbed completion to reveal the filename: % ls Z<hit tab key> % ls Zb^D" (Note that the filenames don't appear so odd by the time they reach your browser. 2008 3:20:00 PM 'while' loops: #!/bin/sh n=0 while [ ! $n = 10 ] do echo $n n=$(expr $n + 1) done #!/bin/tcsh set n = 0 while ($n != 10) echo $n set n = `expr $n + 1` end Oddly-Named Files If you have a filename that shows up like: % ls Z* Z??? It may contain non-standard characters... DB4#^ Danger use the inode number to identify the file. then use 'find' to rename it: % find .) The file can be renamed with: % mv Z<hit tab key> % mv Zb^D" z-file If the type-able portion of the file in not unique: % ls d<hit tab key> DB)b^D"o#?b^B. 'fsck' it Sometimes. -inum 590442 -exec mv {} D-file \.Unix_tricks.htm 578980 tvb 590442 D??????????? 560810 Cantata.

A special command called 'chflags' is used to set and unset flags. % mount -uw / % reboot Deeply Nested Directories Commands (such as fsck) can sometimes fail due to overly nested directories.. 3 . So even root can't delete it. An immutable file has the Immutable flag set . Bit if you do not own some of the files you have to run as root. . Such nesting can be created by buggy scripts. 2 .) Practical Advice . So here goes. (But it can be deleted. no user can change it.. you can't delete it. Look for this on the file-system with: % sudo find / -mindepth 30 (to find nesting greater than 30).Nested Directories You Don't Own One solution is to apply write-permission to all files in the trash.Unix_tricks.The nightmare Scenario A file may have the System Immutable flag set. 1 . if a file resides within a non-writeable directory within the trash. 1 .Immutable Files Files can be made immutable. This means that you cannot change it in any way. And get this . Root cannot change it. 4 . And you can't delete the directory either because it's not empty.Nested Directories You Don't Own This thing about being able to delete any file . And if you run as only applies if you have write permission to the directory containing the file.root can't even unset the of the file flags that exist over and above the attributes (permissions and owners) you see when you list files. So.Open Files If a file is currently in use by some application (open) then it cannot be deleted regardless of your permissions for that file.Trash Destruction Here are four solutions to the four scenarios listed above.txt Page 94 of 271 Printed: Tuesday. 2008 3:20:00 PM To repair the boot volume type: # fsck -y To repair other partitions type: # fsck_hfs -y /dev/disk0sX where X is the slice (partition) number identified by: % pdisk type L read the partition number from the left column against the volume name type q to quit. you may as well simply delete the lot. May 27.

with the 'fstat' command: % ls -al ~/.you will be prompted for each file to delete. ls -alo drwx-----.txt Page 95 of 271 Printed: Tuesday.Trash/* USER . If you are unable to close all open files.  Tell Me More.0 r / /Users/melkor/.Trash 'chflags' changes a file's flags settings.Immutable files You can check which. Close the offending files.Trash/xxx . and I suggest to use this as a safety precaution.15 saruman staff . 2008 3:20:00 PM % sudo rm -ri ~/.Trash % sudo rm -ri ~/.ie to all nested files and directories '-i' is confirm mode . Clear the immutable flag of all files. so you can replace the '-i' with '-f' (force). -rw-r--r-.. warning . May 27. and which application has them open. 'sudo' executes the command as root and therefore overrides all permissions 'rm' removes files '-r' does it recursively . 3 . files have the immutable flag set by specifying the '-o' option to 'ls'.Open Files You can determine which files in the trash are open. then delete: % sudo chflags -R nouchg ~/. 1 saruman staff uchg 0 Feb 20 14:04 user-imutable-set 'uchg' indicates that the User Immutable Flag is set. 510 Jan 6 23:35 osxfaq -rw-r--r-. If you have many. 'nouchg' clears the user immutable flag ('uchg' would set it) '-R' does it recursively.Trash This will zap all files except those that are immutable or in use. resort to the method described in the GUI approach. then attempt to empty the trash again.1 melkor staff 0 Jun 25 16:43 zzz % fstat ~/. many files tedium may set in. 2 .'vi' is editing 'xxx'.that's the one ...Trash total 0 drwx-----.1 melkor staff 0 Jun 25 16:43 yyy -rw-r--r-. drwxr-xr-x 24 melkor staff 772 Jun 25 16:32 .. if any.5 melkor staff 126 Jun 25 16:43 ..1 melkor staff 0 Jun 25 16:32 xxx -rw-r--r-..CMD PID FD INUM MODE SZ|DV R/W MOUNT NAME melkor vi 731 3 368860 -rw-r--r-..Unix_tricks. 'schg' indicates that the System Immutable Flag is set (see point 4 below).

May 27. type: % whoami if the answer is not root type: % su and type control-d twice when you exit. A user without administrative privileges cannot use 'sudo' If you were to accidentally put a space between '. Oops! That's one reason why I suggest the '-i' option % rm -ri ~/. Note: when you enter single user mode. You must enter your admin password (not the root password) when prompted. Alternatively. % cd /Users/your-name-here/.Trash % chflags -R noschg * Then hit control-d to return to multi-user mode. Only by super-root . What!?! When your Mac is up and running in multi-user mode (the normal operating mode) it is running at level 1.' (your home directory) and 'Trash' ( a non-existent directory). Other Trashes . Trash 'rm' would apply to both '~/. such as turn off the System Immutable flag. Do this. You would wipe every single file from your home directory. Some operations even root can't do at level 1. You will lose all services such as network connectivity while in single user mode. you can reboot your Mac.txt Page 96 of 271 Printed: Tuesday. then startup with both the Command (Apple) key and the 's' key held down. the offending files my have the System Immutable flag set. Such files are locked Fort Knox style.the super-duper-user. Hopefully you will spot your mistake on the first confirmation and avoid loosing anything 4 .Trash so you get a prompt for each file. or by reapplying the steps above. 2008 3:20:00 PM sudo sudo (super user do) is a Unix command that allows you to run another command as root.Unix_tricks. Switching into single user mode will allow one to run at level 0 and thus change the System Immutable flag.The Nightmare Scenario If all of the above methods fail. You should now be able to delete the trash with the Finder.' and 'Trash' : %rm -rf ~/. You must run at level 0. Close all applications and issue the command: % sudo shutdown +0 "Bye bye" to shutdown multi-user mode and enter single user mode. The System Immutable flag cannot be changed by root.

Unix_tricks..Trashes may reveal several such 50x directories.Trashes/501 See if they are empty by listing them. The shell special parameter $0 contains the script name. 2008 3:20:00 PM On some occasions. And Finally If you still have problems. '501' is the User ID of the user who did the trashing. Firstly.Trash to /. You can check your own User ID (uid) by issuing the command: % id and substitute this if it is not 501.Trashes/501 . % sudo ls -al /. or else have 'fsck' run automatically with: % fsck -y You can also try commercial disc repair programs.Trashes/501 and /Volumes/disc-name-here/.txt Page 97 of 271 Printed: Tuesday. Display just the script name using ${0##*/} $ cat usage #!/bin/sh echo "Usage 1: $0" echo "Usage 2: ${0##*/}" $ ~/Development/test/usage Usage 1: /Users/saruman/Development/test/usage . part 8 and part 9. The Executable Name This week's tips enhance the Learning Centre tutorial on scripting. you probably have a corrupt file system. enter single user mode and issue: % fsck and answer the prompts. May 27.. trash is placed at the root level of your system disc or another partition or drive. Note that these tips are for 'sh' or 'bash' scripts. but includes the full path name. You can then apply all the techniques given for ~/.Trashes/501. % sudo ls -al /. and you have re-applied the above steps as necessary. The directory is: /.

Unix_tricks.mayo-family.txt Page 98 of 271 Printed: Tuesday. then echo "Mounting /Volumes/$USER" mkdir /Volumes/$USER mount_afp afp://$USER@your-server-address-here/$USER /Volumes/$USER > /dev/null else echo "/Volumes/$USER already mounted" . and 'while' statements can contain AND. it is necessary to check that /Volumes/your-user exists AND is a mount. and NOT elements. then Check if a Volume is Mounted To check if a volume is mounted. OR. for example /Volumes/your-user. 2008 3:20:00 PM Usage 2: usage Complex Conditions Conditions in 'if'. the script: $ cat mount-user #!/bin/sh mounted=$(df | grep "/Volumes/$USER") if [ ! "$mounted" ]. For example. 'elif'. then statement to mount /Volumes/$USER fi because an empty string has the test value FALSE. Note that the environment variable $USER expands to the short name of the current user. Mounting and AFP Volume An AppleShare volume can be mounted using the lines: mkdir /Volumes/$USER mount_afp afp://$USER@carcharoth. To reverse a condition use pling (!). Use '-o' for OR. not a regular file or directory. to test if parameter 1 is "-p" AND parameter 2 is empty use: if [ "$1" = "-p" -a "$2" = "" ]. May 27. One way to do this is to examine the output from df: mounted=$(df | grep "/Volumes/$USER") If the volume is not mounted then the variable 'mounted' will be a null string. then The '-a' represents$USER /Volumes/$USER > /dev/null This mounts the home directory of the current user that is located on the server 'SERVER' at mount point /Volumes/users-short-name. One can use the condition: if [ ! "$mounted" ]. Eg. For example to test if a file is NOT a directory use: if [ ! -d "$file" ].

2008 3:20:00 PM fi Will mount my home directory (saruman) located on my server under /Volumes/saruman: $ ls /Volumes <empty> $ . May 27. Otherwise mount_afp will ask for your password in a pop-up. It uses two environment variables which must be set before the script runs: AFP_MOUNT to say where to mount the volumes (this is usually '/Volumes') SERVER for the network address of the server A password can optionally be given with '-p password'./mount /Volumes/saruman already mounted A Mount-User Script Here is a script using the tips given this week. on the local machine at mount point $AFP_MOUNT/$USER. or extract it from the keychain. then echo "Usage: ${0##*/} [-p password]" echo " give a password" exit fi # if password is given add ":" to the start to comply with mount syntax if [ "$1" = "" ].Unix_tricks. then echo "Usage: ${0##*/} [-p password]" echo " mounts ${USER}'s home directory on the server at $AFP_MOUNT/$USER" exit fi if [ "$1" = "-p" -a "$2" = "" ].txt Page 99 of 271 Printed: Tuesday./mount-user Mounting /Volumes/saruman $ ls /Volumes saruman $ . and the shell programming tutorials in the Learning Centre part8 and part 9. $ cat mount-user #!/bin/sh # mounts the current user's home directory on the server into the afp mount dir # if [ ! "$1" = "" -a ! "$1" = "-p" ]. It mounts the current user's home directory located on the server $SERVER. then pass="" else pass=":"$2 fi .

1 otherwise # if [ ! "$mounted" ].. the link is automatically followed resulting in 'orig-file' being accessed. and has a flag set to say 'I am a symbolic link'.Unix_tricks. When 'link-file' is accessed. If 'orig-file' is moved or renamed the link will be broken.. They come in two varieties: symbolic link and hard link (covered on Tuesday). 2008 3:20:00 PM # check if already mounted # mounted=$(df | grep "$AFP_MOUNT/$USER") # if not mounted. A symbolic link is created using the 'ln' command. specifying option '-s' for symbolic. % cat orig-file I am the original file % cat link-file I am the original file 'link-file' is simply a file that contains some text describing the absolute path to 'orig-file'. then exit 1 else exit 0 fi Symbolic Links A link is a file that 'points' to another file. ensure the mount point (directory) is available # if [ ! "$mounted" ].2 saruman staff 23 . then if [ -e $AFP_MOUNT/$USER ]. May 27. orig-file The symbolic link is shown with the first character as 'l' in a long listing.txt Page 100 of 271 Printed: Tuesday. and as pointing to the original file. partitions. then if [ ! -d $AFP_MOUNT/$USER ]. link-file -> orig-file -rw-r--r-. . then rm -f $AFP_MOUNT/$USER fi mkdir $AFP_MOUNT/$USER fi mount_afp afp://$USER$pass@$SERVER/$USER $ALM_AFP_MOUNT/$USER > /dev/null disktool -r fi # return 0 if no mount was performed.different directories. The two files can be anywhere on the file system .. disks. % echo "I am the original file" > orig-file % /bin/ls -l *-file lrwxr-xr-x 1 saruman staff 9 . or machines..

. orig-file % cat orig-file I am the original file % cat hard-file I am the original file 'hard-file' is not a new file. orig-file . Aliases An alias is managed by the Mac Finder and is not recognized by Unix. Copying an alias using the Unix 'cp' command will create only an empty file.. % ln orig-file hard-file % /bin/ls -l *-file -rw-r--r-. The Finder recognizes symbolic and hard links. May 27...txt Page 101 of 271 Printed: Tuesday... The two files are one in the same. An alias has an empty data fork.Unix_tricks. In technical terms.. alias-file 8 -rw-r--r-.2 saruman staff 23 . (Create and alias (alias-file) to orig-file using the Finder.. alias-file -rw-r--r-. This is because inodes belong to a particular partition and are not unique across partitions or drives. orig-file % cat orig-file I am the original file % cat alias-file % Unix sees only the data fork and hence 'alias-file' as being empty. Either file can be renamed or moved without breaking the link.. hard-file lrwxr-xr-x 1 saruman staff 9 .. with the aliasing information stored in the resource fork.. 2008 3:20:00 PM Hard Links A hard link is created using the 'ln' command.2 saruman staff 23 . hard-file lrwxr-xr-x 1 saruman staff 9 ..2 saruman staff 23 .. and it is not possible to determine which is the original and which is the hard link..1 saruman staff 0 ... the two directory entries reference the same inode.2 saruman staff 23 . link-file -> orig-file -rw-r--r-. % /bin/ls -sl *-file 72 -rw-r--r-. hard-file 8 lrwxr-xr-x 1 saruman staff 9 .1 saruman staff 0 ..2 saruman staff 23 . Note that 'ls' with option -s does recognize the resource fork when calculating the number of blocks the files uses (36k or 72 half-K blocks).) % /bin/ls -l *-file -rw-r--r-.. Hard links cannot be created across partitions or disks. link-file > orig-file -rw-r--r-.. but a new directory entry that points to the original file's data (the same part of the disc). but WITHOUT specifying option '-s'..2 saruman staff 23 .. link-file > orig-file 8 -rw-r--r-..

.2 saruman staff 23 .. . alias-file 678094 -rw-r--r-.. or follow the link and deal with the target file. '-L'... The commands include: * 'chflags' * 'chgrp' * 'chmod' * 'chown' * 'cp' HLP * 'du' HLP * 'find' 'test' (aka '[') can test for a symbolic link with '-L': -L file . Making a Shadow Directory The command 'lndir' creates a shadow directory of symbolic links to another directory. 2008 3:20:00 PM Working with Links Many commands take options to affect the way they treat a link . and '-R' which generally mean: -H If the -R option is specified...) -L If the -R option is specified. a-letter and can show inode numbers with option '-i': % ls -l i *-file 678102 -rw-r--r-.txt Page 102 of 271 Printed: Tuesday..Unix_tricks. This applies only to symbolic links. May 27. (Symbolic links encountered in the tree traversal are not followed.2 saruman staff 23 ... hard-file 678095 lrwxr-xr-x 1 saruman staff 9 .. no symbolic links are followed. a-letter > /Users/saruman/Documents/Letters/Channel Dynamics % /bin/ls -lL a-letter -rw-r--r-..usually to control if they deal with the link itself. symbolic links on the command line are followed. Other commands take options '-H'.1 saruman staff 20480 . -P If the -R option is specified. all symbolic links are followed. 'ls' can be told to follow the link with option '-L': % ln -s ~/Documents/Letters/Channel\ Dynamics a-letter % /bin/ls -l a-letter lrwxr-xr-x 1 saruman staff 49 . orig-file Note that the hard link and original file have the same i-node number.1 saruman staff 0 . link-file > orig-file 678094 -rw-r--r-.True if file exists and is a symbolic link.

Unix_tricks.txt Page 103 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

For example, make the following directory structure:
% mkdir ln-test
% cd ln-test
% touch a b
% mkdir c
% touch c/a c/b
% ls -R
a b c

a b

Make a shadow directory:
% cd ../
% mkdir ln-test-copy
% cd ln-test-copy/
% lndir ../ln-test

Examine the shadow directory created by 'lndir':

% /bin/ls -lR
total 16
lrwxr-xr-x 1 saruman staff 12 Oct 5 11:44 a -> ../ln-test/a
lrwxr-xr-x 1 saruman staff 12 Oct 5 11:44 b -> ../ln-test/b
drwxr-xr-x 4 saruman staff 136 Oct 5 11:44 c

total 16
lrwxr-xr-x 1 saruman staff 17 Oct 5 11:44 a -> ../../ln-test/c/a
lrwxr-xr-x 1 saruman staff 17 Oct 5 11:44 b -> ../../ln-test/c/b

Edit Multiple Files

If you wish to edit a group of files, combine 'find' with your editor of choice ('vim' in my
case). To edit all text files (*.txt) in the current directory hierarchy use:
% vim `find . -iname "*.txt"`

In 'vim' or 'vi', type

to move to the next file, or

to save changes and move to the next file.

To abort changes and move to the next file use:

And finally, to quit before the entire list of files have been edited:

Edit Conditionally on Contents

To search for all files that match a filename pattern AND that contain particular text use the

Unix_tricks.txt Page 104 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM


For example to search for all text files (*.txt) that contain the word "memo", and then edit them
using 'vim':
vim `find . -type f -iname "*.txt" -print0 | xargs -0 grep -ilw "memo"`

* The -print0 and -0 options are to cope with filenames containing spaces.
* 'Find' option -type f is to ensure only regular files are considered.
* 'Grep' options -iw ensure the text search is case insensitive and matches only whole words.

Less (and More) Edits

If you view a file with 'less', then decide you wish to edit that file, just hit 'v'. This will
take you into the 'vi' editor, pre-loaded with the file you were viewing.

If 'vi' is not to your taste, set the environment variable VISUAL to define which editor will be
% setenv VISUAL vim

You can do the same trick with 'more', except that you will need to set environment variable
EDITOR instead. Note that 'less' will use EDITOR if VISUAL is not defined.


If your terminal session gets corrupted, for example after accidentally editing or displaying a
binary file, use 'tput'.
% tput reset

to reset the Terminal, or
% tput init

to initialise it.

The difference between these two depends on the difference between the 'reset' and 'init'
terminal strings in the 'terminfo' database (/usr/share/terminfo/....). Mac OS X uses the
terminfo rather than termcap. Terminfo is a compiled database, and in order to view the
information you must use 'infocmp terminal-name'.

For example:
% infocmp vt100
% infocmp xterm

Cleaning Up

Tidy up a file with 'tr' and 'sed'.

The following command removes duplicate spaces, blank lines, and changes Mac style end of lines
into Unix style.
% tr -s " " < test.txt | tr \\r \\n | sed '/^$/d' > out.txt

The input file is test.txt and the output is out.txt

About POSIX Regular Expressions

REs come in three flavours: older style 'basic' or 'obsolete' as recognized by 'grep', 'sed',

Unix_tricks.txt Page 105 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

'vi', ...; 'extended' or POSIX-compliant as recognized by egrep, gawk, PHP, ...; and Perl regular
expressions as recognised by Perl and PHP.

Perl are more powerful than POSIX, which in turn are more powerful than basic.

Incompatibilities exist between the different varieties, so one should check which form to use
with any given utility.

By default, searching is done a line at a time, and is case sensitive.

This week's tips will concentrate on POSIX-style REs.

Matching and Escaping with POSIX Regular Expressions
% cat test
This is a test file
to illustrate the use of
regular expressions.
Here is a dollar $ symbol.
this is the end of the test file.

To search for a simple string:
% egrep "This" test
This is a test file

You must explicitly make the search case-insensitive: the method will depend on the utility you
are using. For 'egrep' use option -i.

Characters that have a special meaning within regular expressions must be escaped to say "take
the character literally and ignore any special meaning it may otherwise have".

Special characters that must be escaped are:
. ^ $ * + ? [ ( ) | { \

For example:
% egrep 'dollar $' test
(no matches output)

% egrep 'dollar \$' test
Here is a dollar $ symbol.

Use single quotes to prevent the shell from interpreting the $ sign.

Note that it is not always necessary to escape special characters because they may be special
only in a particular context, but to be safe it won't harm to always escape.
% egrep 'dollar $ symbol' test
Here is a dollar $ symbol.

Anchors in POSIX Regular Expressions

An anchor is used to force the pattern to match either the start or end of a line.
% cat test
this is line one
and this is the second line

Searching for 'this' or 'line' will match both lines of the file:

Use '^' to anchor the pattern to the start of a line:
% egrep '^this' test

Unix_tricks.txt Page 106 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

this is line one

Use '$' to anchor the pattern to the end of a line:
% egrep 'line$' test
and this is the second line

Wildcards and Repeaters

The special character '.' will match any character.

To match b<any one character>g use:
% cat test
This is a bag.
This is a bug.
This buug won't match.

% egrep "b.g" test
This is a bag.
This is a bug.

% egrep "b..g" test
This buug won't match.

These special characters say to match if the previous character is repeated a certain number of
<pattern>? matches <pattern> repeated 0 or 1 times
<pattern>+ matches <pattern> repeated 1 or mores
<pattern>* matches <pattern> repeated 0 or more times

'zu?z' matches 'zz' or 'zuz'
'zu+z' matches 'zuz' or 'zuuz', 'zuuuzzzz'
'zu*z' matches 'zz, 'zuz', 'zuuz', 'zuuu...z'

'Hello.*goodbye' will match Hello<0 or more of any character>goodbye

Ranges with POSIX Regular Expressions

Use ranges to match a specific number of characters.
<pattern>'{m,n}' matches <pattern> repeated m to n times inclusive
<pattern>'{m,}' matches <pattern> repeated m or more times
<pattern>'{m}' matches <pattern> repeated m times

'zu{3.5}z' matces 'zuuuz', 'zuuuuz', 'zuuuuuz'
'zu{3,}z' mathes 'zuuuz', 'zuuuuz', 'zuuuuu...z'
'zu{4}z' matches just 'zuuuuz'

Match a Range of Characters with POSIX Regular Expressions

Match any one of a range of characters using a bracketed expression (a range of characters
enclosed in []).
% egrep 'b[aeiou]g'

matches bag, beg, big, bog, and bug.

Invert the match by starting the sequence with the '^' character.
% egrep 'b[^aeiou]g' file

Unix_tricks.txt Page 107 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

matches bbg, bcg, ... bzg, but not bag, beg, etc.

Match a range of consecutive characters using a hyphen, like [a-z] for all lower-case letters, or
[0-9] for all decimal digits. For example:
% egrep '^[a-zA-Z]{3}[0-9]{2}$' file

matches 'letter letter letter digit digit' anchored to the start and end of the line (see last
week's tips for anchors and {} repeaters).

Character Classes in Regular Expressions

A character class is a name for a particular set of characters, is enclosed in :class-name: and
used within a bracketed expression.
[:alpha:] matches any alphabetic character
[:alnum:] matches any alphanumeric character
[:alpha:?] matches any alphabetic character or a '?'.

The classes are:
alnum digit punct
alpha graph space
blank lower upper
cntrl print xdigit

Character classes work in PHP and Perl, but don't seem to work in egrep in OS X 10.2.

Escaping, or Lack Thereof

It is not necessary (in fact it is incorrect) to escape the special characters ^ $ * + ? [ ( ) |
{ \ within a bracketed expression (see Monday's tip).

To match a '^' ensure it is *not* the first character in the list (otherwise it will invert the
match), and to match ']' ensure it *is* the first character in the list.
% egrep '[]%$^]' file

matches any of ] % $ ^

Match non of the above with:
% egrep '[^]%$^]' file

Note: It is permissible to escape special characters in bracketed expressions when using Perl, or
Perl compatible expressions in PHP.

Testing Regular Expressions

The easiest way to test out a regular expression is using 'egrep' and the standard input.
% egrep '<regular-expression-to-test>'

egrep will read from standard input (no filename supplied). Type a line of text. If the regular
expression matches the line of text will be echoed back, if it doesn't it won't.

Repeating using Regular Expressions

If you wish to apply a wildcard or repeater (see last week's tips) to more that a single
character, enclose the section to match in parenthesis to create a sub-expression and apply the
repeater to that.

Previously zipped log files are rolled . including the main system log.hello. You will notice that log files are 'rolled' periodically: daily for the system log.hello hello.Unix_tricks.gz and creating a shiny new (and empty) log file.gz | tail -n35 Create an alias as a useful short-cut to viewing log files: . from viewing them to configuring the logging daemon.that is renumbered 0->1.0.hello. 1->2.hello. wave goodbye. View log files using 'tail' (system. or Say hello. Viewing Log Files This and next week I will cover all things log-file.}' matches: hello.log in the example).hello.log & [1] 683 <--------------. you will see logs for these too. ftp.Say hello.hello. 2008 3:20:00 PM % egrep 'hello(.hello etc % egrep '^(Say hello(. wave goodbye. and ultimately irritate you :-) To stop it issue: % kill nnn where nnn is the number you noted earlier. with the oldest log file being deleted.hello hello. Most log files live in '/var/log'.txt Page 108 of 271 Printed: Tuesday. wave goodbye. The command: % tail -n35 -f /var/log/system.hello. View older log files without the hassle of unzipping and re-zipping with the aid of 'zcat'.2}$' matches: Say hello. and weekly for most others. zcat and tlog Have a look in /var/log.){1. etc. is written to the system log.log Press control-c to stop 'tail' and return to the command line. and log files for netinfo and the lookup daemon. Most general information. If you have enabled any of mail. View log files dynamically with: % tail -n35 -f /var/log/system. and printer services.note the number you see here. % zcat /var/log/system.hello.hello.log using '-n35' to show the last 35 lines instead of the default 10. and critical mail information.0. Rolling involves zipping the most recent log file to <log-file-name>.hello){1.hello){1.log.}. May 27. will constantly interrupt your Terminal screen with the latest log file entry. wave goodbye. statically with: % tail -n35 /var/log/system.

then mv -f "${log}.log do echo -n " $log" if [ -f "${log}. then mv -f "${log}" "${log}.1.3.Unix_tricks. 'apache' and 'named' write their log files to a subdirectory of /var/log.gz".local does not exist create it as: 4.1.2. fi if [ -f "${log}. then mv -f "${log}. you can 'roll' (see Tuesday's tip) them yourself with a simple script placed in /etc/weekly.local The script will look like: #!/bin/sh - ################################## # Local weekly tidy-up script # ################################## host=$(hostname -s) echo "Subject: $host Local weekly run output" for logtype in httpd named do echo "" echo -n "Rotating type $logtype log files:" if [ -d /var/log/$logtype ]. May 27.0k -r-xr-xr-x 1 root wheel 973 Nov 1 14:01 weekly.gz" "${log}.gz". 2008 3:20:00 PM alias tlog 'tail -f -n35 /var/log/\!:1.gz".gz" ].gz".local.log Rolling Your Own If you run 'apache'. fi if [ -f "${log}.gz" ].3.gz" ].gz" "${log}. then mv -f "${log}.0.2. If /etc/weekly.log' allowing one to type: % tlog system rather than: % tail -n35 -f /var/log/system. and so the log files are not rolled by the standard weekly tidy-up scripts. then cd /var/log/$logtype for log in *.txt Page 109 of 271 Printed: Tuesday.0" /usr/bin/gzip -9 "${log}." "${log}. or any daemon that creates many log files. fi if [ -f "${log}" ].0" fi touch "$log" done case $logtype in . fi if [ -f "${log}. or 'named'. then mv -f "${log}.gz" ].gz" "${log}.3.

'syslogd'.. particularly those from device drivers. esac fi done echo "" echo "Complete" Notes: This line specifies the subdirectories of /var/log that will be rolled: for logtype in httpd named These lines restart each daemon to ensure that it recognises the new log file: httpd) apachectl graceful.txt Page 110 of 271 Printed: Tuesday. The Daemonic Syslogd Unix applications make log entries via the system logger . named) ndc restart. May 27.. or cron'. kernel.. are written to the system message buffer instead of log files. and all output to it is written to the file /var/tmp/console. 2008 3:20:00 PM httpd) apachectl graceful. Each log entry includes a facility like 'mail'.log.Unix_tricks. The console is a device under /dev/console. dmesg Log messages from the Kernel. *).app in /Application/Utilities. 'dmesg' may be a valuable source of debugging information. View the console with: % tail -n35 /var/tmp/console. named) ndc restart.log or % tail -n35 -f /var/tmp/console. Syslogd filters message and directs them to particular log files dependant on their facility and .log The console log can also be viewed by Console. The Console Error-level. and a level ranging from 'emergency' to 'debug'... 'syslog'. and mail critical messages are written to the console (some also go to the system log). 'ndc restart' may not work in Panther because it uses a different version of 'bind' to Jaguar. View the system message buffer with: dmesg If you experience start-up or device driver problems.

May 27. Write to the mail log file at level info: logger -p mail. 2008 3:20:00 PM level.kern. warning. user. Change what is logged.. lpr. (See Tuesday.err.auth. Reboot if wider-ranging changes are made. This is useful for testing a new 'syslogd' configuration.mail...authpriv.*. by altering /etc/syslog. All log entries that include a listed facility.crit /var/log/system. The receiving host must have syslogd started with option -u.Unix_tricks.) Facilities are: auth.debug. and its own configuration. uucp and local0 through local7 Levels are: emerg. Log messages are sent to UDP port 514. Note that this is vulnerable to attacks from outside. kern.notice. use @hostname as the destination given in /etc/syslogd. news.emerg Emergency.level' pairs.kern. Restart syslogd and the corresponding deamons(s) to ensure the changes are seen. err.notice. will be sent to the destination. crit.ftp.conf.. and a destination.remoteauth. daemon. alert.crit /dev/console *. Log to Another Host To send to another host.none.log # The authpriv log file should be restricted access. authpriv. Type: cat /etc/syslog.conf *.info Test log message Broadcast an emergency message to all logged in users: logger -p user.none.authpriv. mail. Test Log with Logger The 'logger' command can be used to manually log messages. Each line consists of a list of 'facility. syslog. these .remoteauth. mark.conf ~ % cat /etc/syslog.txt Page 111 of 271 Printed: Tuesday. and a corresponding error level at or above that specified. notice.conf. and where. complete with facility and priority. info and debug Change the Syslogd Configuration The syslogd daemon logs messages according to their facility and level (see Monday) and its own configuration in /etc/syslog. as a cracker is able to continually write to your log files.mail. cron.*. Change ftpd's Logging Change the ftp daemon's (ftpd) logging scheme. '*' is a wildcard in the usual sense.

authpriv.crit /var/log/system.* /var/log/ftp. Change /var/log/ftp. The umask .* /var/log/ftp.remoteauth.*.authpriv.err /var/log/netinfo.debug.mail. must not conflict with any existing rule /dev/console specifies that levels 'err' and above for all ('*') facilities (thus including ftp) are sent to the console. 2300 in this example. The console log can be viewed by the Console. Passive mode is the default.21 to any 1024-65535 in (The rule number.log ftp. Active or Passive Ftp connections come in two flavours: Active .log netinfo. If you wish to use active mode.txt Page 112 of 271 Printed: Tuesday. open a hole in the firewall with: % sudo ipfw add 2300 allow tcp from any 20. all levels (denoted by '*') will be sent to ftp.log.notice.auth.*.ftp. The client must have the appropriate ports open in its firewall.log specifically excludes ftp messages from the system log by specifying ' Passive .* /var/log/mail.authpriv. This line: *.log this mode the server contacts the client to transfer data to this mode the client always contacts the server and therefore does not need to open ports.emerg * Notice the line: ftp.none.) This will enable your client to use active ftp.kern.notice. but will not open your Mac up as an ftp /var/log/lpr. This line: *.log to log to a different file.remoteauth.log which says that facility ftp.none' Change to 'ftp. or find that passive mode hangs.*.*' to get the ftp daemon to log to the system log.none.Unix_tricks.log mail.kern. 2008 3:20:00 PM # messages shouldn't go to terminals or publically-readable # files. May 27.log *.err.crit /var/log/secure. or by viewing /var/tmp/console.

Unix_tricks. Add their account name to: /etc/ftpchroot Note that this feature is broken in OS X 10. to the line server_args = -l Security All Unix account holders may ftp into a server. To prevent a particular user from using ftp add that user's account name to: /etc/ftpusers In order to create a user who cannot log in. Edit the file: /etc/xinetd. 2008 3:20:00 PM The default umask (permissions for created files) set by the ftp daemon is 027.txt Page 113 of 271 Printed: Tuesday. To effect this permanently. May 27.log If you wish to include extra information in the log file then start the daemon with option -d.2. then set the umask: ftp> site umask 022 This affects only the current ftp must do so as the root (super) user.conf (or create it) . you need to configure the ftp daemon. set that user's shell to: /sbin/nologin and ensure that this shell is listed in: % cat /etc/shells # List of acceptable shells for chpass(1). . 'chroot' them. Add a line: umask all 022 FTP Debugging The ftp daemon logs to /var/log/ftp. meaning that there is no world read access to files you upload. A Permanent umask The umask solution from Tuesday affects only the ftp session from which it was issued. /bin/bash /bin/csh /bin/sh /bin/tcsh /bin/zsh /sbin/nologin Finally. for example to allow Apache access to files you have uploaded. but can ftp. # Ftpd will not allow users to connect who are not using # one of these shells. to stop an ftp user from exploring outside of their home directory. for all logins. or any other options you require. Edit the file /etc/ftpd. If you wish to change this.d/ftp and add the debug.

tar.16a. ftpwho. anonymous users disallowed -p 40000:50000 specifies the port range for passive connections -c 5 specifies the number of clients -C 1 specifies the number of connections per IP address -I 5 changes the idle before editing a system level file.macfora. 2008 3:20:00 PM Apple's decision to switch ftp daemons from ftpd in 10.1 to lukemftpd in 10.16a/ .x are left hobbled or with malfunctioning ftp servers (especially in regards to ftpchroot functionality).gz tar xzf pure-ftpd-1. change directories to wherever you keep you downloads or source code: cd /downloads curl -O ftp://ftp. get a root shell so all the following commands don't have to be prefaced with sudo: sudo -s (As always.txt Page 114 of 271 Printed: Tuesday.gz cd pure-ftpd-1. To get the standalone server to launch automagically at startup. quotas. etc.16a. it is wise to create a backup first. One can either fix the broken built-in server by following the instructions here: http://www. efficient and feature-rich ftp server that also boasts no root exploits. The first step is to download and unpack the source./configure --with-everything --with-virtualchroot --without-banner --without-humor --enable-osx --with-tls (This will configure a 'big server' with a plethora of options.php?s=&threadid=6314 xinetd Mode First. default 15 minutes seems excessive -T 25 throttles the bandwidth to 25KB/sec per user Many other switches are available. but will leave off the guady initial banner and the sprinkling of colorful banter in the error messages.d/ cp -p ftp ftp. you would have to write a Startup Item: http://www. a robust.pureftpd.default pico ftp Modify the server and server_args lines as folows: Code: service ftp { .com/forums/showthread. ratios.tar. Their lack of updated documentation to reflect that decision is truly or replace it altogether. including throttling. I elected to replace the built-in ftp server with pure-ftpd. as pure-ftpd can run in either standalone or xinetd mode: Standalone Mode You can run the server in standalone mode with this command: sudo /usr/local/sbin/pure-ftpd & or if you desire. First.chezludo.) cd /etc/xinetd. May 27. with no changes in the man pages to help them configure their new ftp daemon. use command line switches to configure the server at runtime: sudo /usr/local/sbin/pure-ftpd -A -E -p 40000:50000 -c 5 -C 1 -I 5 -T 25 -u 1 & The command line switches I have chosen tell the server the following: -A chroots everyone -E only allows authenticated users.) sudo make install-strip At this point you will need to choose which server type you desire.0. See the documentation for a complete list.2 is of questionable merit.1. Most users upgrading from 10.Unix_tricks.

out Manual Maintenance ..out /var/log/weekly. 2008 3:20:00 PM disable = no socket_type = stream wait = no user = root server = /usr/local/sbin/pure-ftpd server_args = -A -E -p 40000:50000 -c 5 -C 1 -I 5 -T 25 -u 1 groups = yes flags = REUSE } Stop xinetd (if running): killall xinetd Start xinetd to get it to read the new ftp configuration file: sudo /usr/sbin/xinetd -pidfile /var/run/xinetd. The monthly script is run at 5:30 am on the first of each month. Periodic Periodic system maintenance is scheduled by 'cron' (see Week 33). #minute hour mday month wday who command # Run daily/weekly/monthly jobs. 15 3 * * * root periodic daily 30 4 * * 6 root periodic weekly 30 5 1 * * root periodic monthly . The daily script is run at 3:15 am. Otherwise.Unix_tricks.. 220-FTP server ready. The weekly script is run at 4:30 am Saturday.out /var/log/monthly. Your Mac must be running (not sleeping) for these scripts to be run. % cat /etc/crontab . the individual users will be chrooted to their /Users/username home directory..out /var/log/daily. May 27.txt Page 115 of 271 Printed: Tuesday.. To enable the chroot to a single directory. simply assign your ftp users' home directories to your ftp root directory via NetInfo (and possibly put them in a dedicated ftp user group for added flexibility).pid Exit the root shell: exit Test to confirm that it is working: ftp localhost If you get something like this: Code: [gatorparrots:] gator% ftp localhost Connected to localhost. 220 This is a private system .No anonymous login Name (0:gator): Congratulations! Your new FTP server is working as advertised. Cron launches a program called 'periodic' which is responsible for launching the maintenance scripts. The output from these scripts can be found in: % ls /var/log/*.

or monthly . much as the standard weekly script does.local /etc/weekly. weekly.log and /var/log/bind/*.com/info. and once created are run automatically by the default daily. and monthly maintenance are to be found in: # ls /etc/periodic/* /etc/periodic/daily: 100. use the freeware utility MacJanitor http://macupdate. 2008 3:20:00 PM If your Mac is not switched on in the early hours (see Monday's tip).local These are regular shell scripts. May 27. /etc/daily.daily. Periodic Configuration The scripts run by periodic for daily.log files is /var/log. or monthly). edit /etc/crontab and choose times at which your Mac will be running. The standard script only rotates *. weekly.log. Periodic is configured by the file: /etc/defaults/periodic. #!/bin/sh - ######################### # Weekly tidy-up script # .Unix_tricks. weekly. and monthly scripts.local /etc/monthly.then create the following files as necessary.conf See 'man periodic. and it will run all scripts in that directory.daily /etc/periodic/monthly: 500. weekly. Or.conf' for more information. The is done by: % sudo periodic daily % sudo periodic weekly % sudo periodic monthly Alternatively.monthly /etc/periodic/weekly: 500.clean-logs 500.txt Page 116 of 271 Printed: Tuesday. It will rotate log files.weekly Periodic can be called with a full path name to a directory (instead of the words daily. you will have to run the maintenance scripts manually. This script does the same for subdirectories such as /var/log/httpd/*. An Example Log Rotator This script can be placed in /etc/weekly.local.conf You can add your own setting and place them in the file: /etc/periodic.php/id/5856 Roll Your Own If you wish to add your own maintenance tasks .

4. then mv -f "${log}" "${log}.log do echo -n " $log" if [ -f "${log}.0. then mv -f "${log}. then mv -f "${log}. esac fi done echo "" echo "Complete" Filenames With Spaces Filenames with spaces can cause problems in scripts..1. fi if [ -f "${log}.gz". then cd /var/log/$logtype for log in *.. For example: .Unix_tricks. use '*' instead of '`ls`'.gz".3.gz".gz" "${log}.3.gz" ].2.1. 2008 3:20:00 PM ######################### host=$(hostname -s) echo "Subject: $host .gz" ]. named) ndc restart.) When processing all files in a directory with a 'for' loop. fi if [ -f "${log}.0. *)..gz" "${log}.gz" ].local weekly run output" # list the subdirectories to be considered. fi if [ -f "${log}. then mv -f "${log}.3.gz" "${log}.gz" ].2.0" fi touch "$log" done case $logtype in httpd) apachectl graceful.0" /usr/bin/gzip -9 "${log}.2.txt Page 117 of 271 Printed: Tuesday.gz" "${log}.1. in this case # /var/log/httpd and /var/log/named for logtype in httpd named do echo "" echo -n "Rotating type $logtype log files:" if [ -d /var/log/$logtype ]. then mv -f "${log}.gz". fi if [ -f "${log}" ]. May 27. (See week 49 for one such example.

..)) bash$ n1=3.. May 27. 2008 3:20:00 PM bash$ ls -1 file file 2 bash$ for file in `ls`..txt Page 118 of 271 Printed: Tuesday. do echo "File: $file".Unix_tricks. in which base bash will apply arithmetic evaluation without the need for $((. do echo "File: $file".)) expansion: bash$ n=$(( $(( $n1 + $n2 )) * $n3 )) bash$ echo $n 35 I Do Declare Bash allows one to 'declare' shell variables..05a$ n=$(( ($n1 + $n2) * $n3 )) bash-2. One advantage of declaring variables is that they can be given specific attributes. n3=5 bash$ echo $n1 $n2 $n3 3 4 5 This will not work: bash$ n="(($n1+$n2)*$n3)" bash$ echo $n ((3+4)*5) But this will: bash-2.)) (see Tuesday): This will not work: bash$ declare area1 bash$ area1=5*5 bash$ echo $area1 5*5 But using 'declare -i' this will: bash$ declare -i area2 . done File: file File: file 2 Arithmetic Bash will evaluate an arithmetic expression enclosed in $((. Declare a read-only variable: bash$ declare -r pi=3..05a$ echo $n 35 Or use nested $((. done File: file File: file File: 2 bash$ for file in *. n2=4.14 bash$ echo $pi 3.14 bash$ pi=20 bash: pi: readonly variable Variables can also be marked as integers.

otherwise to some default value. 2008 3:20:00 PM bash$ area2=5*5 bash$ echo $area2 25 Default Parameter Values Suppose you wish to set a variable to the value of $1 if $1 is given.${norm}%7d\n" " " "Values:" $1 $2 bash$ tst 1234 5678 Values: 1234. It takes a format string and a set of arguments to print within the context of that format string. and . May 27. indented by a calculated amount. then level=5 else level=$1 fi Or simpler: level=${1:-5} Use Printf The printf command is very much more powerful than echo. 5678 Setting the Prompt The environment variable PS1 contains the bash shell primary prompt. This can be achieved with: #!/bin/bash #set level to $1. This statement will print a line.'. and the first value in red. the label right-aligned. for example: $ declare -x PS1="host=\h\$ " host=saruman$ \h is replaced by the hostname up to the first '.Unix_tricks. \$ is replaced by '$' for normal users. or the default 5 if $1 not given if [ "$1" == "" ]. Change this with.txt Page 119 of 271 Printed: Tuesday. 5678 bash$ tst 1234 5678 8 Values: 1234. bash$ cat tst #!/bin/bash declare -r norm="\033[00m" red="\033[31m" declare -i indent=${3:-4} declare -i spacer=20-$indent printf "%${indent}s%-${spacer}s${red}%7d.

This file is executed when bash is invoked as a non-login shell (when you type 'bash' to start a sub-shell). If you have added the line '.app). set environment variables like PS1 (see Monday's tip) and the search PATH. .. 2008 3:20:00 PM by '#' for root. otherwise a $ \nnn character with octal number nnn \\ a backslash \[ start non-printing characters \] end non-printing characters Bash Startup Files Create a file called '. Also add the line: .bash_profile'.Unix_tricks. .bash_profile' in the root of your home directory. '.g.txt Page 120 of 271 Printed: Tuesday. wanting to return to the original directory.e.bashrc' will then be executed for all (login and non-login) interactive shells.' \H the hostname \j # jobs currently managed by the shell \l basename of shell's terminal device \n newline \r carriage return \s name of shell. version+patchlevel \w the current working directory \W basename of current working directory \! the history number of this command \# command number of this command \$ a # for the root user .bashrc'.bashrc This will execute the second startup file. 'pushd' changes to the new directory.bashrc' to '. 2. 'popd' returns to the original directory.00) \V release of bash.i. Pushing and Popping If you wish to change directories temporarily. This file is executed when bash is invoked as a login shell (when you start a new Terminal session with Terminal. basename of $0 \t time in 24-hour HH:MM:SS format \T time in 12-hour HH:MM:SS format \@ time in 12-hour am/pm format \A time in 24-hour HH:MM format \u the username of the current user \v the version of bash (e. Create a second file called '. but saves the original directory. . This example will display the current username and time in red: $ declare -x PS1="\e[31m\u-\A\e[0m\$ " saruman-10:56$ Other possibilities (taken from the bash manual) \a an ASCII bell character (07) \d date as "Weekday Month Date" \e an ASCII escape character (033) \h the hostname up to the first `. use 'pushd' instead of 'cd'. In here place all settings that are passed to sub-shells . May 27.bashrc' in the root of your home directory. Place all other setting in '.

inputrc' in the root of your home directory and add the line: set completion-ignore-case on Additionally. declare -x HISTFILESIZE=2000. a Terminal session saves the last 500 commands in its history. to cause auto-completion to list alternatives immediately (bash normally rings the bell first) add the line: set show-all-if-ambiguous on Open Files and Folders Use the 'open' command to simulate a double click in the Finder.bashrc' shopt -s histappend This causes histories from different Terminal session to be merged. May 27. If you are in the habit of leaving your Terminal open for long periods and would like more history (say 2000 commands) add the following to '. set the history merge option in '. then to prevent the last exited session from overwriting the history file.Unix_tricks. . If you have more than one terminal session open. ie: $ ls ~/doc <hit tab> will be completed to: $ ls ~/Documents/ create a file called '. declare -x HISTSIZE=2000. Ignore Case If you wish for bash's tabbed auto filename completion to ignore case. Here are some examples of its usage.bash_profile' (see Tuesday's tip). It makes sense to increase the size of the history file too. 2008 3:20:00 PM For example: $ pwd /Users/saruman $ pushd /etc /etc ~ $ pushd /var/log /var/log /etc ~ $ pwd /var/log $ popd /etc ~ $ pwd /etc $ popd ~ $ pwd /Users/saruman $ More History Be default.txt Page 121 of 271 Printed: Tuesday.

app/Contents/MacOS/Clock % ps -auxc | grep "[C]lock" saruman 2405 .. 2008 3:20:00 PM Open a folder into a Finder window: % open Sites Open a document with the default application: % open Sites/index.txt Page 122 of 271 Printed: Tuesday. i.) % sudo -b /Applications/Clock.html pic1.and % ps -auxc | grep "[C]lock" saruman 2405 .. Open an application: % open /Applications/ & The '&' at the end of the command will launch Safari as a separate process allowing you to continue using the current\ Explorer & (Note there are no line-breaks in this command.57 Clock (Clock is running as my user 'saruman'.. May 27. % sudo -b open /Applications/Clock.72 Clock root 2410 .) . without using 'open'.36 Clock (The second Clock is running as root. One must launch manually (see Tuesday's tip).. where the Finder (and 'open') would not do so. Using 'sudo open' will not work. It can also be used to launch multiple copies of an Application. 0:00.) Carbon Applications Launch a Carbon application (in this example Internet Explorer) Launch a Cocoa application manually. 0:00.. NOT as root.jpeg . 0:00.html Open multiple folders/documents/applications in one go: % open *.Unix_tricks. Launch Applications as Root Use the 'sudo' command to launch an application running as root.framework/Versions/ Current/Support/LaunchCFMApp /Applications/ Internet\ Explorer. Manual launching may be quicker in Sites/index.e.. as 'open' runs as root but the application it launches does not.html Open a document with another (non-default) application: % open -a /Applications/Internet\ Explorer.. Launch Applications Launch GUI applications.jpeg pic2. using 'open'. % /Applications/Safari. % /System/Library/Frameworks/Carbon. both Cocoa..

They must be launched by 'LaunchCFMApp'. NEVER use 'sudo'. $ echo ${test2:-not-set} not-set $ echo $test2 Note that test2 remains unset unless one uses ':=' instead of ':-' $ echo ${test2:=not-set} not-set $ echo $test2 not-set Error on Unset Get bash to report an error if a variable or parameter is not set. you may notice that using the '-c' option with 'ps' displays LaunchCFMApp as the application name instead of the application itself. $ echo ${test3:?} bash: test3: parameter null or not set Write a custom error message with: $ echo ${test3:?it is not set} bash: test3: it is not set An error is reported if a variable does not exist. to fail.Unix_tricks. On systems prior to Panther. $ echo $test /hello/world $ echo ${test:-not-set} /hello/world I have used 'echo' to illustrate the expansion. as above. May 27. 2008 3:20:00 PM Carbon applications are not in the correct binary format for Mac OS X. In the code examples I have used a shell variable.framework/Versions/ Current/Support/LaunchCFMApp /Applications/ Internet\ Explorer. or to a default value if the other variable/parameter is not set. Set a shell variable to either the value of another variable/parameter. Launch a Carbon Application as root using: % sudo echo % sudo -b /System/Library/Frameworks/\ Explorer & The 'sudo echo' line gets around password entry problems. but normally one would assign to a variable: $ var=${test:-not-set} In the next example the variable 'test2' is not set and so the default value is substituted. and shell variables like $my_var. Default Values This week's tips apply to both bash script parameters like $1. which does the necessary magic. or is set to null (nothing): $ test4= $ echo ${test4:?} bash: test4: parameter null or not set . This will cause scripts. and commands such as 'killall'. when you've had too many beers. or do anything as root.txt Page 123 of 271 Printed: Tuesday.

to match 0 or more occurrences of any characters. $ echo ${test%/*} /hello $ echo ${test%%/*} (empty string) . use of a double # matches the longest string ('/hello/worl'). 2008 3:20:00 PM Slicing Take a slice of a variable. the shortest string matching '*l' was chopped ('/hel'). $ echo $test /hello/world This example searches for a pattern at the start of the variable ("/hell") and chops it from the expansion: $ echo ${test#/hell} o/world It is permissible to use the wildcard '*'.Unix_tricks. $ echo ${#test} 12 Chopping Chop the head or tail from a variable. This should be used within a bash script. $ echo ${test#*l} lo/world In the above example. Determine the number of characters in a variable's value. May 27.txt Page 124 of 271 Printed: Tuesday. Assuming $0 is /usr/local/bin/script scriptname=${0##*/} echo $scriptname gives: script Use % instead of # (and %% instead of ##) to chop from the end of the string. and the second extracts two characters from the fourth character on. $ echo $test /hello/world $ echo ${test:4} lo/world $ echo ${test:4:2} lo The first example extracts a slice from the fourth character on (the first being counted as 0). $ echo ${test##*l} d Extract just the script name from the full pathname held in parameter 0. In the next.

type: /\<transform\> To search backwards use: ?transform To search for the next occurrence hit 'n'. Use regular expressions in vim searches. To recall a previously entered search-replace type '/' then hit the up-arrow key until the desired search-replace is found. $ echo ${test/l/L} /heLlo/world $ echo ${test//l/L} /heLLo/worLd Use # to anchor the pattern to the start of the string. Search for a line starting with 'The': /^The . 2008 3:20:00 PM Search and Replace Search and replace within a variable.Unix_tricks.$' which performs the replacement from the current line (line .. and for the previous hit 'N'. See weeks 59 and 60 for information on regular expressions. such that 'transformations' is not matched. Replace '%' with an address range such as '. May 27. and % to anchor to the end. $ echo ${test2/#d/D} ldld $ echo ${test2/%d/D} ldlD Basic Search in Vim Search a file for particular text. To search for the first occurrence of 'hello' and replace it with 'goodbye': $ echo $test /hello/world $ echo ${test/hello/goodbye} /goodbye/world To replace all occurences of a pattern use '//'. Search and Replace in Vim Replace typo's such as 'teh' with 'the' using search and replace: :%s/teh/the/g '%' means all lines in the file and 'g' means every occurrence on each line. In normal mode (hit escape to exit from insert mode) type: /transform If you require a complete word match.) to the end of file (line $).txt Page 125 of 271 Printed: Tuesday. To recall a previously entered search type '/' then hit the up-arrow key until the desired search is found. such as 'transform'.

Only whole words will be matched. May 27. Quick Search in Vim Find the next occurrence of the word currently under the cursor.\) captures text in the search pattern. while \1 replays it in the replace string. To perform a search and replace. or '#' to search backwards. but only on lines that begin with 'edit'. and for the previous hit 'N'. This is the same form as used by 'sed'. In normal mode hit '*'.. Customising the Search in Vim Make searches case insensitive with: :set ignorecase Now pattern /the matches The.. THE but pattern /The matches only The Highlight all matches by setting: :set hlsearch and switch off with: :set nohlsearch and use: :noh[ighlight] to temporary turn of the current highlights. the.*\) [I|i]ncludes/Section \1 Contains:/g The construct \(. unless you use 'g*' or 'g#' To search for the next occurrence hit 'n'. Search and Edit in Vim Use global commands to search for lines to edit. THE Set smartcase so patterns in all lowercase match in a case insensitive manner. but patterns including uppercase are case sensitive: :set smartcase The pattern /the matches The. 2008 3:20:00 PM Replace 'chapter <any text> includes' with 'Section <same text> Contains:' : :%s/[C|c]hapter \(. For more information on the available commands use: man ex To delete all lines that start with a './d .txt Page 126 of 271 Printed: Tuesday.Unix_tricks. use: :g/^edit/s/hello/Goodbye/g The general form of this command is: g/search-pattern/command 'search-pattern' is a regular expression and 'command' is an 'ex' command. the.' use: :g/^\.

See week 49. using command 'file': $ file ~/bin/* add-user: Bourne-Again shell script text dhup: Bourne shell script text crontab: ASCII text nodif: Tenex C shell script text test: Mach-O executable ppc $ file ~/sites/./important-info/contact-info. . version 1. Apple Macintosh ./important-info/PW. Apple Macintosh Extracting Filenames Tune Tuesday's example to print only filenames: $ find . | xargs -n1 file | grep -i "rich text" | awk '{print $1}' ../Job/CV. Filter for all rich text files using grep: $ find . $ find ./Job/CV. Friday in particular.rtf: To remove the trailing ':' tell awk that the field separator is ':': ./* data: directory images: directory include: directory index. 2008 3:20:00 PM The command 'd' deletes the line. -print0 | xargs -0 -n1 file | grep -i "rich text" .../important-info/Packing List. The File Command Determine the type of a file. Apple Macintosh .rtf: . based on an analysis of its contents..rtf: . May 27.php: C++ program text NOT QUITE! java-script: directory jf.rtf: Rich Text Format data..html: HTML document text tc.rtf: Rich Text Format data./important-info/PW. -print0 | xargs -0 -n1 file .txt Page 127 of 271 Printed: Tuesday./important-info/contact-info.' to all lines that contain 'edit' :g/edit/normal A.txt: English text .Unix_tricks. version 1.rtf: Rich Text Format data./important-info: directory .rtf: Rich Text Format data..html: HTML document text Using the File Command Use command 'file' in conjunction with other commands to filter file types./important-info/mac-todo. version 1... To select all lines that do not match the pattern use 'g!' instead of 'g' To use normal mode vim commands instead of ex commands use: g/search-pattern/normal vim-command-list This example will add '.

.c. or escaped (\t = tab).. and displays each sequence.rtf ./important-info/PW.v 1.c.] @(#)PROGRAM:ls PROJECT:file_cmds-60 DEVELOPER:root BUILT:Sun Jul 14 03:44:11 PDT 2002 'strings' searches a binary file for a sequence of printable characters. One can print in character format.or in hexadecimal plus character. | xargs -n1 file | grep -i "rich text" | awk -F: '{print $1}' .. The default minimum sequence length is 4.c.: $ hexdump -n32 -c test 0000000 a \t b 001 c \n n o w s o m e t 0000010 e x t a n d t a b s \n \r .31 1998/08/19 01:44:19 thorpej Exp $ 1ACFLRSTWacdfgiklnoqrstuxv %s: directory causes a cycle $NetBSD: print. or a file that contains unprintable characters: $ hexdump -n32 test 0000000 6109 6201 630a 6e6f 7720 736f 6d65 2074 0000010 6578 7420 616e 6420 7461 6273 2020 0a0d The file is displayed in hexadecimal.`find .15 1998/07/28 05:31:25 mycroft Exp $ usage: ls [-1ACFLRSTWacdfgiklnoqrstux] [file ... $NetBSD: cmp..v 1.Unix_tricks. where non-printable characters are either represented in octal. 1993. limited to the first 32 bytes (-n32).` instead of $(find some t| 00000010 65 78 74 20 61 6e 64 20 74 61 62 73 20 20 0a 0d |ext and tabs ..6 1997/07/20 18:53:12 christos Exp $ $NetBSD: util.22 1998/07/28 05:15:47 mycroft Exp $ %s %*u %-*s %-*s $NetBSD: stat_flags.rtf .rtf Finally.c.txt Page 128 of 271 Printed: Tuesday. $NetBSD: ls./important-info/contact-info. 2008 3:20:00 PM $ find .rtf . | xargs -n1 file | grep -i "rich text" | awk -F: '{print $1}') In the tcsh shell use backquotes .c./Job/Adrian-Mayo-CV..b.. May 27.. where non-printable characters are represented by a dot: $ hexdump -n32 -C test 00000000 61 09 62 01 63 0a 6e 6f 77 20 73 6f 6d 65 20 74 |a.c./Job/job-search.14 1998/10/09 02:00:39 enami Exp $ @(#) Copyright (c) 1989. edit all the rich text files: $ open -a textedit $(find . The Hexdump Command Use command 'hexdump' to display the contents of a binary file.v 1.) .v 1. a$ strings -a -16 /bin/ls __dyld_mod_term_funcs __dyld_make_delayed_module_initializer_calls The kernel support for the dynamic linker is not present to run this program.v 1. executable files have . Option -a searches all sections of the file (for example. but can be overridden as in this example (option -16). 1994 The Regents of the University of California. All rights reserved.| Try this: $ hexdump -n4 /bin/* The String Command Search for ASCII text within binary files using command 'strings'.

More Shopt Options Continuing from Monday's tip: (3) Enable case-insensitive globbing: $ ls -d d* ls: d*: No such file or directory Now enable case-insensitive: $ shopt -s nocaseglob $ ls -d d* Desktop Development Documents Note: for case-insensitive completion see: Week 66 . May 27. To set: $ shopt -s <option> To unset: $ shopt -u <option> To list: $ shopt Some useful options are illustrated below.txt Page 129 of 271 Printed: Tuesday.Friday.Unix_tricks. Shell Options Use command 'shopt' to set shell options. (4) Include dot files in globbing: $ ls -d * Desktop Documents Movies Pictures Sites osxfaq Development Library Music Public bin Now enable dot globbing: $ shopt -s dotglob . (1) Use 'cdspell' to correct minor typo's in filenames given to 'cd': $ cd Dekstop bash: cd: Dekstop: No such file or directory Now set shell option 'cdspell': $ shopt -s cdspell $ cd Dekstop Desktop (2) Create short aliases for long directory names: $ shopt -s cdable_vars $ cal="/Users/saruman/Sites/calendar" $ cd cal /Users/saruman/Sites/calendar Note: don't use the '~' alias in 'cdable' variables. 2008 3:20:00 PM code and data sections).

app This can be useful: $ declare -x PROMPT_COMMAND='echo -n "^[]0. To print the date before each prompt: $ declare -x PROMPT_COMMAND='date' Mon Feb 2 12:02:41 GMT 2004 $ Write a message to the title bar of the Terminal window (in this example the output from command date): $ declare -x PROMPT_COMMAND='echo -n "^[]0. got by typing control-V control-G. Aliases With Parameters Unlike tcsh.Unix_tricks.. ^G = control G..d .2 .gvimrc .]' are recognized (as in 'ls *' or 'ls file. } $ word apple apple appleberry appleblossom applecart appledrane applegrower .txt Page 130 of 271 Printed: Desktop . and '[. '?'. (5) Enable enhanced pathname expansion. $ shopt -s extglob The following extended globbing is enabled: ?(pattern-list) Match 0 or 1 occurrence of the given patterns *(pattern-list) Match 0 or more occurrences of the given patterns +(pattern-list) Match 1 or more occurrences of the given patterns @(pattern-list) Match exactly one of the given patterns !(pattern-list) Match anything but one of the given patterns Prompt Commands Have bash execute a command before it issues a prompt.dir_colors .emacs. 2008 3:20:00 PM $ ls -d * .FBCIndex . May 27. Normally just '*'.. got by typing contol-V then escape. This is because bash provides a more powerful mechanism by way of functions.mysql_history Documents . bash aliases do not accept parameters.???'). Write a function called 'word' which searches the Webster's dictionary at /usr/share/dict.. $TITLE^G"' $ declare -x TITLE="Hello" Now any executing command or script can report to the title bar by setting environment variable TITLE. $(date)^G"' Notes: ^[ = escape.DS_Store .web2: $ function word () { grep $* /usr/share/dict/web2. This works for xterm and Apple's Terminal.gimp-1.cnf Development .

given no filename parameters. $1 expands to parameter 1. This text is the result of a 'Here Document' EOD echo "End" The section between '<<EOD' and the line containing just 'EOD' is read by the shell and becomes standard input for the preceding command. fi } Basic HereDoc Redirect text to standard input using Here Documents. etc. This text is the result of a 'Here Document' End $ Variable Expansion Control variable expansion within a HereDoc. will read standard input (the supplied text) and write it to standard output.. Shell variables within a HereDoc are expanded as usual: $ cat hd #!/bin/bash . $ cat hd #!/bin/bash echo "Test" cat <<EOD Hello there. else builtin alias $* . To list all functions use: $ declare -f Extending the 'alias' Command If you wish for the 'alias' command to list functions (see Thursday's tip) declare this function: function alias () { if [ -z $* ] .Unix_tricks.. $* expands to all parameters passed to function 'word'. In this case 'cat'. $ . 2008 3:20:00 PM applejack ./hd Test Hello there. May 27. declare -f. then builtin alias.txt Page 131 of 271 Printed: Tuesday.

$ Read Data Use a HereDoc and a function to set variables. $ But. without including the tab characters in the HereDoc text. EOD $ .txt Page 132 of 271 Printed: Tuesday. 2008 3:20:00 PM today=$(date) cat <<EOD Hello there. $ Remove Indentation Remove shell script indentation from HereDoc text. Today is Mon Feb 16 12:08:16 GMT 2004./hd Hello there. Today is $today./hd Hello there. Non-trivial shell scripts usually nest statements. if the delimiter is quoted no variable expansion is performed: $ cat hd #!/bin/bash today=$(date) cat <<'EOD' Hello there. $ cat hd #!/bin/bash read_data () . $ cat hd #!/bin/bash if [ "$1" = "" ].Unix_tricks. EOD $ . use '<<-'. May 27. EOD fi $ . Today is Mon Feb 16 11:53:46 GMT 2004. then cat <<-EOD Hello there. Today is $today. Today is $today. Today is $(date). To allow the HereDoc to follow the natural flow of indentation. and it is natural to use indentation./hd Hello there.

May 27./hd: var2: parameter null or not set $ ./hd before after $ Here the 'if' statement has been commented out.txt Page 133 of 271 Printed: Tuesday.Unix_tricks. colour: Blue $ Comment Out / Test Variables Thanks to Ian Ferguson for Friday's Tips./hd Make: BMW./hd . 2008 3:20:00 PM { read make read model read colour } read_data <<HEREDOC BMW 3 series Blue HEREDOC echo "Make: $make. model: $model. colour: $colour" $ . Use HereDocs to display an error message if a variable is not set: $ cat hd #!/bin/bash var1=hello : <<ERRORIFNOTSET ${var1?} ${var2?} ERRORIFNOTSET $ . then echo "No parameter given" fi EOD echo "after" $ . model: 3 series. avoiding having to add '#' to the start of every line: $ cat hd #!/bin/bash echo "before" :<<'EOD' if [ "$1" = "" ]. Use HereDocs to comment out code.

.$2.$2. do $1 "$file" "${file%.html to have an extension of .$2 shopt -s nullglob for file in *. Add parameter checking to the script: $ cat ext #!/bin/bash if [ "$3" = "" ] || [ "$4" != "" ]. will let you form such a command: $ cat ext #!/bin/bash shopt -s nullglob for file in *. do pushd $dir > /dev/null ext $1 $2 $3 popd > /dev/null done . 2008 3:20:00 PM Process on Extension Rename a group of files in a single command. but applies the command to all matching files in the directory hierarchy. called 'ext' for extension-based. with the 'ext' script. then echo 'For each file *..$2}.$2 returns null not *.php usage: mv [-fi] source target mv [-fi] source . This will *not* work: $ mv *.$2}.$3" done Recursive Process on Extension Rename a group of files recursively in a single command.Unix_tricks. -type d).ext1. It uses the 'ext' script. such as 'cp'. $ cat rext #!/bin/bash for dir in $(find .html *.ext2"' echo ' eg "cp html php" to copy all html files to php equivalents' echo ' "mv html php" to rename all html files to have a php extension' echo "Usage: ${0##*/} command ext1 ext2" exit fi # this option ensures that no match for *. do $1 "$file" "${file%. directory The following script.$3" done Use 'ext' like this to rename (mv) *.txt Page 134 of 271 Printed: Tuesday. May 27. The script 'rext' does the same as Monday's 'ext' script.ext1 file. executes "command file.php: $ ext mv html php Use any similar command.

html: $ rext cp php html Use any similar command. use the 'each' script to simulate this. -type d).php to equivalent files with an extension of . such as 'mv'. do pushd $dir > /dev/null ext $1 $2 $3 popd > /dev/null done Process Multiple Files If you have a script or command that is not able to take a list of files.html my-script -options . do $* "$file" done Use the script like this to apply 'my-script' to all . 2008 3:20:00 PM Use 'rext' like this to copy (cp) *. May 27. Add parameter checking to the script: $ cat each #!/bin/bash if [ "$2" = "" ]. then echo 'Executes "command file. 'each' executes the given command once for each file. $ cat each #!/bin/bash filetype=$1 shift for file in $filetype. with the 'rext' script.Unix_tricks.. Add parameter checking to the script: $ cat rext #!/bin/bash if [ "$3" = "" ] || [ "$4" != "" ]. then echo "Executes a command on each matching file in the current directory" echo "Usage: ${0##*/} filetype command-to-execcute" exit fi .ext1 file.txt Page 135 of 271 Printed: Tuesday..html files in the current directory: $ each \*.ext2" on each matching file file in' echo ' the directory *hierarchy*' echo ' eg "cp html php" to copy all html files to php equivalents' echo ' "mv html php" to rename all html files to have a php extension' echo "Usage: ${0##*/} command ext1 ext2" exit fi for dir in $(find .

-name "$filetype" -print0 | xargs -0 -n1 $* Use the script like this to apply 'my-script' to all .Unix_tricks. Add parameter checking to the script: % cat reach #!/bin/bash if [ "$2" = "" ]. do $* "$file" done Recursive Process Multiple Files The script 'reach' does the same as Wednesday's 'each' script..php files in the directory hierarchy: $ reach \*. Normally sed uses standard in and out. then echo "Executes a command on each matching file in the directory *hierarchy*" echo "Usage: ${0##*/} filetype command-to-execute" exit fi filetype=$1 shift find . $cat reach #!/bin/bash filetype=$1 shift find . -name "$filetype" -print0 | xargs -0 -n1 $* Sed Extension Create a simple wrapper for the 'sed' command (or any similar command) to enable it to process a file directly.php my-script -options .. . but executes the given command on all matching files in the directory hierarchy. mv $tmp "$2" should not be used as an error in the sed script will cause all files to be wiped. $ cat sedx #!/bin/sh tmp=tmp-file-for-$PPID { sed "$1" "$2" > $tmp \ && mv $tmp "$2" } || rm $tmp Note that the more obvious line: sed "$1" "$2" > $tmp . May 27. and cannot write back to its input file. 2008 3:20:00 PM filetype=$1 shift for file in $filetype.txt Page 136 of 271 Printed: Tuesday.

" echo "Usage: ${0##*/} sed-command file" echo "Or use with (r)each" echo "Usage: (r)each filetype ${0##*/} sed-command" exit fi tmp=tmp-file-for-$PPID { sed "$1" "$2" > $tmp \ && mv $tmp "$2" } || rm $tmp Split Screen This week's tip refer to Apple's Terminal application (/Applications/Utilities/Terminal. and command-V to paste. One can scroll the top frame to view previous commands while working as normal in the bottom This allows copy and paste between the two frames. use the usual command-C (after highlighting text) to copy.html sedx 's/XXX/YYY/' Add parameter checking to the script: $ cat sedx #!/bin/sh if [ "$2" = "" -o "$1" = "-usage" ].. Apply to all files in the current directory: $ each \*. Two command-line utilities are provided to copy and paste: $ pbpaste Two command-line utilities are provided to copy and paste: . Select split screen mode in the Terminal by clicking the icon in the top right corner. May 27.txt Page 137 of 271 Printed: Tuesday. The Terminal also supports drag and drop on the selected text. then echo "Executes a sed command writing back to the original file.. The script as given takes only one input file.html to apply the sed command to index. 2008 3:20:00 PM Use sedx in this way: $ sedx 's/XXX/ZZZ/' index. Copy and Paste Copy text in terminal window by highlighting it as you would in any other application (left click and drag).Unix_tricks.html. Use the scripts from Wednesday and Thursday to process multiple files.html sedx 's/XXX/YYY/' Or to all files in the directory hierarchy (recursively): $ reach \*. The selected text is automatically put into a buffer which can be recalled by clicking the middle mouse button* To paste between different windows or applications.

term Files Create different sets of Terminal window settings (colours. but with control over the Terminal window settings -caux.command' file (see Wednesday). and double clicking the file will open a new terminal and run the shell script in it.command' file.command' file in folder ~ /Library/Application\ Support/Terminal/ and the '.term' file. Use the Unix manual to look up the details. System Information This week I will list useful commands that you might not know about.exit</string> and you get a similar effect to a '. ls -al ~.df -h. window size.Unix_tricks.command Files Create a double-clickable shell script by saving it as a '.command'. . One can also use the command line utility 'open' to run the script from the command line (again.. Save a terminal session using File -> Save As to create a '.' Examine a '. etc). Double-clicking this opens a Terminal window with exactly the same window settings as the Terminal window from which it was saved. Remove '. or might have forgotten about.exit' if you do not wish the Terminal window to close upon completion of the Execution String.command' file will appear in the Terminal's menu 'File -> Library -> . Other Tricks Drag a folder or file onto the Terminal window and the full pathname will be written to the command line. To gather system information: $ arch ppc . Place the '. using either echo or printf. . May 27.txt Page 138 of 271 Printed: Tuesday. Any standard shell script can be saved in a file with an extension '.term' file and you will see: <key>ExecutionString</key> Add shell commands to this: <key>ExecutionString</key> <string>ls -al /.. in a new terminal window). The Terminal title can be set on the command line with: (bssh) declare -x PROMPT_COMMAND='echo -n -e "\033]0. 2008 3:20:00 PM (text previously copied from a GUI application) $ echo "hello there" | pbcopy $ pbpaste hello there * A three mouse button is handy when working in Apple's Terminal or xterm.$user@$host\007"' The above examples display the current user and hostname.$USER@$HOSTNAME\007"' (tcsh) alias precmd 'printf "\033]0.

37. Mach factor: 0. 2008 3:20:00 PM $ machine ppc7450 $ uname -v Darwin Kernel Version 7. For example. root:xnu/xnu-517. $ ls /usr/bin/z* /usr/bin/zcat /usr/bin/zdiff /usr/bin/zgrep /usr/bin/zmore /usr/bin/zprint /usr/bin/zcmp /usr/bin/zforce /usr/bin/zip /usr/bin/znew If the files are compressed with bzip2.0: Fri Mar 5 14:22:55 PST 2004. root:xnu/xnu-517. without going through osascript) to add speech to scripts: $ say "Hello world. 249 threads.00 megabytes.3. 1 processor is physically available. Processor type: ppc7450 (PowerPC 7450) Processor active: 0 Primary memory available: 768.3.e.obj~4/RELEASE_PPC $ hostinfo Mach kernel version: Darwin Kernel Version 7. viewing.0: Fri Mar 5 14:22:55 PST 2004.15." Lookup some words to say with look: $ look zy zyga zygadenine Zygadenus Zygaena . 1 processors Load average: 2.obj~4/RELEASE_PPC Kernel configured for a single processor only. the possibly rezipping.22 $ system_profiler (much output) Working with (B)Zipped Files If you wish to work with zipped files. use the z* set of commands. use 'zmore' to view a file instead of unzipping.3.15.Unix_tricks. Default processor set: 94 tasks. May 27.txt Page 139 of 271 Printed: Tuesday.3. use the bz* set of commands: $ ls /usr/bin/bz* /usr/bin/bzcat /usr/bin/bzegrep /usr/bin/bzip2 /usr/bin/bzmore /usr/bin/bzcmp /usr/bin/bzfgrep /usr/bin/bzip2recover /usr/bin/bzdiff /usr/bin/bzgrep /usr/bin/bzless Talking Scripts Use the 'say' command directly (i.

Note: Formerly most marine alg? were called fuci.] 1.. deceit. {Fuci}. rockweed. .. (Bot. also.1084790981@pan.000u 0. 0. with the Software Update command line tool: $ softwareupdate -l Software Update Tool Copyright 2002-2003 Apple Computer.000s] 221 bye [d/m/c = 0/0/0. Inc.. perhaps remotely. A paint.mime> <1904283. [L. leathery seaweeds. Or search the dictionary with grep: $ grep "^d.alephnull.8. [Obs. false show. May 27.000s] Software Update Perform a software update.) A genus of tough.] 2. Don't forget 'nvram' to manipluate open firmware parameters: To list nvram variables: $ nvram -p To set verbose login (equivalent to Command-V on startup) $ sudo nvram boot args="-v" Use 'bless' to change the boot volume: sudo bless -folder '/Volumes/vol-name/System/Library/CoreServices' -setOF sudo bless -folder '/System/Library/CoreServices' -setOF Image Manipulation Use the Scriptable Image Processing System (sips command) to manipulate images via the command line or AppleScript. Your software is up to date.4. orchil. rock lichen. 2008 3:20:00 PM zygaenid Zygaenidae . pl. . n. a dye.. disguise.0/rf on Linux 2.000r 0.18-14 <auth.000r> 250 ok 150 1 definitions retrieved 151 "Fuci" web1913 "Webster's Revised Unabridged Dictionary (1913)" Fucus \Fu"cus\. used as a red dye.txt Page 140 of 271 Printed: Tuesday.. 0.000u 0. red or purple dictd 220 pan. 250 ok [d/m/c = 1/0/22. usually of a dull brownish green color.Unix_tricks.p$" /usr/share/dict/web2 dadap decap dreep droop Or the online dictionary for the meaning of 'fuci': $ curl dict://dict.alephnull.29426.

issue the command: $ set -o vi and to get back to Emacs: $ set -o emacs Search the Command Line History Quickly find a command line you have issued previously.Unix_tricks. Keep typing until the correct command is showing.html Start typing any part of the command line. or press control-r again to search further back. or the filename. The sequence escape-. control-A moves to the start of the line control-E moves to the end of the line escape-del deletes the word behind the cursor del deletes the character behind the cursor control-d deletes the character forward of the cursor esc-d deletes the word forward of the cursor control-k deletes to the end of line control-u deletes the whole line These commands will be familiar to Emacs users. The command will be executed then the following command will be displayed ready to issue/edit. and bash will back-search its history for the first command that matches what you have typed so far. for example the command. Repeating Repeat a sequence of commands: Hit the up arrow cursor key until the first command in the sequence to repeat is displayed. press return to execute it. When the desired command is showing. anywhere on the line. To use vi-style command line editing.c type t (reverse-i-search)`sit': vim ~/sites/index. 2008 3:20:00 PM There is no manual page. Issue: . and repeat or edit it. so use: $ sips --help Command Line Editing This week's tips help to make entering commands in Bash quicker and easier. or the forward arrow cursor key to edit it. May 27. Hit control-r then start typing the command: $ control-r (reverse-i-search)`': type si (reverse-i-search)`si': vim signal. instead of hitting return. hit control-o. recalls the filename from the previous command.txt Page 141 of 271 Printed: Tuesday. Then. First off is command line editing: Edit the current line (or one from the command line history recalled by pressing the up arrow cursor key) with more than the usual cursor key and back-deleting (del).

Getting the Right Help Use Bash's help system for information on its built-in commands. or a short synopsis if option -s is given. For full details see 'man bash' and search for HISTIGNORE. Similarly. There are two techniques: Set the environment variable HISTCONTROL: $ declare -x HISTCONTROL="ignoreboth" Now command lines starting with space are not entered into the history.particularly useful for long or awkward filenames. lines matching the last line in the history are not added. useful if the command contains sensitive information such as a password. use 'ignorespace' or 'ignoredups' instead of 'ignoreboth'.) Avoiding History Prevent particular lines from entering the command history. then tab completion will complete based on the currently defined shell variables. Additionally.txt Page 142 of 271 Printed: Tuesday.Unix_tricks. For @ to work. $ help <command> to display the help page for the given command. which contains entries in the same format at /etc/hosts. will be changed into $ cd osxfaq .) $ help to list them all. (These will not be found in the Unix man pages. Use Tab to also complete commands. May 27. If the word begins with $. To control these aspects individually. Additionally. and functions. Supply a pattern and Bash will display a help page for each command that matches the pattern. the variable HOSTFILE must contain the name of a file. ~ (tilde) completes based on user names. The second technique is to set the environment variable HISTIGNORE to a colon-separated list of patterns. 2008 3:20:00 PM $ ls -al osxfaq then: $ cd escape-. . aliases. (See 'man hosts'. and @ completes based on hostnames. the shell variable $_ can be used: $ echo $_ osxfaq Command Line Expansion You probably know that hitting tab will complete a filename.

txt Page 143 of 271 Printed: Tuesday. built-ins. Bash may execute an alias. keywords. Aliases List all aliases with: $ alias Set an alias with: $ alias la="/bin/ls -Al" Remove an alias with: $ unalias Here is a trick to alias directory names. commands from $PATH Use the following Bash built-ins to change the search order for a one-off command: * 'command' forces only built-ins or commands from $PATH to be considered. What if you wish to override this? Bash's search order is: aliases. Normally Bash does alias substitution on the first word of a command line (the command) only.. functions. and functions * 'builtin' forces only built-ins to be considered * 'enable' forces only commands from $PATH to be considered For example: $ command ls forces ls from $PATH to be executed. Note: the 'tcsh' shell uses a backslash % \ls to override aliases and built-ins. May 27.. Determine the type of command Bash will execute using the command 'type'. a built-in. overriding any alias or function you may have defined for ls. keywords. $ type ls ls is aliased to `/usr/local/bin/ls --color=tty' View all possibilities with: $ type -a ls ls is aliased to `/usr/local/bin/ls --color=tty' ls is /usr/local/bin/ls ls is /bin/ls Forcing the Right Command Tuesday's tip shows you how to determine which variant of a command Bash will execute. This in analogous to the 'which' command in the tcsh shell. ignoring aliases. 2008 3:20:00 PM $ help -s "re*" Getting the Right Command When you issue a command such as 'ls'. $ pwd /Users/saruman . a function. or one from the search path ($PATH).Unix_tricks.

log.. This example illustrates. 2008 3:20:00 PM $ alias pref="~/Library/Preferences/" $ cd pref -bash: cd: pref: No such file or directory .} And: $ tlog system will view the system log (control-C to exit). Functions In Bash. Bash has the more powerful 'function' construct. Declare a function 'psx': $ psx () { ps axww | grep -i $* | grep -v grep..txt Page 144 of 271 Printed: Tuesday. May 27.. use $1.} The special variable $* is expanded to all parameters passed to the function. .51 /Applications/ -psn_0_2359297 Here is a function to view log files: $ tlog () { tail -f -n40 /var/log/$1. This is exactly the same scheme as that used to expand parameters passed to shell scripts. $2... To Sub-Shell or Not to Sub-Shell Know the difference between running a shell and sourcing it. and now it works.. for the first and second (and so on) parameters. $ psx safari 503 ?? S 12:36.. But alias cd as below (note the trailing space). aliases cannot take parameters. unlike the tcsh shell. $ declare -x NEW_ENV_VAR="XXXX" $ new_var="xxxx" $ echo $NEW_ENV_VAR XXXX $ echo $new_var xxxx $ cat tst ..Unix_tricks. To list all functions: $ declare -f or $ declare -F to list just the function names. $ alias cd="cd " $ cd pref $ pwd /Users/saruman/Library/Preferences . Alternatively.hence this does not work.

$ cat tst2 new_var="xxxx" echo $new_var (new_var="test") echo $new_var $ . You may wish to execute many commands in a sub-shell when the output from all of them is ./tst2 xxxx xxxx You might have expected this script to echo xxxx test Because the second assignment is in parenthesis it is executed as a sub-shell. Note that: $ source . 'Sourcing' a script executes it with the current shell. and hence its environment/shell variables are lost./tst $ echo $NEW_ENV_VAR XXXX $ echo $new_var xxxx Notice the variables haven't changed../tst and $ . 2008 3:20:00 PM #!/bin/bash declare -x NEW_ENV_VAR="TEST" new_var="test" Run the script: $ .and this time they have.Unix_tricks./tst are equivalent. When the new shell exits its environment is not passed back to the parent shell... . and (remembering Monday's tip) has no effect on the parents shell's environment. The environment/shell variables set by the script are those of the current shell and so remain set after the script has completed. Parenthesis Use parenthesis to execute commands in a sub-shell. Source the script: $ source tst $ echo $NEW_ENV_VAR TEST $ echo $new_var test . and those of the original shell remain unchanged. Running a script as in the first example launches a new shell.txt Page 145 of 271 Printed: Tuesday.. May 27.

. Removing the quotes around "$*" won't quite work: $ cat tst #!/bin/bash $* > ~/outfile & This is fine: $ tst 'du -sk *' $ cat ~/outfile 5524 Desktop 1992 Development . May 27. Tuesday's tip executed a list of commands in parenthesis: $ (echo hello. not as a sub shell. Again. But the second command still fails.} > outfile except that the command list is executed as part of the current shell. the shell tokenises before it expands. and hence fails . 2008 3:20:00 PM redirected or piped. (Like the difference between running and sourcing as explained in Monday's tip. echo world > outfile hello $ cat outfile world $ (echo hello.. As a illustrative example: $ echo hello./tst: line 2: du -sk *: command not found $ tst 'ls | grep "^D"' . echo world.) Note the space after the opening brace and the semicolon before the closing brace. $ cat tst #!/bin/bash "$*" > ~/outfile & $ tst 'du -sk *' . The syntax requirements are stricter for braces than for parenthesis.Unix_tricks. Consider this script to run commands in the background. eval Use 'eval' to override the shell's command line parsing order.txt Page 146 of 271 Printed: Tuesday. echo world) > outfile $ cat outfile hello world Braces Use braces to execute a sub-shell./tst: line 2: ls | grep "^D": command not found The script fails because the shell takes the first token ("$*") as the command and *then* expands $*. We need to change the shell's order of evaluation. Eval is a subtle but useful command. echo world) > outfile You can achieve the same effect with braces: $ { echo hello.

Why? The shell expands "$*" as usual. thus giving the shell a second stab at tokenizing the '|' symbol as a pipe. or a script. Normally the shell sends a HUP signal to all its sub-processes when it itself receives a HUP or and 'exit' command. 2008 3:20:00 PM to recognise the '|' as a token and therefore as the pipe symbol. 'Nohup' prevents this. nohup Start a sub-shell command list..) Enable it with: $ shopt -s huponexit $ shopt huponexit huponexit on To disable again: $ shopt -u huponexit $ shopt huponexit huponexit off 'Huponexit' applies to all sub-processes. Alternatively. whereas the 'nohup' command is applied only to the given process. $ tst 'ls | grep "^D"' $ ls: |: No such file or directory ls: grep: No such file or directory ls: "^D": No such file or directory Now try with 'eval': $ cat tst #!/bin/bash eval "$*" > ~/outfile & $ tst 'du -sk *' $ cat ~/outfile 5524 Desktop 1992 Development . $ tst 'ls | grep "^D"' $ cat ~/outfile Desktop Development Documents This works. May 27. Note: Depending on your bash settings. Check it with: $ shopt huponexit huponexit off (HUP on exit is disabled. build a signal handler into the script to catch the HUP signal and ignore it.txt Page 147 of 271 Printed: Tuesday. What eval does is to re-evaluate the command line. using 'nohup' if you want it to remain running after the shell has terminated. .. bash may or may not send a HUP signal to all processes.Unix_tricks.

Restore admin status using 'nicl'. Click on your-user-name. The GUI way: Fire up NetInfo Manager in /Applications/Utilities and navigate to: | / | users | your-user-name | in the top panes. Next. -create /users/saruman shell /bin/bash Re-instate Admin Status Restore admin user status. in the lower pane scroll down until you see 'shell' in the 'Property' pane. The command line way: Use the directory services command line (replace 'saruman' with your own short user name). 12 /users/root: shell /bin/sh 67 /users/saruman: shell /bin/bash nigrep takes a regular expression just like grep. Issue the command: $ mount -uw / to mount the system disc with write permission (it will currently be mounted read-only).Unix_tricks. $ sudo dscl . If your user was created pre-Panther. With nigrep: $ nigrep "/bin/b?a?sh" . Hold down command-s on startup and wait for the text to finish scrolling. You cannot use any of the other utilities like 'niload' because the necessary system services daemon is not running.nidb -merge /groups/admin users your-username-here (All on one line. With nireport: $ nireport . and searches the database. /users name shell nobody /usr/bin/false root /bin/sh . you may still be using the tcsh shell instead of bash. May 27. 2008 3:20:00 PM Change Your Shell Use NetInfo to change your default shell.) The 'ni' Utilities Manage the NetInfo database. you must boot into single user mode and use 'nicl' to add yourself back into the admin group.txt Page 148 of 271 Printed: Tuesday. Change the value in the 'Value(s)' pane to be '/bin/bash'. If your admin user loses admin status for whatever reason (and it has happened to a few people). and you do not have a second admin account. $ nicl -raw /var/db/netinfo/local. Query the NetInfo database to see which users are using the 'sh' or 'bash' shell. You will need to authenticate first by clicking the lock icon in the lower left corner and entering an admin password.

to control how NFS mounts are searched for (by default NetInfo but not the /etc/fstab 'flat file') . nobody:*:-2: nogroup:*:-1: wheel:*:0:root . Issue the commands: $ mount -uw / $ mv /var/db/netinfo/local. saruman:*:501: Load information into NetInfo with niload.nidump Restore a broken database from backup: Boot into single user mode . List NetInfo directories with nidump. most of the Unix flat files are searched..nidump You can do this yourself with: $ nidump -r / -t localhost/local > local. niload will load the database from standard Unix flat files: $ sudo niload -dm group .. May 27. < flat-file Check 'man niload' to see if you require the 'd'elete or 'm'erge options. Backup and Restore The NetInfo database is backed up daily into: /private/var/backups/local.Unix_tricks. saruman /bin/bash nireport takes a directory (/users in this example) and a list of properties to display. Check this search order for services with: $ lookupd -configuration Change this for any service by creating a file in directory '/etc/lookupd/'.hold down command-s on startup and wait for the text to finish scrolling. nidump displays NetInfo directories in the standard Unix flat file format: $ nidump group . They start up NetInfo and create an empty database. $ /usr/libexec/kextd $ /usr/sbin/configd $ /sbin/SystemStarter $ /usr/libexec/create_nidb $ /usr/sbin/netinfod -s local Finally use niload to load the backed up database: $ niload -d -r / . 2008 3:20:00 PM daemon /usr/bin/false ..nidb /var/db/netinfo/corrupt. as well as NetInfo and Directory Services.. < /path/to/backed-up/database Override Lookup Order In Panther. For example.txt Page 149 of 271 Printed: Tuesday. Manage the database with niutil or nicl. I prefer to use 'dscl' for this.nidb The next steps were taken from an article by O'Reilly.

Catch signals in bash scripts: $ cat trap1 #!/bin/bash trap 'echo "On no you dont"' INT trap 'echo "not that way either"' TERM while true. 2008 3:20:00 PM create a file called mounts: $ cat /etc/lookupd/mounts LookupOrder Cache FF NI DS This says to search the Flat Files. However.control-c SUSP . it will execute the command given by the appropriate trap and then continue from where it left off. .. a process can catch the signal and take alternative action...Unix_tricks.. Normally the signal causes the process (executable or script) to terminate. In this example it will display a message on the screen. } Usage: $ killsig TERM processname List some of the more common signals: $ man kill Remember that some signals are mapped to keys: INT .control-z QUIT . then Directory Services. Unix signals and how to handle them in Bash shell scripts Send a signal to a running process using 'ps' and 'kill' with a simple bash function.control-\ List mappings with stty: $ stty -e Change them (map interrupt to control-x instead of control-c): $ stty intr ^x Catch Signals A signal is an interrupt sent to a running Unix process.. then NetInfo.txt Page 150 of 271 Printed: Tuesday." # use sleep to represent a lengthy block of code sleep 1 done If the script receives an INT or TERM signal via 'kill'. May 27. or an INT signal via control-c.. do echo "Hello. Function: $ killsig () { kill -$1 $(ps x | grep $2 | grep -v grep | awk '{print $1}')..

. When the function completes the scrip will continue to execute from where it left off...txt Page 151 of 271 Printed: Tuesday.. Apache will respond to a HUP signal in this way (although one would normally use 'apachectl'). ... $ ....." # use sleep to represent a lengthy block of code sleep 1 done If this script is sent the HUP signal...../trap2 Hello...... it will be caught by 'trap' and function 'handlehup' will be executed... Hello....... } trap "handlehup" HUP while true... ^XOn no you dont Hello. Got a HUP Will now reload my config Hello.Unix_tricks.. May 27.... In another window (using the killsig function from Monday): $ killsig INT trap1 $ killsig TERM trap1 $ killsig KILL trap1 Syntax: trap "command to execute" SIGNAL-LIST Trap Functions Use a function if you need a more complex trap handler: $ cat trap2 #!/bin/bash handlehup () { echo "Got a HUP" echo "Will now reload my config" #...... not that way either Hello....... 2008 3:20:00 PM Run trap1: $ .... do echo "Hello..../trap1 Hello. ^C $ In another window (using the killsig function from Monday): $ killsig HUP trap2 The HUP signal (hangup) is traditionally used to tell a process to reload its configuration and restart.....

Unix_tricks. as Thursday....allow interrupts trap . 2008 3:20:00 PM Trap Over a Section of Code Limit your trap functions to just the critical part of a script. INT and TERM signals around the critical code.SIGNAL-LIST removes a handler from the listed signals and replaces it with the default (if any)./trap3 Critical code Got an INT ^CGot an INT Normal code Hangup In another window (using the killsig function from Monday): $ killsig HUP trap3 (wait a bit until normal code is entered) $ killsig HUP trap3 Trap Over a Section with Sub-Shells Limit your trap functions to just the critical part of a script.. sleep 1 # normal code . $ cat trap3 #!/bin/bash handlesig () { echo "Got an INT" #.stop interrupts trap "handlesig" HUP INT TERM echo "Critical code" # use sleep to represent a lengthy block of code sleep 1. sleep 1. sleep 1 sleep 1. sleep 1. then drops the handler around the normal code. $ . sleep 1.txt Page 152 of 271 Printed: Tuesday. sleep 1.. sleep 1. May 27. sleep 1. often around a number of commands that should not be interrupted mid-way. but use sub-shells.HUP INT TERM echo "Normal code" sleep 1000 The statement: trap . sleep 1. } # critical code . . This script traps HUP. sleep 1.

sleep 1. or start their own window (for example xterm).allow interrupts echo "Normal code" sleep 1000 $ . sleep 1.16 xterm . May 27. sleep 1. $ cat trap4 #!/bin/bash # critical code . [1] is the job number 707 is the PID. This script traps INT (control-c) signals around the critical code. Start a new xterm window with: $ xterm & [1] 707 and once xterm is launched you will regain control of the the original terminal. but the tcsh shell also supports background jobs. sleep 1 sleep 1. sleep 1. sleep 1. sleep 1 ) # normal code . sleep 1.Unix_tricks. The '&' symbol tells bash to run the command in the background. sleep 1./trap4 Critical code ^CCaught by subshell ^CCaught by subshell ^CCaught by subshell Normal code ^C $ List all trap signals with: $ trap -l And find out more aboutth ebash built-in 'trap' with: $ help trap Background Jobs Use background jobs for processes that either take a long time to complete and do not require input/output from the Terminal. These tips apply to the bash shell. 2008 3:20:00 PM A handler assigned to a sub-shell is not inherited by the parent shell.txt Page 153 of 271 Printed: Tuesday. sleep 1. as viewed by 'ps' $ ps x | grep "[x]term" 707 std S 0:00.stop interrupts ( trap 'echo "Caught by subshell"' INT echo "Critical code" sleep 1.

$ xterm (oops! I forgot the &) ^Z [7]+ Stopped xterm $ bg [7]+ xterm & $ (control returned to the current terminal) Swapping Jobs Use 'fg' and 'bg' to manage background jobs.Unix_tricks. $ cat in Adrian $ . Output will be interspersed with output from the current foreground job.. 'fg' with no parameters operates on the most recently . Jobs That Require IO Generally. $ fg %n brings job number n into the foreground.txt Page 154 of 271 Printed: Tuesday. . Use file redirection if a background job produces output. scripts that require input/output are not run in the background. Typing 'jobs -l' on another terminal will not show the job with PID 707. The job will then continue running as if you had launched it in the background using '&'. Background a Running Job Re-gain terminal control from a job you should have backgrounded on launch. then type 'bg' to place it in the background. and input won't work. May 27./script <in &>out & [7] 809 $ When the job completes the terminal will display: [7] Done ../script <in >&out And to prove it worked: $ cat out You claim to be Adrian. Hit control-z to suspend the job. Note that each instance of bash (each terminal window) launches and controls its own list of jobs. or requires predictable input: $ cat script #!/bin/bash read -p "Name: " name echo You claim to be $name. 2008 3:20:00 PM Use: $ jobs -l [1]+ 707 Running xterm & to display a list of jobs and their process numbers and PIDs.

those jobs become orphaned.Unix_tricks. $ bg will send the job back into the background if it needs to run more.. 2008 3:20:00 PM backgrounded job.txt Page 155 of 271 Printed: Tuesday. $ wait %n waits only for job number n. . without the preceding '%'.. $ jobs -l [1]. May 27. You can also specify the PID number to 'fg' by omitting the %. 893 Running xterm & [2]+ 905 Stopped (tty input) . Supply terminal input to a background job (this example uses the script from Tuesday's tip): $ . They can only be stopped with: $ kill -KILL <PID> Prevent a script from creating orphaned jobs using 'wait'. [ vs Test Use: $ man test or: $ man [ to learn about the conditions one can use in Bash and Bourne shell scripts for 'if'. 'while'. You may specify a PID too./script Now supply the input: Adrian You claim to be Adrian. 'bg' can take %job-number or PID as for 'fg'./script . $ wait will prevent the shell (and therefore the script) from exiting until all background jobs under its control have completed... Without any parameters it assumes the most recently foregrounded job. Bring the job into the foreground: $ fg %2 .. Use 'jobs -l' to find out requires input the the terminal '(tty input'). and 'until' statements. the Terminal indicates the job 2 has stopped../script & [2] 905 Name: $ [2]+ Stopped . Waiting. If a shell (or script) exists while background jobs that it controls are still running./script Here.

b=2. then echo "Files are the same" else echo "Files are different" fi Files are the same The return value of a command is held in shell variable $? $ diff f1. and numbers. $ if [ 3 -lt 7 ].. Bash will execute the 'condition' and test the return value. use Bash's AND and OR operators: $ if [ $a -lt $b ] && [ $b -lt $c ]. Why is '.c=3 $ if [ $a -lt $b -a $b -lt $c ]. then echo "a < b < c" else echo "not so" fi a < b < c The test statement uses operators -a for AND and -o for OR.txt f2. Alternatively.' required before 'then'? Conditions and Statements Make more use of Bash's conditions .Unix_tricks. 0 (success) = true not 0 (failure) = false This ties in with Monday's explanation of the 'test' and '[' commands. which can test files. $ a=1.txt. then echo "a < b < c" else echo "not so" .understand that a condition is actually a command. then .. The command is also called '[' which explains the form of a Bash condition.txt $ echo $? 0 AND and AND and OR or OR Use AND and OR constructs for more complex conditions.txt f2. For example: $ if diff f1. 2008 3:20:00 PM Conditions in Bash are evaluated by the command 'test'.txt Page 156 of 271 Printed: Tuesday. strings. then echo "3 is less than 7" else echo 'very odd!' fi 3 is less than 7 An equivalent form is: $ if test 3 -lt 7. May 27.

The expression is enclosed in (( . then echo "a < b < c" fi a < b < c Alternatively: $ if (($a < $b)) && (($b < $c)). )) rather than [ .b=2..Unix_tricks.) $ a=3.. then echo "a < b < c" fi a < b < c Understand the difference between the two forms. 2008 3:20:00 PM fi a < b < c The difference between the two: In the first example the 'test' command '[ . ]. (Though in the simple example below bracketing is not strictly necessary. In the second example 'test' was called twice. Sometimes expressions need to be bracketed to force the correct order of evaluation. $ if (( ($a < $b) && ($b < $c) )). More Complex Tests Understand how to form complex expressions.c=1 $ if [ ( $a -lt $b ) -a ( $b -lt $c ) ].... Bash's Built-in Conditions Use Bash's built-in tests for integer arithmetic (and only integer arithmetic). then echo "a < b < c " else echo "not so " fi a < b < c . Try: $ if [ '( $a -lt $b ) -a ( $b -lt $c )' ]. and bash evaluated 'first test result' AND 'second test result'.. May 27.txt Page 157 of 271 Printed: Tuesday. then echo "a < b < c " else echo "not so " fi -bash: syntax error near unexpected token `$a' Syntax error. we must escape the bracketed expression. ]' evaluated the complete expression. The use of '<' instead of '-lt' makes tests more readable.

Unix_tricks. Figure out why we get the erroneous result in the second form. then echo "a < b < c " else echo "not so " fi not so Much better..) Bash's preferred syntax for command substitution $((. May 27... Here's a handy reference. they are replaced or interpreted.... command separator {. # introduce a comment .) forces execution in a sub-shell && logical and (placed between commands) || logical or (placed between commands) ~ expanded to current user's home directory ~user expanded to user's home directory / directory/filename separator $var expands variable 'var' `... If any of these characters appear naked (unquoted or unescaped) on the command line.` executes a command and substitutes the output $(. Bash Special Characters Know your special characters.. Bash has many characters which it treats specially.)) evaluates an integer arithmetic expression ((....} signifies a command block (..] character set wildcard ? single character wildcard & forces command to be executed in the background < redirect stdin > redirect stdout | pipe ! pipeline logical NOT .txt Page 158 of 271 Printed: Tuesday.)) evaluates an integer arithmetic in a condition ' strong quote " weak quote \ escape next character (cancel special meaning) * wildcard [.. 2008 3:20:00 PM 3 < 2 < 1 ! You jest. Now try: $ if [ \( $a -lt $b \) -a \( $b -lt $c \) ].

as in the awk example below: $ echo "Adrian's" | awk '{print "You cant have " $1 " $1"}' You cant have Adrian's $1 If I wanted to output "can't" instead of "cant"... 3) Double Quotes (weak quoting) $ echo "$a * $b > $c" 5 * 4 > 3 Everything but $ and \ within ". May 27. Use: $ echo "Adrian's" | awk '{print "You can'\''t have " $1 " $1"}' You can't have Adrian's $1 This splits the awk parameter into three adjacent strings: '{print "You can' \' 't have " $1 " $1"}' ...txt Page 159 of 271 Printed: Tuesday.c=3 $ echo \$1 \* \$b > \$c $1 * $b > $c None of $.. a first attempt might be: $ echo "Adrian's" | awk '{print "You can\'t have " $1 " $1"}' > (hit control-c) It is not possible to escape the ' in can't because it lies within a '. or > were interpreted specially. 2) Single quotes (strong quoting) $ echo '$a * $b > $c' $a * $b > $c Everything with in '." is treated literally: this allows you to escape special characters while still expanding variables.b=4..'ed string.Unix_tricks.? Quoting and Escaping Prevent bash from interpreting special characters (see Monday's tip).. *.' is treated literally: the shell does not interpret any special characters. If you wish to mix literal $ and variable expansion $ use either backslash to escape the $: $ echo "\$1 * $b > \$$c" $1 * 4 > $3 Quoting Tricks 1) Quoting Parameters Sometimes it is necessary to use single quotes around parameters. There are three methods: 1) Backslash escaping $ a=5.. 2008 3:20:00 PM Any more.

2008 3:20:00 PM 2) Remember that single quotes are not special within double quotes. The first pass $file is expanded. and has no meaning to the file system. The shell expands '~' before it expands $variables. You might think it is expanded in the expression: ls $file but this is not so. This does not work: $ file="~/Desktop/" $ ls $file /usr/local/bin/ls: ~/Desktop/: No such file or directory $ ls ~/Desktop/ A File Tilde (~) is a shell special character. so file contains the full pathname instead of '~/Desktop' If you need to quote the assignment to variable file. Thursday for examples of 'eval'. . and therefore never sees the '~' until too late. (See Week 78.Unix_tricks.) 2) Don't quote $ file=~/Desktop $ echo $file /Users/saruman/Desktop $ ls $file A File The first line is not quoted. and then tilde is expanded by 'eval'. May 27.txt Page 160 of 271 Printed: Tuesday. Placing it in quotes prevents the shell from expanding it. so you can also use: $ echo "You can't have my \$" You can't have my $ or even: $ echo "You can't have my $" You can't have my $ because $ with nothing following is take as a literal $ Shell Evaluation Understand the shell's evaluation order. use $HOME. $ file="$HOME/Desktop" $ ls $file A File $HOME is expanded on the assignment whereas ~ is not. Ways around this: 1) Use eval $ eval ls $file A File eval gives the shell a second go at expansion.

Unix_tricks. Both must be escaped themselves .mayo-family./script A File Mount AFP Shares Mount an AFP share using the Terminal. The simplest usage is: .we must write escape-escape escape-space: $ echo ls A\\\ Dir > script $ .. An AFP share can be mounted anywhere on the filesystem .com/myuser' is the hostname or IP address of the server followed by the AFP share point as seen in the Finder's 'Connect. It mounts your home directory from the other machine. $ ls A\ Dir A File $ echo ls A\ Dir > script $ . I use: $ cd ~ $ mkdir melkor $ mount -t afp afp://myuser:mypassword@melkor. AFP Mounting Script This script is useful if you have accounts on several machines (desktop and laptop).for example in /Volumes or at the root of your home directory..txt Page 161 of 271 Printed: Tuesday. May 27. The 'mount point' must be an existing directory. 2008 3:20:00 PM Multi-level Quoting Count the number of times the shell will evaluate an expression.' dialogue.mayo-family. to mount my home directory from another machine (melkor) onto the mount point 'melkor' in my home directory on this machine. $ ls Desktop Documents Movies Pictures Sites melkor Development Library Music Public bin osxfaq To unmount use: $ umount melkor and the directory will be deleted. For example./script ls: A: No such file or directory ls: Dir: No such file or directory The script does not work because the shell processed the backslash escaped space when executing echo. The file does not therefore contain the escaping: $ cat script ls A Dir One must escape the escaping by writing escape-space to the melkor mount_afp: the mount flags are 0000 the altflags are 0020 '@melkor. This is a simple example which illustrates the principle.

# # function: usage ([error-message]) # print out the usage lines and exit # takes an options error meeeage to print at the # end of the usage line usage () { echo "mount ${USER}'s home directory from server $ALM_SERVER" echo "" echo "Usage: ${0##*/} [-p password] [-h host] [-u user]" echo " [-p password] .override yourserver as the server to mount [-u user] . 2008 3:20:00 PM $ mount-user $ mount-user -usage mount youruser's home directory from the server yourserver Usage: mount-user [-p password] [-h host] [-u user] [-p password] . May 27.use an AFP password if required [-h host] . but you may want to change it to avoid confusion with Finder-mounted shares (eg to /nfs). fi exit } # function: exit_if_set (current-value.use an AFP password if required" echo " [-h host] . The pathname in $ALM_AFP_MOUNT must exist. The mountpoint will be $ALM_AFP_MOUNT/hostname/username. Your user name is the same on both machines. if not override with option '-u otherusername'. /Volumes is the usual pathname. parameter-description) # called to prevent a parameter being given a # value more than once exit_if_set () { if [ "$1" != "" ]. then usage "$2 already has the value $1" else .txt Page 162 of 271 Printed: Tuesday.Unix_tricks.. where $ALM_AFP_MOUNT is an environment variable.. then echo "Error: $*. #!/bin/bash # mounts the current user's home directory on the server # $ALM_SERVER into the mount $ALM_AFP_MOUNT/.".override youruser as the user to mount It assumes the following. The script will create all the directories as necessary.override $USER as username to mount" echo "" if [ "$*" != "" ].override $ALM_SERVER as server to mount" echo " [-u user] . if not override with option '-h hostname/ip-address'. The server name is defined in the environment variable $ALM_SERVER.

esac # a value has been found for -<char>. password="$1". user="$1".. May 27. "-p" | "-h" | "-u") if [ "$opt" = "" ].. set the value "-p") exit_if_set "$password" "password".. fi # main processing # # if password is given add ":" to the start to comply # with mount syntax . "") usage "invalid value $1". fi if [ "$user" = "" ]. then opt="$1" else usage "no value given for $opt" fi . remember option for next iteration # error if previous iteration was also an option that required a value "-usage") usage. # invalid option *) usage "invalid option $1". "-h") exit_if_set "$host" "host". then # found a -<char> option introducer case "$1" in # these options require a value.txt Page 163 of 271 Printed: Tuesday... then usage "no value given for $opt" fi # set default values for optional parameters if [ "$host" = "" ]. # thus no value has been given if [ "$opt" != "" ]. so reset this flag opt="" fi shift done # this case is where the last option was -<char>.. then host="$ALM_SERVER".. "-u") exit_if_set "$user" "user". 2008 3:20:00 PM return fi } # process each parameter by looping and shifting # $2 -> $1 each iteration # opt="" while [ "$1" != "" ] do if [ "${1:0:1}" = "-" ].Unix_tricks. esac else case "$opt" in # previous iteration was option that required a value. then user="$USER". host="$1".

NFS Manager writes Shares and Connections (Mounts) to NetInfo. drwxr-xr-x 114 root wheel 3876 Aug 15 22:39 . then echo "Error: failed to mount. $ ls -al /etc/lookupd/ total 4 drwxr-xr-x 3 root wheel 102 Apr 18 19:23 .txt Page 164 of 271 Printed: Tuesday. then if [ ! -d $ALM_AFP_MOUNT/$host/$user ]. Create the directory /etc/lookupd and the file mounts as below. then exit 1 else exit 0 fi Enable NFS The easiest way to enable NFS on OS X is to use NFS Manager.Unix_tricks. then rm -f $ALM_AFP_MOUNT/$host/$user 2> /dev/null rm -f $ALM_AFP_MOUNT/$host 2> /dev/null mkdir -p $ALM_AFP_MOUNT/$host/$user fi echo "Mounting afp://$user$password@$host/$user on $ALM_AFP_MOUNT/$host/$user. 1 root wheel 28 Aug 19 13:19 mounts $ cat /etc/lookupd/mounts LookupOrder Cache FF NI . ensure the mount point (directory) is available # delete just in case either exist as files. -rw-r--r-. 2008 3:20:00 PM if [ "$password" != "" ]. This is briefly described in the OSX Tips." mount_afp afp://$user$password@$host/$user $ALM_AFP_MOUNT/$host/$user > /dev/null if [ "$?" != "0" ]. 1 otherwise if [ ! "$mounted" ]. then password=":"$password fi # check if already mounted mounted=$(df | grep "$ALM_AFP_MOUNT/$host/$user") # if not mounted. By default it will use NetInfo (Panther and Jaguar)." exit 255 fi disktool -r fi # return 0 if no mount was performed. and hide # any errors if [ ! "$mounted" ].. force the automount daemon to read the flat files. This tip shows how to enable NFS to work from the traditional Unix flat files /etc/fstab defines which shares to automount (it is necessary on the client machine) /etc/exports defines which directories to share (it is necessary on the server) First. May 27.

unless you know exactly what you are doing '-network. . It is ESSENTIAL that this range includes only trusted machines (see the note below).in /etc/hostconfig: $ grep RPC /etc/hostconfig RPCSERVER=-YES- Next. They are not used. a user's account on the client machine will have the same user id and group id as their account on the server (and all other clients). # NFS mounts # carcharoth:/Users /nfs/Carcharoth nfs -b. I have exported /Users/Shared as read only (-ro). It is assumed that these are consistent across the server and all clients. The last two ( 0 0 ) describe 'dump frequency' and the order for file system checking.255.-i -P are described in 'man mount_nfs'. It is exported Read Only.. That is.. but the actual mountpoint 'Carcharoth' will be created automatically. See 'man exports' for more details. May 27.txt Page 165 of 271 Printed: Tuesday. 'nfs' says that the share is nfs The options -b.0.168.. To define mount points on the client: $ cat /etc/fstab # local mounts # . '-alldirs' allows any subdirectory of /Users to be mounted by a client '-maproot=nobody' ensures that root user on the client does not have root access on the server.Unix_tricks. ESSENTIAL. Note: NFS does not use passwords for authentication.. 2008 3:20:00 PM On Jaguar (but not Panther) it is necessary the force the RPC daemon to start by ensuring the RPCSERVER is set to -YES. and all users are mapped to the unprivileged user 'nobody'.0 -mask=255. reboot the machine and set up the shares (server) and mounts (client) Set Up NFS Shares and Mounts To define share points on the server edit /etc/exports. This is accessible to ALL machines (no IP restrictions).-P 0 0 This line looks for the share /Users on the host carcharoth (you may use either a hostname or IP address) and mounts on the directory /nfs/Carcharoth. This is generally true on a proper server-client network where user account information is held centrally on the server.' and '-mask.. $ cat /etc/exports /Users -alldirs -maproot=nobody -network=192. '/nfs' must exist..' define the IP range that is allowed to mount the share.255.-i.0 /Users/Shared -ro -mapall=nobody In this example directory /Users is exported. but Unix User and Group IDs.

txt Page 166 of 271 Printed: Tuesday. . See weeks 59 and 60 for information on regular expressions. and part 6 of the Learning Centre. Use the same technique with an appropriate regular expression. and worse. Use: $ sed '/^$/d' double-space This is a file with the lines double-spaced and some extra blank lines too. a handy utility to execute a script of editing commands against its input. This example deletes all lines starting with '$'. Add this line in to your bash startup file (/etc/profile. ~/. To write the results back to the original file use this trick: $ sed '/^$/d' double-space > tmp. Hide Commands that Include Passwords Typing a password on a command line can pose a security risk as the password will be saved in Bash's command line history. $ cat double-space This is a file with the lines double-spaced and some extra blank lines too. sed works by matching each line of input against a delimited regular expression (^$ in this example). Delete blank lines from a file.Unix_tricks. $ sed '/^\$/d' text Search for Lines Use sed to search and print matching lines. /etc/bashrc.bashrc): declare -x HISTCONTROL="ignorespace" A command line starting with space will not be added to the history. May 27. Editing command 'p' prints each matched line. If the line matches then sed applies the editing commands that follow. grep-style. 'd' deletes the line. mv tmp double-space Delete lines containing specific text. Option '-n' suppresses this. sed normally echoes each input line to its output. 2008 3:20:00 PM If a client machine connects to your server. it need only have user with the same user id and group id as a user on the server to gain full access to that user's files on the server. written to the history file. Thus start any command that contains a password with space as a security precaution.bash_profile. Delete Lines This week's tips explore sed. or ~/. Combining the two tells sed to print just matched lines.

some of which contain the word which.s/that/twit/g' this twit is twit and twit is twit . and some that do not. some of witch contain the word witch. sed can be made to operate on a range of lines. 2008 3:20:00 PM $ cat text This file has a number of lines. $ sed '/begin-marker/. you may wish to edit the file contents and display just the edited lines: $ sed -n 's/witch/which/p' text of lines. and some Beyond grep.Unix_tricks./end-marker/d' filename Or print them: $ sed -n '/begin-marker/./end-marker/s/witch/which/g' filename Multiple Edits Apply more than one editing command to a matched line.txt Page 167 of 271 Printed: Tuesday. Option -e: $ cat this this is this and that is that $ sed -e 's/this/that/g' -e 's/that/twit/g' this twit is twit and twit is twit Separate multiple commands with '. $ sed -n '/witch/p' text of lines./end-marker/s/witch/which/' filename If you expect the pattern 'witch' to occur more than once on a line use option 'g' (global) to replace all occurrences on the line. $ sed '/begin-marker/./end-marker/p' filename Or edit them: $ sed '/begin-marker/.': $ sed 's/this/that/g. May 27. Delete all lines between begin-marker and end-marker inclusive. or a single line. and some Line Ranges Edit a range of lines using sed. Specify multiple commands to sed using one of two techniques. some of witch contain the word witch. Previous examples have used a regular expression to match lines to be edited.

'print($0)' prints the entire line. a handy utility that executes a script of editing commands against its input. Fields are separated by white space. mv tmp double-space Awk works by matching each line of input against a pattern (in these examples. $0 represents the entire input line. Use option '-f' to specify a scrip file. $n represents field n. $ cat this this is this and that is that $ cat script s/this/that/g s/that/twit/g $ sed -f script this twit is twit and twit is twit Print/Delete Lines This week's tips explore Awk. Print just the pid number (field 1): $ ps ax | grep "[p]ostfix" | awk '{print($1)}' . A common application for this is to print specific information from the output of Unix commands such as ls and ps. $ awk '/. Write the script in a normal text file putting each command on a new line. If no pattern is given then every line is matched. Print Fields Print specific fields of each matched input line.}.+/{print($0)}' double-space To write the results back to the original file use this trick: $ awk '/.+' matching a line containing at least one character).. $ awk '/^\$/{print($0)}' text Delete blank lines from a file.+/{print($0)}' double-space > tmp. as does a simple 'print'. This example prints all lines starting with '$'. Monday's tip used the action {print($0)}. May 27.txt Page 168 of 271 Printed: Tuesday.The Sed Stream Editor: sed has similar capabilities. 2008 3:20:00 PM Script Files Use a script file if the sed script becomes long or complex.. and '. regular expression '^\$' matching a line starting with '$'.Unix_tricks. The above example can be simplified to: $ awk '/^\$/' text Note the difference between printing and deleting is a reversed regular expression. If the input line matches then Awk applies the actions that follow in {. If no action is given then the input line is printed. Print lines containing specific text. Also take a look at Week 85 .

File %s\n". or awk. grep. some of which contain the word witch.$9)}' Sep 13 csv Sep 13 double-space Sep 13 many Aug 30 script Aug 30 text Aug 30 this Sep 13 xxx Use printf to format the output: $ ls -l | awk '{printf("Date: %s %s. . File csv Date: 13 Sep. and some which end-marker Or simply: $ awk '/begin-marker/. File double-space Date: 30 Aug. Previous examples have used a regular expression to match lines. File' results from the first line written by ls -l.$6. File text Date: 30 Aug. This can easily be removed by sed. some of which contain the word witch. Awk can be made to operate on a range of lines. Print all lines between begin-marker and end-marker inclusive./end-marker/{print $0}' text begin-marker of lines. date. File Date: 13 Sep. see: $ man 3 printf Line Ranges Process a range of lines using Awk. and filename from a long listing: $ ls -l | awk '{print($6. File this Date: 13 Sep. File xxx The first line 'Date: ./end-marker/' text Multiple Commands Execute many commands against each input line.$9)}' Date: . $ awk '/begin-marker/. or a single line.Unix_tricks. May 27. To learn about printf.$7. 2008 3:20:00 PM 437 Print the month. $ cat text This file has a number begin-marker of lines. File script Date: 30 Aug.$7.txt Page 169 of 271 Printed: Tuesday. and some which end-marker do not.

/^line/{printf("LINE: %s\n".' and all commands will be executed against each matched line. Note it is different to the first example where two commands were applied to the same pattern match: /pattern/{command1.txt Page 170 of 271 Printed: Tuesday./end-marker/ {next}. and the line itself. command2} Here we are applying the first command to the first pattern. print}' text Multiple Patterns Specify many patterns to match against each line: $ cat many line one the second line and the third the fourth line five the six and the last line $ awk '/^the//^line/' many line one the second line the fourth line five the six Apply a different command to each pattern: $ awk '/^the/{printf("THE: %s\n". which skips the line./end-marker/ {next.. /pattern1/{command1}. To print the length of each line. but illustrates the use of multiple commands. Matched lines execute 'next'. /pattern2/{command2} This does not work: $ awk '/begin-marker/. print}' text 22 This file has a number 12 begin-marker . 2008 3:20:00 PM Separate commands with '. and the second command to the second pattern (which is empty to match all lines).Unix_tricks. Extending Wednesday's tip. use: $ awk '{print(length($0)).$0)}' many MATCHED: line one .$0)}. May 27.$0)}' many LINE: line one THE: the second line THE: the fourth LINE: line five THE: the six Apply a command to lines that match one pattern or the other using the 'or' (||) operator: $ awk '/^the/||/^line/{printf("MATCHED: %s\n". we can print lines that fall outside a specific range: $ awk '/begin-marker/. This could be done with a single print statement. {print}' text This file has a number that do not.. so 'print' is executed. Unmatched lines do not.

" '{print $2.. 2008 3:20:00 PM MATCHED: the second line MATCHED: the fourth MATCHED: line five MATCHED: the six Apply a command to lines that match both patterns using the 'and' (&&) operator: $ awk '/the/&&/line/{printf("MATCHED: %s\n".goddess. Week 85 showed how to use 'sed' to edit files. $3. 5=01 1=adrian.txt Page 171 of 271 Printed: Tuesday.editor-02 jan:forbes. $2.$1. The field separator is actually a regular expression and can take a list of separators. $ awk 'BEGIN {FS=".mr. 2=%s.02 jan.editor in chief-01 adrian:mayo.forbes. $5}' csv2 1=scott. Awk can also edit files. 5=02 1=jan.|:|-"} {printf "1=%s. but not to edit.69 Awk will normally use white space as a field separator. 2=forbes. 2=mayo. 5=69 Editing With Awk Examples so far (including week 86) have used Awk to filter.} following BEGIN are executed before the Awk script 3=mr.sheppard.$0)}' many MATCHED: the second line MATCHED: and the last line Reading CSV Files Change the default field separator used by Awk."} {print $2. 4=editor in chief. $ 5=%s\n".editor in chief. 3=%s.' using BEGIN. Comma Separated Value (CSV) files are a popular method of exporting tables of data (spread sheets can export in CSV format). 3=miss.$4}' csv sheppard editor in chief mayo editor forbes goddess Alternatively. Search and replace with Awk. $ cat text This file is one witch has a number begin-marker of lines.01 adrian.editor.goddess-69 $ awk 'BEGIN {FS=". 3=mr. we can set the Awk variable 'FS' to be '..$4}' csv sheppard editor in chief mayo editor forbes goddess Commands in {.mayo. $ cat csv2 scott:sheppard. 4=goddess. May 27. 2=sheppard.miss. 4=editor. To change this use the -F option: awk -F ". 4=%s. or regular expression wild cards. For example: $ cat csv some of witch contain .

Awk uses variables like any other language. $ awk 'BEGIN {n=0} {n=n+1} END {printf ("Total lines in file %d\n". on its third parameter. t. print $0}' text begin-marker of lines. 'sub' is a function that substitutes its pattern (witch) for the value of a variable (t).. Use option '-f' to tell Awk to read its script from a file: $ awk -f awk-script text . $0). n. t. some of witch contain the word witch. Repeating Wednesday's tip using a script file: $ cat awk-script BEGIN {n=0} /^the/{printf("Line %d: %s\n". and some of witch end-marker This example applies the script {sub('/witch/'. This example uses simple arithmetic to count the number of lines in a file: $ cat text This file is one witch has a number begin-marker of lines. To substitute all patterns on a line use 'gsub'. $2)} {n=n+1} END {printf ("Total lines in file %d\n".} executes its statements just once at the end of the script. and the value of field two: $ awk 'BEGIN {n=0} /^the/{printf("Line %d: %s\n". $0). $2)} {n=n+1} END {printf ("Total lines in file %d\n".txt Page 172 of 271 Printed: Tuesday. Notice that only the first occurrence on each line of pattern 'witch' is substituted. (Global SUBstitute) Line Counting Count lines. n)} Each statement is placed on a new line in the script file. and more. In this case $0 represents the whole input line. print $0} to each line within the markers. $ awk -v t=which '/begin-marker/. n. {n=n+1} is executed unconditionally for each line. but field n could also be specified using $n. and some of witch end-marker do not. Variable t is set to 'which' using the command line option -v.. some of which contain the word which./end-marker/ {sub('/witch/'. n)}' text Line 3: word Total lines in file 6 Use a Script File Complex Awk scripts can be placed in a file. Here is another example which also prints the number of each line beginning 'the'. and some of witch end-marker do not. END {. n)}' text Total lines in file 6 {n=0} is executed once at the start of the script by the BEGIN statement.Unix_tricks. 2008 3:20:00 PM the word witch. May 27.

max. In triplicate: $ cat file line 1 line 2 $ awk '{for (i = 0. and 'if'. with c-style statements such as 'for'. use df to find the device node: $ df Filesystem Size Used Avail Use% Mounted on /dev/disk0s3 12G 6. name). May 27. i++) print $0}' file line 1 line 1 line 1 line 2 line 2 line 2 This is an example using 'if' statements: $ cat posts Mayo 34 posts Forbes 35 posts Sheppard 12 posts Trevor 345678 posts Hollis 17 posts $ cat awk-script BEGIN { print "More than 34 posts".3G 10% /Games /dev/disk0s9 5.5G 5. 'while'. For example.8G 1% /Music (or 'diskutil list') . and diskutil may have to run as root.9G 53% / /dev/disk0s5 12G 3.9G 578M 5. name = $1}} END { printf ("Max posts %d by %s\n".1G 27% /Users /dev/disk0s7 5. i < 3. print "---\n"} $ awk -f awk-script posts More than 34 posts Forbes 35 posts Trevor 345678 posts Max posts 345678 by Trevor --- Repair Discs Use diskutil to verify and repair disks. name = ""} {if ($2 > 34) print $0} {if ($2 > max) {max = $2. max = 0. To repair. the disk must not be in use.txt Page 173 of 271 Printed: Tuesday.Unix_tricks.4G 9.9G 21M 5. 2008 3:20:00 PM Line 3: word Total lines in file 6 More Power Awk is a fully-fledged programming language.

compressing and deleting log files. Checking Extents Overflow file. 'Verify and Repair Permissions'. it may be due to a corrupted preference file.plist preference files in ~/Library/preferences. . $ plutil -lint ~/Library/Preferences/com. 2008 3:20:00 PM and then: $ diskutil verifyDisk /dev/disk0s7 Started verify/repair on disk disk0s7 Games-saruman Checking HFS Plus volume. Checking volume as for Monday's tip. so I don't have OS 9 installed :-) Verify Preferences Verify all those . . Verify/repair finished on disk disk0s7 Games-saruman Or use the mount point (usually /Volumes/name): $ diskutil verifyDisk /Games Use 'diskutil repairDisk' to repair.Unix_tricks. Checking Catalog hierarchy.plist /Users/saruman/Library/Preferences/com.txt Page 174 of 271 Printed: To check all preferences: $ find ~/Library/Preferences -name "*. If you encounter odd problems with system functions or applications not behaving correctly. the diskutil command must be run as root.plist" -print0 | xargs -n1 -0 plutil -lint (This uses -print0 and -0 to cope with spaces in filenames. not the deeper meaning as understood by the owning application.. Repair Permissions Use diskutil to verify and repair file permissions on the system disc.) Periodic Housekeeping OS X tidies the file system by removing old temporary files. Use df to find the device node. May 27. Checking Catalog file. Checking volume information. Use command 'plutil' to verify the preference file. You can only specify an OS X boot volume.plist: OK This will only check the syntax of the file. or specify the mount point. The volume Games-saruman appears to be OK.iChat. If an application is playing up. To repair the current classic boot volume use: $ diskutil repairOS9Permissions You cannot repair OS 9 permissions on this machine (No Classic folders found) OK. and to repair. and then: $ diskutil verifyPermissions /dev/disk0s3 Started verify/repair permissions on disk disk0s3 OSX-saruman Determining correct file permissions..

out /var/log/monthly.out /var/log/daily.daily..out Periodic looks in /etc/periodic/parameter-given-periodic and executes all scripts in the directory. 2008 3:20:00 PM removing old system messages. % cat /etc/crontab .. updating the 'locate' database. to report all fs activity to a log file: $ fs_usage -f filesys > log To monitor just fs activity of the posfix master daemon: $ fs_usage -w -f filesys master ... and more.out /var/log/weekly.. edit /etc/crontab and choose times at which your Mac will be running: If you wish to add your own maintenance tasks .txt Page 175 of 271 Printed: Tuesday.local These are regular shell scripts. Use fs_usage to report system calls in real-time..Unix_tricks. Run the maintenance scripts manually: % sudo periodic daily % sudo periodic weekly % sudo periodic monthly Alternatively.local /etc/monthly. The output from these scripts can be found in: % ls /var/log/*. weekly. and once created are run automatically by the default daily. The daily script is run at 3:15 am. May 27. Periodic system maintenance is scheduled by 'cron' launching a program called 'periodic'.local /etc/weekly. /etc/daily. The weekly script is run at 4:30 am Saturday.then create the following files as necessary. #minute hour mday month wday who command # Run daily/weekly/monthly jobs. weekly. For example. 15 3 * * * root periodic daily 30 4 * * 6 root periodic weekly 30 5 1 * * root periodic monthly . These files will not be overwritten by system updates which might otherwise affect the files in /etc/periodic. The monthly script is run at 5:30 am on the first of each month. backing up the NetInfo database. File System Monitoring Commands Use commands to monitor what is happening in the file system. or monthly . and monthly scripts.

For technical details (ref:5488) see "How it Works".4\})' which is a regular expression looking for ' (ref:' and then 1 to 4 digits. May 27.. It then replays the patterns in a different order. (ref:5488) The pattern to move is matched with: ' (ref:[0-9]\{1. sc_usage. and the second segment (the matched ref). 2008 3:20:00 PM fs_usage must be run as root. $ cat refs For news (ref:1) see "All the News". Use capture patterns to record and replay matched patterns.4\})\)\(. to monitor files opened by user saruman: $ fstat -u saruman To monitor files as they are opened.*\)\( (ref:[0-9]\{1. Move '(ref:nnnn)' from the middle of the line to the end. (ref:223) For technical details see "How it Works". For example. Use fstat to report on open files. HTML to PHP-BB Code Use pattern capture to convert HTML code to PHP-BB code.Unix_tricks. surrounding the segment to capture with \( . HTML looks like: . $ cat capture abc (ref:) def To simply delete (ref:) (see week 85) use: $ sed 's/(ref:)//' capture abc def To capture (all or part of) the pattern. A similar command.txt Page 176 of 271 Printed: Tuesday. (ref:1) For help see "Technical Help". and surrounds each part with a capture pattern. use lsof. For help (ref:223) see "Technical Help". sed is told to substitute the entire line with the first segment.*$\)/\1\3\2/' refs For news see "All the News".\).. To replay it use \1: $ sed 's/(\(ref:\))/\1/' capture abc ref: def And to emphasise what is happening: $ sed 's/(\(ref:\))/\1 again \1 and again \1/' capture abc ref: again ref: and again ref: def The next example uses a pattern to match the entire line in three parts. and then ')'. Capture Patterns This week continues the sed tips started in week 85. $ sed 's/\(^. the third segment. monitors all system calls.

For example.Unix_tricks. 'CONF_SITE_DBPASS').*\)>/[\1]/' html [b><i>Hello in bold italic</i></b] This has not worked because the '. comment out a block of C++ or PHP code using '//' style comments. May 27. $this->user = new DB_Forums_Users('CONF_SITE_DBNAME'.) $ sed 's/<\([^<]*\)>/[\1]/' html [b]<i>Hello in bold italic</i><b> Ah yes. We must modify the pattern to explicitly exclude another '<': (See weeks 59 and 60 on regular expressions. 2008 3:20:00 PM <b><i>Hello in bold italic</i></b> PHP-BB looks like: [b][i]Hello in bold italic[/i][/b] Apply a sed command to file 'html': $ cat html <b><i>Hello in bold italic</i></b> $ sed 's/<\(. log them out and remove their session information if ($nickname != '') { if (! $this->user->isRegistered($nickname)) { $this->session->logout(SESS_FORGET).txt Page 177 of 271 Printed: Tuesday. } } yyyyy } Apply the sed command: $ sed '/xxxx/. // get the user's nickname. check they still exist in the user's databse // and if not. surround the code with markers (eg xxxx and yyyy). if any $nickname = $this->session->name()./yyyy/s/^/\/\//' code.php function Manage_User() { // create a new session and connection to the 'users' database $this->session = new Sess_User().*' is 'greedy' in that it matches the longest pattern it can find. and one must tell sed to 'globally' apply the pattern to the whole line. xxxxx // if the user has a nickname. First. not just the first match: $ sed 's/<\([^<]*\)>/[\1]/g' html [b][i]Hello in bold italic[/i][/b] Comment Out Use sed to apply commands to entire blocks.php function Manage_User() . $ cat code.

txt Page 178 of 271 Printed: Tuesday.script case this is upper case this is lower case y translates letters in the first // to those in the second //. $this->user = new DB_Forums_Users('CONF_SITE_DBNAME'. but keep in the xxxx and yyyy lines in but commented out.php (It is also possible. ./xxxx/d.) Case Insensitivity Make sed pattern matches case insensitive. //xxxxx // // if the user has a nickname. log them out and remove their session information // if ($nickname != '') { // if (! $this->user->isRegistered($nickname)) { // $this->session->logout(SESS_FORGET). we can use function lists and sed's holding space . // get the user's nickname. $ cat case THIS IS UPPER CASE this is lower case $ sed -n '/CASE/p' case THIS IS UPPER CASE $ sed -n '/case/p' case this is lower case Use a script like this to match any case: $ cat case1./yyyy/s/^\/\///. 'CONF_SITE_DBPASS'). Normally. if any $nickname = $this->session->name().Unix_tricks. If the original line needs to be kept. May 27. This matches and prints both lines. // } // } //yyyyy } To remove the comments use: $ sed '/xxxx/. but unfortunately mangles the original line. check they still exist in the user's databse // // and if not. 2008 3:20:00 PM { // create a new session and connection to the 'users' database $this->session = new Sess_User(). to come up with a sed command to remove the comments.see Friday's tip. sed pattern matching is case sensitive. thus making the input line all lowercase.script y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/ /case/p $ sed -n -f case1./yyyy/d' code. but not as easy.

else echo "No".txt Page 179 of 271 Printed: Tuesday. else echo "No". prints the line..} on those lines x swaps the holding space and the current line . The shell interprets '>' as a redirection character.. as usual. They can involve files. illustrating where the problem lies and how to correct it. alphabetically speaking: $ if [ "aaa" > "zzz" ]. $ if [ "zzz" < "aaa" ]. and numbers.script h y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/ /case/ { x p } $ sed -n -f case. 2008 3:20:00 PM Function Lists Use more complex operations on a matched line with function lists. else echo "No".. fi No The first version would have created the file "zzz". /case/ {. then echo "Yes". thus making the input line all lowercase. fi Yes So why 'Yes'? "aaa" is not > "zzz".} matches all lines containing 'case' and executes the commands in {. aka '['. The script from Thursday can be improved: $ cat case2. allows one to make all kinds of tests and comparisons.script case THIS IS UPPER CASE this is lower case h copies the current line to the pattern holding space. then echo "Yes". See Week 82 and: $ man test or $ man [ One can compare strings to test which 'comes first'. May 27. y translates letters in the first // to those in the second //. A command called 'test'. and so it must be escaped: $ if [ "aaa" \> "zzz" ].Unix_tricks. then echo "Yes". it would have given a clue as to what was happening. fi .so having matched in a case insensitive manner we replace the mangled line with the original p. Perhaps if we had tried this next example first. Odd Behavior in Shell Conditions This week highlights a few common problems with shell scripts and commands.. strings.

. else echo "No". then echo "Yes". else echo "No". then echo "Yes". then echo "Yes". ]'). $ echo $((2+7)) 9 $ $( ((2>7)) ) $ echo $? 1 $ $( ((2<7)) ) $ echo $? 0 Interpreting the result of the two conditions. then echo "Yes". else echo "No". else echo "No".)) in a condition. but what about this: $ if [ 17 ">" 2 ]. then echo "Yes". See Week 82 Thursday. fi Yes $ So let's try some numbers: $ if [ 7 ">" 2 ]. fi Yes Odd Behavior in Bash Conditions Bash has the very useful integer arithmetic operator $((.. It can also take $variables as well as literal values like 7.Unix_tricks.. May 27.)). fi Yes As expected. then echo "Yes".. in shell-speak 0 represents TRUE and 1 represents FALSE. else echo "No".. else echo "No". Try this: $ if $((2<7)). An 'if' statement executes the command immediately following it (often '[ . fi bash: 1: command not found No So why '1: command not found'? A common mistake is to use $((. but so does an 'if' statement. then echo "Yes". fi No $ if [ "aaa" "<" "zzz" ]. fi Yes . and so tries to execute '1' (why does it return 1 and not 0?) returned from the evaluation of '2<7'. $ effectively executes the expression to return the value (1). For numeric comparison use '-gt' and '-lt': $ if [ 17 -gt 2 ]. The construct evaluates expressions and relational expressions. Drop the $: $ if ((2<7)). else echo "No". fi No The '>' operator works on strings (or numbers which it interprets as strings).txt Page 180 of 271 Printed: Tuesday.. 2008 3:20:00 PM bash: aaa: No such file or directory No Double quotes can also be used to escape: $ if [ "zzz" "<" "aaa" ].

Whenever a command includes an unescaped *. Anything following is interpreted as an argument.plist . I have a file there called: -BringToFront.. fi No Odd Behavior in Filenames I tried this command: $ cd ~/Library/Preference $ /bin/ls *. else echo "No".what is happening here? The clue lies in the 'illegal option' message. You can see why they work if you understood the explanation above. So how to get around this? Obviously the '*' cannot be escaped: it won't be expanded by the shell. unlink fil Either specify the filename with a path prefix to hide the leading '-'. fi Yes $ if [ $((7<2)) -eq 1 ]. fi Yes $ if [ $((2<7)) -eq 1 ].plist Fire.plist appears to 'ls' to be a list of options. then echo "Yes".. it is the shell that interprets and expands it. 2008 3:20:00 PM The next examples are more long-winded. When specifying a filename directly one will encounter similar difficulties: $ ls -...plist The shell expands *.*. . $ if (($((2<7))==1)).but not OS X.. So the command seen by 'ls' is: ls -BringToFront.plist" ls: *. else echo "No". and 'ls' will complain.. then echo "Yes".trick again: $ rm .. then echo "Yes". $ ls -. or use the -.plist Vim.-* -i $ rm -i usage: rm [-f | -i] [-dPRrvW] file .plist -BringToFront. $ ls "*.plist: No such file or directory Most commands take the special option '--' which says 'end of options'. else echo "No".txt Page 181 of 271 Printed: Tuesday.. -BringToFront..plist Vim..plist ... not an option. $ unlink -i -bash: unlink: command not found .plist Fire. usage: ls [-ABCFGHLPRSTWZabcdfghiklnoqrstuvx1] [file . not the command it is being passed to..plist. which takes no options and thus doesn't get confused by filenames starting '-'.] Odd . but also work.-i Some versions of Unix include the primitive 'unlink' command.plist ls: illegal option -.Unix_tricks./-i $ rm -. May 27.

It is often necessary to use ( . fi Yes This is actually the wrong answer. not an expression. ) to force the correct order of evaluation. then echo "Yes". because bash interprets '!' even within double quotes..Unix_tricks. fi No Odd Behavior in Shell Escaping Here is a file that contains two instance of the sequence '!*'. . Why does it give the wrong answer? Because we are now passing a string into 'test'. May 27. and a string tests as TRUE (or FALSE for an empty (null) string). We could try to escape the entire expression by surrounding it with quotes: $ if [ "1 -eq 0 -a ( 1 -eq 1 -o 1 -eq 1 )" ]. (Very useful in itself sometimes:) $ ls a-very-long-filename-indeed-that-we-might-refer-to-in-the-next-command a-very-long-filename-indeed-that-we-might-refer-to-in-the-next-command $ cat !* cat a-very-long-filename-indeed-that-we-might-refer-to-in-the-next-command Hello To avoid shell interpretation: $ grep "!*" ps grep "ps ps" ps Still no good. else echo "No". 2008 3:20:00 PM Odd Behavior in Complex Shell Conditions The 'test' or '[' command is able to perform 'and' and 'or'.. fi -bash: syntax error near unexpected token `(' The bash shell is interpreting the brackets and not passing them to 'test'. '-a' signifies 'and'. then echo "Yes". The first attempt fails: $ grep !* ps grep ps ps It oddly echoes 'grep ps ps' because the shell interprets '!*' as 'the parameters passed to the last command'. See Week 83 for an explanation of the shell's interpretation of special characters in " " and ' '. else echo "No". The secret is to escape each bracket from the shell. $ cat ps This is a file !* with two !* sequences in it. $ if [ 1 -eq 0 -a \( 1 -eq 1 -o 1 -eq 1 \) ]. '-o' signifies 'or'.txt Page 182 of 271 Printed: Tuesday. it should give 'no' (FALSE and (TRUE or TRUE) = FALSE). then echo "Yes". so the expression expression is passed the 'test' as en expression not a string. else echo "No". Suppose we wish to run the file through grep to search for the lines containing '!*'. See Week 82 Friday. In this example we want: A and (B or C) $ if [ 1 -eq 0 -a ( 1 -eq 1 -o 1 -eq 1 ) ].

two of the most useful being: dscl . create /users/joe and to add property-value pairs: $ dscl . a new group. so to send '\*' to grep we need to escape both with '\\' and '\*'. to say "match '*'. list all users: $ nireport . and in fact in any Directory Services node. 52. create /users/joe name joe $ dscl . and property-value pairs added. See Weeks 9. Remember that grep takes regular expressions. and others.Directory Services Command Line A replacement for the older niutil that manipulates the information in NetInfo. For example. create /users/joe shell /bin/bash $ dscl . 10. don't interpret it as a pattern repeater". nireport . create /users/joe realname "joe jones" Any NetInfo subdirectory can be created.Unix_tricks. Hence '\\\*'.Print tables from NetInfo This reports on the values of the listed properties for specified NetInfo directories.add a new user. /users name nobody root daemon unknown . Still no go. 2008 3:20:00 PM Trying single quotes: $ grep '!*' ps This is a file !* with two !* sequences in it. The '*' needs to be escaped from grep too. There are many commands to query and change the NetInfo database. in a similar manner. $ grep '!\*' ps !* with two !* sequences Or: $ grep \!\\\* ps !* with two !* sequences Here we escaped: '!' from the shell with '\!' '*' from the shell with '\*' Remember that '\' is a special character to the shell. NetInfo Commands This week presents some useful scripts to manage user accounts on the command line . May 27.txt Page 183 of 271 Printed: Tuesday. and add existing users to a group. For example. to add a new user: $ dscl .

. use nicl. List all group names and their group IDs. /users name | grep -w $user)" ]..*$user"). $ nireport . niload. fi Exists $ user=xxxx $ if [ ! -z "$(nireport . then echo "In group". fi $ Convert a group name to GID: $ group=admin $ echo "$(nireport . then echo "Exists". fi root@saruman ~/bin $ Check if a user is in a group (users root then jan in group admin): $ group=admin $ user=root $ in=$(nireport . /groups name users | grep -w "$group.txt Page 184 of 271 Printed: Tuesday. Thursday. while Wednesday. if [ ! -z "$in" ]. when NetInfo is not running. nigrep. /users name | grep -w $user)" ]. and Friday give scripts to add new users and groups to NetInfo. /groups name gid nobody -2 nogroup -1 wheel 0 daemon 1 kmem 2 sys 3 tty 4 . These examples can be used in Bash scripts. Other commands are: nifind. /groups gid name | grep $group | cut -f 1)" 80 Check if this is the user's primary group: . and add users to groups. Tuesday gives some handy one-lines for querying the NetInfo database. Query NetInfo Here are some tips on using nireport to extract information on users and groups from NetInfo. fi In group $ user=jan $ in=$(nireport . May 27.Unix_tricks. nidump In single user mode. 2008 3:20:00 PM smmsp lp postfix .*$user"). if [ ! -z "$in" ]. Check if a user exists: $ user=jan $ if [ ! -z "$(nireport . /groups name users | grep -w "$group.. then echo "In group". These are all documented in the Unix manual.. then echo "Exists".

Lots of checks are made. the desired user id. The script will prompt for a password for the new user. then echo "Primary group".3) #!/bin/bash # Create a user. A home directory will be created so the user is a fully-fledged OS X account holder and can log in in the normal manner. fi Primary group $ user=jan $ group=admin $ gid="$(nireport . then echo " Error: $*".*$gid"). fi exit 1 } . if [ ! -z "$pri" ]. uid. fi $ Get the UID and GID for a given user: $ user=jan $ nireport . /users name gid | grep -w "$user[[:space:]].*$gid"). /groups gid name | grep $group | cut -f 1)" $ pri=$(nireport . if [ ! -z "$pri" ].txt Page 185 of 271 Printed: Tuesday.Unix_tricks. The user id is supplied because normal account creation in OS X does not give this option. as you can see from the script comments. May 27. # Takes the user's firstname (=shortname). and if they are to be a normal (staff) or admin user. /users name uid gid | grep $user jan 520 520 Add a New User This script adds a new user account to OS X. /users name gid | grep -w "$user[[:space:]]. lastname. The user's short name will be equal to their first name. Give the user's first and last names. Grab the script from here. then echo "Primary group". and staff|admin # and creates: # a new user in NetInfo passwd # a new /Users/firstname home directory usage () { echo "Create a new staff or admin user" echo "Usage: ${0##*/} firstname lastname uid staff|admin" if [ "$*" != "" ]. /groups gid name | grep $group | cut -f 1)" $ pri=$(nireport . NOTE: THIS SCRIPT IS WRITTEN FOR PANTHER (10. 2008 3:20:00 PM $ user=jan $ group=jan $ gid="$(nireport .

then usage "User $first already exists at /Users/$first" fi # search NetInfo for the given user . accnt=$4 # check that the users does not already have a home directory if [ -e /Users/$first ]. uid=$3. /users uid | grep -w $uid)" if [ ! -z "$str" ]. then usage fi first=$1. May 27. 2008 3:20:00 PM # The script must be run by root # if [ "$USER" != "root" ]. then usage "User ID $uid already exists" fi # search NetInfo for the given group .it should not exist str="$(nireport . then usage "Group $first already exists" fi # search NetInfo for the given gid . /users name | grep -w $first)" if [ ! -z "$str" ]." exit 1 fi # Check parameters # if [ $# -ne 4 ]. /groups name | grep -w $first)" if [ ! -z "$str" ]. last=$2. /groups gid | grep -w $uid)" if [ ! -z "$str" ].it should not exist str="$(nireport .txt Page 186 of 271 Printed: Tuesday. then usage "Group ID $uid already exists" fi # ensure either staff or admin is given if [ $4 != staff ] && [ $4 != admin ].it should not exist str="$(nireport . then usage "User $first already exists (but does not have a home directory)" fi # search NetInfo for the given uid . then usage "Give account type as 'staff' or 'admin'" fi should not exist str="$(nireport . then echo "Must be run as root.

Unix_tricks.txt Page 187 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

# Add the new user to NetInfo
# add user and essential properties
dscl . create /users/$first
dscl . create /users/$first name $first
dscl . create /users/$first passwd "*"
dscl . create /users/$first hint ""
dscl . create /users/$first uid $uid
dscl . create /users/$first gid $uid
dscl . create /users/$first home /Users/$first
dscl . create /users/$first shell /bin/bash
dscl . create /users/$first realname "$first $last"
dscl . create /users/$first picture "/Library/User Pictures/Fun/Smack.tif"
dscl . create /users/$first sharedDir Public

# add some other properties that are usually in NetInfo
dscl . create /users/$first _shadow_passwd ""
dscl . create /users/$first _writers_hint $first
dscl . create /users/$first _writers_real_name $first

# add the new group
dscl . create /groups/$first
dscl . create /groups/$first name $first
dscl . create /groups/$first passwd "*"
dscl . create /groups/$first gid $uid

echo "New user and group $first created"

# Add admin users to the admin group
if [ $4 = admin ]; then
dscl . merge /groups/admin users $first
dscl . merge /groups/appserverusr users $first
dscl . merge /groups/appserveradm users $first
echo "$first added to groups admin, appserverusr, appserveradm"

# Create the home directory, populate from the template, and set owners
mkdir /Users/$first
if [ ! -d /Users/$first ]; then
echo "Unable to create the user's home directory /Users/$first"

ditto -rsrc /System/Library/User\ Template/English.lproj/ /Users/$first
chown -R ${first}:$first /Users/$first
echo "Home directory /Users/$first created and populated"

Unix_tricks.txt Page 188 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

# Now give the user a password
echo "A password for this account must be given, it is currently blank"
passwd $first

exit 0

Add a New Group

This script adds a new group to OS X. Give the group name and id. The group will be created
without any users belonging to it. Add users to the group with add-user2group (see Friday).

Lots of checks are made, as you can see from the script comments.

Grab the script from here.



# Create a group.
# Takes a group name and gid and creates a new group in NetInfo groups

usage ()
echo "Create a new group"
echo "Usage: ${0##*/} groupname gid"
if [ "$*" != "" ]; then echo " Error: $*"; fi
exit 1

# The script must be run as root
if [ "$USER" != "root" ]; then
echo "Must be run as root."
exit 1

# Check parameters
if [ $# -ne 2 ]; then

Unix_tricks.txt Page 189 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

group=$1; gid=$2

# search NetInfo for the given group - it should not exist
str="$(nireport . /groups name | grep -w $group)"
if [ ! -z "$str" ]; then
usage "Group $group already exists"

# search NetInfo for the given gid - it should not exist
str="$(nireport . /groups gid | grep -w $gid)"
if [ ! -z "$str" ]; then
usage "Group ID $gid already exists"

# Add the new group to NetInfo
# add group and essential properties
dscl . create /groups/$group
dscl . create /groups/$group name $group
dscl . create /groups/$group passwd "*"
dscl . create /groups/$group gid $gid
#dscl . create /groups/$group users "" breaks add-user2group if added as a blank value

echo "New group $group created"
echo "Now add users to it with add-user2group"

exit 0

Add Users to a Group

This script adds one or many users to a group. Give the group names and a list of user short

Lots of checks are made, as you can see from the script comments.

Grab the script from here.



# Add new users to a group.
# Adds a user (or many users) to an existing group in NetInfo

usage ()
echo "Add a user (or several users) to an existing group"
echo "Usage: ${0##*/} group user [user...]"
if [ "$*" != "" ]; then echo " Error: $*"; fi
exit 1

Unix_tricks.txt Page 190 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

# Ensure user is root
if [ "$USER" != "root" ]; then
echo "Must be run as root."
exit 1

# Check parameters
if [ $# -lt 2 ]; then


# search NetInfo for the given group - it should exist
str="$(nireport . /groups name | grep -w $group)"
if [ -z "$str" ]; then
usage "Group $group does not exist"

# get the group number from the name
gid="$(nireport . /groups gid name | grep $group | cut -f 1)"

# Drop the group and loop thro' additional parameters (users) to add to group

for user in $*; do
# check if the user exists
struser="$(nireport . /users name | grep -w $user)"
# check if the user already belongs to the group
stringroup=$(nireport . /groups name users | grep -w "$group[[:space:]].*$user")
# check if this is the user's primary group
strprimary=$(nireport . /users name gid | grep -w "$user[[:space:]].*$gid")

#echo "user $struser, ingroup $stringroup, primary $strprimary"

# ensure that the user exists...
if [ -z "$struser" ]; then
echo "User $user does not exist"
# ...and does not already belong to the group...
elif [ ! -z "$stringroup" ]; then
echo "User $user already belongs to group $group - not added again"
# ...and this is not the user's primary group
elif [ ! -z "$strprimary" ]; then

Unix_tricks.txt Page 191 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

echo "This is the user's primary group - not added"
# add user to the group
dscl . merge /groups/$group users "$user"
echo "$user added to group $group"

exit 0

Executables as Conditions

These examples are written for the Bash shell. They are applicable to other shells but the syntax
may need modification. The construct $(...) should be replaced by `...` in tcsh.

See also weeks 56 and 65.

Incorporate the result of executing a command into a condition.

For example, I recently had to write a script that waited (looped) until a particular volume was
mounted. 'df | grep' is an easy way to test this:
$ volname=Games
$ df | grep $volname
/dev/disk0s7 5.9G 578M 5.3G 10% /Games

The technique is to loop while the output from the above command is null. Note that '[ -z string
]' is true when the string is zero length (null).

Attempt 1:
$ while [ -z df | grep $volname ]; do echo -n "."; done
-bash: [: missing `]'
grep: ]: No such file or directory

'[' expects a string, so we must capture the output of executing 'df | grep $volname' and test
that string.

Attempt 2:
$ while [ -z $(df | grep $volname) ]; do echo -n "."; done
-bash: [: too many arguments

The output from grep is a string with spaces, so we must quote the output so it appears to '[ -z
... ]' as a single string.

Use "$(...)" to return a single string from the output of a command or pipe:
$ while [ -z "$(df | grep $volname)" ]; do echo -n "."; sleep 2; done;

This loops until the volume $volname appears.

Note 'echo -n' is used to avoid a newline for every dot.

Use 'sleep n' in the body of a wait loop to avoid hogging CPU time. 'n' is in seconds.

Tricky Quoting

Continuing from Monday's tip.

sleep 2. and the very simple solution is: $ while [ -z "$(df | grep "$volname")" ]..^C We must quote $volname.)'.)" Bash re-sets its idea of nested quotes.. $ volname="New Games" $ while [ -z "$(df | grep $volname)" ]. echo . do echo -n ". how can we quote $volname too? Using single quotes '... If the string is not all digits grep returns a null string.. done.. The first technique uses grep and a regular expression to search for a string of all digits. You might expect this to work: $ while [ -z "$(df | grep \"$volname\")" ]. fi number The second technique relies on 'test' (or '[') throwing an error and returning '2' if -eq is ."..Unix_tricks. do echo -n ". 2008 3:20:00 PM Understand quotes within quotes. Quotes in Quotes This useful trick is not always obvious until it is pointed out. $ var=12 $ if [ "$(echo $var | grep "^[[:digit:]]*$")" ].grep: Trailing backslash It doesn't because Bash is being quite clever. do echo -n ".' for the outer set will not work because the shell won't interpret '$(. If you know of a better method let me know. echo grep: Trailing backslash . then echo number.". Here are two indirect ways of making this test. These should be obvious: $ word=Hi $ echo "He said $word" He said Hi $ echo $word Hi $ echo '$word' $word What's the output from this command.. and why? $ echo "He said '$word'" Test for a Numeric Variable Bash does not seem to provide an easy way of testing if a variable is numeric ('123' as opposed to 'abc' or 'a2'). echo grep: Games: No such file or directory . Within "$(... which is interpreted as false. as the expression: "$(df | grep $volname)" is itself already quoted.txt Page 192 of 271 Printed: Tuesday. May 27.. But. done.". sleep 2. sleep 2. Suppose the volume name included spaces (or shell special characters). done.

then treats the resulting 'system_BSD=yes' as a command. fi number $ $ var="12AB" $ [ $var -eq 0 ] 2> /dev/null $ if [ $? -eq 0 -o $? -eq 1 ]. which may be of any length. in a round-about way. we want to construct a variable name from another variable: From: $var=BSD We want to create a variable called: system_BSD Let's set to 'yes' variable 'system_???'. $ cat . then echo number. then echo number. where ??? is the value of $var: $ var=BSD $ system_$var=yes -bash: system_BSD=yes: command not found This fails because Bash interprets $var as BSD./underline #!/bin/bash read -p 'Give a string to underline: ' string . To set a variable variable use: $ var=BSD $ eval "system_$var=yes" $ echo $system_BSD yes We can't echo the value in the same manner: $ eval "echo $system_$var" BSD We want $var to be 'eval'uated. but not $system: $ eval "echo \$system_$var" yes $ Underline a String Underline a generated string. fi $ Variable Variables Bash can implement variables variables.txt Page 193 of 271 Printed: Tuesday. 2008 3:20:00 PM given a non numeric value. Basically. then tell Bash to re-parse the line from scratch using 'eval'.Unix_tricks. We need to evaluate $BSD. PHP programmers will be familiar with variable variables. Otherwise '0' or '1' is returned for TRUE or FALSE. $ var="123" $ [ $var -eq 0 ] 2> /dev/null $ if [ $? -eq 0 -o $? -eq 1 ]. May 27.

do echo -n "-" ((len = len ..) is usually used to capture the output from commands./underline #!/bin/bash # Function underline # # Generate an underline string # # $1: the string to underline # $2: the optional underline character. This little script uses Bash built-in integer tests and arithmetic: while ((len != 0)). Return Arbitrary Values Return an arbitrary value from a Bash function. but it works equally well for functions. Here I have converted the underline example from Monday's tip into a function that returns the underline string.. This is testable via the shell special variable $?. or even a string? One way to do this is to get the function to 'echo' the return value.Unix_tricks./underline Give a string to underline: This is a test. do ((len = len .1)) done echo $ . and use the $(.) construct to capture the output from the function. But how do we return an arbitrary number. $ cat . Bash functions return an exit status between 0 and 255. --------------- The construct: ${#var} returns the length in characters of 'var'..txt Page 194 of 271 Printed: Tuesday. $(.1)) Notice anything unusual/unexpected? 'echo -n' is used so as not to follow each dash with a newline. This is a test. This example would normally be coded as a Bash function and called whenever the main script needs to underline a string.. default value is '-' # # return: the underline string # underline () . 2008 3:20:00 PM len=${#string} echo $string while ((len != 0)). May 27.

Unix_tricks. This is a test..../underline Give a string to underline: This is a test. May 27.1)) done echo $line } # Main script # read -p 'Give a string to underline: ' string echo $string dashes=$(underline "$string") echo $dashes $ . ----------------- Function underline returns its value with: echo $line and this value is captured in 'dashes' by the main script using: dashes=$(underline "$string") This is a useful construct: char=${2:-"-"} Return many values using arrays. Here is a simple example using an array to receive many return values from a function. 'count' returns four strings. 2008 3:20:00 PM { local -i len # to hold the length of the string in $1 local line # to hold the generated underline string local char=${2:-"-"} # set the underline to $2 or the default if not supplied len=${#1} while ((len != 0)).. $ cat numbers #!/bin/bash # Function count # count () { #return one two three four echo one two three four } .txt Page 195 of 271 Printed: Tuesday. do line=${line}$char ((len = len .

txt Page 196 of 271 Printed: Tuesday. May 27. but the text is echoed. $ cat getpass #!/bin/bash read -p "Give your name: " name stty -echo read -p "And your password: " pass./getpass Give your name: Adrian And your password: And your shoe size: 9 Access the Last Parameter Get the value of the last parameter passed to a script/function when the number of parameters is *not* known. echo stty echo read -p "And your shoe size: " ss $ . $2 id the second parameter. ./numbers one two three four Stop Echoing for Passwords Stop input being echoed when a script reads a password. The secret is to use: stty -echo to switch off echoing. read text from the user. 2008 3:20:00 PM # Main # declare -a numbers numbers=($(count)) echo ${numbers[0]} echo ${numbers[1]} echo ${numbers[2]} echo ${numbers[3]} $ . but how do we get the n'th parameter without knowing 'n'? The number of parameters is given by the special parameter $#. A script can print a prompt.Unix_tricks. If a password is being read. We need '$$#'. it would be nice to prevent this from being echoed.

txt Page 197 of 271 Printed: Tuesday. if globbing (wildcard expansion) expands to nothing. done . This can be effected by setting Bash's 'nullglob' option: $ shopt -s nullglob $ for file in *.txt expands to two files: $ for file in *. Change the Shell's Null Globbing Change the shell's behaviour for glob expansion./lastp one two three four four four The first parse effected by eval changes \$ into $ and $# into 4./lastp one two three 3 2428# 2428 ${3} The secret is to evaluate {S#} first. May 27. For example. done $ (no error message) At the end of the script.txt.Unix_tricks. do echo -n $file ": ". The resulting line is then parsed normally echoing $4 (or assigning $4 to last).text. done *. This is not always what is required in a shell script. do echo -n $file ": ". $ cat .text : head: *.txt : Hello g. letting Bash parse the line twice. head $file. the shell returns the glob instead of null. *. head $file.text doesn't match: $ for file in *.text. the default behaviour should be restored. Normally. head $file. $ shopt -u nullglob $ for file in *.txt : there but *.text: No such file or directory We'd prefer the loop to be silent rather than give a spurious error message. do echo -n $file ": "./lastp #!/bin/bash eval echo \$$# eval last=\$$# echo $last $ . head $file.text. 2008 3:20:00 PM None of these attempt work: $ cat lastp #!/bin/bash echo $# echo $$# echo ${$#} echo \${$#} $ . then evaluate $-the-result-of-the-first-evaluation. Bash's 'eval' function does just this. done f. leaving $4. do echo -n $file ": ".

Unix_tricks.txt Page 198 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

*.text : head: *.text: No such file or directory

Enable FTP

This week's tips show how to set up a simple FTP service complete with 'chrooting' and access
classes. Note: read the complete week's tips before exposing your FTP set-up to the Internet.

Enable FTP the GUI way. In System Preferences select Sharing, tab Services, and check FTP Access.
The fire-wall will be automatically be opened.

Enable FTP the command line way.

1) Open the fire-wall ports 20 and 21, and all high-numbered ports connected to from FTP clients
(from source ports 20, 21).
$ sudo ipfw add 03000 allow tcp from any to any 20-21 in
$ sudo ipfw add 03010 allow tcp from any 20,21 to any 1024-65535 in
$ sudo ipfw list
03000 allow tcp from any to any 20-21 in
03010 allow tcp from any 20,21 to any 1024-65535 in

Rules 3000 onwards are not used by the OS X default firewall set-up. You might like to 'ipfw
list' to check this first if you use a non-standard set-up.

Note: System Preferences will sulk when it sees you have been playing with its fire-wall rules.
If you want SP to play again, do:
$ sudo ipfw flush
Are you sure? [yn] y

Flushed all rules.

(re-)Launch SP and enable the firewall.

To start FTP edit the 'ftp' service in xinet.d. (FTP is set up as an on-demand service launched
by xinet.d, not as a permanent daemon.)
$ sudo pico /etc/xinetd.d/ftp

Set the disable line to 'no'
disable = no

Re-set xinet.d so it re-reads the edited file (bash):
$ sudo kill -HUP $(cat /var/run/

or (tcsh, bash)
$ sudo kill -HUP `cat /var/run/`

Test by connecting with 'ftp user@hostname/IP-address':
$ ftp saruman@carcharoth
Connected to
220 FTP server (tnftpd 20040810) ready.
331 Password required for saruman.
Welcome to Darwin!
230 User saruman logged in.
Remote system type is UNIX.
Using binary mode to transfer files.

Unix_tricks.txt Page 199 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

ftp> quit
Data traffic for this session was 0 bytes in 0 files.
Total traffic for this session was 480 bytes in 0 transfers.
221 Thank you for using the FTP service on

Allow and Deny Explicitly

Disallow FTP access by default.

The standard Apple-supplied ftp configuration allows all users to use FTP, except those
explicitly listed - i.e. allow by default and deny explicitly. A more security-conscious set-up
would be to deny by default and allow explicitly.

Change the /etc/ftpusers file from:
$ cat /etc/ftpusers
# list of users disallowed any ftp access.
# read by ftpd(8).

$ cat /etc/ftpusers
# list of users allowed ftp access.
# read by ftpd(8).
saruman allow
loraine allow
... list all the accounts that may ftp...

# deny all other users
* deny

This file must be edited as root, or via 'sudo'.

Jail and Umask

Jail users to their home directory.

The standard Apple-supplied ftp configuration allows all users the same access to the file system
they would enjoy by when logging in. That is, they can 'cd /' and view many system files. A more
security-conscious set-up would 'jail' most users such that 'cd /' took them to their home
directory. For theses users, their home directory is the root of their file system and they can
never move to a level above it.

Edit/create the file /etc/ftpchroot:
$ sudo pico /etc/ftpchroot

so that it looks like this:
$ cat /etc/ftpchroot
# all users are chrooted

Unix_tricks.txt Page 200 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

Tomorrow's tip shows how to jail some users, and not others (without explicitly listing all
chrooted users).

Change the default umask.

The default ftp umask in 027, as opposed to the more usual 022. This means that users uploading
files to be served by Apache, for example, create files that Apache cannot read.

FTP provides a 'chmod' command:

ftp> help chmod

chmod change file permissions of remote file

but a more convenient solution may be to change the default umask for all users to be 022.

Edit/create the file /etc/ftpd.conf:
$ sudo pico /etc/ftpd.conf

so that it looks like:
$ cat /etc/ftpd.conf
# normally 027
umask all 022

Test with:
$ ftp loraine@carcharoth
Connected to
230 User loraine logged in.
ftp> umask
200 Current UMASK is 022
ftp> quit

Define Classes of Users

To get finer control over FTP access, umask settings, chrooting, etc, FTP allows one to define
classes. Settings can be defined against a class, and each user assigned to the appropriate
class. A user will then inherit the attributes set for the class to which they belong.

Edit the ftp config file to define classes.
$ sudo pico /etc/ftpd.conf

In the configuration below, I have defined two classes, 'free' and 'restricted'. I've 'chrooted'
the free class to '/' so they can view the entire file system (the default setting). Restricted
users can only view their home directory (%d). Note that 'homedir' for restricted users is set to
'/', which is their home directory because of their chroot setting.
$ cat /etc/ftpd.conf

# users of class 'free' (see /etc/ftpusers) chroot to /
# with their ftp home directory set to their login home
chroot free /
homedir free %d

# users of class 'restricted' chroot to their home directory
# with their ftp home directory set to their new root (ie their login home)
chroot restricted %d
homedir restricted /

Unix_tricks.txt Page 201 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

# change the umask from the default of 027
umask all 022

The last directive sets the umask for 'all', which represents all classes.


FTP is not the most secure of services. Set up carefully it should be fine, but remember that
passwords and all communications are sent unencrypted.

If you run ssh (Secure SHell) then you might like to take advantage of secure copy and secure
FTP. You must already have ssh set up in order to use these services.

scp is like the regular Unix copy, but copies across networks using an ssh login. To copy the
file 'test' in my home directory to the host saruman.wless I can use:
$ scp ~/test saruman.wless:~/test

To use scp precede the path on the remote machine with 'hostname:' or 'ip-address:'.

The equivalent of 'ftp' is 'sftp'. As far as I know it is not possible to configure sftp to the
same degree as ftp. For example, it is not possible to jail users to their home directories.

Note: To use scp ensure that file /etc/sshd_config has the line:
# override default of no subsystems
Subsystem sftp /usr/libexec/sftp-server

Finally, to restrict the range of ports to which a client can call-back, use (for example):
# set port range for passive for all classes
portrange all 40000 40999

The default is 1024-65535. After having made this change you can then restrict the number of open
ports in the firewall configuration (see Monday's tip) to:
03010 allow tcp from any 20,21 to any 40000-40999 in

Strings and Vis

Use 'strings' to remove junk characters.

The strings command searches a file for printable strings and displays them. It can be used on a
text file to remove spurious non-printable characters.
$ strings text-file > text-file.clean

Peek inside commands.

Use strings on commands such as 'ls' to discover embedded text:
$ strings /bin/ls
The kernel support for the dynamic linker is not present to run this program.
$FreeBSD: src/bin/ls/cmp.c,v 1.12 2002/06/30 05:13:54 obrien Exp $
@(#) Copyright (c) 1989, 1993, 1994
The Regents of the University of California. All rights reserved.
$FreeBSD: src/bin/ls/ls.c,v 1.66 2002/09/21 01:28:36 wollman Exp $

Unix_tricks.txt Page 202 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

Use 'vis' to display messy files.

Vis displays control characters as escape sequences allowing you to view files that might
otherwise spew junk to the terminal.

File and Hexdump

Use 'file' to discover file types.
$ file safari-bm.html
safari-bm.html: UTF-8 Unicode English text, with very long lines

$ file safari-bm.txt
safari-bm.txt: OS/2 URL object text (WWW) <

$ file /bin/ls
/bin/ls: Mach-O executable ppc

$ file /
/: sticky directory

$ file ~
/Users/saruman: directory

$ file /dev/disk0
/dev/disk0: block special (14/0)

$ file /dev/rdisk0
/dev/rdisk0: character special (14/0)

and so on...

Use 'hexdump' to display a file in hexadecimal.
$ hexdump /bin/ls | head -n 5
0000000 feed face 0000 0012 0000 0000 0000 0002
0000010 0000 000f 0000 0778 0000 0095 0000 0001
0000020 0000 0038 5f5f 5041 4745 5a45 524f 0000
0000030 0000 0000 0000 0000 0000 1000 0000 0000
0000040 0000 0000 0000 0000 0000 0000 0000 0000

hexdump is useful for revealing control characters in text files, or examiming the contents of a
binary file

Note that Mach-O executables start with the hex sequence 'feed face' :-)

When is it Time to Leave?

Use 'leave' so you never again forget lunch.

Lunch at one:
$ leave 0100
Alarm set for Mon Dec 27 13:00. (pid 10494)

Coffee in 10 minutes:
$ leave +10
Alarm set for Mon Dec 27 12:38. (pid 10497)

. Time to leave! ..Unix_tricks. $ history -c $ cd ~ . Time to leave! ..clean Development Library Music Public bin test typescript $ rm test $ rm test. Option -a appends to an existing script results file. $ Just one more minute! .clean (at this point press control-D to exit the script session) $ exit Script done.clean Development Library Music Public bin test typescript $ rm test $ rm test..clean $ exit Script done on Mon Dec 27 12:45:34 2004 $ Of course.. If you are doing something you may want to repeat later. or wish others to do. Time to leave! . output file is typescript $ cd ~ $ ls Desktop Documents Movies Pictures Sites osxfaq test. use history. kill leave using its pid as echoed back when the reminder was set. .. $ script Script started.. May 27.. $ You have to leave in 5 minutes. output file is typescript View the script file: $ cat ~/typescript Script started on Mon Dec 27 12:45:12 2004 $ cd ~ $ ls Desktop Documents Movies Pictures Sites osxfaq test. use the script command to record you terminal session to a file (~/typescript if no filename is given). $ kill -KILL 10494 To kill all leaves use: $ kill -KILL $(ps | grep ' [l]eave ' | awk '{print $1}') (in tcsh use:) $ kill -KILL `ps | grep ' [l]eave ' | awk '{print $1}'` Record a Terminal Session Use 'script' to record a sequence of commands.. When you log out the nagging will finish. if all you want is the commands you typed. and with increasing frequency as the event approaches. 2008 3:20:00 PM Leave will remind you periodically about the forthcoming event.txt Page 203 of 271 Printed: Tuesday. If you don't log out..

.0.1 Thu Dec 23 21:19 . . and delete a user. here are some tips on commands to query the user accounts in NetInfo.. delete a user from a group. load averages: 0. logged in: $ last | head -n 5 ferdi ttyp2 10. 2:22.2. You can send that user a message direct to their console: $ write ferdi [ttyp2] get off The tty number is optional if they are logged in just once. 2 users. First.28 Directory Service Commands More useful scripts to manage user accounts on the command line .1 Thu Dec 23 21:30 still logged in saruman ttyp2 10.21 0. 2:36.21:25 (00:05) saruman ttyp3 Thu Dec 23 21:01 still logged in saruman ttyp3 Thu Dec 23 21:01 .21 USER TTY FROM LOGIN@ IDLE WHAT saruman console .1 Thu Dec 23 21:25 . load averages: 0.2. Use 'dcsl list' to display groups and users..Sun17 4days - ferdi p2 middle-earth.21:01 (00:00) 'uptime' is a good to boast about how long your server has been up and running. finish the session $ history > ~/what-i-typed Who's Logged In There are several command for monitoring which users are currently logged into you Mac or server.2. Command 'w' gives a little more information.21:26 (00:00) saruman ttyp2 10.. 2008 3:20:00 PM $ ls Desktop Documents Movies Pictures Sites osxfaq .1) User 'ferdi' is logged in from another machine.. $ w 21:30 up 6 days.wle 21:30 .36 0.. $ uptime 21:16 up 6 days. $ dscl .0. as given by the IP address in brackets. $ who saruman console Dec 19 17:29 ferdi ttyp2 Dec 23 21:30 (10.delete a group. 2 users.Unix_tricks.-bash Command 'users' lists only local users: $ users saruman Command 'last' give a history of who was.0. and who still is. list /groups admin appserveradm appserverusr bin claire .txt Page 204 of 271 Printed: Tuesday.21 0.2.0. May 27.45 0.

etc) exists.txt Page 205 of 271 Printed: Tuesday..Unix_tricks. search /groups name claire Search results for dsAttrTypeStandard:name Search results for dsAttrTypeNative:name claire dsAttrTypeNative:name = (claire) Use 'dscl read' to display properties and values. $ dscl . Grab the script from here. list /users appserver claire cyrus daemon . $ cat del-group #!/bin/bash # Delete a group.. user. $ dscl . Use 'dscl search' to check for property-value pairs. 2008 3:20:00 PM $ dscl . /users/claire found in .quiet: no warnings or prompts for confirmation" . May 27.. $ Delete Group Use this script to delete a group from NetInfo. # Takes the group name and deletes it from NetInfo declare quiet="no" # -q option not specified declare group # hold te given group name declare gid # hold the group id derived from the group name declare ans # reply from prompt usage () { echo "Delete a group" echo "Usage: ${0##*/} [-q] groupname" echo " -q . read /groups name: dsRecTypeNative:groups $ dscl . id = 90 $ nifind /users/xxxxxxxxx . read /groups/claire AppleMetaNodeLocation: /NetInfo/DefaultLocalNode GeneratedUID: 0066A0DA-ED62-11D8-B80E-000393B2D604 GroupMembership: Password: * PrimaryGroupID: 507 RecordName: claire Use 'nifind' to check if an entry (group. $ nifind /users/claire .

then quiet="yes" shift fi if [ $# -ne 1 ]. then # get the group number from the name gid="$(nireport .)" if [ -z "$str" ]. echo "Error: $*". then echo "WARNING: this is a primary group:" nireport . 2008 3:20:00 PM echo " otherwise a warning is issued if the group to" echo " be deleted is a user's primary group" if [ "$*" != "" ].txt Page 206 of 271 Printed: Tuesday. then usage fi group="$1" # search NetInfo for the given group . then usage "Group $group does not exist" fi # Check if this is a primary group for some user and warn if so # but not in quiet mode if [ $quiet = "no" ]. then echo "Must be run as root" exit 1 fi # Check parameters # if [ "$1" = "-q" ]. /groups name gid | grep -w "^$group" | cut -f 2)" # print a warning if it is any user's primary group str="$(nireport .it should exist str="$(nifind /groups/$group . May 27. /users gid name | grep -w "^$gid" . /users gid | grep -w $gid)" if [ ! -z "$str" ].Unix_tricks. then echo. fi exit 1 } # The script must be run as root # if [ "$USER" != "root" ].

Unix_tricks.txt Page 207 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

read -p "Type a to abort: " ans
if [ "$ans" = "a" ]; then
echo "Aborted"

# Delete the group from NetInfo
# sanity check
if [ "$group" = "" ]; then exit; fi

dscl . delete /groups/$group

echo "Group $group deleted"
exit 0

Delete User From Group

Use this script to delete a user from a group, or from all groups.

Grab the script from here.

$ cat del-user4group


# Remove a user from a group, or all groups
# Removes an existing user from an existing group in NetInfo, or from
# all groups to which that user belongs (but not their primary group)

declare groups # hold the given group name or the list of groups
declare user # hold the user account name
declare gid # hold the group id derived from the group name
declare str strgroup stringroup strprimary # working

usage ()
echo "Remove a user from a group or all groups"
echo "Usage: ${0##*/} group|all user"
echo " for 'all' the user is removed from all but their primary group"
if [ "$*" != "" ]; then echo; echo "Error: $*"; fi
exit 1

Unix_tricks.txt Page 208 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

# Ensure user is root
if [ "$USER" != "root" ]; then
echo "Must be run as root"
exit 1

# Check parameters
if [ $# -lt 2 ]; then

groups="$1"; user="$2"

# If group is all, expand into the list of groups to which the user belongs
if [ $groups = "all" ]; then
groups="$(id -Gnr $user)"

# Loop to remove the user from each group
for group in $groups; do

# get the group number from the name
gid="$(nireport . /groups name gid | grep -w "^$group" | cut -f 2)"

# check if the group exists
strgroup="$(nifind /groups/$group .)"
# check if the user is listed for the group (not listed in own primary)
stringroup="$(nireport . /groups name users | grep -w "^$group[[:space:]].*$user")"
# check if this is the user's primary group
strprimary="$(nireport . /users name gid | grep -w "^$user[[:space:]]*$gid")"

# ensure that the group exists...
if [ -z "$strgroup" ]; then
echo "Group $group does not exist"
# ...and this is not the user's primary group
elif [ ! -z "$strprimary" ]; then
echo "Not removing from primary group $group"
# ...and that the user is listed in the group
elif [ -z "$stringroup" ]; then
echo "User $user not listed in $group"
# remove user from the group

dscl . delete /groups/$group users $user

Unix_tricks.txt Page 209 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

echo "User $user removed from group $group"

exit 0

Delete User From Group

Use this script to delete a user from a group, or from all groups.

Grab the script from here.

$ cat del-user4group


# Remove a user from a group, or all groups
# Removes an existing user from an existing group in NetInfo, or from
# all groups to which that user belongs (but not their primary group)

declare groups # hold the given group name or the list of groups
declare user # hold the user account name
declare gid # hold the group id derived from the group name
declare str strgroup stringroup strprimary # working

usage ()
echo "Remove a user from a group or all groups"
echo "Usage: ${0##*/} group|all user"
echo " for 'all' the user is removed from all but their primary group"
if [ "$*" != "" ]; then echo; echo "Error: $*"; fi
exit 1

# Ensure user is root
if [ "$USER" != "root" ]; then
echo "Must be run as root"
exit 1

# Check parameters
if [ $# -lt 2 ]; then

Unix_tricks.txt Page 210 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM


groups="$1"; user="$2"

# If group is all, expand into the list of groups to which the user belongs
if [ $groups = "all" ]; then
groups="$(id -Gnr $user)"

# Loop to remove the user from each group
for group in $groups; do

# get the group number from the name
gid="$(nireport . /groups name gid | grep -w "^$group" | cut -f 2)"

# check if the group exists
strgroup="$(nifind /groups/$group .)"
# check if the user is listed for the group (not listed in own primary)
stringroup="$(nireport . /groups name users | grep -w "^$group[[:space:]].*$user")"
# check if this is the user's primary group
strprimary="$(nireport . /users name gid | grep -w "^$user[[:space:]]*$gid")"

# ensure that the group exists...
if [ -z "$strgroup" ]; then
echo "Group $group does not exist"
# ...and this is not the user's primary group
elif [ ! -z "$strprimary" ]; then
echo "Not removing from primary group $group"
# ...and that the user is listed in the group
elif [ -z "$stringroup" ]; then
echo "User $user not listed in $group"
# remove user from the group

dscl . delete /groups/$group users $user
echo "User $user removed from group $group"

exit 0

Delete User

Use this script to delete a user.

The user and their primary group are deleted. The user is removed from any other groups. The
users home folder is archived then deleted.

Unix_tricks.txt Page 211 of 271
Printed: Tuesday, May 27, 2008 3:20:00 PM

It uses the previous two scripts.

Grab the script from here.

$cat del-user


# Delete a user.
# Takes the account name (short name) and:
# removes the user from all groups
# removes the user's primary group (of the same name)
# removes the user's account in NetInfo
# archives and deletes the user's home directory in /Users/shortname

declare user # to hold user's account name
declare str # working

usage ()
echo "Delete a user account, group, and group membership"
echo "Usage: ${0##*/} username"
if [ "$*" != "" ]; then echo; echo "Error: $*"; fi
exit 1

# The script must be run as root
if [ "$USER" != "root" ]; then
echo "Must be run as root"
exit 1

# Check parameters
if [ $# -ne 1 ]; then


# search NetInfo for the given user - it should exist
str="$(nifind /users/$user .)"
if [ -z "$str" ]; then
usage "User $user does not exist"

The command: $ scselect location-name changes the current network location to the selected one. New Since . 2008 3:20:00 PM # Delete the user from NetInfo # # delete the user from all groups del-user4group all $user # delete the user's primary group del-group -q $user # delete the user from NetInfo dscl . May 27.txt Page 212 of 271 Printed: Tuesday. then rm -rf /Users/${user}/ fi fi echo "User's home directory archived and deleted" exit 0 Change Network Locations Switch network locations from the command line. delete /users/$user echo "User $user deleted" # Archive the user's home directory # # check that the user has a home directory if [ -e /Users/$user ].Unix_tricks.tgz $user cd - # delete it CHECKING THAT AN ARCHIVE WAS CRESATED if [ -e /Users/${user}-archive. as viewed in system Preferences::Network. then # archive it cd /Users tar -czf ${user}-archive. The command: $ scselect lists all network locations.tgz ].

15M Jan 15 12:12 ferdi-coll3. Find all new or changed directories. 30M Jan 23 11:59 ferdi-cool. and noting the modification dates and sizes: $ ls -l total 49M . but up to a maximum nesting depth of 2. /mach..psd ..psd . 13.sym This finds all files in your home that are newer than /mach.. $ touch ~/marker then later.psd .g. 2008 3:20:00 PM List all files created since the last restart: $ find ~ -newer /mach.txt Page 213 of 271 Printed: Tuesday. '-ls' lists the details of each file./ferdi-polish. 31.7M Jan 15 12:12 ferdi-gala.Unix_tricks.6M Jan 23 15:59 ferdi-polish. May 27. -size -10000 . -size -2000 for less than 1MB. E../ferdi-gala.psd Find those files modified more than 1 day ago: $ find . or against files which are smaller than the specified size by specifying a '-' sign. -mtime -1 . Complex Conditions (See also weeks 1.psd Find those files less than 5M in size: $ find . 1.sym -size +2000 -ls '-size' can match against an exact number of blocks by omitting the '+' sign./ferdi-gala. $ find ~ -newer ~/marker More examples.psd .psd Find those files modified less than 1 day ago: $ find .see 'man find'. 'l' for symbolic link. .psd . Given the following files./ferdi-polish. '-size' is specified in blocks which are a 512 bytes or a half K./ferdi-coll3.psd .. and 49) Use complex selection criteria with 'and' and 'or' constructs.. -mtime +1 . $ find ~ -newer /mach.sym is one of the files created when OS X boots. $ find ~ -newer /mach./ferdi-cool. Create a file at the start of the day to keep track of all files you have created or edited that day..sym -type d -maxdepth 2 '-type' can also be 'f' for file.. or other special file types . Find all new files larger than 1MB. 1. ...psd .sym.

/ferdi-coll3. Find with no recursion.psd ./ferdi-gala.psd Find those files modified less than one day ago AND bigger than 5M OR modified more than one day ago AND smaller than 5M: $ find ./ferdi-cool.txt Page 214 of 271 Printed: Tuesday. May 27. because 'find' evaluates AND before OR. Compare running this command without option '-x'. -mtime +1 -a -size -10000 .psd Find those files modified more than one day ago AND smaller than 5M: $ find . $ find * -prune -size +10 -ls . -mtime -1 -a -size +10000 . Use option '-x' to stop 'find' from looking inside mounted file systems: $ ls /Volumes/ Inside Mac Media OSX-saruman guest $ find -x /Volumes /Volumes /Volumes/guest /Volumes/Inside Mac Media /Volumes/OSX-saruman Only /Volumes is searched./ferdi-cool./ferdi-gala.Unix_tricks.psd If the expression were of the form: (a -o b) -a (c -o d) the brackets would be necessary. 2008 3:20:00 PM Find those files greater than 5M in size: $ find . $ find ./ferdi-cool./ferdi-cool.psd . \( -mtime -1 -size +10000 \) -o \( -mtime +1 -size -10000 \) . -mtime -1 -size +10000 -o -mtime +1 -size -10000 . recursing.psd Find those files modified less than one day ago AND bigger than 5M: $ find ./ferdi-gala.psd Note the use of brackets (escaped from the shell) to ensure that the AND and OR expressions are evaluated in the correct order ./ferdi-cool.psd Also. then the results ORed. the three mounted file systems within it are not. the expression in this case can be shortened to: $ find .psd ./ferdi-gala. -size +10000 .psd .that is the two ANDs are evaluated. Curtailing Find Stop 'find' from traversing other file systems. \( -mtime -1 -a -size +10000 \) -o \( -mtime +1 -a -size -10000 \) . the brackets can be omitted. Because 'find' assumes AND by default. and following symbolic links.

psd .psd .psd . -name "*.psd .psd . Jan 23 11:59 ferdi-cool. -prune -size +10 -ls will not work because 'find' will not recurse into the specified directory './ferdi-coll3.psd ..'./ferdi-polish.psd" This will: $ find -H * -name "*.psd" ./ferdi-gala.Trash .psd ./ferdi-polish. -maxdepth 1 -size +10 -ls Make find follow symbolic links./ferdi-coll3.. -name "*.. '-prune' stops find from recursing into each directory.. rather than symbolic links found during the search.psd . Option '-H' tells 'find' to follow symbolic link given on its command line. Jan 15 12:12 ferdi-coll3.psd . Find does not by default follow symbolic links ..psd . This will not follow 'home': $ find -H . Jan 15 12:12 ferdi-gala. Jan 23 15:59 ferdi-polish./home/Pictures/complete/ferdi-gala..psd" ../ferdi-gala./home/Pictures/complete/ferdi-polish./ferdi-cool. Jan 23 16:46 home -> /Users/saruman $ find .psd . and lists their details with '-ls'...psd .'home' in this example..psd . $ du -sk ~/* | sort -nr | head -n 10 2252708 /Users/saruman/Pictures 490664 /Users/saruman/Library 186164 /Users/saruman/Sites 132596 /Users/saruman/Development 46928 /Users/saruman/Documents 33216 /Users/saruman/osxfaq 14372 /Users/saruman/Movies 12304 /Users/saruman/../home/Pictures/complete/ferdi-coll3./ferdi-cool. This will work: $ find .psd" Find and Du Find the top ten largest files/directories.psd Use option -L tell 'find' to follow symbolic links: $ find -L . $ ls -l total 49700 .. Note that: $ find .psd .psd .Unix_tricks. May 27./home/Pictures/complete/ferdi-cool. -name "*.txt Page 215 of 271 Printed: Tuesday. 2008 3:20:00 PM This finds all files in the current directory greater than 5K.

Trash 10252 /Users/saruman/Desktop (options '-print0' and '-0' cater for filenames with spaces .) Summarise to a greater depth.see the man pages or previous weeks.txt Page 216 of 271 Printed: Tuesday. Find all directories called test. May 27.Unix_tricks. '-s' summarises presenting just the total size for directories. 'sort -nr' sorts into numeric reverse (descending) order.gimp-2.*/test/.0 'du' (Disk Usage) displays the size of all files/directories (~/*). Previous week's covered '-name' and '-iname' which match against names of found files. Improve this with 'find' to include only directories. 2008 3:20:00 PM 10252 /Users/saruman/Desktop 580 /Users/saruman/. $ find ~ -type d -maxdepth 2 -print0 | xargs -0 /usr/bin/du -sk | sort -nr | head -n 20 3181012 /Users/saruman 2252708 /Users/saruman/Pictures 1772768 /Users/saruman/Pictures/iPhoto Library 490664 /Users/saruman/Library 378168 /Users/saruman/Library/Caches 308072 /Users/saruman/Pictures/complete 186164 /Users/saruman/Sites 142848 /Users/saruman/Sites/albums 132596 /Users/saruman/Development 80224 /Users/saruman/Development/developer-downloads 77160 /Users/saruman/Pictures/people 69472 /Users/saruman/Library/Mail 54920 /Users/saruman/Pictures/web-site 50224 /Users/saruman/Development/test 46928 /Users/saruman/Documents 33216 /Users/saruman/osxfaq 26304 /Users/saruman/Pictures/Photoshop-effects 20076 /Users/saruman/Library/Application Support 19100 /Users/saruman/Documents/1dot1 19012 /Users/saruman/osxfaq/sf03 Searching the Path Use '-regex' to filer against the whole path. $ find ~ -regex ". wherever they are in the directory hierarchy. $ find ~ -type d -maxdepth 1 -print0 | xargs -0 du -sk | sort -nr | head -n 10 3181012 /Users/saruman 2252708 /Users/saruman/Pictures 490664 /Users/saruman/Library 186164 /Users/saruman/Sites 132596 /Users/saruman/Development 46928 /Users/saruman/Documents 33216 /Users/saruman/osxfaq 14372 /Users/saruman/Movies 12304 /Users/saruman/. and 'head -n 10' displays just the first 10 lines.*" will find: /Users/saruman/Development/test/* . '-regex' matches the entire path.

Unix_tricks. and '-iregex' says to match in a case-insensitive manner.psd . $ ls a* a"b a\b $ cat a'"'b double quote in name $ cat a\"b double quote in name $ cat a'\'b back-slash in name $ cat a\\b back-slash in name Find. type the first part of the filename and hit tab to auto-complete it.psd ferdi-cool.. ferdi-coll3. However. often by passing the filenames to 'xargs'.psd /Users/saruman/Development/test/find/ferdi-gala.psd ferdi-polish.psd ferdi-gala.psd /Users/saruman/Development/test/find/ferdi-cool.psd /Users/saruman/Development/test/find/ferdi-polish. Escaping can also be used to address a filename that includes odd characters such as " or \. Escape spaces in filenames. 2008 3:20:00 PM and /Users/saruman/Pictures/Gimp/test/* Also. May 27.txt Page 217 of 271 Printed: Tuesday. it will fail for filenames with spaces: $ ls -1 file* file one file two .psd" -execdir echo {} \. $ find ~ -name "*. Easily remove the full path name.psd vs the regular: $ find ~ -name "*. This week presents some solutions.psd" /Users/saruman/Development/test/find/ferdi-coll3. option '-E' says to use extended regular expressions. and Spaces 'Find' is a command often used to root out particular files in a directory hierarchy and process each file. To enter a filename that includes spaces on the command line use: my\ file "my file" or 'my file' Alternatively.. Xargs. On the Command Line Coping with spaces in filenames is problematic for both the command line beginner and the more experienced scripter.

May 27. 1 saruman saruman 0 Feb 7 14:34 . do echo "File: $i" done $ ./file: No such file or directory ls: two: No such file or directory Fix this using the '-print0' option of 'find'. To process these two files: $ ls -1 file* file one file two This will not work: $ cat loop #!/bin/bash ls=$(ls file*) for i in $ls. 1 saruman saruman 0 Feb 7 14:34 .txt Page 218 of 271 Printed: Tuesday. 2008 3:20:00 PM $ find ./loop File: file File: one File: file File: two This will: $ cat loop #!/bin/bash for i in file*./loop File: file one File: file two When filename expansion is done directly in the loop each filename is fed to the loop individually. and the '-0' option of 'xargs'. Going Loopy Choose the correct looping method to preserve spaces in filenames. hence there is no confusion between spaces in filenames and spaces as separators in ./file two '-print0' tells 'find' to null-separate filenames when it produces the list of found files. '-0' tells 'xargs' to expect null-separated filenames. do echo "File: $i" done $ . -name "file*" -print0 | xargs -0 ls -l -rw-r--r-./file one -rw-r--r-. instead of space-separating them. $ find . -name "file*" | xargs ls -l ls: .Unix_tricks./file: No such file or directory ls: one: No such file or directory ls: .

/file one File: . do echo "File: $i" done $ . $ cat find-and-loop #!/bin/bash list=$(find . Normally the separator is any of space. This will not work: $ cat . tab. File: . ./find-and-loop File: . 'Find' (and 'ls' ) produce new-line separated lists. -atime -1). Bash's IFS Wednesday's example cannot be used if the list of filenames is generated by a method more complex than wildcard expansion: for example using 'find./find-and-loop #!/bin/bash list=$(find ./file File: one File: . May 27. 2008 3:20:00 PM a list. When an intermediate variable is used the loop is fed a space-separated list./find-and-loop File: . do Get clever with Bash's Internal File Separator (IFS). -atime -1) for i in $list.txt Page 219 of 271 Printed: Tuesday. -atime -1 ) IFS=" " for i in $list./file two Note: There is a single newline character between the quotes in IFS=" " Parameters With Spaces Process parameters with spaces. so changing the IFS to just new-line does the trick. do echo "File: $i" done $ . or newline./file File: two Neither will using: for i in $(find . File: .Unix_tricks.

Unix_tricks. do echo "Parameter: $i". do echo "Parameter: $i". done echo echo 'For "$@"' for i in "$@". do echo "Parameter: $i"./params "param one" "param two" For $@ Parameter: param Parameter: one Parameter: param Parameter: two For "$@" Parameter: param one Parameter: param two . 2008 3:20:00 PM This will not work: $ cat params #!/bin/bash echo 'For $*' for i in $*.txt Page 220 of 271 Printed: Tuesday. we use Bash's special $@ symbol. do echo "Parameter: $i". done $ . done $ . $ cat params #!/bin/bash echo 'For $@' for i in $@. done echo echo 'For "$*"' for i in "$*"./params "param one" "param two" For $* Parameter: param Parameter: one Parameter: param Parameter: two For "$*" Parameter: param one param two Neither solution gives us: Parameter: param one Parameter: param two Instead of $* to represent all parameters. May 27.

0. using the following escape sequences.0.t^[[ and X11 xterm.t"' Then use: $ sz 50 100 $ sz 25 80 These escape sequences work in both Apple's Terminal. For example. or within a script.\!:1.t" When re-sizing to the maximum screen width and/or height.50.10. (See Monday's tip too.0.t^[[8. May 27.0. Focus and Dock with Escape Sequences Move a window to the background or the foreground from the command line.$2.Unix_tricks. using the following escape sequences. it is advisable to position the screen at the top left: $ echo -n "^[[3.\!:2.0.0.) . or from a script.100t" (Note: the sequence displayed as ^[ represents the escape character.0t" Re-size to the maximum screen height by giving rows as 0: $ echo -n "^[[8. Re-size and Move with Escape Sequences Re-size and move a terminal window from the command line.txt Page 221 of 271 Printed: Tuesday.) Re-size to the maximum screen width by giving columns as 0: $ echo -n "^[[8.0. 2008 3:20:00 PM The second variant "$@" give us what we want.0.100t" Move to a specific position (eg 10 pixels from the left.t" > } (tcsh) alias sz 'echo -n "^[[8. obtained by typing control-v then the 'esc' key.0.$1.0.0t"' To size a screen use a Bash function or tcsh alias: (Bash) $ sz () > { > echo -n "^[[8.0t" Define Aliases and Functions Define aliases and functions to move and resize.50. and 100 pixels from the top): $ echo -n "^[[3. Re-size to a specific number of rows and columns (eg 50 rows by 100 columns): $ echo -n "^[[8.0t"' (tcsh) % alias big 'echo -n "^[[3.t^[[8. to make a big screen use an alias: (Bash) $ alias big='echo -n "^[[3.

$ echo -n "^[[2t. Check 'Option click to position cursor. waits 5 seconds. Drag and Drop. Most visual aspects of the terminal can be changed from menu item Terminal::Window Settings. (Remember to add the & otherwise the first terminal will be locked running Xterm as a foreground task. then brings it back into has a few nifty features that you may or may not have encountered." The above command hides the terminal window. For example.) Xterm is hugely configurable as witnessed by 'man xterm' Start new Xterm terminals with Command-N." The above command docks the window. and you can type: $ xterm& to get a new Xterm window. You can now option click anywhere on the current command line to position the cursor at that point.20) . Terminal. Split Screen. Set a background.".) Move a window into the Dock. including setting a background picture. but there are many other alternatives. May 27.". obtained by typing control-v then the 'esc' sleep 5. Alternatively Alt-drag a picture onto the Terminal window to instantly set a background image. Xterm is started automatically when X11 starts up. Drag any file or folder (or multiple files and folders) onto the terminal window and the full path name will be written to the prompt line. For example. Select X11 menu Applications::Customise to add new commands to the Applications menu. If you have installed X11. This option has to be enabled in menu item Terminal::Windows Settings::Emulation. Click to position cursor. to list the contents of a folder. When the script completes the terminal window will re-appear to let you know. 2008 3:20:00 PM echo -n "^[[6t. Click the little 'split' icon situated top left above the vertical scroll bar. echo -n "^[[5t. positioned (40. then drag the horizontal divider to adjust the top/bottom split ratio. sleep 5. type 'ls ' then drag the folder from the Finder onto the Terminal window and hit return. add: Name: xterm Command: /usr/X11R6/bin/xterm -sb -sl 5000 -rightbar -fg white -bg black -geometry 100x50+40+20 Shortcut: n Now Command-N will start a new Xterm window sized 100 columns by 50 rows. These sequences can be places in bash/tcsh aliases. echo -n "^[[ Nifty Features Apple's Terminal. Alternative Terminals The standard terminal for OS X is Apple's Terminal. The top pane can be used as a scroll-back to view and copy previous commands without disturbing the main window. (Note: the sequence displayed as ^[ represents the escape character. you will have xterm. then after 5 seconds brings it back into focus. The 'sleep 5' part could be a script that takes a while to run.txt Page 222 of 271 Printed: Tuesday.Unix_tricks. This also works in text editors such as Pico and Vim.

The . AppleHighlightColor = "0.776500 0. It interprets the XML encoding allowing one to easily read and write property values. The 'defaults' command provides easy access to the OS X preference files. like many routers. watch the eyes watching your cursor. It supports language encodings. For newer Macs that don't have a normal serial port.Trouble-Shooting II (System). ZTerm can talk to ports on USB to serial adapters. . '-sb -sl 5000 -rightbar' sets a scroll bar on the right-hand side.Trouble Shooting IV (Advanced). Xeyes is fun. Its still a useful utility for those systems that only offer dialup connections and for connecting to devices through a serial port. AppleCollationOrder = "en_GB".776500 0. Zterm ZTerm is a terminal emulation program for the Macintosh. many people used it to connect to Bulletin Board Systems and download files. iTerm iTerm is a full featured terminal emulation program written for OS X using Cocoa.Unix_tricks. and a scroll-back buffer of 5000 lines. $ man defaults View and change the NSGlobalDomain.Plist Files This week's tips are an accompaniment to the Daily Mac OS X tips comprising weeks 28 February - Trouble-Shooting (Applications).txt Page 223 of 271 Printed: Tuesday. Set up a customised command as follows: Name: xman Command: /usr/X11R6/bin/xman -bothshown -notopbox Shortcut: whatever-you-want or blank iTerm is an alternative terminal. Now we have the internet. a nice front end to the Unix manual. through the appropriate driver software supplied with the adapter. In its day. $ defaults read -globalDomain { AppleAntiAliasingThreshold = 8. Set up a customised command as follows: Name: eyes Command: /usr/X11R6/bin/xman -bothshown -notopbox Shortcut: whatever-you-want or blank Xman is useful. May 27. Zterm is useful if you need to talk via a serial or USB port (to configure routers for example). 14 March - Trouble-Shooting III (Hardware). 7 March . '-fg white -bg black' sets white text on a black backgound. 2008 3:20:00 PM pixels from the top left of the screen (-geometry 100x50+40+20).776500". or create new property values. and 21 March . This contains default preferences that apply to an application if it doesn't override them it its own domain. The Unix trouble-shooting tips give command-line alternatives to methods discussed in the OS X Daily tips. AppleAquaColorVariant = 1. VT100/ANSI/XTERM emulation and many convenient GUI features.

1 = true/yes. "56C0BB88-C7D2-11D6-950E-000393B2D604:ABPerson" )..iChat AutoLogin 0 Note: 'defaults' automatically looks in ~/Library/Preferences and adds the .... May" = 3.iChat AutoLogin 0 $ defaults read com.'dd".. 3 = "d' 'MMMM'. View a single property: $ defaults read com. Use less to view: $ defaults read com.plist file with XML decoding. AutosaveChats = 1. AutoLogin = 1.. WARNING: don't modify the . use: $ defaults write domain 'the-plist-from-defaults-read' Use vim to view: $ defaults read com. "BuddyList. This translates the XML encoding into something more readable. 'yyyy". .iChat | less .txt Page 224 of 271 Printed: Tuesday. .apple. 'd' 'MMMM'. 2008 3:20:00 PM AppleICUDateFormatStrings = { 1 = "yyyy'.PLIST FILE DIRECTLY.plist Change a property: $ defaults write CardsBlockingPresentityPictures = ( "60765A82-A567-11D7-A842-000393B2D604:ABPerson".Unix_tricks. BuddyInfoSelectedTab = 2 = "d' 'MMM'. 4 = "EEEE'.'MM'.SecondarySortOrder" = 1. 'yyyy". "BuddyList. " NOTE: DO NOT WRITE THE DECODED VERSION BACK TO THE ORIGINAL . $ defaults read com. ..iChat | vim - Vim: Reading from stdin.Visible" = 1. 'yyyy".iChat { ABDirectoryResultColumnTitle = "Instant Messaging".plist of a running application.iChat AutoLogin 1 Note: 0 = false/no. $ defaults read -globalDomain AppleLocale en_GB $ defaults write -globalDomain AppleLocale en_US $ defaults read -globalDomain AppleLocale en_US View a .

Problems involving incorrect icons or documents failing to open (with the proper application) can be solved using 'lsregister' $ /System/Library/Frameworks/ .) Killall can match process names by regular expressions too .Unix_tricks.iChat. The 'kill' command requires a process ID. The 'lsregister' command can be launched from the command-line.67 iCal (Put the application name in quotes if it contains spaces or characters special to the shell. not an application name. $ plutil ~/Library/Preferences/com. The Finder also uses this information to associate icons with documents.txt Page 225 of 271 Printed: Tuesday.check out the man page. and the 'open' Kill a process by name. You must either be root or use 'sudo' to issue the 'kill' command.plist Option '-s' reports only error conditions. Discover it with: $ ps xc | grep -iw ical 6577 ?? S 0:41. } $ killer ical Kill applications belonging to other users. May 27. 2008 3:20:00 PM Use plutil to verify the syntax of .plist /Users/saruman/Library/Preferences/com.plist: OK $ plutil -s ~/Library/Preferences/*. To shut down the computer from the command line: $ sudo shutdown -r now Launching and File Association The Quit it with: $ kill -QUIT 6577 Force-quit it with: $ kill -KILL 6577 Do it on one line by creating a bash function: $ function killer () { kill -KILL $(ps xc | grep -wi "$*" | awk '{print $1}').iChat. $ killall ical No matching processes belonging to you were found $ killall iCal (Killall is case-sensitive.plist files. and gives scope for manual maintenance of the Launch Services database. Force-Quit Use 'kill' or 'killall' to force-quit errant applications. and is silent on success. Add option 'a' to the 'ps' command so it lists processes owned by other users too. both use Launch Services to associate documents with applications. Use the 'killall' command.

May 27. but it does have potential for querying.the/long/path/lsregister -kill -r -domain system -domain local -domain user will repair the Launch Services database. -lazy n Sleep for n seconds before registering apps if the local cache is aleady populated. weekly. Scheduling is controlled by the system 'cron' scheduler. -h Display this help. Keep It Tidy Unix is pretty much self-maintaining. 15 3 * * * root periodic daily 30 4 * * 6 root periodic weekly 30 5 1 * * root periodic monthly If your Mac is switched off during these hours you may want to re-schedule the periodic maintenance. I've not investigated this command much. -dump Display full database contents after registration... which reads its scheduling information from /etc/crontab: $ cat /etc/crontab # /etc/crontab SHELL=/bin/sh PATH=/etc:/bin:/sbin:/usr/bin:/usr/sbin HOME=/var/log # #minute hour mday month wday who command # # Run daily/weekly/monthly jobs.txt Page 226 of 271 Printed: Tuesday.. -r Recursively register directory contents. and is scheduled to run in the early hours daily. ask CF for the list of application locations in the given domain(s). maintaining.framework/Versions/A/ Frameworks/LaunchServices. -kill Reset the global Launch Services database before doing anything else -lint Print information about plist errors while registering bundles -convert Register apps found in older LS database files -load Load the LaunchServices service plugin if it's not already loaded...Unix_tricks. and monthly. -R Recursively register directory contents. For domain specifications. do not recurse into packages or invisible directories. -f force-update registration info even if mod date is unchanged -v Display progress information. A little periodic maintenance is required. 2008 3:20:00 PM ApplicationServices. Issuing the command: . Search the paths for application bundles and add each found item to the Launch Services database.. and resetting the LS database. See: $ man 5 crontab .framework/Versions/ Current/Support/lsregister (all on one line) lsregister: [OPTIONS] [-domain { system | local | user | network }]. [path]. including the contents of packages and invisible directories.

$ fink describe anacron Information about 1951 packages read in 4 seconds.slt | head -1)/Cache \ && echo removed. Internet Explorer. #!/bin/bash if [ "$1" = "-usage" ]. iCab. Opera.3-4: Periodic command scheduler Anacron executes commands at intervals specified in days. May 27. then echo "Cleans the caches for the browsers:" echo "Camio. As a consequence browser caches greatly increase the time required for incremental backups.. It is like cron but catches up on scheduled events that were missed when your Mac was asleep or not running. For example. This can be obtained from the many Unix ports.log Get more details with: $ find /var/log -size +20000 -ls Install 'anacron'. Opera 7.log /var/log/secure. iCab." { cd ~/Library/Application\ Support/Chimera/Profiles/default/ \ && rm -r $(ls -d *. I run almost all browsers to test my web development. OmniWeb5. including fink.. Mozilla.. Firebird. it does not assume that the system is running continuously. Omniweb 5. Alternatively. I have written a simple script that removes the caches for Camio. IE. } 2> /dev/null \ || echo "none" . find all log files over 10 Meg: $ find /var/log -size +20000 /var/log/named/default. and Safari. Unlike cron..Unix_tricks." # Camio echo -n " Camio . Remove Browser caches. run the maintenance manually with: $ sudo periodic daily $ sudo periodic weekly $ sudo periodic monthly Other candidates for removal include log files in /var/log that are not periodically zipped and rotated.txt Page 227 of 271 Printed: Tuesday. It's ideal for machines such as laptops. Safari" echo "Usage: ${0##*/}" exit fi # just in case! cd ~/Library || exit 1 echo "Removing caches. anacron-2. Mozilla. 2008 3:20:00 PM for an explanation of the format of crontab. FireFox.

" { rm -r ~/Library/Preferences/iCab\ Preferences/iCab\ Cache/ \ && echo removed. } 2> /dev/null \ || echo "none" # Safari echo -n " Safari ." { cd ~/Library/Mozilla/Profiles/default/ \ &&| rm -r $(ls -d *.Unix_tricks. } 2> /dev/null \ || echo "none" echo "Done." { rm ~/Library/Caches/MS\ Internet\ Cache/IE\ Cache.omnigroup." { rm -r ~/Library/Caches/Opera\ Cache \ && echo removed. } 2> /dev/null \ || echo "none" # Mozilla echo -n " Mozilla ." { rm -r ~/Library/Caches/com.waf \ && echo removed. } 2> /dev/null \ || echo "none" # Opera echo -n " Opera ." . May 27. } 2> /dev/null \ || echo "none" # iCab echo -n " iCab .OmniWeb5/Cache.txt Page 228 of 271 Printed: Tuesday. 2008 3:20:00 PM # Firefox echo -n " Firefox ." { rm -r ~/Library/Caches/Safari \ && echo removed.slt | head -1)/Cache \ && echo removed." { cd ~/Library/Application\ Support/Firefox/profiles/*default/ \ && rm -r Cache \ && echo removed.bundle \ && echo removed. } 2> /dev/null \ || echo "none" # OmniWeb echo -n " Omniweb . } 2> /dev/null \ || echo "none" # Interner Explorer echo -n " Internet Explorer .

Checking Extents Overflow file. Checking volume bitmap. Verify/repair finished on disk disk0s9 Music-saruman $ diskutil verifyDisk /dev/disk0s9 Started verify/repair on disk disk0s9 Music-saruman Checking HFS Plus volume. Checking Catalog hierarchy. May 27. Checking volume bitmap. fsType = 'hfs'. specify its mount point. and verify and repair the structure of other volumes. but I'm not sure with the Mozilla set of browsers how much the random folder names vary. Disc Drives /Applications/Utilities/Disk Utility allows one the verify and repair permissions on the system volume. volName = 'Music-saruman') $ diskutil verifyDisk disk0s9 Started verify/repair on disk disk0s9 Music-saruman Checking HFS Plus volume.. volName = 'Users-saruman') ***Disk Appeared ('disk0s7'. Verify/repair finished on disk disk0s9 Music-saruman To repair use: $ diskutil repairDisk /Music Started verify/repair on disk disk0s9 Music-saruman Checking HFS Plus volume. fsType = 'hfs'. but in my case I have mounted directly under /. volName = 'Games-saruman') ***Disk Appeared ('disk0s9'. or device node.. fsType = 'hfs'.. The mount point is usually /Volumes/disc-name. . ***Disk Appeared ('disk0s3'.Mountpoint = '/Games'.Mountpoint = '/'. Checking volume information. Checking Catalog hierarchy.Mountpoint = '/Music'. These functions can be performed from the command line using 'diskutil' To refer to a specific disc/volume.. Checking Extents Overflow file..Unix_tricks. . Checking volume information. Checking Catalog file. as below. Checking Catalog file.Mountpoint = '/Users'.txt Page 229 of 271 Printed: Tuesday. The volume Music-saruman appears to be OK. $ disktool -l . volName = 'OSX-saruman') ***Disk Appeared ('disk0s5'. Verify/repair finished on disk disk0s9 Music-saruman To discover the disc identifier and device node use disktool. fsType = 'hfs'. Verify/repair finished on disk disk0s9 Music-saruman . disc identifier. 2008 3:20:00 PM The locations of the caches work for me. $ diskutil verifyDisk /Music/ Started verify/repair on disk disk0s9 Music-saruman Checking HFS Plus volume. The volume Music-saruman appears to be OK..

txt Page 230 of 271 Printed: Tuesday. or use command 'df' in single user mode. May 27. list the contents of the current directory Option -l (ell) to list with details Option -a to list hidden dot files. or type: $ reboot Repair non-boot volumes. -A to miss out '. for journalled filesystems (the default in Pather): $ fsck -fy If fsck reports problems with something like: ***** FILE SYSTEM WAS MODIFIED ***** re-run fsck until it reports that no modifications were necessary. It is not possible to repair a volume that is mounted and cannot be unmounted (ie is always in use). ls . or Option -G to identify with colour . don't forget the Unix manual 'man'. Restart holding down the Command-S key combination and wait until the text has finished scrolling.Unix_tricks. Next mount the system volume as writable (it is currently mounted as read-only at this stage of the boot process). There are so many commands available that it is impossible to know them all. For example. Sometimes it is not possible to repair other volumes because they cannot be unmounted (for example if you have your home directory on another partition). use: diskutil verifyPermissions /OSX-saruman diskutil repairPermission /OSX-saruman NOTE: the name of your system volume will differ from mine.' Option -L to follow symbolic links Option -F to identify different types of files. To repair the boot volume you must boot into single user mode. to repair HFS disc 0 partition 9: fsck_hfs -fy /dev/rdisk0s9 Refer to 'disktool' above to get the disc and partition numbers.' and '.. See the rest of the tips for lots of example usage of these commands.. and explanations of Unix permissions. And of course. Hit control-d to continue the boot process. Type: $ mount -uw / Then check/repair the file system with: $ fsck -y or. Basic File and Directory Commands The next four weeks will form a Unix command reference by listing lots of commands under specific areas of usage. or even remember all those one knew and used just last month. 2008 3:20:00 PM To verify and repair permissions (of the boot volume).. See the Learning Centre for tutorials on these commands. In this case the offending volume has to be repaired in single user mode just like the system volume.

..... move it to another directory.. move or rename files and directories Rename a file... change current directory 'cd. and -h as for 'chmod' Change both user and group owners by specifying user-owner:group-owner chgrp . delete files and directories Option -R or -r to recursively move whole directory subtrees Option -f to override permissions where possible Option -i for interactive confirmation of each deletion cd . move back to the last pushed directory 'pushd' and 'popd' work on a stack system where the last directory pushed is the first to be popped dirs . change file and directory permissions Option -R to change permissions in a whole directory subtree Option -L to follow symbolic links Option -h to change the mode of a link instead of the file it points to chown .. change owners of files and directories Options -R.. display a file .. and -h as for for 'chmod' chflags . that can be set using this command Options -R and -L as for for 'chmod' Commands to View Files cat ..txt Page 231 of 271 Printed: Tuesday...... print the directory stack This is the stack is written by 'pushd' and read by 'popd' (See tips weeks 30 and 32) mkdir . print working directory Display the current directory pushd . archived. delete a directory The directory must be empty Use 'rm -r' to delete non-empty directories chmod ... or do both at the same time rm .. -L...Unix_tricks.. change directory and stack it Each time pushd is issued.. create a new directory Option -p to make many (nested) directories in one go rmdir ... May 27.' to move up a directory 'cd' to move to your home directory 'cd -' to move back to the last directory pwd ... change group owner of files and directories Options -R. copy files and directories Option -R for recursive copy to copy whole directory subtrees mv . 2008 3:20:00 PM (See tips week 24) cp . the current directory is remembered on the directory stack Use popd to retrace your steps popd . append-only.. change the file flags Unix files and directories have many flags such as immutable.. -L..

word.. display a file in a paged manner Like more. not just binary sections (applicable mainly to executables) hexdump .. display a file in a paged manner Scroll up and down with the cursor keys Page up and down with the 'u' and 'd' keys Search for text with /text-to-search-for And lots. cat... powerful text editor vi/vim .. and 69) . display files with pagination Display files page by page with headers and footers vis . but much more than more See 'man less' lesskey .... display binary files The output shows the hexadecimal value of each byte Option -n len to display only the first 'len' bytes xxd .. zmore . May 27... display the last 10 lines of a file Option -n specifies a different number of lines to display (See tips week 4) zcat. bzless . display files with non-printable characters Does a better job than 'cat -v' strings ... search a binary file for printable strings Option -a to search the whole file.. 53. and -c to control what is counted file . cat and more zipped files bzcat. determine the type of a file pr . 2008 3:20:00 PM Cat actually concatenates many files and writes the result to standard out (the screen) Redirect to a file to actually join files Option -b to number non-blank lines Option -s to collapse multiple blank lines Option -v to display non-printing characters so they are visible more . specify key binding for less Specify alternative key binding for controlling less head . line.. more. display the first 10 lines of a file Option -n specifies a different number of lines to display tail .. the best text editor (See tips weeks 50. see 'man more' less .Unix_tricks. bzmore.txt Page 232 of 271 Printed: Tuesday..... less on bzipped files wc ..' for non-printable characters) (See tips week 95) emacs ... lots *more*.. display binary files Like 'hexdump' but displays both the hexadecimal and character representation of each byte (or '. -w.. and character count files Options -l........

..Unix_tricks... deleted lines Use to compare two versions of the same file Option -b to ignore differences in white space Option -B to ignore differences in blank lines Option -i to ignore changes in case diff3 . diff zipped.. or key Option -t to specify the key separator (space.. file 2 only.txt Page 233 of 271 Printed: Tuesday. display lines common to two files The two files must be sorted first Three columns are produces for lines that are in file one only. compares three files Comparison between three files f1... compare and merge two files Compare the differences between two files and merge them to a third file Option -b to ignore differences in white space Option -B to ignore differences in blank lines Option -i to ignore changes in case zdiff.. bzdiff . a simple text editor Commands to Compare Text Files split . -3 to suppress printing of their respective columns diff .. so you can sort on any column..... perform a database join on files Specify a field (as for 'sort') by which the files are compared A line is displayed for each pair of lines in the two files where the compared fields match paste . compare two files Compare two files and print the differences in terms of small differences in lines...and more... new lines. orig... command. tab..this acts like a merge operation to merge the changes made to f2 (relative to orig) into f1 sdiff . and f2 compares the differences between f1 and f2 relative to orig Option -A to incorporate all the changes between orig and f2 into f1 . filter out repeated lines of a file The file must be sorted first For repeated lines only the first copy is written out Option '-f fields' to ignore the first 'fields' fields of each line in the comparison Option '-s chars' to ignore the first 'chars' characters of each line in the comparison Option -c to precede each output line with the number of repetitions in the original input join .. -2. uniq . etc) Option -k to specify which key/column to sort on Option -b to ignore leading blanks Option -f to fold case (make the sort case insensitive) .. merge corresponding lines of two file comm . split a file into many files Split a large file into many small files each 1000 lines long Option -l to specify a different number of lines for the small files sort .. sort the lines of a file into alphabetic order Each line is considered a number of columns.. and both files Options -1.. bzipped files Commands to Search for Content . May 27.. 2008 3:20:00 PM pico .

2008 3:20:00 PM grep .txt Page 234 of 271 Printed: Tuesday. search a file for text The same as 'grep -E'. extracting information from those text files.. (See tips weeks 85 and 89) tr ..... strip chars Col can be used to clean-up files Option -b to remove multiple backspaces as output by the man pages . stream editor Sed reads a text file line by line and applies a series of search. pattern scanning and manipulation language Awk reads a text file line by line and applies a series of search. bzgrep . so redirection is necessary to work on files Don't write the output back to the input file. nawk . or a pattern specified by a regular expression (see 'man re_format') Option -i to specify case-insensitivity Option -w to match whole words Option -r to search all files in a directory subtree Option -v to display lines that do NOT match Options -n.. It's also a good language for many system management tasks.. replace... search a file for text Search a file for specific text. complete) rather than beautiful (tiny... otherwise it will end up empty col .Unix_tricks.. fgrep a bzipped file perl ... i. Awk can read a script file for its processing commands the language combines a 'C'-like programming language with regular expression search and replace. (See tips weeks 86 and 87) gawk.. replace. minimal). elegant. efficient. and line numbers Option -E to use extended regular expressions instead of basic Option -F to use fixed strings (this is much quicker than pattern matching) egrep . translate on character into another Reads a text file and translates all occurrences of a given character into another By default reads and writes standard in and out.. and edit operations to lines that match given criteria Sed can read a script file for its processing commands the language uses regular expression search and replace functionality. The language is intended to be practical (easy to use.. bzipped file bzegrep ... egrep a bzipped file bzfgrep . -L. a text manipulation language From the man pages: Perl is a language optimized for scanning arbitrary text files. May 27. and edit operations to lines that match given criteria. and printing reports based on that information.. -H to control the what is printed in terms of matched text.. -h. grep a zipped. variations of awk Not available in the standard Mac OS X install sed . filenames..e it used extended regular expressions fgrep . but lacks the 'C'-like programming language... search a file for text The same as 'grep -F' zgrep. Commands to Change File Content awk . -l.

fold long lines Simple version of fmt which breaks long lines to 80 characters or less Option -w to specify a maximum width other than 80 Commands to Search for Files Last week and the next three weeks will form a Unix command reference by listing lots of commands under specific areas of usage.Unix_tricks. or optionally do not come in under a minimum specified length Option -c to centre text Option -p to allow indented paragraphs Option -s to collapse multiple white space inside lines fold . There are so many commands available that it is impossible to know them all. '-f 3-5' for 3 to 5.. like '-d " "' for space....3-5' for 1 and 3 to 5. use: find . or '-f 1. permissions. which specify which files are matched based on many. size. May 27.. type. See the rest of the tips for lots of example usage of these commands.txt Page 235 of 271 Printed: Tuesday. like '-f 3.. See the Learning Centre for tutorials on these commands. update the locate database in (See tips week 1) The best way to update the locate database database to avoid revealing private filenames is by ."' for comma expand ... many criteria such as name... filters columns from files Cut can process a file that has columns of text (for example tab-separated tables) and extract just the specified columns Option -f to list the columns to be output. owners.5' for 3 and 5. 97...updatedb . compress spaces to tabs fmt . To search for files in the files system. 13. don't forget the Unix manual 'man'. 2008 3:20:00 PM Option -x to convert tabs characters to multiple spaces (See tips week 10) cut . or '-d ". find files matching a number of criteria Option -E to specify extended regular expressions Option -L to follow symbolic links for found files Option -X when find is used with xargs Option -s to search hierarchies in lexicographical order Check out the many primaries. format a text file Read a file and write a version who's lines do not exceed a maximum specified length. 98) locate ... 31.. or even remember all those one knew and used just last month. search a database of files This relies on the locate database being up to date (See tips week 1) locate. time/date.. expand tabs to spaces unexpand . 49. etc (See tips weeks 1. And of course. Option -d to specify the field separator (by default tab). and explanations of Unix permissions.

-K to timeout the user's password to sudo users ... find an executable (search system path) Also: ls -r * .plan and ~/...project files to advertise more information to finger passwd ...... switch to another user's identity sudo . change the password of the current user chpass.see under 'Networking Commands' later in the series. display the current user's login name finger ..........Monday) whereis . display identity of the current user Option -p to display in a more readable format Option -G to display group numbers. 2008 3:20:00 PM the command: $ sudo periodic weekly (See tips weeks 64... useful for shell scripts groups .. . as 'users' but with more detail . 88) type .... list all files in the current directory Commands About and For Users id .. change the umask for this and child processes (See week 47 . log into the machine Option -p to inherit the current environment rlogin .Wednesday) login ..... display groups to which a user belongs (See tips week 17) logname .. execute a command as root Option -s to run a shell as the new user Option -u to run as a specified user rather than root Option -l to list what you are allowed to sudo Options -k. May 27.Unix_tricks. add or change user information Check the man pages for compatibility with OS X umask . log into a remote machine This is depreciated in favour of the more secure 'ssh' .txt Page 236 of 271 Printed: Tuesday. find an executable (bash shell only) Option -a to find all versions of the executable (See tips week 77 . chsh. su .... find an executable (csh and tcsh) (See tips weeks 20 and 47 . display who is currently logged in who .. chfn. recursive directory listing echo * . display information on a user Create ~/.Tuesday) which ...

... remove an 'at' job form the queue atrun . send a message to another user Use 'who' to list other users and their consoles (See tips week 95 .... list the pending 'at' jobs atrm . execute a command at a specific time atq . scheduling and reminder service (See tips week 41) cron . allow/disallow messages from other users Useful Utility Commands man ..txt Page 237 of 271 Printed: Tuesday. chat to another user Use 'who' to list other users and their consoles mesg ... as 'who' but with even more detail whoami.... May 27.. send a message to all logged in users talk ..... and status (See tips week 17) write .. the Unix manual Option -k to search the manual Option -f to report a one-line summary of a command apropos ....... 2008 3:20:00 PM w ....Unix_tricks. the daemon that launches 'at' jobs This should be added to the system crontab and run periodically (say every 5 minutes) to schedule 'at' jobs. list all logins with date. the equivalent of man -k This and 'man -k' search for the given command as a string and will display all manual entries containing the string whatis . useful for amnesiacs last .. system task scheduling daemon ... info .Friday) wall ... calendar .. create formatted version of man pages This speeds up the 'man' command. who am i . (re)build the whatis database (See weekly tips weeks 2 and 20) catman .. time.. the GNU on-line manual Try 'info info' for more details Try 'info command-name' for information on the (GNU) command at .... the equivalent of man -f This and 'man -f' search for the given command as a whole word makewhatis .

....txt Page 238 of 271 Printed: Tuesday.... display a monthly.. user task scheduling daemon (See tips week 33) leave .Wednesday) banner . reset the terminal display Useful after accidentally printing a binary file that messes up the terminal's settings tty .. yearly calendar Option -j displays day numbers units .. an arbitrary precision calculator dc . compare and print terminfo descriptions captoinfo ..... query the terminfo database Commands to Control the Shell alias . convert terminfo description to termcap description tput .. May 27........ convert termcap description to terminfo database infotocap .... assign a function to a keyword . 2008 3:20:00 PM crontab .. write banner text to a printer bc .. convert quantities from one units system to another Commands to Control the Terminal clear . change the setting for the terminal Option -a to display the current settings Option -g to display in a form that can be used by -f Option -f to set the display reading commands from a file reset . display all terminfo entries by name Reads the compiled terminfo database in /usr/share/terminfo/. resize the terminal window Option -s to simply specify rows and columns $ resize -s 50 10 stty ..... infocmp .... clear the terminal window resize .... display the user's current terminal toe ....Unix_tricks.... alias a command line to a keyword (Bash and tcsh) function . tic . an arbitrary precision reverse polish calculator cal .. remind you when it's time to leave Do we need reminding of this? :-) (See tips wee w95 . terminfo compiler Reads terminfo descriptions and compiles them into the terminfo database in /usr/share/terminfo/.

set a shell variable unset . The load level is controlled by atrun. quit a login shell Commands for Info on Machine and Discs The last two weeks and the next two weeks will form a Unix command reference by listing lots of commands under specific areas of usage. quit non-login shell logout . remove a shell environment variable (tcsh only) (See tips week 54 ... remove a shell variable setenv . don't forget the Unix manual 'man'...txt Page 239 of 271 Printed: Tuesday.. 77. 20.. See the Learning Centre for tutorials on these commands.Tuesday) tee . refresh shell's executable list (tcsh) (See week 21 .... display the machine architecture . See the rest of the tips for lots of example usage of these commands. 71. 2008 3:20:00 PM (Bash only) (See tips weeks 12.. including hardware...... executes command in place of shell nohup ..Monday) expect .Unix_tricks. execute commands in non-busy periods The command is delayed until system load levels are low. execute a command immune to hangups The command will not be stopped bu 'kill -HUP' or when the current shell exits (See tips week 78) lockfile . operating system.. see Wednesday's tip. May 27.... or even remember all those one knew and used just last month... prevent file from being accessed by more than one process rehash .... script a dialogue with interactive programs script . to change shell key functionality batch . 99 .. and explanations of Unix permissions. split a pipe into to streams Can be used to direct the output of a command to a file and to terminal at the same time set .. For information on the machine (host). make a transcript of a terminal session bindkey .. exec .. set an shell environment variable unsetenv .Friday) hashstat . and kernel use: arch ... There are so many commands available that it is impossible to know them all.... And of course. give hash hit statistics (tcsh) exit .

. M-.. Commands for System and Kernel Stats These commands display information on system. display live system call stats For example. display for how long system has been running date . May 27.... and G-bytes Option -s * to display information for each file df ... 2008 3:20:00 PM machine ..txt Page 240 of 271 Printed: Tuesday... verify and repair filesystems Option -f to force check. display information on the host kernel The above commands are often used by the 'make' and '. display o/s name.. show the i/o registration for all hardware For information on disc usage use: du . set and display the host's network name Option -s to display hostname without domain name uptime . display and set the system date AppleSystemProfiler . display the processor type uname ... display disc usage statistics Option -a to report for all files in the hierarchy Option -c to display a grand total Option -h to display file sizes in K-. It is incorporated into the System Profiler command in Application:Utilities (or from the About This Mac box)..... display disc free information for all discs Option -h to display file sizes in K-. list discs Disktool is part of Apple's command line interface to the file system.Unix_tricks. disktool -l . display live file system usage For example. useful for journaled devices Option -y to complete check without manual intervention (See tips weeks 55.. version..... and processor type Option -a to print full (all) information hostinfo . (See tips week 11) ioreg ... and G-bytes Option -i to display inode statistics too fsck .. 100) mount . display information on mounted devices As well as mounting devices.. the mount command on its own shows information on each mounted device../configure' system to help build the correct binaries for the host platform. . and file usage sc_usage . 'sudo fs_usage TextEdit' then use text edit and open/save files... memory. 'sudo sc_usage TextEdit' fs_usage .. display comprehensive details This command is no longer included in Panther. M-. including whether it is local and whether it is journaled. hostname .

... stop/restart processes by PID Sends a signal to a process.. display system message buffer (See tips week 61 ... kernel extension load kextunload ... display a 'ktrace....txt Page 241 of 271 Printed: Tuesday.. ktrace . display user connect time for accounting These commands operate on the kernel.Friday) ac . display context switches &interrupts This must be run as root dmesg . 2008 3:20:00 PM lsof . Give the signal number or name..out' file human readable For example 'kdump' if the trace file is in the current directory.. display virtual memory statistics Watch out for page-outs increasing by 100's per second: you need to buy more memory or run fewer processes. 'ktrace ls' executes 'ls' and trace it's kernel activity. kernel extension unload kextstat ..Unix_tricks. killall ... Option -f to specify the trace file zprint .. perform kernel tracing For example. May 27. (See tips week 45) ps . latency . display and set kernel state variables Option -a to display all state variables and their current settings kextload . kdump . information on signals Use this command to learn about the possible signals that 'kill' can send to running processes.. list running processes Option -x to also list processes not attached to the terminal Option -a to list processes belonging to other users Option -ww to make a wide listing.out'. and the PID (Process ID) of the process to kill. display file status For example 'fstat' or 'fstat ~' or 'fsstat ~/letter.. otherwise the output is truncated Option -c to list only the process (command) name... This produces the file 'ktrace..doc' vm_stat .. list open files For example 'lsof' or 'lsof /dev/console' fstat . not the full command line .. display kernel extension statistics Commands for Running Processes kill . display information on kernel zones sysctl .... stop/restart processes by name Like kill but a case sensitive process name is required instead of a PID. man 2 sigaction .......

file transfer protocol Command line ftp client to get and fetch files from remote servers (See tips week 63) ftpd . execute a process with a given priority renice . grab a URL from a remote server Like curl... TELNET.. get mail from SMTP.... GOPHER.... LDAP. alter the pripority of an existing process exec . ftp daemon This runs an ftp service (See tips week 94) telnet ..... ssh daemon . fetchmail .. sshd .. read Usenet newsgroups ftp . execute a command immune to hangups The command will not be stopped by 'kill -HUP' or when the current shell exits (See tips week 78) leaks .. FTP.. start a secure shell on a remote machine The session is encrypted.. or connect to specific ports to test services such as FTP. May 27. connect using the telnet protocol Connect to remote machines that run telnet.. display information on running processes (See tips week 19) nice .apple. read Usenet newsgroups trn . execute a command in place of shell (See tips week 47) nohup . SMTP.. check process for memory leaks For example 'leaks TextEdit' heap .. IMAP etc tin. grab a URL from a remote server Supports the following protocols: HTTP.Unix_tricks.. HTTPS... DICT..txt Page 242 of 271 Printed:' wget . including the transfer of password information used to log into the remote machine.. POP..... list allocated memory in process's heap For example 'heap TextEdit' printenv ... and HTTP by hand. ssh ... Available for OS X via ports such as Fink. 2008 3:20:00 PM top .. FILE For example: 'curl http://www. display environment of the current shell Commands for Network Fetching curl . but not included with OS X..

.. remote execution Superseded by 'ssh' pine ... May 27. 2008 3:20:00 PM This runs an ssh service scp ... configure network interfaces Without any parameters displays the configuration for all network interfaces For example... maintain address resolution protocol tables The tables map IP addresses to Ethernet addresses Option -a to display all table entries service ...Unix_tricks... hostname is normally set automatically from /etc/hostconfig ifconfig ... manage TLS/SSL certificates certtool .... maintain network routing tables The routine information for a destination can be queried with: route -v get apple. display the network status Shows the state of all network connections (sockets) Option -I to specify a particular interface like en0 tcpdump . command line email client Commands for Network Information hostname . superseded by 'ssh' rsh . ipconfig .. rlogin . remote file copy Copy files to from a remote host running an rlogin server. secure version of FTP The ftp session is run over an encrypted ssh session openssl .. create new key pairs for certificates rcp . -vvv to increase verbosity of reports . start and stop xinetd services option --list to list the services The possible commands are 'start' and 'stop'.txt Page 243 of 271 Printed: Tuesday.set the IP mode of an interface Usage is: ipconfig set <interface> BOOTP|MANUAL|DCHP|NONE|INFORM route . netstat .. dump tcp activity Option -v. 'ifconfig en0 down' then 'ifconfig en0 up' resets en0.. Superseded by 'scp'.... set or display the hostname Option -s to display hostname without domain name part. route -v get localhost arp .. secure copy files to/from a remote machine Files are copied over an encrypted ssh session sftp . login to a remote machine The remote machine must be running an rlogin server. execute a shell on a remote machine Superseded by 'ssh' rexec .......

. or even remember all those one knew and used just last month.. and (Unix) if they are standard Unix commands.. to listen to network traffic with apple..txt Page 244 of 271 Printed: Tuesday. look up DNS information Option -t to specify the type of DNS record to look up. dump and analyse tcp activity Like tcpdump but not included with OS X.) $ a=$(expr 1 + 1 ) $ echo $a (See tips week 54 Friday . 'a'.` or $(expr .. query ntp time server ntptimeset . Option -a to request all DNS information for the host named . set shell options (bash) unset .. See the rest of the tips for lots of example usage of these commands. interactive domain name query (DNS) dig ... DNS daemon This runs a DNS service ntpd . set network time Miscellaneous Commands The last three weeks and the this week will form a Unix command reference by listing lots of commands under specific areas of use 'sudo tcpdump -vvv host apple. And of course... May 27. unset a shell variable (tcsh and bash) setenv ... display environment variables (tsch and Unix) (See tips week 54) These commands are useful in shell scripts.. There are so many commands available that it is impossible to know them all. network time protocol daemon This runs an ntp server ntpq . set and display the environment (Unix) printenv ... 2008 3:20:00 PM For example. See the Learning Centre for tutorials on these' to fetch name server records for apple. For example 'host -t ns apple. To set and display environment and shell' tcpflow ... or logical Used in scripting as `expr . set an environment variable (tcsh) unsetenv .... relational.. The commands are marked as (tcsh) and (bash) if they are built-in for the respective shells... set . unset an environment variable (tcsh) env . 't can also be 'mx'. look up DNS information Works both as a one-off command or interactively. nslookup .. Available from ports such as Fink. don't forget the Unix manual 'man'.... set a shell variable (tcsh) set . evaluate an expression The expression can be arithmetic.Unix_tricks.. 'soa'. host . expr .

archive files into a single file Option -c to create a new archive Option -x to extract from an existing archive Option -r/-A to append files to an existing archive Option -z to (un)compress with gzip Miscellaneous commands.... vipw .ws ) getopt . Used if the generated command line is too long (has too many arguments). time . (See tips weeks 49.. -1 is fastest but with the least compression (See tips weeks 5 Friday.. vipw locks the file and does sanity checks.. May 27. 75 Tuesday) tar . visudo . formatted print to display See 'man 3 printf' for more details of the formatting string.. get and parse command line options This is the standard command line parsing utility used by many commands to process options and command line arguments.. time the execution of a command limit/unlimit . tests an expression returning 0 (TRUE) or 1 (FALSE) Use in conditional statements such as 'if' and 'while' xargs ..... visudo locks the file and does sanity checks.txt Page 245 of 271 Printed: Tuesday. 37. 2008 3:20:00 PM http://www. change system resource limits . This command is redundant as OS X uses shadow passwords and therefore does not use /etc/passwd except in single user mode. uses a better compression algorithm Option -1 to -9 to adjust the level of compression.. prevent file from being accessed by more than one process These commands compress files and make archives.. zip unzip . GNU version of zip bzip2 bunzip2 . Option -0 the expect null-separated filenames. 98 Thursday) lockfile mktemp . safe edit of /etc/sudoers 'sudoers' should not normally be edited by a standard test editor.... sleep .. Used with 'find -print0' to cope with filenames with spaces Option -n to limit the number of arguments on each invocation. compress/uncompress files and folders gzip gunzip . pause processing of a script the script is paused for the given number of seconds to let other processes execute [ and test ..... generate unique temporary file names Option -d to make a directory Option -t to include a prefix printf ....osxfaq.. safe edit of /etc/passwd 'passwd' should not normally be edited by a standard editor...

. repair.. copy with resource forks Copies resource forks like ditto. mount or unmount diskutil . manipulate disc images cd9660. generate detailed system information This is a command line equivalent to 'About This Mac....Commands for Disc and File These commands are OS X specific and are not found in other Unix flavours.... mount ISO 9660 volumes udf. Invisible OSX Special .. partition table editor List all partitions of mounted discs with: sudo pdisk Top level command (? for help): L vsdbutil .. mount UFS (Unix File System) volumes hfs. Locked..util ... More Info. refresh. Normally done as part of an install or software update.. When new executables and libraries are added...... system_profiler ... 2008 3:20:00 PM OSX Special . redo_prebinding .... It generates textual information to standard output.System Commands These are command line equivalents for System Preferences panes. May 27. get attributes of HFS+ files SetFile ....' in the Apple menu. sets attributes of HFS+ files Sets HFS+ attributes such as Alias. disktool . This is part of the Developers Tool and not included in a standard install.. interact with CD and DVD burners hdiutil . eject.util . update_prebinding .. scselect .. journal discs drutil . read/write enable permissions on HFS+ volumes ditto .txt Page 246 of 271 Printed: Tuesday. Disc manipulation and maintenance... copy files/sync folders with resource forks Option -rsrc to copy preserving resource forks cpMac .util . command line version of S/W update This is useful to software update via a remote connection to a server. change network location This is equivalent to changing the Network Location in System Preferences pane Network .... mount DVD volumes ufs.util .. (See tips weeks 16 and 18) softwareupdate . GetFileInfo . mount HFS (Standard Apple) volumes pdisk ....Unix_tricks.. these commands can be used to update the binding information so commands launch faster.

Dump NetInfo information in unix FF format niload ..Unix_tricks. read and write .. set power management parameters (screen. performs a regular expression search on NetInfo nireport . nvram .. read and write domain in plain text nifind . NetInfo command set to query and change NetInfo (See example usage in Friday's tip) nicl . 2008 3:20:00 PM pmset .... defaults ...plist in preferences folders). scriptable image processing system appleping .plist utilities Miscellaneous commands. (See Tips weeks 91 and 96 for scripts which use NetInfo and Directory Service commands... manage TLS/SSL certificates and keychain access (See 'man openssl') opendiff .. opens two files and compare . text to speach converter sips .. interface to configd to manage Network Locations etc OSX Special ... NetInfo Command Line Utility nidump .. . part of Open Directory ni* .. DS daemon...... say . interface to xinetd services . directory services command line utility dscl is preferred over the ni* set of commands (See example usage in Friday's tip) scutil script These commands query and manipulate information in NetInfo and Directory Services... pings and appletalk network certtool .txt Page 247 of 271 Printed: Tuesday.bless a system folder. prints tables from NetInfo dscl ....) DirectoryService ....plist files plutil .. manipulation of Open Firmware NVRAM variables ioreg ..... find a NetInfo directory nigrep ...Utility Commands open . optionally make it the boot disc Can be used to change the startup disc... open a file as if double-clicked screencapture ......... and to 'register' cloned systems.. disc etc) bless . display IO reg hierarchy service . May 27. screen and window capture (See example usage in Friday's tip) These commands are for maintaining and checking Mac OS X preference files (those that end in . Load NetInfo information from unix FF format niutil ...

examine object files (=Unix ldd) Example Usage These examples are taken from my own notes and have not been tested for a while.nidump From single user with / mounted rw. 2008 3:20:00 PM lsbom . perl. and then burn it onto a CD (directly in the commandline). Screen Capture Command When you use the interactive version (command-shift-4).Unix_tricks.toggle between mouse selection and window selection modes escape key . interprets contents of binary bom files See 'man 5 bom' otool .Backup a NetInfo database nidump -r / -t localhost/local > backup.) using the new screencapture command found in /usr/sbin/ usage: screencapture [-icmwsWx] [file] [cursor] -i capture screen interactively. Screen captures can be triggered from the command line (and therefore AppleScript. etc.dmg hdiutil mount ~/Desktop/backup. May 27. move old /var/db/netinfo/local. hdiutil create -fs HFS+ -volname Backup -size 50m ~/Desktop/backup. the Dock or the menu bar.dmg ditto -rsrc -V ~/Desktop/datatobackup /Volumes/Backup >> /Volumes/Backup/backup..log hdiutil unmount /Volumes/Backup hdiutil burn ~/Desktop/backup..Restore a NetInfo database from backup periodic daily backs-up the NI database to /var/backups/local. backup data to it..cancels interactive screen shot -c force screen capture to go to the clipboard -m only capture the main monitor. undefined if -i is set -w only allow window selection mode -s only allow mouse selection mode -W start interaction in window selection mode -x do not play sounds file where to save the screen capture hdiutil Command Here is how to create a 50MB drive image.dmg -noverifyburn -noeject NetInfo Command Set . the space bar will toggle between the regular marquee selection mode and a new mode to capture just a single window.txt Page 248 of 271 Printed: Tuesday.nidb Start essential services /usr/libexec/kextd /usr/sbin/configd /sbin/SystemStarter Create blank NetInfo database and start NetInfo /usr/libexec/create_nidb /usr/sbin/netinfod -s local .nidump . The cursor changes to a camera icon when you are in this new mode and as you mouse over different areas of the screen the screen capture target is highlighted for you. by selection or window control key ..causes screen shot to go to clipboard space key .

.profile is sourced. set verbose mode on boot tcpflow sudo tcpflow -i en1 -c port 80 whois whois -h whois. print manual pages mount mount_afp [-o option1[.terminal TerminalOpaqueness '0.arin.85'.bash_profile (If ~/.bash_login is sourced..Unix_tricks..... < /var/backups/local.nidump .. A login shell sources the login scripts: /etc/profile ~/. 2008 3:20:00 PM Load the backup into NetInfo: /usr/bin/niload -d -r / .Re-create NetInfo via OS X first-time setup rm /var/db/.net ip-address Background Information Bash can be started as a login shell or as a non-login shell... fsck In single user mode. ~/. ~/. file system hierarchy man cmd | col -b . to print variables sudo nvram boot-args="-v" . and if this can't be read.bash_profile cannot be read.txt Page 249 of 271 Printed: Tuesday.option2 .]] afp://[username:password]@rhost[:port]/volume node mount_cd9660 /dev/disk2s1 /Volumes/cd umount /Volumes/cd mount -t hfs -w /dev/disk0s10 /Volumes/Grima nvram nvram -p .apple. One of the most significant differences between the two modes is the initialisation scripts sourced when Bash starts up.. May 27. ASCII table man 7 hier ..AppleSetupDone reboot Other Commands bless Set OS X as a startup sudo bless -folder '/System/Library/CoreServices' -setOF Set OS 9 as a startup sudo bless -folder9 '/Volumes/Mac OS 9/System Folder' -setOF defaults defaults read/write/delete <file> <parameters> defaults write com. to repair HFS disc 1 partition 5 fsck_hfs -y /dev/rdisk1s5 man man 7 ascii ..) .

Customise the look of an xterm terminal by adding more lines to ~/. and a login shell by 'logout'.Xdefaults $ cat ~/. To help rationalise this behaviour and get a consistent environment we can change X11 xterm to launch a login shell. When Apple's Terminal. For example: XTerm*background: black XTerm*foreground: white XTerm*backdrop: white XTerm*rightScrollBar: True XTerm*ScrollBar: True xterm*saveLines: 5000 XTerm*VT100*geometry: 100x55+10+10 . or by the X11 Applications menu. and make the same change. Typing 'exit' in a login shell will run 'logout'. A non-login shell is exited by 'exit'. Edit /etc/X11/xinit/xinitrc and change the line that invokes xterm: xterm & to: xterm -ls & Alternatively. When you run a shell script. When X11's xterm starts a new terminal session it starts a Bash non-login shell.txt Page 250 of 271 Printed: Tuesday.LoginShell: True Now xterm will start a login shell whether it is invoked by X11 on startup. or by typing 'xterm &' on the command line. To ensure all xterm windows are login shells too.xinitrc which is a copy of /etc/X11/xinit/xinitrc. to affect only your own starts a new terminal session it starts a Bash login shell. Make X11 Xterm Launch Login Shells From Monday's tip it can be seen that Apple's launches a login shell while X11 xterm launches a non-login shell. The new shell is a non-interactive (non-login) shell. A non-login shell sources the following scripts: /etc/bashrc ~/.Xdefaults # Default settings for X Applications # # xterm XTerm*. May 27.Xdefaults. It does not source any scripts on startup. 2008 3:20:00 PM On exit a login shell sources ~/. /etc/bashrc is not sourced. When you type 'bash' at the command line a non-login shell is started. Now X11 will start a login shell.Unix_tricks. Non-interactive shells.bash_logout. instead of the above create a file called ~/. a new shell is launched to execute the script. Make X11 launch login shells. create a file ~/. Even better. but it is.bashrc NOTE: according to the bash manual.

bashrc to set shell variables and other shell settings.Unix_tricks. Finally.bash_profile should source ~/. to add a new item. Hence it is better for xterm to startup login shells so that they source the login scripts. /etc/profile and ~/.) Initialisation Scripts The four different initialisation scripts (see Monday's tip) must be set up correctly to ensure shells have the correct environment. ~/. Shell variables are not inherited by non-login shells and therefore need to be set each time. Typing 'bash' at the command line runs a non-login shell which sources /etc/bashrc and ~/. Use these scripts to set up the environment that is common to all users. /etc/profile should source /etc/bashrc: source /etc/bashrc and ~/. for example: . use the X11 menu bar Applications::Customise.bash_profile are sourced by login shells and therefore applied once for each terminal session.bashrc: source ~/.. Environment variables are inherited by non-login shells and don't need to be setup again.bash_profile to set environment variables.bashrc invoke on the command line as: bash --norc To create a login shell from the command line use: bash --login To have a 'noprofile' shell option in X11. May 27..txt Page 251 of 271 Printed: Tuesday. /etc/profile and /etc/bashrc are system-wide. Fink. Call the item something like xterm-np and make the command: /usr/X11R6/bin/xterm -e /bin/bash --noprofile Note that we tell xterm to run a command with the -e option. Beware! If you source the Fink initialisation script. In OS X Aqua controls the root window. Use /etc/bashrc and ~/. Note that these must be 'sourced' and not simply called like: /etc/bashrc Prevent Execution of Initialisation Scripts To prevent bash from running /etc/profile and ~/. A good strategy is to do the following: Use /etc/profile and ~/.bashrc. First off. If no option is given it runs bash in a default manner.bashrc are specific to each user and are used to add personal settings to the environment.bash_profile when it is invoked as a login shell use: bash --noprofile To prevent a non-login shell from running /etc/bashrc and ~/. 2008 3:20:00 PM (The reason X11 xterm launches non-login shells is that in most Unix systems X11 controls the root window and the correct environment has already been set up. Second.bashrc to avoid having to set shell variables in both the login and non-login scripts.bash_profile and ~/.

bash_profile. use the following technique. To /etc/profile add: declare -x STARTUP="/etc/profile.log.0 zcat: /var/log/system. name-of-script Executing an initialisation script like /etc/bashrc instead of sourcing it will not work.log.4 (Tiger).gz .sh' ]&& source /sw/bin/init.log. " and similarly to ~/. But historically.0. like other z-commands.log.Z). the zcat command was. If you want a script executed by the current shell you must 'source' it: $ source name-of-script or $ .Z: No such file or directory $ gzcat /var/log/system. or else make an alias to the correct bash.bashrc. This can confuse matters if you try to start a login shell by typing 'bash --login'. $ zcat /var/log/ the script checks to see if '/sw/bin' and '/sw/sbin' are already in the path.Z: No such file or directory $ zcat /var/log/system.0.' Executing a script in the usual manner from the command line: $ name-of-script executes the script in a new bash shell.bashrc add: STARTUP="${STARTUP}~/.0.log. " Any other sourced files such as those from Fink should also have similar statements added. " To /etc/bashrc add: declare -x STARTUP="${STARTUP}/etc/bashrc.gz. When the new shell ends all its environment and shell variables are destroyed. and the GNU version is named gzcat. May 27. 2008 3:20:00 PM [ -r /sw/bin/init. If not it adds them to the start of the path.txt Page 252 of 271 Printed: Tuesday. It could be used to directly display gzip compressed files. If you need to keep track on which initialisation scripts have been run in any one shell environment. The command was often overwritten by the GNU version. Therefore. or else invoke it directly with '/bin/bash' Script vs 'source' vs '. always use gzcat unless the target file really was created by compress.bash_profile add: STARTUP="${STARTUP}~/. the original zcat is preserved. " and to ~/. In Tiger.Unix_tricks. Fink includes its own version of bash which does not read /etc/profile but reads /sw/etc/profile. Zcat in now Gzcat In versions of Mac OS X prior to 10.gz zcat: /var/log/system. zcat has existed to directly display files compressed by the Unix compress command (the extension used for this sort of file is . part of the GNU zip tool set. To avoid any problems add '/sw/bin' and '/sw/sbin' to the end of your path before you call '/sw/bin/init.

.. It has been replaced by the GNU nano editor..nano-editor. If /usr/local/bin is not in your search path. or else invoke nano using its full pathname.. Jan 1 18:11:49 Sauron kernel[0]: AFP_VFS afpfs_unmount: .. $ tar xzf nano-1.Unix_tricks.5.nanorc in your home directory. but offers extra features. 2008 3:20:00 PM .much output here.5/ $ . Syntax Colour in Nano Nano is capable of syntax highlighting. we can tell it to display HTML tags such as <head> in blue. Type the following lines into the Terminal window (the version number may differ from the one shown here).tar.. Enter the following into the nanorc file. For example. called .. you will either need to add it. Nano is compatible with pico. Also included the very useful zless. The source builds right out of the box in Panther. Note that in pre Tiger systems gzcat was installed as zcat and gzcat. Use gzcat in scripts for portability.. lacking in earlier versions of Mac OS X..Z files to .. and as such had some critical limitations such as a lack of search and replace.txt Page 253 of 271 Printed: Tuesday.. The executable will be installed into /usr/local/bin/nano. May 27.. the site provides free downloads of the program./configure # . #HTML Syntax Highlighting . and the man pages into /usr/local/man/. Remember too that bzip2 compressed files have a similar set of utilities. which didnt include nano..gz files. $ make $ sudo make install Password: # . . The official home site for nano.2. Jan 1 18:11:36 Sauron kernel[0]: AFP_VFS afpfs_mount: . Here's a reminder of the many z-variant commands available: zcat zdiff zfgrep zgrep zmore zprint zcmp zegrep zforce zless znew Note that znew can be used to recompress .. and then clicking on the file nano-[versionnumber]. Most important for users of pre-10. Here are instructions for installing it: Download the source by clicking the link at the bottom of the screen labeled Get Nano.. www. and escaped characters such as & in red..gz $ cd nano-1.4 releases of Mac OS X. It was part of the PINE email toolset.2.output is a useful resource for nano users. bzcat bzdiff bzfgrep bzip2 bzless bzcmp bzegrep bzgrep bzip2recover bzmore Pico is Nano The pico editor was a very popular and easy to learn Unix text editor.

rtf. It searches through meta data for attribute values. ]*.txt Page 254 of 271 Printed: Tidy with Tidy . and the fourth says that everything between & and . 1) The Korn shell in /bin/ksh. and prefix any with the word bright. Check the man pages for textutil and mdfind for full details.Unix_tricks. May 27.sample. Try loading an HTML file into a new instance of nano. $ man -t bash | open -f -a /Applications/Preview. 2008 3:20:00 PM syntax "HTML" "\.app. The string to match-and-color is specified using an extended regular expression. displays gzip compressed files directly using the less command. and how to write syntax highlighting instructions. Here's a handy tip to convert postscript output to PDF using Preview. 4) mdfind is a command line Spotlight.doc. This file gives examples of what can be placed in the configuration file. useful if you ever want to run a Korn shell script (or if you prefer to use Korn as your interactive shell!) 2) zless. type: $ textutil -convert doc index. black. and you should see the syntax highlighting described above in action. red. The third says that everything between < and > (possibly across multiple lines) is to be coloured blue. written to index. Some examples of using textutil are: To convert the HTML file index. cyan.html To convert the same file to an MS Word document. green. More New Commands Tiger includes the following commands that were not in earlier versions of Mac OS X. or for specific attributes with given values. written to The -cat option is handy to concatenate all the input files into one big output file.html to rich text format. blue. (when they are not separated by white space or semicolon) is to be colored red. magenta. (The text between [ and ] includes a space character and a tab characters.html$" color blue start="<" end=">" color red "&[^. It's actually a very simple wrapper script: $ cat /usr/bin/zless #!/bin/sh PATH="/usr/bin:$PATH". export PATH LESSOPEN="|gzip -cdfq %s". part of the GNU zip toolkit. such as ~/Sites/index. Download the nano source files (see yesterday's tip) and you'll find a file called nanorc.html. which is a command line interface to the Cocoa text system." The second line says that the HTML syntax is applied to all files who's name ends with . export LESSOPEN exec less "$@" $ 3) textutil. type: $ textutil -convert rtf index.html. In this example we pipe the raw postfix man page for bash into Preview. yellow.) You may specify any of the colors: white.

First off. " " Map to check html and display error in a new window " noremap \err :1<CR>yG :20new<CR>p :setfiletype html<CR> :%!tidy -i >/dev/null<CR> " Map to tidy html " noremap \tidy :%!tidy -i -f /dev/null<CR> " END " The new commands are invoked by typing \err and \tidy in normal mode. May 27. tidy can be used to great effect to check and tidy HTML source files directly within vim. $ cat ~/.txt Page 255 of 271 Printed: Tuesday. output is sent to standard out.vim " MAIN URL: http://www. " " This file should be placed in ~/. We want our scripts to honour this convention.Echo to Standard Error They're of most use in shell scripts. Add the following file type plugin for html files." to standard error.vim/ftplugin/html/ on Unix systems " and will automatically be sourced for filetype = html. Redirection .mayo-family. but can be used directly on the command line too. Tidy can be installed in earlier versions of Mac OS X from sourceforge and presumably other ports like Fink and DarwinPorts. let's write the error message "Can't open the Unix commands such as 'ls' and 'mount' write output to standard out and error messages to standard " " This file defines maps to call the html tidy program. Type: $ tidy -h to see a help screen. See sourceforge.vim/ftplugin/html/tidy. Why would we want to write to standard error instead? Traditionally. As an example.vim """""""""""""""""""""""""""""""""""""""""""""" " Adrian's vim filetype plugin for html tags " """""""""""""""""""""""""""""""""""""""""""""" " " Maintainer: Adrian Mayo " URL: http://www. 2008 3:20:00 PM Tidy is a utility to syntax check and tidy HTML source code. This convention ensures that if we were to pipe the command's output to another command (or redirect it to a file) error messages would still displayed on the terminal screen and would not get mixed up with the 'normal' output. here's how a script might send an error message to standard error instead of standard out.Unix_tricks. when your script writes any type of message using a command such as 'echo'.sourgeforge. see: " http://tidy.mayo-family. $ cat example . If you're into the vim editor.

May 27. we'll test an 'ls' command line that lists 'file' (which fails) and the directory 'test' that contains 'file': $ ls test file ls: file: No such file or directory test: file $ Next.Redirect and Pipe Both Monday's tip showed how we can make our scripts write error messages to standard error instead of standard out. fi >out Redirection ./example >out 2>err $ cat out $ cat err Can't open the file." 1>&2 The key is this after the echo command: 1>&2 which merges standard out (stream 1) into standard error (stream 2).. Naturally. not standard out. We run it and see the error message: $ . echo "Can't open the file.txt Page 256 of 271 Printed: Tuesday. suppose we wish to send both the standard out and standard error of 'ls' to the same file. $ To prove the message went to standard error and not standard out. we redirect standard out to the file 'out' and standard error to the file 'err'. 2008 3:20:00 PM #!/bin/bash # some processing here. but still see the error message on the screen. Today. fi > out An error occurred Notice the use of '1>&2' to merge standard out into standard error. $ Here's an example in which we issue a command and want to report an error if the command failed. For example.. hence causing 'echo' to write to standard error.Unix_tricks. displaying the contents of each file: $ . we want to send our error message to standard error. just like 'real' Unix commands. First. Notice also that we took advantage of Bash's integer expressions in: (($?!=0)) More conventionally we would have written: $ if [ $? != 0 ]. as in Monday's tip. we'll look at how to negate this convention by merging the standard error from a command back into its standard out./example Can't open the file. if its non-zero the command failed. We use the special redirection '&>'. To prove the message was sent to standard error and not standard out. we redirect standard out to the file 'out'. $ ls zzz ls: zzz: No such file or directory $ if (($?!=0)). We test the special shell variable '$?': if it's zero then no error occurred. then echo "An error occurred" 1>&2. then echo "An error occurred" 1>&2. we'll redirect standard out to 'out': . Here's how we achieve this in an example where we test the error status returned from the 'ls' command.

only standard out is piped. when we pipe the output of one command to another. you must be careful combining the merging of standard out and standard err (1>&2 or 2>&1) with redirection. If you need to pipe both standard out and standard error. we pipe only standard out: $ ls test file | awk '{print "Piped: " $0}' ls: file: No such file or directory Piped: test: Piped: file $ Next. Our first attempt fails: $ ls test file 1>&2 2>err test: file $ Reversing the order of the redirect and the merge will yield the desired behaviour: $ ls test file 2>err 1>&2 $ cat err ls: file: No such file or directory test: file $ The reason the first attempt fails is that out is merged with error at the time error is still . 2008 3:20:00 PM $ ls test file > out ls: file: No such file or directory $ Finally.txt Page 257 of 271 Printed: Tuesday. we pipe both standard out and standard error. First. May 27. Here's an example using the same 'ls' command as above. This example uses an 'ls' command line that lists 'file' (which fails) and the directory 'test' that contains 'file': $ ls test file ls: file: No such file or directory test: file We merge standard out into standard error and redirect standard error (hopefully this is now both out and error) to the file 'err'. We pipe to the 'awk' command. we'll redirect both to the same file using the special redirection '&>': $ ls test file &> out $ Normally. first merge standard error into standard out. $ ls test file 2>&1 | awk '{print "Piped: " $0}' Piped: ls: file: No such file or directory Piped: test: Piped: file $ Combine Redirections Because of the order in which Bash processes a command line and redirection syntax.Unix_tricks. then pipe standard out as normal. which then prints its input lines with "Piped: " added to the beginning.

at source line 1 context is >>> <<< $ Note that other shells such as 'sh' and 'tcsh' use a different syntax for command substitution - `command` instead of $(command): $ device="`df | awk '/\/Volumes\/pw/ {print $1}'`" Mysql .. We'll look at installing the latest stable release of MySQL . $ device="$(df | awk '/\/Volumes\/pw {print $1}' 2>&1)" $ echo $device awk: non-terminated regular expression \/Volumes\.. at source line 1 context is >>> <<< $ echo $device $ To capture standard error too. too late for out to be redirected too. and how to integrate it with the Apache-PHP module.5. (I've always though that the two syntaxes work the opposite way around to what seems natural. May 27. For example. First.0. Only later is error redirected to the file. if we like only good news: $ ls test file 2>/dev/null test: file $ Redirect to a Variable It's often useful to capture the output of a command into a shell variable..Grab Mysql This week's tips show you how to install and set up the MySQL Database server.Unix_tricks. 2008 3:20:00 PM going to the screen. download the binary distribution from MySQL. For example.txt Page 258 of 271 Printed: Tuesday. $ device="$(df | awk '/\/Volumes\/pw {print $1}')" awk: non-terminated regular expression \/Volumes\. only standard out is captured.) However. . this command returns the device identifier of the device mounted at '/Volumes/pw' $ df | awk '/\/Volumes\/pw/ {print $1}' /dev/disk3s2 $ We capture the output from this command into a shell variable called 'device' by typing: $ device="$(df | awk '/\/Volumes\/pw/ {print $1}')" $ echo $device /dev/disk3s2 $ (We might use $device in a 'mount' command later in the script. as shown in this next example where we form an invalid 'awk' command. redirect to the special file /dev/null. we simply merge it with standard out using the syntax '2>&1'.) To throw away either or both of standard out and standard error..

You have a second choice too. These instructions assume 10.1.4-powerpc. The line pertinent to MySQL is '/usr/local/mysql/bin:\'. also. I've had no problems with the non-64 bit version running on an iMac G5.3 (Power PC).16-apple-darwin8.Create a Database The database directory. is stored. is created under the actual installation in /usr/local/mysql/data. although the 64 bit version works.4 Power PC.0-powerpc mysql This uncompresses the archive and makes a symbolic link (not a copy) called mysql to the actual installation. It's a good idea to add the path of the MySQL binaries to your PATH variable so you don't have to type the full path name to execute a MySQL command. Next.10. It's ok to break it like this in the file. I've assumed the tar file was downloaded to ~/Desktop) type: # cd /usr/local # tar -xzf ~/Desktop/mysql-standard-5.) The path is '/usr/local/mysql/bin'.0.gz # ln -s mysql-standard-4. 10. It's better to create it elsewhere. The database will be created in a more useful location that the default /usr/local/mysql/data. I've tried both the standard and the 64 bit versions for 10. perhaps under /var where most server generated data.Unix_tricks. apply the correct owner and permissions to the install: # cd mysql # chown -R root:mysql .txt Page 259 of 271 Printed: Tuesday. such as incoming mail and log files. A typical PATH variable would be set in /etc/profile and look like this: declare -x PATH="\ /usr/local/bin:/usr/bin:/bin:\ /usr/local/sbin:/usr/sbin:/sbin:\ /usr/X11R6/bin:\ /usr/local/mysql/bin:\ " The statement is broken into six lines.4 (Power PC). We'll install MySQL in the standard location of /usr/local. May 27.4 Power PC Tar package.4 (x86). provided '\' is added to the end of each broken line. To become the root user (give your admin password) type: $ sudo -s Password: # . Type the following commands: To become the root user (give your admin password) type: $ sudo -s Password: # To install MySQL (the name of the Tar archive will be different for other distributions.18-osx10. (This assumes you are using the Bash shell. I could not compile PHP against that distribution. 2008 3:20:00 PM Choose the appropriate Mac OS X Binary . Tuesday's tip will show you how to create the MySQL database and test the installation. between a Mac-style Package installer and a Unix style Tar package (found towards the bottom of the HTML page in the 'special' section). However. Mysql . by default. Other shells vary.2. or 10.tar.

2008 3:20:00 PM Create the database directory /var/mysql-data (you can place it at another location if you wish).cnf [mysqld] datadir=/var/mysql-data Next. start a new terminal session. it's necessary to give the option -p to most MySQL commands and type root's password at the prompt. /etc/my. Once you have done so. Alternatively.. Type: # mkdir /var/mysql-data # chown mysql:mysql /var/mysql-data # chmod 770 /var/mysql-data Create the MySQL configuration file. Obviously. see Monday's tip to add the MySQL executable directory to your PATH.domain.domain. .cnf. create the necessary administrative databases by typing: # /usr/local/mysql/scripts/mysql_install_db --user=mysql --datadir=/var/mysql-data This should create all the necessary files with owners mysql:mysql.) NOTE: From this point password my-password Replace 'hostname.. stop the server by typing: # mysqladmin -u root -p shutdown Enter password: STOPPING server from pid file /var/mysql-data/. which tells mysql where the database it' with your host's name (eg ' /usr/local/mysql/bin/mysqld_safe. This is what it should contain: # cat /etc/my.local').Unix_tricks.they are entirely separate. replace my-password with your chosen MySQL root password.start the server by typing: # mysqld_safe --user=mysql & [1] 19718 # Starting mysqld daemon with databases from /var/mysql-data (If mysqld_safe cannot be found. we must create a Mac OS X startup item so MySQL will be launched when your Mac starts.txt Page 260 of 271 Printed: Tuesday. Test the installation .) Check that the three default databases are there # mysqlshow +--------------------+ | Databases | +--------------------+ | information_schema | | mysql | | test | +--------------------+ MOST IMPORTANT: enable the root password by typing: # mysqladmin -u root password my-password # mysqladmin -u root -h hostname. May 27. (Don't confuse the Unix root user with the MySQL root user . You might get away with typing just the first line. 060111 00:51:57 mysqld ended [1]+ Done mysqld_safe --user=mysql # Next. As a further test of successful installation. type the full path name to this and other MySQL commands .

then echo "Starting MySQL server" $mysqlctl start fi } StopService () { echo "Stopping MySQL server" $mysqlctl stop } RestartService () { echo "Restarting MySQL server" $mysqlctl restart } if [ -x $mysqlctl ]. Requires = ("DirectoryServices". { Description = "MySQL Startup". } Make sure the script is executable: # chmod +x MySQL . but not in the Unix style Tar package. 2008 3:20:00 PM Create a directory by typing: # mkdir /Library/StartupItems/MySQL Create two files in the directory. then RunService "$1" else echo "Cannot execute $mysqlctl" fi -.common mysqlctl="/usr/local/mysql/support-files/mysql.MySQL -- #!/bin/sh # # Startup Item for MySQL # ###################### . /etc/rc. "NetworkExtensions"). May 27. OrderPreference = "Last". Change to the appropriate directory: # cd /Library/StartupItems/MySQL Create the files with the following contents: -.Unix_tricks.txt Page 261 of 271 Printed: Tuesday.StartupParameters.server" StartService () { if [ "${MYSQLCOM:=-NO-}" = "-YES-" ] . Provides = ("MySQL").plist.plist -. called MySQL and StartupParameters. They are included in the Mac-style Package installer.

but must be the mysql root user. 0 rows affected (0. Apple's PHP install. 1 row affected (0. and give that user permissions on the jan database. If you wish to use MySQL with Apache-PHP. create a new user with an old-style password. MySQL should automatically start up. at the time of writing. May 27. you might have problems because PHP-BB adds odd invisible characters to the code sections. $ mysql -u root -p Enter password: mysql> create database jan. Alternatively.. and how to add a user to the database. either don't give a password by missing out: identified by 'password'.00 sec) Then create a user. Database changed mysql> Select * from user\G . when you create a new user. +--------------------+ | Database | +--------------------+ | information_schema | | jan | | mysql | | test | +--------------------+ 4 rows in set (0. NOTE: If you copy and paste this code from the OSXFAQ forums.txt Page 262 of 271 Printed: Tuesday. Query OK.00 sec) mysql> show databases.01 sec) To view all users (user information is held in the mysql database) type: mysql> use mysql. also but not necessarily called jan. To do this. You need not be the Unix root user.Unix_tricks.Add a Database and a User Let's create a database called jan. whereas MySQL 4. To allow PHP to connect to a database.* to jan@localhost identified by 'jans-password'. and you are using Apple's PHP install. Type: mysql> grant all on jan. mysql> quit Bye $ Mysql.. Firstly. then you'll have problems.1 and later use the new-style encryption. create the user as given in Wednesday's tip and then type (this example shows a user called jan): # mysql -u root -p Enter password: . 2008 3:20:00 PM When you reboot. PHP and Passwords The first three tips showed how to install MySQL and how to create a new database and user. Wednesday's tip will show how to add a new database. Query OK. was built to use old-style MySQL password encryption. Mysql .

1).4. and also the socket is to be found at /var/mysql instead of / /usr/libexec/httpd/libphp4. and until Apple corrects it. or by re-installing and re-configuring MySQL. Download the (bzip2) source from: PHP. for a more fully featured PHP.sock Alternatively.1. re-build PHP.4. and until Apple corrects it./configure --prefix=/usr/local --mandir=/usr/local/share/man --with-apxs --with-mysql=/usr/local/mysql --with-mysql-sock=/tmp/mysql. configure by typing: $ . The problem lies in the configuration of Apple's PHP build. as shown in Friday's tip. 2008 3:20:00 PM mysql> set password for jan@localhost = old_password('jans-password').txt Page 263 of 271 Printed: Tuesday. Extract it by typing: $ tar xf php-4.4 update. This can be corrected by building your own PHP install.4. This can be corrected by building your own PHP install. May 27. applied when Apple builds the PHP module.sock' meaning MySQL is expected to be found at /usr instead of /usr/local.Build the Apache PHP Module As of the 10.sock --with-openssl=/usr --with-ldap=/usr --with-kerberos=/usr --with-zlib-dir=/usr --with-iodbc=/usr --with-curl=/usr --enable-exif --enable-ftp --enable-dbx --enable-sockets Build it by typing: $ make Install it by typing: $ sudo -s Password: # mv /usr/libexec/httpd/libphp4.Unix_tricks. These instruction apply to PHP-4.bz2 --bzip2 $ cd php-4.4 update. the Apache-PHP module was built in a way that's incompatible with most MySQL installs. as of the Tiger 10.1 (You version may be different to 4. If you are running 5. you'll already know how to install PHP-5. Mysql . Configure it by typing (on a single command line): $ . Apple used the configuration commands: '--with-mysql=/usr' '--with-mysql-sock=/var/mysql/mysql.tar.4.orig # make install . In particular.1. mysql> quit Bye $ Also. You can solve this by moving the MySQL installation and starting the MySQL daemon (in the startup items) using the option: --socket=/var/mysql/mysql. the Apache-PHP module was built in a way that's incompatible with most MySQL installs./configure --with-mysql=/usr/local/mysql --with-apxs --mandir=/usr/local/share/man

When the script executes sudo. Restart Apache (if it's already running) so the new PHP module is read.Authentication Suppose you have a script that uses 'sudo' to execute a command as root./configure --help for a full list of the configure options. If the user fails to authenticate then sudo will not run the command and we must terminate the script. } How does it work? Option -p causes sudo prints a prompt.txt Page 264 of 271 Printed: Tuesday. Then consider using 'printf' over echo.Fancy Printing Ever wanted to spice up the output from a script. like aligning numbers or zero padding them. (Why?) It can be installed by user root using pear. The install does not include the DB package by default. If authentication is successful. sudo returns success (or True) and the second statement is skipped. Similarly. . even though the command that requires it occurs later.True OR anything = True. Place this line at the start of the script: sudo -p Admin password echo 2> /dev/null || { echo Incorrect . True or False. exit. The sudo command just executes a null echo command. If authentication is not successful. To do this we rely on the fact that a sudo authentication is valid for five minutes. An OR operator executes its statements only in order to produce a Boolean result. Scripting one-liners . sudo returns failure (or False) and the following statement (after the || 'OR') operator is executed. the user must authenticate with a password. # apachectl graceful Install extra packages with PEAR. these should point to directories within /usr/local pear list # list all installed packages With any luck. Scripting One-Liners . It reasons that if the first statement yields True. Ideally. Type: # pear install DB NOTE: Some handy pear commands: pear # lists all pear commands pear help command # get help on a specific command pear config-show # shows the config settings. If the script is already part-way through execution we might have to undo some of the stages its already gone through. 2008 3:20:00 PM NOTE: Run . then the second statement needn't be executed because it won't affect the overall result . the AND operator (&&) doesn't execute its second statement if the first yields False - False AND anything = False.Unix_tricks. you should have an Apache-PHP module that talks to the MySQL 5 installation from the earlier tips. May 27. we want to achieve authentication at the start of the scrip. This statement prints an error message and exits.

Unix_tricks. %5d tells printf to look outside the format string and interpret what it finds as a decimal number and to write it to a column 5 characters wide. which is documented by: $ man 3 printf Scripting One-Liners . And to left justify: $ printf "Value: %-5d units\n" $num1. Because the format string has two occurrences of %s. $ num1=4. we can read the values from a file. it takes three sweeps at the string. 2008 3:20:00 PM Here's how to print numbers in columns. It uses the printf library function. . num2="123" $ printf "Value: %5d units\n" $num1.. but we supply six values.More printf We can take advantage of a very useful feature of printf to print tables of data. $ echo "Apples 7 Pineapples 5 Pears 6" > fruit-count $ printf "<tr> <td> %s</td> <td>%s</td>\n</tr>\n" $(cat fruit-count) <tr> <td>Apples</td> <td>7</td> . May 27. printf "Value: %05d units\n" $num2 Value: 00004 units Value: 00123 units You can also change the format stored in a variable: $ num1=$(printf "%05d" $num1) $ echo $num1 00004 The printf function can do much than than shown here. right justified. printf "Value: %-5d units\n" $num2 Value: 4 units Value: 123 units And to print leading zeros: $ printf "Value: %05d units\n" $num1. printf "Value: %5d units\n" $num2 Value: 4 units Value: 123 units Printf prints what's inside its format string. %s tells printf to look outside the format string for a value to substitute. Using the next trick.. Check this example: $ printf "<tr>\n <td>%s</td> <td>%s>/td>\n</tr>\n" Apples 7 Pineapples 5 Pears 6 <tr> <td>Apples</td> <td>7</td> </tr> <tr> <td>Pineapples</td> <td>5</td> </tr> <tr> <td>Pears</td> <td>6</td> </tr> $ The printf statement prints what's in its format string.txt Page 265 of 271 Printed: Tuesday.

$ ( >ls >df >cat number-list >) > out $ You can use a group-command instead of a sub-shell by replacing (.} > out Use the same trick to re-direct standard input to a group of commands. Create a sub-shell around the commands who's output you wish to redirect. $ cat infile 1234 full $ { read code.conf File: snmp. May 27.txt Page 266 of 271 Printed: Tuesday. do echo "File: $f". For example.conf File: treewalk-dot $ (A more useful example would do something more than simply echo each filename. making use of metadata.conf File: snmp. You can process each field in a file with: $ for f in $(cat file-to-process). an image created . Spotlight . Metadata is data about data. done File: a file with spaces. A sub-shell is executed by a new instance of the shell.) > out with {.Reveal the Metadata I'm sure you're familiar with Apple's Spotlight and its powerful search capabilities.. 2008 3:20:00 PM Scripting One-Liners ..Batch Process Files Here's a handy way to process every file in a directory: $ for f in *. and without having to redirect the output of each command individually.txt File: snmpd. } < infile $ echo $code $membership 1234 full $ Note: use a group-command. It searches file content rapidly using a pre-built content index database. do echo $f.txt File: db-snmpd.Bulk Redirection We all know how to redirect output to a file using the redirection symbol '>'.Unix_tricks. if you wish to retain the values of variables. not a sub-shell. $ ls > file-list Here's an example in which we redirect the output of many commands in a shell script.) It'll even cope with filenames that contain spaces. read membership. done Scripting One-Liners . and its environment (including all variable) dies when the sub-shell completes. It also searches for information about files. as witnessed by the first file in the list...

(Note: Spotlight was introduced in Mac OS X Tiger 10. "public.text". May 27.rtf" kMDItemFSNodeCount = 0 kMDItemFSOwnerGroupID = 501 kMDItemFSOwnerUserID = 501 kMDItemFSSize = 7607 kMDItemFSTypeCode = 0 kMDItemID = 923169 kMDItemKind = "Rich Text Format (RTF) document" kMDItemLastUsedDate = 2006-02-06 00:59:21 +0000 kMDItemUsedDates = ( 2006-01-14 00:00:00 +0000.rtf". and such like form the metadata.rtf" and user and group owner (shown as numeric IDs . "public. such as filename name: kMDItemFSName = "schedule. 2006-02-05 00:00:00 +0000. 2006-02-06 00:00:00 +0000 ) You'll see all the usual metadata normally displayed by 'ls -l'. but the filename. such as: kMDItemDisplayName . which often means without the extension.the name as shown in the Finder. date stamp.the type of content. 2006-01-20 00:00:00 +0000. .rtf schedule. For example. if we mdls a TextEdit file saved in Rich Text format we see: $ mdls schedule.txt Page 267 of 271 Printed: Tuesday. kMDItemKind . "public. owners. 2006-01-21 00:00:00 +0000.rtf ------------- kMDItemAttributeChangeDate = 2006-02-06 00:59:21 +0000 kMDItemContentCreationDate = 2004-06-03 17:17:17 +0100 kMDItemContentModificationDate = 2006-02-06 00:59:21 +0000 kMDItemContentType = "public. has opened up the power of Spotlight to the command line. 2008 3:20:00 PM by Photoshop contains data that is the image itself. "public. The 'mdls' command (metadata ls) shows all metadata attached to a file or directory.content") kMDItemDisplayName = "schedule" kMDItemFSContentChangeDate = 2006-02-06 00:59:21 +0000 kMDItemFSCreationDate = 2004-06-03 17:17:17 +0100 kMDItemFSCreatorCode = 0 kMDItemFSFinderFlags = 16 kMDItemFSInvisible = 0 kMDItemFSIsExtensionHidden = 1 kMDItemFSLabel = 0 kMDItemFSName = "".the translation to textual names is performed by ls): kMDItemFSOwnerGroupID = 501 kMDItemFSOwnerUserID = 501 You'll notice other metadata too. 2006-01-23 00:00:00 +0000.Unix_tricks.rtf" kMDItemContentTypeTree = ("public.item". being a nice Unix player. Let's look at what's available in Terminal land. file size.4) Apple.

psd ------------- kMDItemAttributeChangeDate = 2006-01-03 00:21:09 +0000 kMDItemBitsPerSample = 40 kMDItemColorSpace = "RGB" kMDItemContentCreationDate = 2004-02-06 02:12:09 +0000 kMDItemContentModificationDate = 2004-02-06 02:12:09 +0000 kMDItemContentType = "com.Pictures and Songs Today's tip looks at file content metadata.0" kMDItemDisplayName = "heart" kMDItemFSContentChangeDate = 2004-02-06 02:12:09 +0000 kMDItemFSCreationDate = 2004-02-06 02:12:09 +0000 kMDItemFSCreatorCode = 943868237 kMDItemFSFinderFlags = 1024 kMDItemFSInvisible = 0 kMDItemFSIsExtensionHidden = 1 kMDItemFSLabel = 0 kMDItemFSName = "jj1. Spotlight stores its content index and metadata store are held in a directory at the top level of each file system: $ sudo ls /.plist store. we'll look at what metadata is available for other types of file content.plist _rules. Let's pass a photoshop file to mdls: $ mdls heart. May".Unix_tricks.item". BTW.psd heart. "public.psd" kMDItemFSNodeCount = 0 kMDItemFSOwnerGroupID = 501 kMDItemFSOwnerUserID = 501 kMDItemFSSize = 2204679 kMDItemFSTypeCode = 943870035 kMDItemHasAlphaChannel = 1 kMDItemID = 923966 kMDItemKind = "Adobe Photoshop file" kMDItemLastUsedDate = 2006-01-03 00:21:09 +0000 kMDItemLayerNames = (jan. Spotlight . Other metadata types concern the file contents and is not information that ls reports. "public.plist Each file system has its own Spotlight store ensuring that this information travels with the files to which it belongs. hair) kMDItemOrientation = 0 kMDItemPixelHeight = 718 kMDItemPixelWidth = 450 .photoshop-image" kMDItemContentTypeTree = ( "com.db .adobe.Spotlight-V100/ .content" ) kMDItemCreator = "Adobe Photoshop 7. "public.journalHistoryLog ContentIndex.txt Page 268 of 271 Printed: Tuesday.image".db _exclusions. Different types of file content have different types of metadata.photoshop-image".db _IndexPolicy. 2008 3:20:00 PM Metadata with the types kMDItemFS* is information from the filesystem. "public.

Unix_tricks..PDF /Users/Shared/Books/Folders/Shakespear/COMEDY/MEASURE.txt Page 269 of 271 Printed: Tuesday. pass the item name as an argument to the name option: $ mdls -name kMDItemAuthors "12 DARE.PDF /Users/Shared/Books/Folders/Shakespear/COMEDY/TEMPEST. kMDItemMediaTypes = (Sound) kMDItemMusicalGenre = "Alternative & Punk" kMDItemStreamable = 0 kMDItemTitle = "Like It Or Leave It" kMDItemTotalBitRate = 191384 . you might be tempted to form a 'find/xargs/grep' command line to search through..PDF? Because Spotlight is looking for kiss AND me AND kate .swx /Users/saruman/1dot1/Miri/emulator/design/Emulator.. Tomorrow's tip will look at spotlighting on the command line.PDF /Users/Shared/Books/Folders/Shakespear/COMEDY/SHREW.PDF Why does this search produce more results than just SHREW.m4a" 12 DARE. To extract a specific metadata item.00006 kMDItemUsedDates = (2005-11-02 12:18:02 +0000.. say. and bit rate. kate" /Users/Shared/Books/Folders/Shakespear/COMEDY/LOVESLAB. But Apple has provided a much better way of doing this.00006 kMDItemResolutionWidthDPI = 72. Try passing a song file to mdls.PDF Or: $ mdfind "kiss me. 2006-01-03 00:00:00 +0000) The last few items show metadata describing the image size. title. May 27. 2008 3:20:00 PM kMDItemResolutionHeightDPI = 72. use mdfind. The metadata includes the genre. Here's another example: $ mdfind "miranda" /Users/Shared/Books/Folders/Shakespear/COMEDY/TEMPEST. resolution. all music for songs a particular Genre.m4a ------------- kMDItemAuthors = (Gorillaz) $ At this point. and even on what photoshop layers it contains. .txt /Users/saruman/1dot1/Miri/emulator/proposals/Emulator...a command line spotlight.psd /Users/saruman/Sites/adrian/data-base/rfc-index. A tip to save you life: "Remember to buy her a card!" Spotlight . To search the file system for specific content or metadata.pdf These results should match those found by pressing Command-Space and entering the same term into the Spotlight bar.Spotlight from the Command Line So far we've seen how to examine file metadata using the mdls command. Here's an example to search for the text "DOCSIS MIB" (it's a cable TV term): $ mdfind "DOCSIS MIB" /Users/saruman/1dot1/Miri/emulator/design/figure-1.

txt /Users/saruman/splotlight-test/file3. we have three simple text files: $ cat file1. $ mdfind -onlyin ~/splotlight-test "kiss kate" /Users/saruman/splotlight-test/file3. that document will match the search.txt /Users/saruman/splotlight-test/file3.Boolean Expressions Yesterday. contain.m4a /Users/sharing/Music/iTunes/iTunes Music/Compilations/Encore/03 Never Enough. .txt Kate $ cat file3. May 27. $ mdfind -onlyin ~/splotlight-test "kiss|kate" /Users/saruman/splotlight-test/file1. Spotlight will return those files that contain ALL the terms . If all three words appear somewhere in a document.m4a /Users/sharing/Music/iTunes/iTunes Music/Compilations/Encore/05 Like Toy Soldiers.txt There must be no white space around the '|' symbol. To illustrate.txt If we specify both terms. You must specify the full path name of the directory. Tuesday's tip showed us that songs include the metadata item kMDItemMusicalGenre Let's search for all songs that are 'Hip Hop/Rap': $ mdfind "kMDItemMusicalGenre == 'Hip Hop/Rap'" /Users/sharing/Music/iTunes/iTunes Music/Compilations/Encore/01 Curtains Up.m4a /Users/sharing/Music/iTunes/iTunes Music/Compilations/Encore/02 Evil Deeds. we saw how to use mdfind to find files that contain particular text.txt /Users/saruman/splotlight-test/file3.Unix_tricks.txt Note use of the option -onlyin to limit searches to files rooted in a particular directory hierarchy. 2008 3:20:00 PM (it ignores the comma) but not necessarily consecutively.m4a /Users/sharing/Music/iTunes/iTunes Music/Compilations/Encore/04 Yellow Brick Road.txt To test for "kiss" OR "kate". we separate the terms with '|'. kate Let's look for files that contain the word "kiss": $ mdfind -onlyin ~/splotlight-test "kiss" /Users/saruman/splotlight-test/file1. Tomorrow's tip will show how to perform more complex searches using mdfind. We can also look for specific metadata. And files that contain the word "kate": $ mdfind -onlyin ~/splotlight-test "kate" /Users/saruman/splotlight-test/file2.txt /Users/saruman/splotlight-test/file2. Spotlight .txt Kiss me.m4a Use mdls to discover other types of metadata for other content types.txt Page 270 of 271 Printed: Tuesday.txt Kiss me $ cat this example "kiss" AND "kate". or should not. or have particular metadata. Today we'll look at how we can be more exacting about what words a file should.

txt The comma is ignored by spotlight.PDF$" We escaped the dot because it has a special meaning to regular expression. make sure there is no white space between the terms. grep. Specify option -0 (zero) to both mdfind and xargs to fix this. but not "me".e.Piping its Output Just like the regular Unix find.txt:Kiss me. not just have .pdf". kate $ This is the same trick as used with "find | xargs" as is described here. and xargs.txt: No such file or directory $ (Tip: Use grep to make the search term case sensitive. we'll combine mdfind with regular Unix commands such as find.txt Again. 2008 3:20:00 PM What about not kissing kate? $ mdfind -onlyin ~/splotlight-test "kiss(-kate)" /Users/saruman/splotlight-test/file1. kate $ mdfind -onlyin /Users/saruman/splotlight-test "kiss me." | xargs -0 grep -i "Kiss me. $ mdfind "kiss kate" | grep -i "\." /Users/saruman/splotlight-test/file3.txt Page 271 of 271 Printed: Tuesday.txt:Kiss me. and used the dollar symbol to anchor matching text to the end of the term (i.txt Kiss me $ cat file3.txt:Kiss me.txt /Users/saruman/splotlight-test/file3.pdf in the middle).) The command choked on the filename "file 4. $ mdfind -onlyin ~/splotlight-test "kiss|kate(-me)" /Users/saruman/splotlight-test/file2.txt Tomorrow. To open all the files in Preview type: $ mdfind -onlyin /Users/shared/books "kiss kate" | xargs open To make the search term more selective.Unix_tricks. Spotlight . . "kiss" or "kate". And finally.txt Kiss me. the list of files returned by mdfind can be piped to other Unix commands for further processing." | xargs grep -i "Kiss me.pdf. kate /Users/saruman/splotlight-test/file 4. $ mdfind -0 -onlyin /Users/saruman/splotlight-test "kiss me. use grep and regular expressions. matching filenames must end in . kate grep: /Users/saruman/splotlight-test/file: No such file or directory grep: 4. May 27." /Users/saruman/splotlight-test/file1. The next command will filter the results from find leaving only files that have the extension ". Let's use grep: $ mdfind -onlyin /Users/saruman/splotlight-test "kiss me.txt" because it contains a space. Here's a simple example: $ cat file1." /Users/saruman/splotlight-test/file3.