Author: Unknown
Email Address:  barnum@nym.alias.net
Date: 21 July, 1997

Question: Help with some general Unix commands\concepts.


         Note on Unix: Peter Allan
 ("Unix" and some other words in this article are trademarks.)
      Feel free to copy this file and inform
      me of any errors you find. I am not responsible for anything
      you do before, during or after reading any of this file.
 ----------------------------------------------------------------------------
The latest version of this file can be collected by email
to peter.allan@aeat.co.uk  Subject="send goodies" Body="guide to unix"
 ----------------------------------------------------------------------------
 These books have been helpful to me:
       Peter Nortons's Guide to Unix
             by Peter Norton and Harley Hahn
                especially good on vi
       Unix Shell Programming
             by Steve Kochan and Patrick Wood
                especially good on filename and variable substitution and quotes 
 
 Also some of the famous O'Reilly Nutshell series.
 ----------------------------------------------------------------------------
 
 Note that there are several Unix-like operating systems.  These usually run on
 workstations, but there are version for high-end PCs.
 
 True Unix is a command-line system where you type your
 commands with their options and parameters.  To some
 extent it resembles DOS, but has more power.  Actual operating
 systems you buy, based on Unix, have graphical extensions.
 
It is a multi-user system, with many commands.  It is of course
possible to survive with only a few.  Whole books are written
about single commands.  This file makes no pretence at being complete.
 
##################################################################
 
 Each human user is known to the machine by a userid or id.
 A password is usually (should be) set for each id.
 
 Everything is case-sensitive.  ('A' and 'a' are different.)
 
 Here are the essential commands for beginners:
 
 ls
 ==  'ls' is the command to list filenames.
 It can have options (Flags) added after the command with a minus sign '-'.
 Arguments (parameters) can also be added.
 
     ls                 ordinary file listing
     ls -l              long file listing (file sizes, change times,..)
     ls -a              include filenames starting with '.', normally not listed
     ls -l -a           applies both the above options
     ls -la             identical to the previous command
     ls *.fort          list files called "ANYTHING.fort"
     ls -lat *.fort     the t means list them in time order, not alphabetically
 
 These examples show the main points of all Unix commands.  They are
 short, thay take options, they take parameters, you can combine
 things together.
 
 rm
 ==
 This command removes files (and directories).
 BY DEFAULT YOU GET NO SECOND CHANCE, AND ONCE THEY ARE GONE THEY ARE GONE.
    rm     filename(s)
    rm -r  filename(s)   Remove directories and all their contents.
    rm -i  filename(s)   Prompt for a y/n choice before each file.  GOOD IDEA !
    rm -f  filename(s)   Do not complain if the file does not exist.
                         Also overrides the -i flag.
 You cannot remove a directory unless it is empty.

 { To keep your files from accidental loss
 in csh
    alias rm 'rm -i'
    set noclobber
 in ksh
    alias rm="rm -i"
    set -o noclobber
 
 Do similar aliases for 'mv' and 'cp' if you wish. See below.

   (Note that not everybody likes this alias rm 'rm -i' idea.
    alias del 'rm -i'
    has an advantage if your alias goes missing
    or if your rm-happy habits grip you on another system.)

A script to wrap round rm is presented in the script
examples section below.
 }
 
 
 cp
 == This command copies files.
 cp file1 file2              copies the file1 to a new file file2
 cp /vmunix /home/allen      copies the file /vmunix to another directory,
                             so making the new file /home/allen/vmunix.
 cp -r mydir mynewdir        With the -r flag, copies directories too.
 
 mv
 ==  This moves and renames files and directories.
 mv file2 file3                    renames a file (or directory)
 mv /home/allen/vmunix /tmp        moves a file (or directory)
 
 pwd
 ===
 Unix has directories to help you organise your files.
 pwd   tells you your 'current working directory' or CWD
 An example directory name would be '/home/allen/handy/scripts'.
 The full name starts with a slash "/" and directory names
 are separated by slashes.
 (This is different from DOS, which uses the backslash \.)
 
 cd
 ==
 Changes your CWD.
    cd     takes you to your 'home directory'
    cd  /usr/lib    takes you to the dir '/usr/lib'
    cd  ..          takes you to the parent directory (eg /usr)
    cd /            The root directory - there are none above this.
 Your 'home directory' is the one you log in to.
 It is defined in the file /etc/passwd (unless some special
 network system is in use and then it is found in the output from
 'ypcat passwd').
 
 whoami
 ======
 This tells you your id name, eg 'allen'
 So you can type
 grep `whoami` /etc/passwd
 to see your home directory and some other things.
 allen:!:5037:1:P. Allan:/u/allen:/bin/csh
 ( The command 'grep' is for searching text files for strings. )
 ( The backquotes `` will be covered later. )
 The ! in the second field tells the computer that the encrypted password
 is held in another file.  This makes it a bit harder for dishonest people
 to get hold of your password.
 
 passwd
 ======
 This should be one of the first commands you use when given a new
 id.  It sets your password, and can do a few other things too.
 passwd
    You will be prompted for the old password (if any)
    and will be asked for the new password twice. (length = 8 characters)
 EXAMPLE
 % passwd
 Changing password for "allen"
 allen's Old password:
 allen's New password:
 Enter the new password again:
 %
 (There are restrictions on what you can choose, but they
 are in /etc/security and so not readable!)
 Good passwords are hard to guess.  I suggest always using all 8 characters
 and including at least one of [a-z], [A-Z] and [0-9].  There are programs
 around that guess many passwords, so unwise passwords include any based on
   names, real words, the Unix id, the hostname......
 
 
 man    Important !  Use it often.
 ===
 This is the traditional Unix command for finding things out.
 You may prefer to use the InfoExplorer which does the same thing.
 'man ls'      Tells you a lot more about ls than I did above.
 (correct use of 'man' requires you to have a variable MANPATH set
 so if it doesn't work for you that is probably the explanation:
 use InfoExplorer instead until you have fixed the problem!)
 I recommend reading the man page on at least every
 command in this article, and new ones as you meet them.
If you cannot find man pages on your system, ask your administrator.
 
 
 exit
 ====
 This stops your session or just one window (if in a window).
 When using a Graphical Interface a menu accessed with the mouse
 will be used to end your whole session.  The window schemes are
 not part of Unix, and vary greatly between manufacturers.
 'exit' detects jobs that are stopped (by CTRL-Z for instance)
 because they would die if you used exit.  Type exit a second
 time if you are happy with that.   Running background jobs
 continue.
 
 chmod
 =====
 Every file and directory has a 'filemode'.
 You can see this with 'ls -l'.
 
 cd ; ls -la
 (The ; separates two commands on the same line.)
 That shows you the modes of the files in your home directory.
 Included in the list should be two entries called '.' and '..'.
 These are:
      . the CWD
      .. the parent of the CWD
 
 You will see the owner (your id probably for all except ..)
 and the group (perhaps 'staff'), a file size,
 a file change time/date and at the start of the line
 a set of characters giving the filemode.
 EG
 total 312
 drwxr-x---  16 allen    staff       1024 Oct 21 14:07 .
 drwxr-xr-x  21 sys      sys          512 Oct 13 16:25 ..
 -rwxr-----   1 allen    staff        896 Oct 20 14:44 .cshrc
 drwx------   2 allen    staff        512 Oct  6 08:51 .elm
 -rw-r--r--   1 allen    staff         59 Oct 17 13:59 .exrc
 -rwxr-----   1 allen    staff        461 Oct 17 12:18 .login
 -rwxr-----   1 allen    staff        254 Sep 22 08:33 .profile
 drwxr-xr-x   2 allen    staff        512 Oct 18 15:02 cases
 drwxr-xr-x   3 allen    staff        512 Oct 21 14:07 doc
 drwxr-xr-x   5 allen    staff        512 Oct 11 13:54 handy
 drwxr-xr-x   4 allen    staff        512 Sep 23 09:23 info
 drwxr-xr-x   2 allen    staff        512 Oct  5 15:07 man1
 drwxr-xr-x   7 allen    staff        512 Oct 20 14:45 sasfort
 
 A d in column 1 means the entry is a directory. A - means it is a file.
 See 'man ls' under the -l flag for other possible file types.
 
 There are 9 other characters in 3 sets of 3.
 Columns 2-4  are for the file's OWNER.  (named later on the line)  (eg allen)
 Columns 5-7  are for the file's GROUP.  (named later on the line)  (eg staff)
 Columns 8-10 are for every other id.
 
 r means read permission is granted.
 w means write permission is granted.
 x means execute permission is granted.
            (=search permission if a directory)
 - means the permission is not granted.
 See 'man ls' under the -l flag for other possible modes.
 Notice that write permission on a directory allows you to
 remove files from that directory EVEN IF YOU DO NOT OWN THEM.
 
 chmod 640     file1      sets file1 to -rw-r-----
 chmod 755     file2      sets file2 to -rwxr-xr-x
 chmod go=     file3      sets file3 to -???-------  (removes group & other)
 chmod -R go-w $HOME      GOOD SAFE COMMAND: only you can write in your home dir.
 
 
 more, pg
 ========
 These two commands allow you to look at a text file.
 more .cshrc
 more /etc/motd
 They allow you to page through a file
 and search for strings.
 When in a more or pg session type 'h[enter]' to see what commmands you can use.
 
 file
 ====
 This makes an attempt to recognise files,
 classifying them as English Text or executable or data,  .....
 type 'file *' to see some examples.
 XLF:		ascii text
 doc:		directory
 handy:		directory
 unix:		ascii text
 .Xauthority:	data or International Language text
 .cshrc:		commands text
 .exrc:		English text
 
 diff
 ====
 This important command tells you the differences between
 two text files.  It also tells you whether two binary files
 differ.
 
 wc
 ==
 This does word counts etc.
 In fact it counts lines, words and characaters unless you tell it not to.
 % wc .login
       6       7     461 .login
 My .login file has 6 lines, 7 words, and 461 characters.
 Of course the computer has a strange idea of what makes a "word".
 
 find
 ====
 To list all files or directories named 'core' under the CWD .
 find . -name core -ls

 To remove all files named 'core' under the CWD.  Do this:- core files are
 rarely useful because you need the matching source code to use them easily.
 find . -name core -exec rm {} \;

 To list all files or directories named SOMETHING'.core' under the CWD .
 find . -name '*.core' -ls
 To name all directories called 'man' under /usr.
 find /usr -name man -type d -print
 The next 2 examples are useful.  They remove temporary files you no longer want.
 find /tmp /var/tmp -mtime +3 -type f -user allen -exec rm {} \;
 find /var/preserve -mtime +8 -type f -user allen -exec rm {} \;
 
 The next example names files and directories
 which EITHER have changed in the last 2 days,
 OR have execute permission for the owner.
 find . \( -mtime -2  -o -perm -100 \) -print
 
 Checking what has changed in the last N days is a great
 way to find out why your computer isn't working any more !
 
 pr
 ==
 This adds titles and page numbers to your text files.
 Try it on a few example files.
 pr /etc/sendmail.cf | more   (keep pressing enter to watch the file
  go past slowly)
 
 lpr
 ====
 This prints. (Use lprm to cancel a print job.  lprm will fail when a
 printer is remote.  rlprm handles this. rlprm is available from Peter Allan)
 
 
 Text Editors    
 ============
 You have on your machine the editors 
       'vi'      Bill Joy's editor, much liked by me
       'emacs'   A more recent graphical interface editor
   and 'e'       IBM's own offering.
 I am describing here vi (and ex) briefly.
                Norton and Hahn's book highly recommended!
 
 Both vi and ex start by reading your file
 (if any) called $HOME/.exrc .  You can put in it 
 any commands you like to control your use of these editors.
 EG 'set nu' , 'map .....'
 
 ex:
 ===
 ex is an old line editor and I do not expect anyone to use it interactively
 (except maybe to practise it).  
 It is useful in shell scripts, and it forms a part of the vi editor.
 Option:  A useful option is '-' that keeps the program 'silent'
 so that it does not report its actions.  I use this in shell scripts
 (after debugging).  It is important if the script is to be run in
 the background.
 
 Commands in ex are entered at the colon : prompt 
    [return]    The return key moves you to the next line.
    1           A number (and return) takes you to the line of that number.
    s/one/two/
                This is the most important ex command I think. 
 As used here it changes the string 'one' into the string 'two', only once on the line. 
 Other examples are:
 :1,10s/one/two/          change 'one' to 'two' on lines 1 - 10
 :1,$s/one/two/           change 'one' to 'two' on lines 1 - the end
 :1,$s/one/two/g          change ALL 'one's to 'two's (on lines 1 - the end)
 :1,$s/one/two/gc         as above, but with a y/n choice for each change
 :1,$s/ //g               change all ' 's to ''s (deleting all spaces)
 :1,$s/^ //               delete single spaces at start of line
 :1,$s/ $//               delete single spaces at end of line
 :1,$s/ /^M/g             Type Ctrl-V Ctrl-M here.
                          Changes all spaces to newlines.
 
 Special characters: . * ^ $ &  [ ]  \< \>
     (Use \ if you want to escape special meaning)
     .  in the first string means any one character
     *  in the first string means Zero or more of the preceding character
     ^  in the first string means start of line
           (The key is shift_6, this doesn't print correctly.)
     $  in the first string means end of line
     &  in the second string means repeat the first string
   [ ]  as for the shell section below
    \<   beginning of word
    \>   end of word  (use as e\> to find 'e's at the end of a word.)
 :1,$s/  *$//             delete multiple spaces at end of line
 :1,$s/^/      /          add 6 spaces at start of line
 :1,$s/.*/mv & &.old/     & means the first string  (use after ':re !ls')
 :1,$s/[a-z]/\u&/g        Changes all [a-z] to [A-Z]. (\l& for lower case)
 
    g/search/             This gives another command to lines
                          with the string 'search'
 :1,$ g/search/s/ /X/g    Lines with 'search' get ' 's changed to 'X's.
 
    !                     Shell escape ( :! ls )
    set                   There are variables you can set
 :set nu                  display line numbers
 :set nonu                No line numbers
 :set directory=/eprexe   Choose a directory for temporary files.
      (/var/tmp is the default.) (useful if a disk gets full)
 :set all                 show you all settings
    re                    Read in a file in after the cursor. (only one file, * may fail ) 
 :re !date                Read a command's output after the cursor.
    vi                    Switch from ex to vi.
    w                     Write file to disk (can give filename)
 :w!                      Insist on writing, even if ex was started read-only.
 :w >> longerfile         This appends to a file.
 :1,10 w filename         Write only lines 1 - 10.
    q                     Quit.  (q! to quit if file has been altered.)
    x                     Write file and quit.  ( can also be done by 'wq')
    n                     go on to next file ( after 'ex file1 file2 file...')
 
 
 vi: ex mode
 ===
 In vi you can do these ex commands (as well as those listed above).
 :w      do one command (w) in ex mode
 :vi     begin vi mode
 
 vi: command mode
 === 
 These commands are typed without the return key.
 Most of them can be preceded by a number.
 (eg 10w means do 'w' 10 times,so move forwards by 10 words.)
 They take a bit of learning, but once you are familiar with
 them you can do almost anything in a few keystrokes.
 
 u                        'u' undoes the last command.
 U                        'U' tries to restore that one line
                           to its original state.
  ~                         Changes case of letters.  a->A , A->a
  .                         Repeat last commmand done in command mode.
 ma                        'm' is for mark.  Here we mark (invisibly) a place
                            as 'a', so we can return to it later. (with 'a)
 ZZ   write and quit  ( = :wq)
 Q                         begin ex mode (and stay in it)
 J                         Join next line to this one.
 ^G                        describe current file
 !(M)command               Introduces shell command (ending in return).
 The text moved over is replaced by the command's output.
 (eg  '!10+sort' sorts 10 lines into order (starting at current line)).
 
 Movement  '(M)' is used in this note to denote a movement command.
 This is a big subject !
 Cursor keys and hjkl move you one character left, down, up and right.
 ^u  ^d   up and down  (scrolling in half-screenfuls)
 ^f  ^b   forward and back (jumping in whole screenfuls)
 G        to start of last line
 1G       to start of line 1 (or other chosen number) 
 w        to next start of word
 e        to next end of word
 b        back to last start of word
 f F      find (on current line) the character typed (EG fa forwards, Fa back) 
 return, +  down one line
 -          up one line
 ^        to first non-blank character on line   (^ is shift_6)
 n|       n is a number , go to column n  (=0 if n not given)
           (The | is the pipe character outside of vi.)
 0 or |   to start of line
 $        to end of line
 'a       to place previously marked with 'ma'   
 ''       to the last place moved to (try for example G'')
 (  )     to the start (end) of 'sentence'
 {  }     to the start (end) of 'paragraph'
 
 [iaArRsScCoO] (Insert/Overwrite/Change)
 These commands (except 'r') involve string input which is ended with 'escape'.
 The letters have the following meanings:
    i   insert before cursor.
    a   insert after cursor.
    A   insert at end of line.
    r   replace one character with the one typed next
    R   replace text with the string typed next
    s   substitute string for the current character
    S   substitute string for the current line
    c(M)   change (with movement command): change the text moved over
    C   change to end of line
    o   insert new line after current line (o is for open) 
    O   insert new line before current line
 
 Deletion (keeping the deleted text in a buffer)
 dd           delete current line
 D            delete from cursor to end of line (= d$)
 d(M)         deletes from cursor to new position after move
 x            deletes current character
 X            deletes character to the left
 
 Yank
 yy            copies the current line into a buffer
 Y             = yy
 y(M)          copies into a buffer 
 "a(a yank)    copies into a named buffer ('a')
 
 Put
 p      put after current position (same line or next line)
 P      put before current postion
 "3     from chosen buffer (3 buffers ago)
 "a     from chosen buffer ('a')
 
 Examples using deletion, yanking and putting:
     xp               swap 2 characters
     ddp              swap 2 lines
     5yyP             copy 5 lines
     Dpj$p            copy the ending of a line to the next line
 
 Screen control
 z.             position current line in screen centre
 z-             position current line at screen bottom
 z+             position current line at screen top
 ^L             redraw screen
 
 vi:
 ===
 Searching a file   ( [] . * ^ $ have the meanings you'd expect.)
 /          a search from the current position to the end
            and from the beginning to the current position again
 ?          like / except it searches backwards
     n      find next  in direction of the search
     N      find next against the direction of the search
     (so '?string' 'N' will look down the file)
 Can be made case-insensitive (a=A)  by 'set ignorecase'.
 
 
 map       You can define a key to have the meaning you want.
           This is useful to save typing (and to correct the meaning
           of the cursor keys on some keyboards). 
 EG
    map t JJJj                    Join 3 lines and go down one
    map ^V(left arrow) h          Make left arrow key move you left
 
 
 
 
 Compiling etc
 =============
 f77,xlf   These are two names for the IBM  Fortran 77 compiler.
 f90       This is the NAG Fortran 90 compiler {really a C translator}.
 Fortran source files should have names ending in '.f'.  Object files
 have names ending in '.o'.
 
 ld        A linking or binding program to join modules.
 You would normally use the compiler to do this.
 
 make      A configuration tool - used to organise compile and link steps.
 See my example Makefile.  The file consists of a list of 'targets'
 with the input steps and actions to perform.  This means that if you
 have several source files and you change one of them you can compile
 only the new source and link it to the unchanged objects.
 The Makefile needs tabs (not spaces) at the start of some lines.
 
 sccs      A code control tool, keeps each version of the files edited.
                    (eats up disk space !)
 
 cc        A C compiler.
 
 
 
 
 File Transfer
 =============
 mail:  This is a fast and convenient way to send files, provided they
        are not too large.
 ftp:
 To get a file from the mainframe to the workstation:
    % ftp hdimvsp
    > get 'irs970.euref.pds(alpc)' alpc.f
    > bye
 Other ftp commands include 'put' and '?' (for help).  With 'mget' there
 is filename globbing.  Getting files onto the workstation with () in
 their names is a nuisance because of the meaning the shell gives to ().
 When transferring files between workstations it is easier to use the command:
       > get alpc.f
               No name given for the new file => same name will be used.
 Provided you are happy with the security of your unix id you can
 keep a .netrc file in your home directory, holding the password
 to another (EG mainframe) id.  This saves you from typing your password
 for ftp sessions.  If .netrc is not used, the ftp program prompts you
 for the username (type return for the same username)
 and password.  ftp only works to id's that have a password.  
       % ftp hdimvsp
       Connected to hdimvsp.kfk.de.
       220-FTPSERVE IBM MVS V2R2 at hdimvsp.kfk.de, 15:11:48 on 11/03/94
       220 Connection will close if idle for more than 5 minutes.
       Name (hdimvsp:allen): irs0c7
       331 Send password please.
       Password:
       230 IRS0C7 is logged on.
       ftp> get 'irs0c7.pma.text(sasic2)' sasic2.f
       200 Port request OK.
       125 Sending data set IRS0C7.PMA.TEXT(SASIC2) FIXrecfm 80
       250 Transfer completed successfully.
       60516 bytes received in 0.275 seconds (214.9 Kbytes/s)
       ftp> bye
       221 Quit command received. Goodbye.
 
 The mode of ftp can be varied between text (ascii) and binary.
 Sending binary files in text mode results in rubbish.
 
 You can get a bit of unix-emulation out of the mainframe
 by using commands inside ftp such as :  
      cd 'irs970'
      cd euref
      ls
      help
      cd ..
 
 "Anonymous ftp" is a means by which people can make their files available for anyone else
to collect.   To fetch a file you use the login name 'anonymous' and as a password use your mail
address.  Many products (both useful and silly) can be obtained like this from computers
offering the service. Obviously it would be possible for someone to plant a harmful
program there.

ftp relies on another program ftpd existing on the target host.  So you can run ftp
on a PC to a unix workstation, but not usually in the other direction.
 
 uuencode, uudecode
 uuencode can convert binary files into printable character files
 so they can be sent through the mail, and uudecode can convert them back.
 
 
 ln
 ==
 Using 'ls -l ' you will see something like this
     -rw-r--r--   1 allen    staff      12386 Oct 21 15:24 XLF
 The first number is the LINK COUNT. It shows that the file XLF has
 only one name.  I can give the file a second name like this:
 ln XLF also.xlf
 ls -l  XLF also.xlf
     -rw-r--r--   2 allen    staff      12386 Oct 21 15:24 XLF
     -rw-r--r--   2 allen    staff      12386 Oct 21 15:24 also.xlf
 This means that if I remove one of these links the other remains.
 If I change one (with an editor, say) they BOTH change.
 You can tell they are both links to the same file because they have
 the same INODE NUMBER (ls -i, and find -inum).  This is an index
 arrangement used by the computer to get from the filename in the directory
 to the parts of the disk where the file is actually stored.
 % ls -li XLF also.xlf
 38999 -rw-r--r--   2 allen    staff      12386 Oct 21 15:24 XLF
 38999 -rw-r--r--   2 allen    staff      12386 Oct 21 15:24 also.xlf
 
 'ln -s ' makes a SYMBOLIC LINK.  This is merely a pointer to the other
 filename as can be seen with 'ls -l'.   Removing the 'pointed-to' file
 makes the symbolic link useless.  They are good for giving alternative
 names to directories.  (See /bin and /u.)
 They can also point across to another filesystem, so saving space on this one.
 A lot of the unix commands have this arrangement.
 The file /dev/null is a null file that :
      if you write to it, your writing is thrown away
      if you read from it you get an end-of-file
 'ln -s /dev/null fort.40'  therefore makes a file 'fort.40' that you 
 can write to as much as you like but it won't waste space on your disk.
 
 which  commandname(s)
 EG which ex
 This will tell you which 'ex' command you will execute if you
 call the command without a pathname.  There are commands called
 'ex' in /usr/bin and /usr/ucb.  If you get problems with a command
 (including  a new script of yours) it might be because the name clashes
 with a name found earlier in your path.
 
 
 crontab
 This command allows you to execute commands at the time of your choice,
 especially regular commands.
 This is my crontab file produced with the  command 'crontab -e'.
 I am using crontab here to remove unwanted files and save disk space.
    It allows me to make temporary files and directories called 'core'
    knowing they will be removed automatically.
 
 # remove (useless) core files every night at 23:15
 15 23 * * * find . -name core -exec rm -r {} \; 2> /dev/null
 # remove my files unchanged for 3 days under /tmp and /var/tmp
 30 23 * * * find /tmp /var/tmp -mtime +3 -type f -user allen -exec rm {} \;  2> /dev/null
 # remove my files unchanged for 8 days under /var/preserve
 45 23 * * * find /var/preserve -mtime +8 -type f -user allen -exec rm {} \;  2> /dev/null
 
 You could get it to send you mail on your birthday, warn you of
 Friday 13th's and do other amazingly useful things.
 
 The command 'at' is similar, but for one-off timed commands.
 
 
 SOME MORE TO READ ABOUT 
 These include text processing which can save you quite a bit
 of high-level programming. Also monitoring of disk space and
 printer queues, file conversion,...
 
       awk, cat, colrm, cut, dd, df, du, echo, egrep, elm, fgrep, finger, grep,
       gzip, head, limit, lpq, lprm, mkdir, mesg, nice, nl, paste, read, rmdir,
       rsh (sometimes known as remsh) , script, sed, strip, sum, tail, tar, touch,
       umask, w, write, xsend

 perl is a hot (newish) language worth learning.
 
 ##############################################################
 The Shell 
 =========
 
 The output from the command
 grep `whoami` /etc/passwd 
 should resemble this:
 allen:!:5037:1:P. Allan:/u/allen:/bin/csh
                                  ^^^^^^^^---------this is my login shell
 
 The entries in /etc/passwd are separated by colons ":".
 The last entry is the "login shell" or the program you run automatically
 when you log in.  There is a choice of shells such as:
   /bin/csh    The C shell
   /bin/sh     The Bourne shell
   /bin/bsh    The Bourne shell  (again)
   /bin/ksh    The Korn shell
 
 You choose with the command 'chsh' or 'passwd -s' which you want.
 The Korn shell is an extended version of the Bourne shell, and
 is the standard one under AIX.
 This author likes the C shell because of the history substitution
 he is used to, but the Korn shell has inline editors and you probably
 want to use that.
 
 The important point about your shell is that it is the program that reads
 your commands.   If I do the following  ( % is the prompt):
 
 % alias rm 'rm -i'
 % ls
 file file.old list list.old prog prog.old wigwam.old
 rm *.old
 rm: Remove file.old? y
 rm: Remove list.old? y
 rm: Remove prog.old? y
 rm: Remove wigwam.old? y
 
 I have first aliased the command 'rm' to be 'rm -i'.
 This is the safe advice I gave earlier to prevent accidental loss.
 The shell is the thing that remembers that, and when I later type 'rm'
 knows that it should prompt me for each file I am about to remove.
 The shell is also the thing that recognises the special symbol *
 and looks for all the files called *.old.
 So in the above example the shell changes the command to
 
 rm -i  file.old list.old prog.old wigwam.old
 
 and only then executes the 'rm' command.  When a command fails to
 do what you expect it could be because the shell has altered it.
 
 The shell also had to find the command 'rm'.  It searches a path
 which is defined in a variable.  Getting your path set correctly
 is important for security and to make sure that all the commands
 you want are available to you.
 
 When you have variables you always want set, it is a good idea to set them in 
 a file executed as you log in.
 
   For C    shell:    .cshrc
   For Korn shell:    .kshrc
 
 This is a path set in the C shell.
 You can see your variables with the command 'set'.
      (LINES ARE BROKEN HERE FOR CONVENIENCE IN PRINTING - NOT REALLY)
 path	(/usr/ucb /usr/bin /etc /usr/sbin /usr/bin/X11 /sbin
          /usr/lib /usr/lpp/x3270/bin /usr/local/kfk/bin /u/allen
          /u/allen/handy/bin /u/allen/handy/scripts
          /u/allen/sasfort/sasbin .)
 
 It was set in the file .cshrc with the line
 set path=(/usr/ucb /usr/bin /etc /usr/sbin /usr/bin/X11
           /sbin /usr/lib /usr/lpp/x3270/bin
           /usr/local/kfk/bin ~  ~/handy/bin
           ~/handy/scripts ~/sasfort/sasbin .)
 
 Sadly, the Bourne and Korn shells set their variables differently from the Csh.
 
 Directories are separated by colons. The word 'set' is not used.
 PATH=/bin:/usr/bin:/usr/local/kfk/bin:.
 
 In the C shell example above, the user wanted some directories listed 
 in relation to his own home directory.  The tilde (~) is the C shell symbol
 for a home directory.  You can see it has been expanded to /u/allen in
 the path variable.
 
 In both examples above the current directory is included in the path,
 BUT IT COMES LAST.  You should not include a . in your path near the
 beginning because it might give somebody else an opportunity to break
 into your id.
  
 The shell has a number of other handy features.....
 
 Standard Output redirection: >
 date
 date > my.date.file
 The date command alone tells you the time and date.
 With the > sign it doesn't !  But it does make a new file
 containing the output from 'date'.
 This will create a new file or overwrite an existing one.
 Setting "noclobber" in csh or ksh  will prevent this from overwriting
 existing files.
 
 Appending from standard output: >>
 date     >  my.date.file
 hostname >> my.date.file
 whoami   >> my.date.file
 echo  This   should   be   quite   clear.   >> my.date.file
 echo 'This   should   be   quite   clear.'  >> my.date.file
 
 Standard Input Redirection: <
 colrm 17 19  < my.date.file
 This gives the file as input to the colrm command.
 
 Standard Error Redirection : 2>   (with bsh and ksh)
 cat /etc/fstab   : this should fail with permission denied.
 cat /etc/fstab  > stdout
 cat /etc/fstab 2> stderr
 Now which file holds the error message ?  stderr
 Error messages come on a different output stream.
 
 Here Documents: <<
 Run this script 
 (scripts need execute permission:  "chmod u+x filename" )
         ##### next line is the beginning #####################
 #!/bin/sh
 myDFfile=/tmp/df.$$
 df > $myDFfile
 cat $myDFfile
 echo '.... press enter .............'
 read aaa
 ex - $myDFfile  << ENDMARKER
 1,$ s/[a-z]/\u&/g
 x
 ENDMARKER
 cat $myDFfile
         ###### previous line was the end #####################
 Notice how the '<< ENDMARKER' (or whatever word you choose)
 allows you to define input on the lines before the ENDMARKER appears.
 Here 2 lines of input are given to the command 'ex'.
 {ENDMARKER must be at the start of the line.}
 
 Pipes: |
 du -s -k  * | sort -n
 Output from 'du' is given to 'sort' as input.
 This is amazingly useful.  You will soon wonder how you
 managed on the mainframe.
 
 Return codes, && and ||
 Commands usually return "return codes" to state whether
 they worked, or how they failed.
 These symbols come between commands.
 && means continue if the last command succeeded.
 || means continue if the last command failed.
    ls && date
    ls || date
    rmdir /etc/WHATDIR  && date
    rmdir /etc/WHATDIR  || date
 
 Metacharacters : Some of these these allow expansion ("globbing') of filenames
 *     Acts as a part of a filename  (but not an initial '.')
 ?     Acts as a single character in a filename  (but not an initial '.')
 []    Acts as a single character, depending on the contents.....
         file[12345] : file1 file2 file3 file4 file5
         file[1-5]   : same as above
         file[a-cA-C]: filea fileb filec  fileA fileB fileC 
 
 ;     End of commmand - like a newline
 
 ()   This produces a second shell.  In the example below 
      the first shell talks to the second through a pipe.
            cd /home/allen
            tar cbf 20 - handy | ( cd ; tar xvbBpf 20  - )
      Why is this different from a "cp -r" command ?
 
 
 Quotes
 ''   Single Quotes   These protect all the contents from being
                      read by the shell.
       echo '     one    two three *    '       is different from
       echo       one    two three *    
            
 ""   Double Quotes   As above except that  a few (4 ?) symbols still
                      get read by the shell.   $`  Can't remember
                      the others just now!
 
 \    Backslash    Quotes the next single character IF it has a
                   special meaning.
                   Also used in other commands such as vi and grep.
 Notice how I used it in my 'find' commands in 2 ways:
 to keep the () from being read by the shell
             'find' uses the () in its OR condition
 to keep the ; from being read by the shell
             'find' uses the ; to mark the end of the '-exec' command
 It can be used to quote 'return' characters too, if they come immediately
 after the \,  and let you type a command over several lines.
 EG
 f77 -v -qextchk -qlistop -qnomaf -qrndsngl -qspnans -qxflag=alias \
     -bloadmap:LOADMAP -bnso -bI:/lib/syscalls.exp  \
     -NB100 -NC2048 -ND8192 -NN1048576 -NP500 -NA16384 -NT65536 *
 
 ``   Backquotes      Is substituted by the output of the command.
                      EG   myvar=`date`
 
 Foreground and Background and other controls
 
 command &   runs command in the background - You don't wait for it
             to finish - useful on long commands that don't need your input.
 jobs        Tells you about current background and suspended jobs.
 %1          Brings job 1 back to the foreground.
 ^Z          This is CONTROL-Z  and it suspends the current foreground job.
 bg %1       Restarts a suspended job in the background.
 ^D          End of input - for Bourne shell and commands reading standard input.
 ^C          Terminates current foreground job.
 ^U          Clears current input line - quickest way to erase typing errors.
 ^Q,^S       ^S=SLOW ; stop the screen scrolling  ^Q=QUICK ; start it again
 ^V          Prevent the next character from having its normal action.
             EG ^V^M allows you to type a "return" character in your text.
             ^V is used in this way elsewhere, including vi and wordperfect.
 ps          Tells you about running and stopped processes.  Needs arguments.
 kill        Sends signals to processes - to kill them for instance.
                (Best way to check success is to repeat the kill.
                 the second try will fail if the first succeeded.)
 
 
 Shell Scripts  (These need execute permission  
                 chmod u+x, chmod 750 or similar)
 I am assuming you choose the Bourne or Korn shells for scripts.
 I do this and reserve the C shell for interactive use.
 ======================================================
 #!/first_line
 The first line of the shell script normally selects
 the shell program to run.  EG '#!/bin/sh' or '#!/bin/ksh'
 There can be options, of course, such as '#!/bin/sh -x'
 
 Variables.
 These are very important in shell programming. They begin with a $ sign.
 Legal characters in the names are [A-Za-z0-9_],  "." is NOT allowed.
 You can set  and use your own variables like this.
         myname='the string I choose'    (The $ is NOT used when setting.)
         echo $myname
         echo $mynameX          : NOTHING! 'mynameX' is not set
                       (May cause errors as some commands need an argument. e.g. test)
         echo ${myname}X        : OK = 'the string I chooseX'
         echo $myname$myname    : OK = 'the string I choosethe string I choose'
         echo "$mynameX"        : OK 'mynameX' is not set so = ""
  
 
 
 Read-only parameters
 $0      The name of the script. 
 $1 (n)  The 1st (nth) argument given to the script at run time.
 $*      All the arguments 1-n.
 $@      As $*, but whitespace in an argument is preserved.
 $#      The number of arguments given to the script at run time.
 $$      The process number of the shell program as it runs.  This is useful
         for making output files with names that are likely to be unique.
 There are further environment variables such as $USER and $HOME
 that you can see by typing 'sh -c set'.
 
test    Normally used in an "if" command  : if ( test "$myname" = ZZZ ) then
'[' is another name for the 'test' command, so you often see  [ "$myname" = ZZZ ]

if ( ) then
       target1
else
       target2
fi                   ( 'if' backwards ! )

 
lc=0
while ( test "$lc" -lt 11 ) do
{
lc=`expr  $lc + 1`
echo loop counter is $lc
}
done


for file in  1 2 3 *.txt
{
more $file
}

read  myvar
echo $myvar

myfunc ()   You can make your own functions inside a shell script.
{
ls $1
} 


daynum=`date +%w`
case $daynum in 
0)  echo sunday;;
1)  echo monday;;
2)  echo tuesday;;
3)  echo wednesday;;
4)  echo thursday;;
5)  echo friday;;
6)  echo saturday;;
esac





#!/bin/sh
# You could call this script "del" for deleting.
# An "rm" that won't delete anything if you intend to delete something
# like "xx*" and inadvertantly give it "xx *", assuming "xx" doesn't exist
f=-f
i=-i
r=-r
for a in $@ 
{
case "$a" in
$f);;
$i);;
$r);;
*)
f=""
i=""
r=""
flag=""

   if [   -f "$a"  ]; then
      flag=TRUE
   fi
   if [  -d "$a"  ]; then
      flag=TRUE
   fi
   if [ !  "$flag" ]; then
    echo "File or directory $a is not found.   No removals."
    exit 1
   fi
;;
esac
}
rm "$@"





 It is legal for one script to call another.
 It is also legal to use recursion as one script calls itself,
 but this needs to be controlled as you would expect.
 
"getopts" is a command useful for options in scripts.
 
Errors in scripts are usually reported with the name of the script
and the name of the offending command.

How many of your operating system's commands are actually scripts ?
And what can you learn from them ?

 
 Problems
 ========
 Is my command just taking a long time ?
 Is it waiting for me to type something ?  (such as CTRL-D) (try 'cat')
 Has this worked before ?
 When did it last work ? What has been changed recently ?
 Do other id's have the same problem ?
            (Or have I set my environment badly ?)
 Am I executing the right command ?
            (path variable, which shell am I using ?)
 Have I recorded accurately what I did and what happened ?
 Where are my error messages going ? ('script' command)
 Is there a way to test PART of the action I tried ?
 Have I read the manual entries for the command ?
 Do all relevant files exist ?  With suitable filemodes ?
 What if I log out and log in again ?

 And for those who can interpret the output.....
 Do dbx, trace, strace, truss, or par reveal anything useful ?
 

 
 
 The administrator and his role
 ==============================
 There is a priviledged id called 'root' which can do many things
 not open to most users.  None of your files are safe from reading
 and writing by this superuser.  It is good policy to have
 only 1 or 2 trusted and skillful people using the root id, and only
 when they need it; to prevent the severe consequences of mistakes made
 as root.
 
 You should try hard not to annoy your administrators.
 
 Among other things the administrator will:
      Choose and install (or reject) new hardware and software
      Monitor the security situation:
         catching people who try to crack passwords, and
         who search for files with particular filemodes
         and who try other devious tricks
      Look after resources such as printers and disk space
      Make backups of files, from which he can recover if there is a
         disaster of any kind. (The top disaster is theft of all hardware
         including the tape drive containing the just-completed backup.)
      Advise users of planned changes
         (So keep up to date with reading your mail.)
   
 If there is a problem such as a printer out of action,
 or a disk is full, or a disk seems to be missing,
 you can tell the administrator about it politely.
 
 He can reset your password if you forget it.
 
 You should be aware of his back-up arrangements so that if
 you lose/damage a file of yours, you know what you can expect
 from him.  (If the backups are done overnight (at 1 o'clock)
 he will not have a copy of today's new file.)  It is also good to
 know how long each tape is kept, as this tells you the time during
 which you must report a lost file.
 
 You may also find him helpful with your unix questions,
 AFTER you have looked in the manual/infoexplorer/answerbook and any
 notices your administrator publishes : "Frequently Asked Questions".
 
 He may complain to anyone "hogging" resources like disk space or
 CPU time.  You might find him asking you to run processes at a
 different time or on a different machine.  He may ask to move
 some files to tape to free disk space.
--------------------------
And a poem for the admins....

      /usr/games/admin
      ================
      (All trademarks acknowledged.)
      
      I was working in the office and the time was getting late
      when they told me of the problems that I must investigate
      as the PostScript wasn't scripting and the DAT drive had no lights
      I could see as it was starting it would be one of those nights.
      
      There were deadlocks in my mutex, all my pointers missed the node.
      There were children with no parents, and I think they wrote the code.
      My recursion never ended, nothing ever got to free().
      My machines were going crazy, it was either them or me.
      
      Power cycles were forbidden as the uptimes had to rise,
      all the manuals were hidden far away from searching eyes.
      Once again I chose a printer, but I really had no choice,
      so I telephoned the helpline and they recognised my voice.
      
      With the server not responding and the network up the creek
      (someone changed the DNS box in the middle of last week)
      I was reaching for the media for a spooler reinstall
      when I saw a station crashing and it wouldn't boot at all.
      
      But the boot disk wasn't altered, all the files were still in place.
      I said this to the computer and it laughed right in my face.
      In the end I got it started, but it only mounted root.
      There were few commands to play with but it had begun to boot.
      
      Forty minutes or so later, with my inodes mostly cleared,
      I was rather short of data, over half had disappeared.
      But at least I had my backups, surely they were home and dry,
      write-protected, cycled, labelled - so my confidence was high.
      
      I'd forgotten that the tape drive was in a disgusting mess.
      All my  diagrams were PostScript and I hadn't DPS.
      There's an answer to these crises TFM does not relate:
      Go and hide inside the libraries of a distant friendly state.

