Processes and Multitasking Terminology can be confusing Multiprogramming - the system can execute more than one program at the same time Multiuser - system can simultaneously service more than one online terminal Multitasking - system can execute two or more tasks at the same time In common usage, these all refer to the same thing UNIX System Programming
Multitasking Operating Systems Multitasking OSs are designed to perform a complex juggling trick They must: Allocate resources, such as CPU cycles and memory, and assign priorities so each process receives adequate attention UNIX System Programming
Multitasking Operating Systems Higher priority jobs need more or larger CPU time- slices without neglecting lower priority jobs Jobs that are waiting for some resource (such as user input, input from disk, or a shared output such as a printer) need to handled without wasting CPU time UNIX System Programming
Multitasking on a Single CPU Obviously, a single CPU cannot run multiple process simultaneously The OS simulates simultaneity by switching between tasks at a high rate Each switch is a time-slice Since thousands or hundreds of thousands of CPU cycles can go by between user keystrokes, this gives the appearance of simultaneous operation This resource allocation, priority processing, and timeslicing is all done by the scheduler UNIX System Programming
Process States Unix uses several process states to determine the current condition of a process Runnable Stopped Page Wait UNIX System Programming
Process States Unix uses several process states to determine the current condition of a process Runnable Stopped Page Wait UNIX System Programming
Process States Non-Interruptable wait Typically for disk I/O or NFS requests Sleeping Idle Terminated UNIX System Programming
Unix Scheduling Algorithm Unix schedules tasks in this order: Highest priority task that is Ready-to-Run and loaded in memory and preempted Ties for priority are broken by time spent waiting (also known as Round-Robin scheduling) If no one is ready to run, the kernel idles until the next time-slice UNIX System Programming
Unix Images and Processes Each process receives a unique numerical process identifier (pid) when it is started Even if the same program is run multiple times, each instance will have a unique PID UNIX System Programming
Unix Images and Processes A process has an image in RAM Sharable Text Segment Data Segment Stack Segment UNIX System Programming Context Saved Registers Open Files Process State
The Shell is a Process The principle process you interact with is the shell The shell can run some commands (built-ins) itself but for most commands, it creates a separate process It usually waits for the command process to finish and then gives you a new shell prompt UNIX System Programming
The Shell is a Process What if you could tell the shell not to wait? You could then instruct the shell to do something else while the first command was running in the background Voila! Multiprocessing in action! UNIX System Programming
Background Execution Caveats Some processes do not lend themselves to background execution vi, for example, expects user input and regular screen output Commands that produce screen displays can create havoc with each other if running concurrently UNIX System Programming
Background Execution Caveats However, Unix processes such as print spooling seem to be made for background processing Background processing should normally only be used with commands that are getting their input from some place other than standard input and writing their output to other than standard output UNIX System Programming
Background Processes Using & The simplest way to create a background process is to invoke the command with a trailing & Example: % sort addresses > mail_list & [1] 21885 % UNIX System Programming
Background Processes Using & This will sort the file addresses in the background and place the results in the file mail_list 21885 is the unique pid of this instance of the sort process [1] is the job number that is assigned by the current shell UNIX System Programming
wait What if you initiate several background processes and then you decide that you need to wait until they have all finished before doing something? UNIX System Programming
wait will make the shell wait on all the background processes to complete before giving you another prompt If wait immediately returns a prompt, there are no processes executing in the background UNIX System Programming
show Proces. Ses - ps Common options: a - include processes that are not owned by you x - show processes that don’t have a controlling terminal r - show only running processes f – full details u – for a specific user e - everyone Example: clyde% ps -a UNIX System Programming
Example of ps ps -a PID TT STAT 26129 p 1 S 26632 p 1 R 25408 p 2 IW TIME COMMAND 0: 00 -csh (csh) 0: 00 ps -a 0: 00 -csh (csh) 25937 p 2 IW 0: 00 vim lab 3 S -> Sleeping R -> Running IW -> Idle (and swapped out) UNIX System Programming
jobs Lists the active jobs under shell job control Syntax: jobs [-l] -l List process IDs in addition to normal information clyde% [1] [2] [3] + clyde% jobs Running fist. scr Stopped (tty output) vi Stopped (tty input) ed + is the current process - is the previous job UNIX System Programming
Controlling Background Processes Commands available for job control bg - background fg - foreground stop kill ^Z Reference to a job begins with a %. By itself, % refers to the current job (and most of the commands default to the current job) UNIX System Programming
Back. Ground - bg Run the current job (or job_no) in the background Syntax: bg [%job_no] - the job number assigned when the task was initially put in the background, can be viewed with the jobs command job_no Example: bg %3 will make job number 3 resume execution in the background UNIX System Programming
stop Stop the current or specified background job Syntax: stop [%job_no] - the job number assigned when the task was initially put in the background, can be viewed with the jobs command job_no Example stop %1 will stop background job number 1 until either an fg or bg command is executed UNIX System Programming
^Z or Ctrl-Z ^Z stops the current foreground job and puts it in the background To re-enable execution, either use bg %job_no or fg %job_no Example %vi wordcount. c ^Z will suspend execution of this instance of vi and place it into the background until an fg command is executed UNIX System Programming
kill - signal a process kill is somewhat strangely named Syntax: kill [-sig_no] pid kill -l (display list of signals) -sig_no - signal number to send pid - process id of process to receive signal Default sig_no is 15, or request-process-termination UNIX System Programming
kill - signal a process kill -9 pid terminates the process with extreme prejudice As usual, you can only kill your own processes unless you are the superuser UNIX System Programming
nice - Modify Process Priority nice increments or decrements the process priority value Syntax: nice [+n|-n] [command] +n|-n - increments or decrements the process priority value by n UNIX System Programming
nice - Modify Process Priority The higher the process priority value, the lower the priority Only the superuser can lower the process priority value (raise the process priority) UNIX System Programming
find - Find files find recursively descends the directory hierarchy for each pathname in in the pathname-list looking for files that have certain filenames, permissions, or modifications times. find can also execute a program for each file it locates. UNIX System Programming
find - Find files Syntax: find directory-list expression - the pathnames of one or more directories to be searched. All subdirectories of all listed directories are searched to all levels. directory-list - a logical expression of one or more criteria as described later. A space only separating criteria is a logical AND operator. A -o separator is a logical OR operator. An ! negates any criteria it precedes. expression UNIX System Programming
Caveats Special characters within expression must be escaped () to keep the shell from interpreting them. These are characters such as parenthesis (), square brackets [ ], question marks ? , and asterisks *. UNIX System Programming
Caveats Each element within expression is a separate argument and must be separated from the others with spaces. There must be a space on both sides of each parenthesis, exclamation point, criterion, or other element. When you escape special characters with a , the spaces go on each side of the character pair [ UNIX System Programming
find Criteria -name filename matches if the file being evaluated matches filename -type filetype matches if the file has a matching file type d - directory f - ordinary l - symbolic link UNIX System Programming
find Criteria -links +/-n -user name matches if the file has the number of links specified by +/-n matches if the file is owned by name (login name) -group name matches if file belongs to group UNIX System Programming
find Criteria -inum n matches if the files inode number is n -size +/-n[c] matches if file size is the size, in blocks, specified by +/-n, or characters if +/-nc UNIX System Programming
find Criteria -atime +/-n matches if the file was last accessed the number of days ago specified by +/-n. The access time of searched directories is changed when this option is used. -mtime +/-n matches if the file was last modified +/-n days ago UNIX System Programming
find Criteria -newer filename matches if file is newer than filename -print all files match, causes find to print the pathname UNIX System Programming
find Criteria -ls all files match, similar to -print but displays more information such as inode number, size, permissions, number of lines, etc. UNIX System Programming
find Criteria -exec command ; included at the end of a criteria list, command is executed on all files that meet the preceding criteria. Must be terminated with an escapes semicolon (; ). {} within the command string represents the name of the file being evaluated. E. g. , find ……. –exec rm { } ; UNIX System Programming
find Criteria -ok command ; same as -exec only it displays the commands about to be executed and waits for a y from standard input before proceeding UNIX System Programming
More Caveats You must explicitly tell find to display matching file names with the -print criteria. You can use a -a (-o)between criteria for logical AND (OR) conditions. + is equivalent to greater than (>) - equates to less than (<) UNIX System Programming
find Examples find. -name a* finds all files in your current directory and all of its subdirectories with names beginning with an a. However, since no -print criteria was given, it does not display them. find. -name a* -print same as before but prints the pathnames of the files it finds UNIX System Programming
find Examples find. -type f -mtime -1 ! -name *. o -print | cpio -o. B > /dev/rmt 0 finds all files that have been modified within the past day (-mtime -1) excluding object files (! -name *. o) and pipes them to an archive program (cpio) UNIX System Programming
find Examples find. ( -name core -o -name junk ) -print -exec rm {} ; used to clean up a disk, finds all files whose name is core OR junk (notice the -o), displays their pathname, and deletes (rm) them if used with -ok instead of -exec, find will prompt you with an > character and wait for you to enter a y before deleting each file UNIX System Programming
find Examples find. ( -name *. cc -o -name a. out ) -print –ok chmod 700 {} ; find. -name [0 -9]* -ok chmod 700 {} ; UNIX System Programming
What does this do? % chown –R us. /base All your base are belong to us UNIX System Programming
Multi-Command Lines Multiple commands can be entered on a single line by separating them with a ; wc chapter. * ; who | wc will operate on chapter. *, then wc will operate on who UNIX System Programming
Multi-Command Lines You can also group commands using parenthesis (wc chapter. * ; who) | wc The first wc will operate on chapter. *, then who will run, and then the final wc will operate on the combined output of (wc chapter. * ; who) The character, with space separators, can be used to extend commands across multiple lines UNIX System Programming
Multi-line Command Example who | wc is equivalent to who | wc Q. What happens if you do: who ? (i. e. , a space and enter follows ) UNIX System Programming
show file DIFFerences - diff Displays the differences between two files Syntax: diff [ -biw ] [-c | -e ] file-name 1 file-name 2 -b - ignore trailing blanks (spaces and TABs at end of and treat all other strings of blanks as equivalent UNIX System Programming line)
show file DIFFerences - diff -i - ignore the case of letters -w -ignore all blanks (spaces and TABs); i. e. , a=b will compare equal to = a b -c - produce listing of differences with lines of context UNIX System Programming
list UNIQue items - uniq Examines a file for duplicate lines. Duplicate lines are only detected if they are consecutive. Syntax: uniq [ -cdu] [ +|-n ] input-file [ output-file] -c - Supercede the -u and -d options and generate an output report with each line preceded by an occurrence count -d - write one copy of only the duplicated lines UNIX System Programming
list UNIQue items - uniq -u - write only those lines which are not duplicated NOTE: the default output is the union (combination) of -d and -u The n arguments specify an initial portion of each line to skip +n - skip the first n characters -n - skip the first n fields and any blanks before them UNIX System Programming
C Shell Variables There are two types of shell variables: Environmental shell variables Ordinary shell variables These both have variables that either take on values or act as switches Switches are on if they are declared and off if they are not UNIX System Programming
C Shell Variables Environmental shell variables are inherited by all child shells while ordinary shell variables must be defined for each instance of the shell Shell variables are usually defined in either the. login or the. cshrc files UNIX System Programming
What Shell Variables Are In Use? set will display all ordinary shell variables and there values setenv will display all currently defined environmental variables and their values Traditionally, the ordinary variables have lowercase names and the environmental variables are all uppercase UNIX System Programming
Setting Variables Environmental shell variables are set using setenv variable[=value] Ordinary shell variables are set using set variable[=value] UNIX System Programming
Setting Variables Note that the text shows spaces on both sides of the =. This will not work. To unset a defined shell variable, use the unset and unsetenv commands unset variable unsetenv variable UNIX System Programming
Referring to Shell Variables To refer to shell variables, preface the variable name with a $ Examples echo $HOME echo This is the value of the history variable: $history echo environment path is: My $PATH echo current path is: My $path echo favorite shell, My $shell UNIX System Programming a great shell. is
Common Environmental Variables HOME - your home directory as defined in /etc/passwd PATH - your directory search path TERM - your terminal type SHELL - your default shell, used when forking other shells USER - your user name UNIX System Programming
Common Ordinary Shell Variables history - enables the history function and defines number of commands to savehist - specifies the number of commands saved when logging out shell - pathname of the shell home - your home directory prompt - sets the prompt string UNIX System Programming
Common Ordinary Shell Variables echo - if set, the shell displays each command before executing it (switch) filec - enables file name completion (switch) ignoreeof - tells shell to ignore ^D so you must use exit or logout to leave a shell (switch) noclobber - prevents you from accidentally overwriting a file when you redirect output (switch) UNIX System Programming
Redirecting Standard Input Redirection of standard input works in a similar manner < is the standard input redirection operator command [options] [arguments] < input_filename UNIX System Programming
Redirecting Standard Input options and arguments are the same as you are familiar with command is any executable program input_filename is the place (file) that contains the data you want to use for input to command Obviously, input_filename must exist UNIX System Programming
Using Input and Output Redirection Standard input and standard output redirection can be used together on the same command [options] [arguments] < input_filename > output_filename E. g. , crypt < password > cryptext sort < unsorted_file > sorted_file UNIX System Programming
Using Input and Output Redirection or command [options] [arguments] < input_filename >> output_filename E. g. , foo < input_file >> output_file UNIX System Programming
Redirecting Standard Error Under the C shell, you can redirect both standard output and standard error by using the > or >> symbols followed by an & UNIX System Programming
Redirecting Standard Error If you don’t do this, commands like find may clutter up your screen with error messages while you are trying to accomplish other work. find / -name stdlib. c -print >& whereitis or find / -name stdlib. c -print >>& whereitis UNIX System Programming
Pipes (Inter-Process Communication) Suppose you want a printed sorted list of who is currently logged on to the system You could issue the following command sequence who > temp_file sort < temp_file > sorted_file lpr sorted_file rm temp_file rm sorted_file UNIX System Programming
Pipes A pipe will let you do this in one command line without having to use temporary files who | sort | lpr UNIX System Programming
So What Is a Pipe? The shell uses a pipe ( | ) to connect the standard output of one command directly to the standard input of another This has the same effect of redirecting the standard output of one command to a temporary file, and then using that file as the standard input to another command UNIX System Programming
So What Is a Pipe? You can use pipes with any command that accepts input from either a file specified on the command line or standard input Pipes can also be used with commands that only accept data from standard input UNIX System Programming
So What Happens in a Pipe? Consider the command string ls | wc -l The shell opens a pipe for communication between ls and wc are then executed as concurrently running tasks UNIX System Programming
So What Happens in a Pipe? As ls writes data to the pipe, wc can read the data from the pipe Neither process knows that it is using a pipe instead of a standard file UNIX System Programming
Pipes Because the processes are running concurrently and not writing to intermediate files, the entire sequence is more efficient and should complete quicker. UNIX System Programming
Pipes If the writing process produces data much more quickly than the reading process and accept it, the pipe can fill up In that case, the kernel puts the writing process to sleep until the reader has a chance to catch up UNIX System Programming
Common Mistakes Confusing redirection and pipes is a common error that can be deadly to your files A pipe ( | ) will take the output from a command connect it to the input of another command Redirection ( > ) will take the output from a command create or overwrite a file with the data UNIX System Programming
tee Copy standard input to standard output and one or more files Unix Command Standard output file-list UNIX System Programming
tee Syntax: tee [ -ai ] file-list -a - append to output file rather than overwrite, default -i is to overwrite (replace) the output file - ignore interrupts file-list - one or more file names for capturing output UNIX System Programming
tees and Pipes So, what happens if you use tee in a pipe? ls | tee dir_list | sort -r What will this do? Why would I want to do it? UNIX System Programming
tees and Pipes What about this ? a. out | tee output_file Why do this? UNIX System Programming
Filters A filter is a command that process an input stream of data (from standard input) to produce an output stream (on standard output. ) A command line that uses a filter use a pipe to connect the filter’s input to the standard output of another command or filter UNIX System Programming
Filters Another pipe may connect the filter’s standard output to another command or filter’s standard input Interactive utilities, such as mail and vi, cannot be used as filters UNIX System Programming
Commonly Used Filters sort - sort a file tr - translate characters to different characters spell - check the spelling of a list of words wc - count the number of words, characters and lines in a file UNIX System Programming
Commonly Used Filters grep - search files for a pattern head and tail - list first/last part of a file sed - stream editor awk - pattern-action pair programming language UNIX System Programming
Filter Examples who | sort - display a sorted list of who is logged on ls | wc -w - display the number of entries in a directory tr A-Z a-z <myfile >yourfile - make all characters in myfile lower case and saves output in yourfile UNIX System Programming
TRanslate - tr Copies standard input to standard output with substitution or deletion of selected characters Syntax: tr [ -ds ] [ string 1 ] [ string 2 ] -d - delete all input characters contained in string 1 UNIX System Programming
TRanslate - tr -s - squeeze all strings of repeated output characters that are in string 2 to single characters tr provides only simple text processing. It does not allow the full power of regular expressions. If you need the power of regular expressions, you need to use sed UNIX System Programming
TRanslate - tr reads from standard input. Any character that does not match a character in string 1 is passed to standard output unchanged Any character that does match a character in string 1 is translated into the corresponding character in string 2 and then passed to standard output UNIX System Programming
TRanslate - tr Examples tr ‘s’ ‘z’ replaces all instances of s with z tr ‘so’ ‘zx’ replaces all instances of s with z and o with x tr ‘a-z’ ‘A-Z’ replaces all lower case characters with upper case characters UNIX System Programming
tr As you can see, tr establishes a single character to single character (or 1 -to-1) mapping. This mapping is case sensitive. The output of tr can be redirected to a file or piped to another filter and its input can be redirected from a file or piped from another command UNIX System Programming
tr This implies that certain characters must be protected from the shell by quotes or , such as: spaces : ; & ( ) | ^ < > [ ] ! newline TAB Example tr o replaces all o’s with a blank (space) UNIX System Programming