Software I Utilities and Internals Lecture 1

Software I: Utilities and Internals Lecture 1 – UNIX for Beginners * Credit to Dr. Robert Siegfried and http: //www. upscale. utoronto. ca/General. Interest/Harrison/Learn. Linux/ by David M.

What is UNIX? • UNIX is a time-sharing operating system with userchosen shells (user interfaces) and one kernel (operating system core, which allocates and control resources such as CPU time, memory and I/O devices). • UNIX includes: – kernel – tools, including compilers, editors, etc. – shell, which includes its own programming language

Unix OS Layers Microsoft, Corporation, . Unix Application Migration Guide (Patterns & Practices) Apr 2, 2003

Unix Philosophy • • Simplicity – KISS Focus – one program -> one task Reusable components – libraries Filters – transform input to produce output – combine unix programs easily that way • Open file formats – config and data not binary • Flexible – assume others will use it differently than you intended

What is LINUX? • Linux is an open-source operating system, indirectly based on the last public release of UNIX. • Linux is available in many different versions and different releases and is also closely associated with the GNU project, and through the GNU project has many tools comparable to those found in a UNIX distribution. • GNU – open license (though read details); collaborative project; www. gnu. org

GNU Project software for Linux • • GCC: The GNU Compiler for C G++: The GNU Compiler for C++ GDB: A source code-level debugger GNU Make: compilation instructions Bash: A command shell GNU Emacs: An editor Linux is the kernel, and you add in projects to make a complete Linux Installation

History of UNIX • AT&T Bell Labs (1969) • one of the first time-sharing operating systems. • It was originally developed on a DEC PDP-7 and later redeveloped on a DEC PDP-11 in C, • the first OS written in a high-level language. • popular at colleges and universities and later in the financial industry. • Standards: POSIX (portable Operating System Interface Specs) by IEEE - 1988 • The Single UNIX Specification by The Open Group

History of Linux • Andrew Tanenbaum developed MINIX from the last public distribution of UNIX for use with his operating Systems textbook. 1987 • When version 2 of MINIX was released, it was not well adapted for 32 -bit processors. This inspired Linus Torsvald to begin work on what became Linux. 1991 • Torsvald welcomed suggestions; this gave way to the community approach to software development that became a hallmark of Linux. • Kernel should contain only freely distributable code

Logging In motd (message of the day) doesn't appear when you type login as: pepper@panther. adelphi. edu's password: Last login: Sat Aug 30 21: 58 2014 from pool-1 Welcome to Panther! If you experience problems or have questions, please contact the IT Help Desk at (516) 877 -3340, email support@adelphi. edu, or visit us in the Information Commons on the Second Floor of Swirbul Library. ** Reminder: Your Website is http: //home. adelphi. edu/~pe 16132 PEPPER@panther: ~$ prompt

Where are you? • Home – cd ~ or just cd • Present working directory – pwd • What do you have – ls • Moving around directories – cd directory – cd. .

Commands to Try Case Sensitivity • date – gives date and time • who – tells you who is on the system and connected to what terminal • who am i – tells you who you are and to what terminal you are connected • whoami – tells you who you are • w – displays what the system users are doing

UNIX and Terminals • Unix is full duplex – communications between computer and terminal goes in both directions simultaneously and the computer controls terminal display, using a process called echo. • Example echo hi there (turn echo off with stty –echo and back on with stty echo)

stty • UNIX gives the user a way of adjusting terminal settings • stty – set terminal – change and print terminal line settings. • Example PEPPER@panther: ~$ stty -a speed 38400 baud; rows 24; columns 80; line = 0; intr = ^C; quit = ^; erase = ^? ; kill = ^U; eof = ^D; eol = ; eol 2 = ; swtch = ; … …

Standard Codes intr erase werase kill ^c quit stop start eof suspend resume ^ ^h ^w ^u ^s ^q ^d ^z ^y Stops a program backspaces erases last word typed kills the current input line stops the program and saves core in a file pause screen output resumes screen output no more data temporarily stops (i. e. , suspends) a program resumes running a program

Try one Standard Code • • • ^s to stop Type why don’t I see this ^q to resume Type a bit more without pressing enter Press ^u to erase

Stopping A Program • Programs can be stopped by pressing the Break, usually the ^c. • ^z stops a program temporarily. – You can restart it by listing jobs with jobs. – You can bring it to the foreground using fg %1 • ^d indicates end of file. – In your shell, it exits, logging you off – After cat command it stops input mode

Logging Out • You can log out by typing logout or exit. If you use the wrong one, the system will let you know. • You can logout by simply typing ^d.

Online manual • Most UNIX (and Linux) systems have an online manual that can be accessed by typing: man commandname • Example man who – shows the manual page on who man – shows the manual page on man (g to top, G to bottom, /

Info manual • Same information as man but with navigation • See tutorial with ctrl + H and exit that with l • Place cursor and enter to reach section • [ to return

Files and File-Oriented Commands • A great deal of work on the system involves files (data moving into or out of the computer), which makes file-oriented commands particularly important. • File commands include: – vi, ex, ed, emacs – file editors – cat, more, pr – printing and display commands – mv, cp, rm – file manipulation commands – grep, sort, diff, tail - filters

Create a file using cat • cat concatenates one or more files • Can add file content • Syntax: – cat > filename to create – Lines in the file – Ctrl d to end cat > hello Hi there I am here Ctrl d

Append to a file • > sends output to a new file – creates – Overwrites • >> adds to the end – Creates – Appends cat >> hello Put this at the end Ctrl d

Text editors • There are 4 text editors that we will concern ourselves with: – ed – the original line-oriented editor – ex – the extended line-oriented editor – visual editor (screen-oriented) - emacs – rich visual editor

A Sample ed Session Start the ED editor: ed [filename] Add mode: a Write: w End/Quit: q $ed a add text … type lots of stuff …. ends adding text w junk write text to a file called junk 39 number of characters saved q quit

Changing A File Print current line: p Print all lines: 1, $p start at 1, end at last line [SIEGFRIE@panther ~]$ ed junk: No such file or directory a To be or not to be. a That is the question. p That is the question 1, $p To be or not to be That is the question

Changing A File (continued) Substitute: s/original/new s/the/The/ 1, $p To be or not to be That is The question w 40 q [SIEGFRIE@panther ~]$

ls – Listing Files [SIEGFRIE@panther bbb]$ ls junk temp [SIEGFRIE@panther bbb]$ ls -l total 8 -rw-r--r-- 1 SIEGFRIE users 40 Jun 9 16: 49 junk -rw-r--r-- 1 SIEGFRIE users 40 Jun 9 17: 03 temp [SIEGFRIE@panther bbb]$ ls -t temp junk [SIEGFRIE@panther bbb]$ ls -l -t total 8 -rw-r--r-- 1 SIEGFRIE users 40 Jun 9 17: 03 temp -rw-r--r-- 1 SIEGFRIE users 40 Jun 9 16: 49 junk [SIEGFRIE@panther bbb]$ ls -lt total 8 -rw-r--r-- 1 SIEGFRIE users 40 Jun 9 17: 03 temp -rw-r--r-- 1 SIEGFRIE users 40 Jun 9 16: 49 junk [SIEGFRIE@panther bbb]$ ls * - all contents below

ls – l • ls –l – provides a long listing of the files [SIEGFRIE@panther bbb]$ ls –l long listing total 8 -rw-r--r-- 1 SIEGFRIE users 40 Jun 9 16: 49 junk -rw-r--r-- 1 SIEGFRIE users 40 Jun 9 17: 03 temp Permissions # of links owner to file owner's group # of bytes in file date & time of last modification file name

Permission -rw-r--r-regular file Owner has read and write but not execute permission Owner's group has read but not write nor execute permission The rest of the world has read but not write nor execute permission

Displaying File Content • cat - displays multiple files in order • more – displays one page at a time – – – Space for next page b to back up (not classic) q to quit / to search G to bottom and g to top • less – similar to more • pr – display as pages for a printer

mv, cp and rm • mv – move (or rename) a file – mv origname 1 newname – mv origname 2. . /newplace/newname – mv origname 3 folder 1/ – mv origname 4. . / • cp – copy a file – cp file 1 file 2 • rm – remove (or delete) a file – rm file 1 – rm * - deletes everything in the directory! – rm –rf folder 1 – deletes folder and its contents

Rules Governing File Names • Much older systems may limit names to 14 characters. • UNIX is case-sensitive; junk, Junk and JUNK are three different files. • File names should not (but unfortunately can) include unprintable characters (like escape characters) or characters with special meanings. – E. g. , how would you print the file –t?

Name completion • Put in the first few characters – Tab • Fills if only one choice • If not unique, tab again for choices

Command History • All commands from this shell and then prior fully exited shell • Up and Down keys to see them – Enter to execute – Or Edit with left and right arrow keys • history command – See command number – !command number to execute it

Quick VI introduction • • Create a file with vi filename See contents Type contents (starting in insert mode) Save: escape : wq – Means escape into command mode – W for write to the filename given – Q for quit

Mid PPT summary • • Signon Entry of commands Terminal settings Standard control characters Creating files Displaying files Copy and Move files

Exercise • Start putty and enter panther. adelphi. edu then open • Exercises 1 -3 – Use ls not lc – Use cat, ed and vi to create the files when asked

A Few Helpful File Processing Commands • There are several file processing commands that will become useful: wc word count – displays line, word & character counts grep sort general regular expression program – recognizes text within a file. sorts lines of text within a file. tail prints the last line(s) of text within a file. head Prints the first line(s) of text within a file. diff compares two files, printing each pair of lines that differ.

grep Search inside a file [SIEGFRIE@panther bbb]$ grep fleas poem Great fleas have little fleas And little fleas have lesser fleas, And the great fleas themselves, in turn, have greater fleas to go on; [SIEGFRIE@panther bbb]$ grep -v fleas poem upon their backs to bite 'em and so on ad infinitum. While these again have greater still, and great still and so on.

diff Line by line difference [SIEGFRIE@panther bbb]$ diff poem newpoem 2 c 2 < upon their backs to bite 'em --> upon their backs to bite them 4 c 4 < and so on ad infinitum. --> and so til ad infinitum. [SIEGFRIE@panther bbb]$

Command Summary ls ls filenames ls –t ls –l ls –u ls –r ed filename cp file 1 file 2 mv file 1 file 2 rm filenames list filenames in current directory lists only these files lists in reverse chronological order long listing list by last time used list in reverse order edit a file listed by name copy file 1 to file 2 move (or rename) file 1 to file 2 delete these files

Command Summary cat filename(s) display file contents pr filename(s) display and format file contents pr –n filename(s) display and format file contents in n columns pr –m filename(s) display files side by side wc filename(s) count words and bytes for these files wc –l filename(s) count lines for these files grep pattern filename(s) print lines containing the pattern grep -v pattern filename(s) print lines not containing the pattern diff file 1 file 2 print each pair of differing lines more file 1 Print the file one page at a time tail file 1 Print the end of the file only

Directories • The system knows how to distinguish between different files with the same name by recognizing that they are listed in different directories. • Each user has his/her own directory, which can be divided into subdirectories by the user. • You can use the cd (change directory) command to switch between different directories and pwd (print working directory) to display the current directory being used.

Directory Movement • Move up one directory – cd. . (up one) – cd. . /anotherdirectory (get to parallel dir) – cd. . /yetanotherdir (up 2 and down one) – cd directory (goes down) – cd /bin (absolute path) • Move to home – cd ~ • Where are you? – pwd

A Sample File System / bin dev etc you junk usr mike tmp paul junk unix boot mary temp junk data • /bin : binaries, programs used in booting • /usr/bin: user binaries, standard programs available to users • /usr/local/bin: local binaries, programs specific to installation

Managing Directories • mkdir dirname – Makes a directory inside your present working directory • rmdir dirname – Delete an empty directory • Use file commands to rename or copy dirs – Rename a directory: mv olddir newdir – Copy a directory: cp olddir newdir

The Shell • The shell is another name for the command interpreter, which runs all the commands that we type in a session on a UNIX (or Linux) system. • It provides 3 benefits: – File name shorthands - a whole bunch of files specified using wild card characters, can be specified at one. – Input-output redirection – a file can replace either keyboard or screen or both. – You can personalize your environment.

Using * * Expands to all matching files • pr * - displays all the files in print format. • rm * - deletes all files in current directory. (do not do this) • rm *. sav – deletes all files ending with. sav • ls * - displays all files and folder contents except. And. . • ls. * - displays all files and folder contents above and all below you

Metacharacters – An Example [SIEGFRIE@panther junk]$ echo * cookie temp See all files except hidden that * suppresses [SIEGFRIE@panther junk]$ echo. *. . mybad Just the hidden files PEPPER@panther: ~/271/tobetar$ ls * junk 99 link. To. Menu 5. c menu 4. c menu 5. c insidedir: afileinsidedir 2 All files inside dir and subdir PEPPER@panther: ~/271/tobetar$ ls. *. : insidedir junk 99 link. To. Menu 5. c menu 4. c menu 5. c. . : alphavowels dict. txt All files inside the. Files of. And. . – so all above

[ ] • [ ] matches a single occurrence of one of the characters in the brackets. • pr ch[12346789] prints every whole chapter except 5. • pr ch[1 -46 -9] prints every whole chapter except 5. • rm temp[a-z] – deletes tempa, tempb, …, tempz if they exist.

? • ? – replaces any single occurrence of a single character. • ls ? – lists single-character file names. • ls –l ch? . 1 lists ch 1. 1, ch 2. 1, … ch 9. 1 if they exist. • rm temp? – deletes temp 1, temp 2, …tempa, tempb, …, tempz if they exist.

Final Word on Metacharacters • All file names must exist for the metacharacters to be used: mv ch. * chapter. * Won’t work because the chapter files don't already exist. • Metacharacters also can match other names in the path, e. g. , usr/*/calendar. • How do you use a file name that has a metacharacter in it? • ls '? ' or ls ?

Redirection • Redirection replaces standard input (or standard output) with a file. • ls – lists the files in the directory on the screen. • ls > filelist – creates a file containing the directory listing • cat f 1 f 2 f 3 >temp – places the files' contents in a file called temp. If the file already exists, it is overwritten. • cat f 1 f 2 f 3 >>temp – places the files' contents in a file called temp. If the file already exists, the output is placed at the end.

Redirection – Some Other Examples who > temp sort < temp • Alphabetical list of users who > temp wc –l < temp • Counts number of users ls > temp pr -3 < temp • Prints filenames in 3 column format who > temp grep mary < temp • Is Mary logged in? sort < temp sort temp • Does the same thing

Normal Flow of Data 0 stdin 1 Command options 2 stderr stdout

Redirect errors Error text is sent to stderr, accessed by 2> PEPPER@panther: ~/270$ ls cannotfindthis ls: cannot access cannotfindthis: No such file or directory PEPPER@panther: ~/270$ ls cannotfindthis > temp ls: cannot access cannotfindthis: No such file or directory PEPPER@panther: ~/270$ ls cannotfindthis 2> temp PEPPER@panther: ~/270$ more temp ls: cannot access cannotfindthis: No such file or directory PEPPER@panther: ~/270$

Pipes • As though it outputs to a temporary file and then the piped program reads the temporary file who | sort who | wc –l ls | pr -3 who | grep mary • Any program reading the keyboard and displaying on the screen can use a pipe (|)" who | grep mary | wc –l How many times did Mary log in?

Processes • A process is the act of running a program • Proper use of the shell will allow you to run 2 processes with one command: (with ; separating them) Example: date; who Yields: Wed Jun 10 19: 11 EDT 2009 SIEGFRIE pts/0 Jun 10 10: 07 (pool… …. net)

Background Processes • Proper use of the shell will allow you to run 2 processes concurrently: Run in the background [SIEGFRIE@panther bbb]$wc ch* > wc. out & [1] 12909 [SIEGFRIE@panther bbb]$ cat wc. out 1 4 21 ch 1. 1 8 47 260 ch 1. 2 8 47 258 ch 2. 1 1 6 19 ch 2. 2 18 104 558 total [1]+ Done wc ch* >wc. out [SIEGFRIE@panther bbb]$

Pipes and Background Processes pr ch* | lp & 6951 $wait kill -9 6944 Applies to the whole pipe Waits until all the processes are finished Kills the lp process

ps • ps – process status [SIEGFRIE@panther bbb]$ ps PID TTY STAT TIME 6660 tty 1 Ss+ 0: 00 6661 tty 2 Ss+ 0: 00 6662 tty 3 Ss+ 0: 00 6663 tty 4 Ss+ 0: 00 6664 tty 5 Ss+ 0: 00 6665 tty 6 Ss+ 0: 00 12115 pts/0 Ss 0: 00 12944 pts/0 R+ 0: 00 [SIEGFRIE@panther bbb]$ ag COMMAND /sbin/mingetty /sbin/mingetty -bash ps ag tty 1 tty 2 tty 3 tty 4 tty 5 tty 6

Creating Processes • Processes can create processes. • If a process is running, in the background when you log out, it will terminate unless you submitted with nohup (no hang-up). nohup command & • Output is saved in the file nohup. out nice expensive-command – cuts the process's priority (20 highest, -20 lowest) nice –n 19 ls *

Summary • • Introduction to linux command entry Create and move files and directories Navigate directories and history A small command set – wc, grep, sort, tail, head, cmp, diff, less, cut • • Metacharacters * and. Piping Process Help (info and man)

Exercise • http: //home. adelphi. edu/~pe 16132/csc 271/ assignment/ex. Basic. Files. html