Monday 18 November 2013

Unix Commands

Man ual command.
man man This is help command, and will explains you about online manual pages you can also use man in conjunction with any command to learn more about that command for example.
  • man ls will explain about the ls command and how you can use it.
  • man -k pattern command will search for the pattern in given command.
Banner command.
banner prints characters in a sort of ascii art poster, for example to print wait in big letters. I will type
banner wait at unix command line or in my script. This is how it will look.
  
 #    #    ##       #     #####
 #    #   #  #      #       #
 #    #  #    #     #       #
 # ## #  ######     #       #
 ##  ##  #    #     #       #
 #    #  #    #     #       #
 
Cal command
cal command will print the calander on current month by default. If you want to print calander of august of 1965. That's eightht month of 1965.
cal 8 1965 will print following results.
    August 1965
 S  M Tu  W Th  F  S
 1  2  3  4  5  6  7
 8  9 10 11 12 13 14
15 16 17 18 19 20 21
22 23 24 25 26 27 28
29 30 31
 
Clear command
clear command clears the screen and puts cursor at beginning of first line.
Calendar command
calendar command reads your calendar file and displays only lines with current day.
For example in your calendar file if you have this
12/20   Test new software.
1/15    Test newly developed 3270 product.
1/20    Install memory on HP 9000 machine.
On dec 20th the first line will be displayed. you can use this command with your crontab file or in your login files.
Nohup command.
nohup command if added in front of any command will continue running the command or process even if you shut down your terminal or close your session to machine. For exmaple, if I want to run a job that takes lot of time and must be run from terminal and is called update_entries_tonight .
nohup update_entries_tonight will run the job even if terminal is shut down in middle of this job.
Tty command
Tty command will display your terminal. Syntax is
tty options
Options
  • -l will print the synchronous line number.
  • -s will return only the codes: 0 (a terminal), 1 (not a terminal), 2 (invalid options) (good for scripts)

File Management commands.


Pwd command.
pwd command will print your home directory on screen, pwd means print working directory.
 /u0/ssb/sandeep
is output for the command when I use pwd in /u0/ssb/sandeep directory.
Ls command
ls command is most widely used command and it displays the contents of directory.
options
  • ls will list all the files in your home directory, this command has many options.
  • ls -l will list all the file names, permissions, group, etc in long format.
  • ls -a will list all the files including hidden files that start with . .
  • ls -lt will list all files names based on the time of creation, newer files bring first.
  • ls -Fxwill list files and directory names will be followed by slash.
  • ls -Rwill lists all the files and files in the all the directories, recursively.
  • ls -R | more will list all the files and files in all the directories, one page at a time.
Mkdir command.
mkdir balaji will create new directory, i.e. here balaji directory is created.
Cd command.
cd balaji will change directory from current directory to balaji directory.
Use pwd to check your current directory and ls to see if balaji directory is there or not.
You can then use cd balaji to change the directory to this new directory.

Cat command
cat cal.txt cat command displays the contents of a file here cal.txt on screen (or standard out).

Head command.
head filename by default will display the first 10 lines of a file.
If you want first 50 lines you can use head -50 filename or for 37 lines head -37 filename and so forth.
Tail command.
tail filename by default will display the last 10 lines of a file.
If you want last 50 lines then you can use tail -50 filename.
More command. more command will display a page at a time and then wait for input which is spacebar. For example if you have a file which is 500 lines and you want to read it all. So you can use
more filename
Wc command
wc command counts the characters, words or lines in a file depending upon the option.
Options
  • wc -l filename will print total number of lines in a file.
  • wc -w filename will print total number of words in a file.
  • wc -c filename will print total number of characters in a file.
File command.
File command displays about the contents of a given file, whether it is a text (Ascii) or binary file. To use it type
file filename. For example I have cal.txt which has ascii characters about calander of current month and I have resume1.doc file which is a binariy file in microsoft word. I will get
file resume.doc
resume1.doc:    data   
file cal.txt
cal.txt:        ascii text

Cp command.
cp command copies a file. If I want to copy a file named oldfile in a current directory to a file named newfile in a current directory.
cp oldfile newfile
If I want to copy oldfile to other directory for example /tmp then
cp oldfile /tmp/newfile. Useful options available with cp are -p and -r . -p options preserves the modification time and permissions, -r recursively copy a directory and its files, duplicating the tree structure.

Rcp command.
rcp command will copy files between two unix systems and works just like cp command (-p and -i options too).
For example you are on a unix system that is called Cheetah and want to copy a file which is in current directory to a system that is called lion in /usr/john/ directory then you can use rcp command
rcp filename lion:/usr/john
You will also need permissions between the two machines. For more infor type man rcp at command line.

Mv command.
mv command is used to move a file from one directory to another directory or to rename a file.
Some examples:
  • mv oldfile newfile will rename oldfile to newfile.
  • mv -i oldfile newfile for confirmation prompt.
  • mv -f oldfile newfile will force the rename even if target file exists.
  • mv * /usr/bajwa/ will move all the files in current directory to /usr/bajwa directory.

Ln command.
Instead of copying you can also make links to existing files using ln command.
If you want to create a link to a file called coolfile in /usr/local/bin directory then you can enter this command.
ln mycoolfile /usr/local/bin/coolfile
Some examples:
  • ln -s fileone filetwo will create a symbolic link and can exist across machines.
  • ln -n option will not overwrite existing files.
  • ln -f will force the link to occur.

Rm command.
To delete files use rm command.
Options:
  • rm oldfile will delete file named oldfile.
  • rm -f option will remove write-protected files without prompting.
  • rm -r option will delete the entire directory as well as all the subdirectories, very dangerous command.

Rmdir command.
rmdir command will remove directory or directories if a directory is empty.
Options:
  • rm -r directory_name will remove all files even if directory is not empty.
  • rmdir sandeep is how you use it to remove sandeep directory.
  • rmdir -p will remove directories and any parent directories that are empty.
  • rmdir -s will suppress standard error messages caused by -p.

Comparison and Searching


Diff command.
diff command will compare the two files and print out the differences between.
Here I have two ascii text files. fileone and file two.
Contents of fileone are
This is first file 
this is second line
this is third line
this is different    as;lkdjf 
this is not different

filetwo contains
This is first file
this is second line
this is third line
this is different    xxxxxxxas;lkdjf
this is not different

diff fileone filetwo will give following output
 4c4
< this is different    as;lkdjf
---
> this is different    xxxxxxxas;lkdjf

Cmp command.
cmp command compares the two files. For exmaple I have two different files fileone and filetwo.
cmp fileone filetwo will give me
fileone filetwo differ: char 80, line 4 

if I run cmp command on similar files nothing is returned.
-s command can be used to return exit codes. i.e. return 0 if files are identical, 1 if files are different, 2 if files are inaccessible.
This following command prints a message 'no changes' if files are same
cmp -s fileone file1 && echo 'no changes' .
no changes

Dircmp Command.
dircmp command compares two directories. If i have two directories in my home directory named
dirone and dirtwo and each has 5-10 files in it. Then
dircmp dirone dirtwo will return this
 Dec  9 16:06 1997  dirone only and dirtwo only Page 1
 
 
./cal.txt                                   ./fourth.txt
./dohazaar.txt                              ./rmt.txt
./four.txt                                  ./te.txt
./junk.txt                                  ./third.txt
./test.txt 

Grep Command
grep command is the most useful search command. You can use it to find processes running on system, to find a pattern in a file, etc. It can be used to search one or more files to match an expression.
It can also be used in conjunction with other commands as in this following example, output of ps command is passed to grep command, here it means search all processes in system and find the pattern sleep.
ps -ef | grep sleep will display all the sleep processes running in the system as follows.
     ops 12964 25853  0 16:12:24 ttyAE/AAES  0:00 sleep 60
     dxi 12974 15640  0 16:12:25 ttyAH/AAHP  0:00 sleep 60
     ops 12941 25688  0 16:12:21 ttyAE/AAEt  0:00 sleep 60
     ops 12847 25812  0 16:11:59 ttyAH/AAH6  0:00 sleep 60
     ops 12894 25834  0 16:12:12 ttyAE/AAEX  0:00 sleep 60
     dxi 13067 27253  2 16:12:48 ttyAE/ABEY  0:00 sleep 1
     ops 13046 25761  0 16:12:44 ttyAE/AAE0  0:00 sleep 60
     dxi 12956 13078  0 16:12:23 ttyAG/AAG+  0:00 sleep 60
     ops 12965 25737  0 16:12:24 ttyAE/AAEp  0:00 sleep 60
     ops 12989 25778  0 16:12:28 ttyAH/AAHv  0:00 sleep 60
     ssb 13069 26758  2 16:12:49 ttyAH/AAHs  0:00 grep sleep
     pjk 27049  3353  0 15:20:23 ?           0:00 sleep 3600
Options:
  • -b option will precede each line with its block number.
  • -c option will only print the count of matched lines.
  • -i ignores uppercase and lowercase distinctions.
  • -l lists filenames but not matched lines.
other associated commands with grep are egrep and fgrep. egrep typically runs faster. for more information type man egrep or man fgrep in your system.

Find command.
Find command is a extremely useful command. you can search for any file anywhere using this command provided that file and directory you are searching has read write attributes set to you ,your, group or all. Find descends directory tree beginning at each pathname and finds the files that meet the specified conditions. Here are some examples.

Some Examples:
find $HOME -print will lists all files in your home directory.
find /work -name chapter1 -print will list all files named chapter1 in /work directory.
find / -type d -name 'man*' -print will list all manpage directories.
find / -size 0 -ok rm {} \; will remove all empty files on system.
conditions of find
  • -atime +n |-n| n will find files that were last accessed more than n or less than -n days or n days.
  • -ctime +n or -n will find that were changed +n -n or n days ago.
  • -depth descend the directory structure, working on actual files first and then directories. You can use it with cpio command.
  • -exec commad {} \; run the Unix command on each file matched by find. Very useful condition.
  • -print print or list to standard output (screen).
  • -name pattern find the pattern.
  • -perm nnnfind files whole permission flags match octal number nnn.
  • -size n find files that contain n blocks.
  • -type c Find file whole type is c. C could be b or block, c Character special file, d directory, p fifo or named pipe, l symbolic link, or f plain file.
Text processing


Cut command.
cut command selects a list of columns or fields from one or more files.
Option -c is for columns and -f for fields. It is entered as
cut options [files]
for example if a file named testfile contains
this is firstline
this is secondline
this is thirdline
Examples:
cut -c1,4 testfile will print this to standard output (screen)
ts
ts
ts
It is printing columns 1 and 4 of this file which contains t and s (part of this).
Options:
  • -c list cut the column positions identified in list.
  • -f list will cut the fields identified in list.
  • -s could be used with -f to suppress lines without delimiters.

Paste Command.
paste command merge the lines of one or more files into vertical columns separated by a tab.
for example if a file named testfile contains
this is firstline
and a file named testfile2 contains
this is testfile2
then running this command
paste testfile testfile2 > outputfile
will put this into outputfile
this is firstline       this is testfile2 
it contains contents of both files in columns.
who | paste - - will list users in two columns.
Options:
  • -d'char' separate columns with char instead of a tab.
  • -s merge subsequent lines from one file.

Sort command.
sort command sort the lines of a file or files, in alphabetical order. for example if you have a file named testfile with these contents
zzz
aaa
1234
yuer
wer
qww
wwe
Then running
sort testfile
will give us output of
1234
aaa
qww
wer
wwe
yuer
zzz
Options:
  • -b ignores leading spaces and tabs.
  • -c checks whether files are already sorted.
  • -d ignores punctuation.
  • -i ignores non-printing characters.
  • -n sorts in arithmetic order.
  • -ofile put output in a file.
  • +m[-m] skips n fields before sorting, and sort upto field position m.
  • -r reverse the order of sort.
  • -u identical lines in input file apear only one time in output.

Uniq command.
uniq command removes duplicate adjacent lines from sorted file while sending one copy of each second file.
Examples

sort names | uniq -d will show which lines appear more than once in names file.
Options:
  • -c print each line once, counting instances of each.
  • -d print duplicate lines once, but no unique lines.
  • -u print only unique lines.

Awk and Nawk command.
awk is more like a scripting language builtin on all unix systems. Although mostly used for text processing, etc.
Here are some examples which are connected with other commands.
Examples:
df -t | awk 'BEGIN {tot=0} $2 == "total" {tot=tot+$1} END {print (tot*512)/1000000}' Will give total space in your system in megabytes.
Here the output of command df -t is being passed into awk which is counting the field 1 after pattern "total" appears. Same way if you change $1 to $4 it will accumulate and display the addition of field 4
which is used space.
for more information about awk and nawk command in your system enter man awk or man nawk.

Sed command.
sed command launches a stream line editor which you can use at command line.
you can enter your sed commands in a file and then using -f option edit your text file. It works as
sed [options] files
options:
  • -e 'instruction' Apply the editing instruction to the files.
  • -f script Apply the set of instructions from the editing script.
  • -n suppress default output.

for more information about sed, enter man sed at command line in your system.

Vi editor.
vi command launches a vi sual editor. To edit a file type
vi filename
vi editor is a default editor of all Unix systems. It has several modes. In order to write characters you will need to hit i to be in insert mode and then start typing. Make sure that your terminal has correct settings, vt100 emulation works good if you are logged in using pc.
Once you are done typing then to be in command mode where you can write/search/ you need to hit :w filename to write
and in case you are done writing and want to exit
:w! will write and exit.
options:
  • i for insert mode.
    • I inserts text at the curson
    • A appends text at the end of the line.
    • a appends text after cursor.
    • O open a new line of text above the curson.
    • o open a new line of text below the curson.
  • : for command mode.
    • <escape> to invoke command mode from insert mode.
    • :!sh to run unix commands.
    • x to delete a single character.
    • dd to delete an entire line
    • ndd to delete n number of lines.
    • d$ to delete from cursor to end of line.
    • yy to copy a line to buffer.
    • P to paste text from buffer.
    • nyy copy n number of lines to buffer.
    • :%s/stringA/stringb /g to replace stringA with stringB in whole file.
    • G to go to last line in file.
    • 1G to go to the first line in file.
    • w to move forward to next word.
    • b to move backwards to next word.
    • $ to move to the end of line.
    • J join a line with the one below it.
  • /string to search string in file.
  • n to search for next occurence of string.

14 comments:

  1. Quite informative collection of unix commands

    ReplyDelete
  2. Good crisp precise please share an ETL project or link for any reference.

    ReplyDelete
  3. A Dedicated Server provider that meets your hosting needs can be a game-changer.

    ReplyDelete