Unix command 积累

UNIX is a multi-user multitasking-optimized operating system that can run on various hardware platforms. Linux is basically an open-source clone of UNIX，而Unix的所有components都来自于同一个vendor，所以更稳定。

A shell in the world of computers refers to a program that allows the user to interact with the computer through some kind of interface. It allows you navigate directories, see lists of files and edit them using like 'cd', 'ls' and 'nano'.

Shells: sh, bash[linux, mac os], ksh, tcsh...

http://www.suso.com/infosheets/shell-commands20050327.png

1,basename, 当向basename传递一个path name时，它会删除任何前缀，直到最后一个斜线（'/'）字符，然后返回结果。

$ basename /home/jsmith/base.wiki         ->base.wiki

2, symbolic link 就是一个已有文件的别名

In computing, a symbolic link (also symlink or soft link) is a special type of file that contains a reference to another file or directory in the form of an absolute or relative path. Programs that read or write to files named by a symbolic link will behave as if operating directly on the target file.

ln -s target_path link_path

To find all the symbolic links in current directory: ls -lrt | grep '^l'  //因为symbolic link以小写l开头，所以可以用^l选出来

　　lrwxrwxrwx  1 java67 Domain Users  4 Sep 19 12:31 version_1.0 -> java/

　　lrwxrwxrwx  1 java67 Domain Users  4 Sep 21 13:59 os -> unix/

除了用ls -lrt|grep '^l' 还可以用find . -type l 来找出当前目录以及子目录的所有soft links，用到-type 小写l。查看某个softlink指向哪里： readlink /path.../rel

3, Hard links always refer to an existing file.

A file in the file system is basically a link to an inode. ‘inode’ is a ‘data-structure’, which is used for file identification on Linux. Each file on an Unix System has a separate ‘inode’ and an ‘Unique’ inode Number.
A hard link then just creates another file with a link to the same underlying
inode. Symbolic uses different inode. Deleting renaming or moving the original
file will not affect the hard link as it links to the underlying inode. Any
changes to the data on the inode is reflected in all files that refer to that inode.
$ ln /tmp/file link-here

4, chmod u=User, g=Group, o=Others
是一条unix system中用于控制用户对文件的权限的命令,
有两种方式:

$ chmod 664 myfile

$ ls -l myfile

-rw-rw-r--  1   57
Jul  3 10:13  myfile

$ chmod ug+rw mydir

$ ls -ld mydir

drw-rw----   2 unixguy 
uguys  96 Dec 8 12:53 mydir
chmod的八进制语法的数字说明：r 4， w 2， x 1， - 0， 所以664表示User和group都有rw权限，而others用户只有读权限。

 5, ls

-l, details[包括right, owner, creation time,
name,path]             -r, reverse order    
      -t, order by modification time,   $ ls -lrt

ls -l | grep someString         //List files which contains someString

ls -l file.txt         ...which will display output that
looks like the following:

-rwxrw-r-- 1  
hope   hopestaff  123  
Feb 03 15:36   file.txt

Here's what each part of this information means:

-	The first character represents the file type: "-" for a regular file, "d" for a directory, "l" for a symbolic link.
rwx	The next three characters represent the permissions for the file's owner: in this case, the owner may read from, write to, and/or execute the file.
rw-	The next three characters represent the permissions for members of the group that the file belongs to. In this case, any member of the file's owning group may read from or write to the file. The final dash is a placeholder; group members do not have permission to execute this file.
r--	The permissions for "others" (everyone else). Others may only read this file.
1	The number of hard links to this file.
hope	The file's owner.
hopestaff	The group to whom the file belongs.
123	The size of the file in blocks.
Feb 03 15:36	The file's mtime (date and time when the file was last modified).
file.txt	The name of the file.

6, grep

grep [OPTIONS] PATTERN [FILE...]

regular expression example: ls | grep
"^[ab]", list all files and then list only the ones starts with
character a or b.

grep -R //recursively read all files under each directory
following symbolic links

grep -R "ERROR.*IOEXCEPTION" | wc -l  //wc是统计字节数， -l是统计行数。

grep 'AppId' autoCfg.xml     //把autoCfg.xml这个文件中包含AppId的行都显示出来

grep “AppI." autoCfg.xml

grep -w "hope" myfile.txt   //找出word hope，而不是hop123, 123hope23

grep -cw "hope" myfile.txt  //显示行数count

grep -l "hope" /www/*       //显示包含hope的文件名字

grep "hope" *      
        //搜索当前目录

grep -i "hope" myfile.txt   //搜索不计较case大小

grep -v EFM　　　　　　　　//不包含EFM的

grep -R -l 'hello world' *     //Find the file in this system which has the phrase "hello world"

$ export GREP_OPTIONS='--color=auto' GREP_COLOR='100;8'    //用GREP_OPTIONS可以设置字体颜色

By default grep will show the line which matches the given pattern/string, but if you want the grep to show out only the matched string of the pattern then use the -o option.

grep -v Sales employee.txt //grep all the lines except those that contains "Sales"

7, file 

create file: touch file, or cat>file, cat>>file是append到file里。

cat: reading file(cat file1) , concatenating files(cat file1 file2 file3), creating files(cat>file1)

change file permission: chmod 400 file,

copy file: cp file file1

remove file: rm file file1 file2

rename filename: mv file newfile

8, directory

create dir: mkdir sampledir

copy file to dir: cp file sampledir

move file to dir: mv file sampledir

remove empty directory: rmdir sampledir

remove non-empty directory: rm -r sampledir

9, navigating directory

cd ~: immediately go back to home

cd ../config: navigate back to upper level and then go to the config folder

cp ../log4jUS.xml . : copy the log4jUS.xml in upper directory to current directory

10, question: difference between unix and other main-frame os.

11, Jobs

A process is an executing program identified by a unique PID (process identifier).

Jobs are processes which are started by a shell. The shell keeps track of these in a job table. The jobs command shows a list of active background processes. They get a jobspec number which is not the pid of the process.

When using Unix or related operating systems via a terminal, a user will initially only have a single process running, their login shell. Most tasks (directory listing, editing files, etc.) can easily be accomplished by letting the program take control of the terminal and returning control to the shell when the program exits; however, sometimes the user will wish to carry out a task in the background while using the terminal for another purpose. Job control is a facility developed to make this possible, by allowing the user to start programs in the background, send programs into the background, bring background processes into the foreground, and start and stop running programs. Processes under the influence of a job control facility are referred to as jobs.

Jobs和process的主要区别是jobs是由shell start的，shell可以做Job control，Job control主要就是开启background task或把foreground task转成background等。

查看:  ps,              jobs

Run foreground and background processes: sleep 10,  sleep 10 &

Background a current foreground process: sleep 10,  bg,

Foreground a background process:  fg %1,  1 is the job number

kill process and jobs:  kill pid,        kill %jobNumber,  kill -3 pid(terminate with core dump), kill -9 pid(force termination).

12, scp

securely copy files and directories between remote hosts without starting an FTP session or logging into the remote systems explicitly.

scp ~/rebels.txt dvader@deathstar.com:~/revenge
//copy the rebels under home directory to deathstar.com home directory/revenge.

13, top
top is a task manager program found in many Unix-like operating
systems. It produces an ordered list of running processes selected by
user-specified criteria, and updates it periodically. Default ordering by CPU usage, and only the top CPU consumers
shown (hence the name). top shows how much processing power and
memory are being used, as well as other information about the running
processes. 
The ps program
is similar to top, but instead produces a snapshot of processes taken at
the time of invocation.

14, ps

ps -ef will omit process with very long command
line while ps -auxwww will list those process as well.

ps aux | grep {process-name}

pfiles pid //list the opened files of the
process or use lsof -p pid

参数包括display User, PID, %CPU, %Mem,
Command.RSS_Real memory usage, Start time, TTY_Terminal associated with the
process, STAT_Process status code.

ps has many options. On operating systems that
support the SUS and POSIX standards, ps
commonly runs with the options -ef, where "-e" selects every
process and "-f" chooses the "full" output format. Another
common option on these systems is -l, which specifies the "long"
output format.

Most systems derived from BSD fail to accept
the SUS and POSIX standard options because of historical conflicts (for
example, the "e" or "-e" option will cause the display
of environment variables). On such systems, ps
commonly runs with the non-standard options aux, where "a" lists
all processes on a terminal, including those of other users,
"x" lists all processes without controlling terminals and
"u" adds a column for the controlling user for each process. Note
that, for maximum compatibility when using this syntax, there is no
"-" in front of the "aux". Also you can add 'ww' after aux,
like "ps auxww" for complete information about the process including
all parameters.

15, find

find where-to-look criteria what-to-do

find a file in current directory: find .
-name "*.log"

find in the whole system: find / -name scripts

find . –name "*.java" –print | xargs
grep “MemoryCache”  // this will
search all java files starting from current directory for word
"MemoryCache".
==grep "MemoryCache" `find . -name "*.java"

find . -maxdepth 1 -type f -newer first_file
 //Find file only in current directory

find . -size +1000c -exec ls -l {} \;           //-exec is to execute command ls -l, '{}' is to replace content, \ is to end command.

find /tmp -name core -type f -print | xargs /bin/rm -f     //rm the result

find $HOME -mtime 0                                            //find in home directory which have been modified in the past 24 hours.

16, df & du

df (abbreviation for disk free)
is a standard Unix computer program used
to display the amount of available disk space for filesystems on
which the invoking user has appropriate read access. df is usually
implemented by reading the mtab file
or using statfs.

du (abbreviated from disk usage)
is a standard Unix program used
to estimate file space usage—space used under a particular directory or files on
a file system.

17，more 显示那种多于一页内容的文件, less
more 123.txt

Less is a program similar to more, but which
allows backward movement in the file as well as forward movement. In less, if
you press F, it will wait for new updates.

G – go to the end of file                             g – go to the
start of file

q or ZZ – exit the less pager

/ – search for a pattern which will take you to
the next occurrence.

n – for next match in forward                     N – for previous match in
backward

? – search for a pattern which will take you to
the previous occurrence.

n – for next match in backward direction    N – for previous match in forward direction

18, echo
echo "Mary" 只是把Mary这个string打印出来, echo "Mary" | rev 是将Mary
reverse
19, uname
uname -a 打印出来system information：

Linux nykpsr10803 2.6.32-431.17.1.el6.x86_64 #1
SMP Fri Apr 11 17:27:00 EDT 2014 x86_64 x86_64 x86_64 GNU/Linux

20, tail

The following command will display the last 10
lines of messages and append new lines to the display as new lines are
added to messages:

tail -f /var/adm/messages
head -n 5 file1.txt
tail -n 5 file1.txt //by default is 10

21, vi https://www.ccsf.edu/Pub/Fac/vi.html
vi has two modes: the command mode and the insert mode.vi always starts out in
command mode.You can type i to enter the insert mode. If you wish to
leave insert mode and return to the command mode, hit theESC key.You must
be in command mode if you wish to move the cursor to another position in your
file. The command ZZ (notice that it is in uppercase) will allow
you to quit vi and save the edits made to a file. You will then return to a
Unix prompt. Note that you can also use the following commands:

:w      to
save your file but not quit vi (this is good to do periodically in

case
of machine crash!).

:q       to
quit if you haven't made any edits.

:wq     to
quit and save edits (basically the same as ZZ).

:e!      reads
the original file back in so that you can start over.

:q!      wipes
out all edits and allows you to exit from vi.

22, lsof

You can list only the processes which opened a
specific file, by providing the filename as arguments.
# lsof /var/log/syslog

COMMAND 
PID   USER   FD  
TYPE DEVICE SIZE/OFF   NODE NAME

rsyslogd 488 syslog    1w  
REG    8,1     1151 268940 /var/log/syslog

# lsof -p 1753

COMMAND 
PID       USER   FD  
TYPE DEVICE SIZE/OFF    NODE NAME

bash   
1753 lakshmanan  cwd    DIR   
8,1     4096  393571 /home/lakshmanan/test.txt

bash   
1753 lakshmanan  rtd    DIR   
8,1     4096       2 /

23, nslookup

nslookup 204.228.150.3       nslookup is a program used to query Internet domain
name servers. ifconfig is used to configure or view the configuration
of a network interface.
24, 4 stages of Linux process.

Waiting: Linux Process waiting for a resource.

Running : A Linux process is currently being
executed.

Stopped : A Linux Process is stopped after
successful execution or after receiving kill signal.

Zombie : A Process is said to be ‘Zombie’ if it
has stopped but still active in process table.

25, 补充

What is the difference
between Swapping and Paging?

Swapping:

Whole process is moved from the swap device to
the main memory for execution. Process size must be less than or equal to the
available main memory. It is easier to implementation and overhead to the
system. Swapping systems does not handle the memory more flexibly as compared
to the paging systems.

Paging:

Only the required memory pages are moved to main
memory from the swap device for execution. Process size does not matter. Gives
the concept of the virtual memory. It provides greater flexibility in mapping
the virtual address space into the physical memory of the machine. Allows more
number of processes to fit in the main memory simultaneously. Allows the
greater process size than the available physical memory. Demand paging systems
handle the memory more flexibly.

ping - sends packets of info to specified IP address and returns status

 sed s/22:06/21:55/g Marks.txt >> Filter.txt    //In a file called "Marks.txt", search for all occurrences of 22:06 in the file and replace them with 21:55 and append the output to an existing file called Filter.txt

When server is running slow, because of processes are eating up available memory and CPU as either it has gone in infinite loop or bad programming practices like not doing memory cleanup etc. To find out bad process, we should use the top command. From there we can get process id and other useful details. There afterifthe process is not critical or junk then we may decide it to terminate by using the kill command.

TCP (transmission control protocol) and UPD (user datagram protocol) both are internet protocol. TCP is a connection o riented protocol where as UDP is a simple message based connectionless protocol. TCP is reliable, ordered and heavyweight whereas UDP is unreliable, not ordered and lightweight. So we should choose protocol depending upon application type.

http://www.freejobalert.com/it-interviewtechnical-questions/unix-interview-questions-and-answers/

mount命令

unix系统文件并不是只放在同一个区域，例如/var就是挂载到了另一disk区域，通常都有多个mount points.