---------------------------------------------------------------------------------------------------
SORT COMMANDS
---------------------------------------------------------------------------------------------------
Uppercase will appear first than lowercase
Sort alphabetically : sort file.txt
Sort reverse alphabetically : sort -r file.txt
Sort ignore case : sort -f data.txt
Sort data numerically : ls -1v data*.txt || find -name "data*.txt"
Read multiple files at once : find -name "data*.txt" -print0 | sort --files0-from=- | nl
Sort by column : sort -k [column-number] file.txt
Sort by column that have number : sort -nk2 file.txt
Sort and ignore duplicate line : sort -u file.txt
Sort and save output to new file : sort -o outputfile.txt inputfile.txt
Sort data reverse numerically : sort -nr file.txt
Sort ip public based on rang : sort -V ip_list.txt
---------------------------------------------------------------------------------------------------
HEAD & TAIL
---------------------------------------------------------------------------------------------------
Display first ten line of file : head file
Display first 20 line of file : head -20 file
Display last ten line of file : tail file
Display last 20 line of file : tail -20 file
Reading file from number x line : tail +100 file
---------------------------------------------------------------------------------------------------
UNIQ COMMANDS
---------------------------------------------------------------------------------------------------
cat data.txt
line1
line1
line2
line3
line4
line4
line4
line5
line1
Ignore duplicate line : uniq data.txt
Ignore but count duplicate line : uniq -c data.txt
Display which line is duplicate : uniq -d data.txt
Display all duplicate line : uniq -D data.txt
---------------------------------------------------------------------------------------------------
CUT COMMANDS
---------------------------------------------------------------------------------------------------
cat data.txt
My name is Darin
I am Linux Sysadmin
Display 1st and 3rd character : cut -c 1,3 data.txt
Display 4 character : cut -c 1-4 data.txt or cut -c -4 data.txt
Cut if there is space : cut -d " " -f 1 data.txt
Cut second field : cut -d " " -f 1,3,4 data.txt
Cut 3rd field & change space to (-) : cut -d " " -f 1,3,4 data.txt --output-delimiter='-'
Cut 3 character : cut --complement -c 3 data.txt
---------------------------------------------------------------------------------------------------
FMT COMMANDS
---------------------------------------------------------------------------------------------------
cat data.txt
hello,
my name
is darin
Put all text into single line : fmt data.txt (max 75 character)
Create one line is for 10 char : fmt -w 10 data.txt
---------------------------------------------------------------------------------------------------
TR COMMANDS
---------------------------------------------------------------------------------------------------
cat data.txt
hello my name is Darin, i am 21 years old.
i am linux sysadmin with experience as technical support
Convert all strings to uppercase : cat data.txt | tr "a-z" "A-Z"
Convert all strings to lowwercase : cat data.txt | tr "A-Z" "a-z"
Convert space to tab : cat data.txt | tr " " "\t"
Delete specific character : cat data.txt | tr -d "D"
Delete digit : cat data.txt | tr -d [:digit:]
Print just digit (21) : cat data.txt | tr -cd [:digit:]
---------------------------------------------------------------------------------------------------
NL COMMANDS
---------------------------------------------------------------------------------------------------
cat data.txt
line1
line2
line3
line4
Print number at 1st line : nl data.txt
Print number at every first line : nl -b data.txt
Print number and increment is +2 : nl -i 2 data.txt
Print number and use dash (-) : nl -s - data.txt
Print number and the star is 5 : nl -v 5 data.txt
---------------------------------------------------------------------------------------------------
GREP COMMANDS
---------------------------------------------------------------------------------------------------
Print how many times strings found : grep -c -i 'accepted' /var/log/secure
Display line of matched pattern : grep -n "root" ps_result.txt
Sort multiple file in directory : find -name "data*.txt"
---------------------------------------------------------------------------------------------------
SED COMMANDS
---------------------------------------------------------------------------------------------------
cat books.txt
1) A Storm of Swords, George R. R. Martin, 1216
2) The Two Towers, J. R. R. Tolkien, 352
3) The Alchemist, Paulo Coelho, 197
4) The Fellowship of the Ring, J. R. R. Tolkien, 432
5) The Pilgrimage, Paulo Coelho, 288
6) A Game of Thrones, George R. R. Martin, 864
Delete line 1 : sed -e '1d' books.txt
Delete multiple line : sed -e '1d' -e '6d'
Delete line x from file : sed -i 'xd' books.txt
Duplicating each line : sed -e 'p' books.txt
Duplicationg line 2 : sed -e '2p' books.txt
Replace text 'The' with 'A' : sed 's/The/A/' books.txt
Only show line 5-8 : cat ps_result.txt | sed -e '1d' | nl | sed -n 5,8p
Replace text in line 5 : sed '5!s/hendarin/root/' file.txt
If boom is found rplc aaa with bb : sed '/boom/!s/aaa/bb/' file.txt
Delete lines matching pattern : sed '/pattern/d' file.txt
Get mail header : sed '/^$/q'
Get mail body : sed '1,/^$/d'
Get mail subject : sed '/^Subject: */!d; s///;q'
Replace text but backup first : sed -i .bak 's/old/new/g' [file]
---------------------------------------------------------------------------------------------------
AWK COMMANDS
---------------------------------------------------------------------------------------------------
cat top.txt
PID users PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1 root 20 0 8304 56 28 S 0.0 0.0 0:00.23 init
119 root 20 0 8304 52 16 S 0.0 0.0 0:00.00 init
120 darin 20 0 17092 2708 2600 S 0.0 0.1 0:01.21 bash
1056 darin 20 0 16948 3696 3620 S 0.0 0.1 0:01.12 bash
1174 darin 20 0 17620 2020 1500 R 0.0 0.0 0:00.35 top
1179 darin 20 0 17620 2040 1500 S 0.0 0.0 0:00.14 top
1180 darin 20 0 17864 1756 1416 S 0.0 0.0 0:00.03 awk
AWK Structure :
awk ' BEGIN{ print "start" } pattern { commands } END{ print "end"}' file
awk ' BEGIN{ statements } { statements } END { end statements }'
awk ' BEGIN{ i=0 } { i++ } END { print $i }' file
read file
awk '{print}' top.txt
awk 1 top.txt
show only column x
awk '{print $x}' top.txt
show 2 column
awk '{print $2 " " $4}' << with space
awk '{print $2 "\t" $4}' << with tab
awk '{print $2 "\n" $4}' << with new line
print last column
awk '{print $NF}' top.txt
show line which contain 'darin' only
awk '/darin/' top.txt
awk '/darin/ {print $0}' top.txt
show line which contain 'darin' column 2 and 12 only
awk '/darin/ {print $2 "\t" $12}' top.txt
counting and printing matched pattern
awk '/darin/{++cnt} END {print "Count = " cnt}' top.txt
giving header
awk 'BEGIN{print "PIDnumber\tKolomuser\n"} {print}' top.txt
awk 'BEGIN{print "1\t2\t3\t4\n"} {print}' top.txt
awk 'BEGIN{print "This is TOP header"} {print}' top.txt
giving footer
awk 'END {print "This is TOP footer"} {print}' top.txt
remove header
awk 'NR>1 {print}' top.txt
awk 'NR>4 {print}' top.txt (if there is 4 line of header)
erase specific strings
awk '{print}' RS='stringhere' top.txt
change all text to UPPERCASE
awk '$0 { print toupper($0); }' top.txt
change all text to LOWWERCASE
awk '$0 { print tolower($0); }' top.txt
assign data to variable
echo | awk -v home=$HOME '{print "My home is " home}'
ignore column 1
awk '{$1=""; print}'
NR: It keeps a current count of the number of input line.
NF: It keeps a count of the number of fields within the current input record.
FS: It contains the field separator character which is used to divide fields on the input line.
RS: It stores the current record separator character.
OFS: It stores the output field separator, which separates the fields when Awk prints them.
ORS: It stores the output record separator, which separates the output lines when Awk prints them.
---------------------------------------------------------------------------------------------------
XARGS COMMANDS
---------------------------------------------------------------------------------------------------
cat example.txt
1 2 3 4 5 6
7 8 9 10
11 12
AxBxCxDxExF
cat example.txt | xargs
1 2 3 4 5 6 7 8 9 10 11 12 AxBxCxDxExF
cat example.txt | xargs -n 6 | grep -v AxBxCxDxExF
1 2 3 4 5 6
7 8 9 10 11 12
cat example.txt | xargs -d x
1 2 3 4 5 6
7 8 9 10
11 12
A B C D E F
echo folder1 folder2 folder3 | xargs -t mkdir
ps auxf | awk '{print $2}' | xargs
pgrep perl | xargs kill -9
ls | xargs wc
find . -type f | xargs grep "accepted"
find . -name "*.xyz" -type f | xargs rm -rf
find . -name "foo*" -print0 | xargs -0 vim
find . -name "*.xyz" -type f -print0 | xargs -0 tar -cvzf file.tar.gz
- -print0 : to procedure an ouput delimited by the null character
---------------------------------------------------------------------------------------------------
SPLIT COMMANDS
---------------------------------------------------------------------------------------------------
#create 1G of dummyfile
fallocate -l 1G dummyfile
#split to 10 file, each file is 100M (alphabetic)
split -b 100M dummyfile; ls -larth
-rw-rw-rw- 1 darin darin 104857600 Jan 14 21:22 xaa
-rw-rw-rw- 1 darin darin 104857600 Jan 14 21:22 xab
-rw-rw-rw- 1 darin darin 104857600 Jan 14 21:22 xac
-rw-rw-rw- 1 darin darin 104857600 Jan 14 21:22 xad
-rw-rw-rw- 1 darin darin 104857600 Jan 14 21:22 xae
-rw-rw-rw- 1 darin darin 104857600 Jan 14 21:22 xaf
-rw-rw-rw- 1 darin darin 104857600 Jan 14 21:23 xag
-rw-rw-rw- 1 darin darin 104857600 Jan 14 21:23 xah
-rw-rw-rw- 1 darin darin 104857600 Jan 14 21:23 xai
-rw-rw-rw- 1 darin darin 104857600 Jan 14 21:23 xaj
-rw-rw-rw- 1 darin darin 25165824 Jan 14 21:23 xak
#split to 10 file, each file is 100M (numeric)
split -b 100M dummyfile -d -a 4; ls -larth
-rw-rw-rw- 1 darin darin 100M Jan 14 21:26 x0000
-rw-rw-rw- 1 darin darin 100M Jan 14 21:26 x0001
-rw-rw-rw- 1 darin darin 100M Jan 14 21:26 x0002
-rw-rw-rw- 1 darin darin 100M Jan 14 21:26 x0003
-rw-rw-rw- 1 darin darin 100M Jan 14 21:26 x0004
-rw-rw-rw- 1 darin darin 100M Jan 14 21:26 x0005
-rw-rw-rw- 1 darin darin 100M Jan 14 21:26 x0006
-rw-rw-rw- 1 darin darin 100M Jan 14 21:26 x0007
-rw-rw-rw- 1 darin darin 100M Jan 14 21:26 x0008
-rw-rw-rw- 1 darin darin 100M Jan 14 21:27 x0009
-rw-rw-rw- 1 darin darin 24M Jan 14 21:27 x0010
#split to 10 file, each file is 100M (numeric+filename)
-rw-rw-rw- 1 darin darin 100M Jan 14 21:28 file0000
-rw-rw-rw- 1 darin darin 100M Jan 14 21:28 file0001
-rw-rw-rw- 1 darin darin 100M Jan 14 21:28 file0002
-rw-rw-rw- 1 darin darin 100M Jan 14 21:28 file0003
-rw-rw-rw- 1 darin darin 100M Jan 14 21:28 file0004
-rw-rw-rw- 1 darin darin 100M Jan 14 21:28 file0005
-rw-rw-rw- 1 darin darin 100M Jan 14 21:28 file0006
-rw-rw-rw- 1 darin darin 100M Jan 14 21:28 file0007
-rw-rw-rw- 1 darin darin 100M Jan 14 21:28 file0008
-rw-rw-rw- 1 darin darin 100M Jan 14 21:28 file0009
-rw-rw-rw- 1 darin darin 24M Jan 14 21:28 file0010
