Bash Shell Learning Notes

Bash: Bourne-again shell, referring to its objective as a free replacement for the Bourne shell.

Lynda online tutorial: http://www.lynda.com/Bash-tutorials/Up-Running-Bash-Scripting/142989-2.html

Push to backend run

bash filename.sh > log.txt 2>&1

Linux command:

pwd: current working directory

ls 

ls directory

ls -l directory: info

man ls: help info

rmdir directory : delete

clear

cp file1 file2: copy 1->2

rm: delete file
cat file: see what's inside

more file: view page by page

head file: first couple line

tail file: last couple line

chmod mode file: change mode

eg: chmod 000 *_015*

#	Permission	rwx
7	read, write and execute	111
6	read and write	110
5	read and execute	101
4	read only	100
3	write and execute	011
2	write only	010
1	execute only	001
0	none	000

Brace Expansion: {} 

touch file{1,2,3} :create multiple files

touch file_{1...100}: create 100 files 

touch {apple,banana}_{01..03}{a..b} : note no space inside

echo {1..10..2}: 1,3,5,7,9

rm *: delete all

rm -I * : delete all with prompt once

ls -l | wc -l : count files

ls | more : list view page by page

Changing where things go with pipes and redirection

cp -v * ../otherfolder 1>../success.txt 2>../error.txt

copy all files from current folder to another folder, -v: output what is being done; 1 successful copy, 2 error message, & all message

/dev/null: nowhere, eg: ls > /dev/null

Manipulating output with grep, awk, and cut

Grep: Search

Awk:  Awk is both a programming language and text processor that can be used to manipulate text data in very useful ways.

Cut : Select certain items with deliminator

grep search_term file: search term in file

grep -i : None case sensitive

grep  search_term file | awk {'print $12'}: search for lines, and only return the 12th item in line

linux command | grep 'search term': search in an linux command output, eg: ping -c l example.com | grep 'bytes from'

Cut: Eg: ping -c 1 example.com | grep 'bytes from' | cut -d = -f 4 : 4th thing counting by deliminator '='.  Counting method: 1st=2nf=3rd=4th

BASH Script

#!/bin/bash

# comment

commands

Two ways executing the bash file:

1. bash my.sh

2. chmod +x my.sh

    ./my.sh

Displaying text with echo

echo statement : need \ before special char, eg \(

echo 'statement' : take everything as text

echo "statement" : middle way, $ as special, everything else as text, \$ to take $ as text 

greeting='hello'
echo $greeting, world \(planet\)!
echo '$greeting, world (planet)!'
echo "$greeting, world (planet)!"

Output:
hello, world (planet)!
$greeting, world (planet)!
hello, world (planet)!

Variables

Define variable

Variable name should start with letter

a=Hello: No space

b="Hello World": if space, then put in quote

Use variable

$a: put $ before variable

Adding attributes to variables

declare -i d=123 #d is an integer

-i: integer

-r: read only

-l: lower case

-u: upper case

Built-in variables

$HOME : home dir

$PWD : current dir

$MACHTYPE : machine type

$HOSTNAME: system name

$BASH_VERSION: version of Bash

$SECONDS : time in seconds the bash session has run

$0 : file name

Command substitution

d=$PWD or d=$(pwd)

a=$(long command)

Working with numbers

((expression))

val=$((expression))

Operators

* / %       #(times, divide, remainder)

+ -         #(add, subtract)

** #Exponentiation

Example:

#!bin/bash

a=2

b=1

e=$((a+b))

echo "$a plus $b = $e"

((e++))

echo $e

((e--))

echo $e

((e+=5))

echo $e

((e*=2))

echo $e

Float Number (bc command)

f=$((1/3)) #0

g=$(echo 1/3 | bc -l) #.333

Comparing values

[[expression]]

1:FALSE

0:TRUE

For Char:

< > <= >=   #(the obvious comparison operators)

== !=       #(equal to, not equal to)&&          #(logical and)

Examples

[["cat" == "dog"]] #1, False

[[20 > 100]]  #0, True, as it compares it as string

For int numbers:

-lt #less than

-gt 

-le #less than or equal to

-ge

-eq #equal

-ne

 

For Logic:

&& #and

|| #or

! #not

For string

-z #is null?

-n #is not null?

Working with strings

Concatenation $a$b

a="hello"

b="world"

c=$a$b #helloworld

echo $c 

Length of string: ${#a}

Substring: 

${a:3} #substring starting from 3rd char 'lo' (index starts from 0)

${a:3:1} #start at 3, ask for 1 char after that

${c: -4} #4chars from end of the string 

${c: -4:3} #first 3 letters of the last 4 letters 

Search and Replace

1. search and replace once

${str/search_term/replace_term}

eg. 

fruit='apple banana banana cherry'

echo ${fruit/banana/durian} #first banana gets replaced

2. search and replace for all

${str//search_term/replace_term}

eg.

fruit='apple banana banana cherry'

echo ${fruit//banana/durian} #all banana gets replaced

3. Matching 

wildcard: *

eg.

fruit='apple banana banana cherry'

echo ${fruit//b*/durian} # 'banana banana cherry' gets replaced by 'durian'

4. search and replace once only if it is in certain location

${str/#search_term/replace_term} # only if search_term at the beginning of the string

${str/%search_term/replace_term} # only if search_term at the end of the string

Coloring and styling text

Echo with escaping

eg: echo -e '\033[34;42mColor Text\033[0m'

format: echo -e '\033[text style ANSI; foreground color; background colorm Text \033[clear format using 0 m'

Begin escaping: \

End escaping: m

Style: Foreground/Background color number code

Date and printf

today: date

format: date+"%d-%m-%Y"

          date+"%H-%M-%S"

printf "Name:\t%s\nID:\t%04d\n" "Scott" "12"

eg. 

today=$(date+"%d-%m-%Y')

time=$(date+"%H-%M-%S")

printf -v d "Current User:\t%s\nDate:\t\t%s @ %s\n" $USER $today $time

echo "$d"

Array (0 based)

a=()

b=("apple" "banana" "cherry")

echo ${b[2]}

b[5]="kiwi"

b+=("mango")

echo ${b[@]} #whole array

declare -A myarray

myarray[color]=blue

myarray["office"]="HQ West"

echo ${myarray["office"]} is ${myarray[color]}

Reading and Writing text files

Write

echo "Some test" > file.txt #Write by replacing

echo "Some test" >> file.txt #Write by adding

>file.txt  #delete content in the file

Read

while read f; do

echo $f

done<file.txt

Putting command in file and execute 

ftp -n < ftp.txt

Here documents

cat << EndOfText

This is a 

multiline

text string

EndOfText

cat << EndOfText >file.txt

This is a 

multiline

text string

EndOfText

<<- eliminate tab at the beginning of the each line

Control Structures

If...Else...

(1) if((expression))

(2) if expression

     then

          echo "True"

     elif expression2; then

          echo "False"

     else

          echo "Null"

     fi

a=5
if [ $a -gt 4 ]
then
        echo $a is greater than 4!
else
        echo $a is not greater than 4.
fi

#Note:  [ $a -gt 4 ] space needed

Case

a="dog"

case $a in

     cat) echo "Feline";;

     dog|puppy) echo "Canine";;

     *) echo "No match!";;

esac

Loop

while

i=0 

while [ $i -le 10 ]; do

     echo i:$i

     ((i+=1))

done

until

counterpart of while loop, once qualify -> exit

for

#M1

for i in 1 2 3

do 

          echo $i

done

#M2: 1~100, counting by 2

for i in {1..100..2}

#M3

for((i=1; i<=10;i++))

#M4 array

arr=("apple" "banana" "cherry")

for i in ${arr[@]}

#M5

declare -A arr

arr["name"]="Scott"

arr["id"]="1234"

for i in "${!arr[@]}"

do

          echo "$i: ${arr[$i]}"

done

#M6 command

for i in $(ls)

Loop

Example:

startdate='20140929'

for i in $(seq 1 1 7)
do
dati=$(date -d "$startdate $i days" +'%Y%m%d')
echo "$dati"
done

Function

#Note: must be a space between function name and {

function greet {

     echo "Hi $1!"

}

echo "And now, a greeting!"

greet Scott

function numberthings {

     i=1

     for f in $@; do

          echo $i: $f

          ((i+=1))

     done

}

numberthings $(ls)

numberthings pine birch spruce

Arguments

#manual argument: 

echo $1

echo $2

#put all argument as an array

for i in $@

do 

          echo $i

done

echo There are $# arguments.

Flags (Arguments by matching char, not by order)

while getopts u:p: option; do

     case $option in

          u) user=$OPTARG;;

          p) pass=$OPTARG;;

     esac

done

     echo "User: $user / Pass: $pass"

./my.sh -u scott -p secret

#u:p: there will be two flags

#u:p:ab a,b optional to be here

#:u:p:ab Other key in wildcard ?) 

Getting input during execution

Method 1

echo "What is your name?"

read name

echo "What is your password?"

read -s pass

read -p "What's your favorite animal?" animal

echo name: $name, pass: $pass, animal: $animal

Method 2

select animal in "cat" "dog" "bird" "fish"

do

     echo "You selected $animal!"

     break

done

select option in "cat" "dog" "quit"

do

     case $option in

          cat) echo "Cats like to sleep.";;

          dog) echo "Dogs like to play catch.";;

          quit) break;;

          *) echo "I'm not sure what that is.";;

     esac

done

Ensuring a response of number of arguments

if [$# -lt 3]; then

          cat <<- EOM

          This command requires three arguments: a1, a2, a3.

     EOM

else

          #the program goes here

          echo "Username: $1"

          echo "UserID: $2"

          echo "Favorite Number: $3"

fi

Setting default value

read -p "Favorite animal? [cat]" a

#if a is empty

while [[-z "$a"]]; do

          a="cat"

done

echo "$a was selected."  

Checking input format

read -p "What year? [nnnn]" a

while [[! $a=~[0-9]{4}]]; do

     read -p "A year, please! [nnnn]" a

done

echo "Select year: $a"

#Project Guessing Game

#Input a number

echo "Input A Secret Number: " 

read -s a

while [[ ! $a -lt 100 && $a -gt 0 ]]; do

     echo "A number in 0-100, please! " 

     read -s a

done

#Or a random Number 

rand=$RANDOM

a = ${rand:0:1}

function guess {

     if [ $1 -eq $2 ]; then

     echo "Bingo! The number is $1!"

     echo "You are the best!"

     echo "You are amazing!"

     echo "You are YOU DIAN NI HAI!"

     elif [ $1 -lt $2 ]; then

     echo "$1 is smaller than the number."

     else 

     echo "$1 is greater than the number."

     fi

}

read -p "Guess a number from 0 to 100: " b

guess $b $a

while [[ ! $a -eq $b ]]; do 

read -p "Guess again: " b

guess $b $a

done

Next step

man bash

tldp.org/LDP/abs/html/gotchas.html