Shell scripting

Choose a unique name for the scripting file. Try following commands to see if an executable with the same name exists:

type myscript
type -a myscript
which myscript

Then we need to store our executable in the system PATH. Check the existing paths:

echo $PATH

If we want to include a new path to our $PATH variable, we can include following in our .bashrc:

PATH=$PATH:/new/path

or, issue this every time you restart your terminal

export PATH="/new/path:$PATH"

We can write our script in a file:

myscript.sh

#!/bin/bash
echo "Hello world"

Next step is to grant executable permission:

chmod +x myscript.sh

We can execute the script by:

./myscript.sh

We can also run the script by:

sh myscript.sh

bash myscript.sh

info

sh and bash are not exactly the same, and might give slightly different result in some cases. bash refers to Born Again SHell while sh refers to the original UNIX shell. To know which shell interpreter is currently running, issue: echo $0.

Variables:

my_var="/home/user/pranab"
my_num=0
my_num=$((my_num + 1))

echo ${my_var}
echo $my_num

tip

bash uses capitalized variable names for global variables (e.g., PATH, PWD). Perhaps it is a good idea to name local variables with lower case letters to avoid conflict.

Array:

my_arr=("one" "two")

# add item
my_arr+=(three)

# length of an array
len=${#my_arr[@]}

For loop:

for i in "${my_arr[@]}"
  do printf "${i}\n"
done

Use seq to create array with increment:

$ my_arr2=`seq 0 2 10`; echo $my_arr2
0
2
4
6
8
10

tip

In bash the array index starts from 0, while in zsh it starts from 1. If you want your script to be cross compatible, you may use ${array[@]:offset:length} e.g., only first element ${array[@]:0:1}; only second element ${array[@]:1:1}; first and second element ${array[@]:0:2}.

If condition:

# if a file exists
if [ -e ".DS_Store" ] ; then
    rm ".DS_Store"
fi

# if a directory exists
if [ -d "tmpdir" ] ; then
    rm -rf tmpdir
fi

Common file attribute operators:

Operator	True if
-a or -e	file exists
-d	is a directory
-f	is a regular file (e.g., not a directory)
-s	not empty
-r/w/x	have read/write/execute permission

Brace expansion:

$ echo {1..10}
1 2 3 4 5 6 7 8 9 10

$ echo {2020..2021}-{01..12}
2020-01 2020-02 2020-03 2020-04 2020-05 2020-06 2020-07 2020-08
2020-09 2020-10 2020-11 2020-12 2021-01 2021-02 2021-03 2021-04
2021-05 2021-06 2021-07 2021-08 2021-09 2021-10 2021-11 2021-12

A bash script to convert degree Celsius to Fahrenheit:

src/deg_f_to_c_conv.sh

#!/bin/bash
# this reports undefined variables
shopt -s -o nounset

# declare integer variables
declare -i tempF
declare -i tempC

printf "Fahrenheit-Celsius Conversion\n\n"

# read value via prompt
read -p "Enter temperature in Celsius (integer only): " tempC
tempF="9*tempC/5+32"
printf "The Fahrenheit temperature is %d\n" "$tempF"
exit 0

String substitution

Rename files example:

$ ls
item_01.txt item_03.txt item_05.txt item_07.txt item_09.txt
item_02.txt item_04.txt item_06.txt item_08.txt item_10.txt

# match string at end with %
$ for i in $( ls ); do mv ${i} ${i%*.txt}.md; done; ls
item_01.md item_03.md item_05.md item_07.md item_09.md
item_02.md item_04.md item_06.md item_08.md item_10.md

# match string at beginning with #
$ for i in $( ls ); do mv ${i} slide_${i#*_*}; done; ls
slide_01.md slide_03.md slide_05.md slide_07.md slide_09.md
slide_02.md slide_04.md slide_06.md slide_08.md slide_10.md

# or
$ for i in $( ls ); do mv ${i} slide_${i#item_*}; done

Replace spaces in the filenames with underscores:

for file in *\ *; do mv "$file" "${file// /_}"; done

Running python code in bash script

You can include python, ruby, or any other language code in a bash script. Here is an example:

src/github_emoji.sh

if [ -d "assets" ]; then
  rm -rf assets
fi
mkdir assets

wget https://api.github.com/emojis -O data.json

python3 <<EOF
import json

fid = open('data.json')
data = json.load(fid)
fid.close()

fid = open('wget_urls', 'w')
for key, value in data.items():
    fid.write("wget {0} -O assets/{1}.png\n".format(value, key))
fid.close()

fid = open('README.md', 'w')
fid.write("# GitHub emoji assets\n\nThis repository contains the GitHub emoji assets in PNG format. Here are all the {0} emojis:\n\nName | Emoji\n---- | -----\n".format(len(data)))
for key in data.keys():
    fid.write('{0} | <img src="assets/{0}.png" alt="{0}" width=20/>\n'.format(key))
fid.close()
EOF

sh wget_urls
rm data.json wget_urls

Input password

Here is a solution for the bash:

#!/bin/bash
read -s -p "Password: " passwd; echo
# do something with the passwd
echo $passwd
unset passwd

A POSIX compatible alternative:

#!/bin/sh
# disable echo
stty -echo
printf "Password: "
read passwd
# enable echo
stty echo
printf "\n"
echo $passwd

New line in echo:

echo -e "First line.\nSecond line."

Check command availability

COMMANDS=("git" "node" "python3" "ruby")

for COMMAND in $COMMANDS; do
  if ! command -v "$COMMAND" &> /dev/null; then
    echo "Please install $COMMAND";
    # exit 1;
  fi
done

Check environment variable

if [ -z "$HOME" ]; then
  echo "Seems "\$HOME" is not defined :(";
  # exit 1;
fi

Assign variable using basic calculator

Make sure you have basic calculator installed:

apt install bc

var2=$(echo "scale=7;${var}/2" | bc)

Parameter expansion

Strip everything after the query string ?:

url="https://example.com/resource?query=123&filter=abc"
path=${url%%\?*}
echo $path  # https://example.com/resource

An alternate of the above could be:

path=(${url//\?/ })  # substitute `?` with spaces, then assign array
path=${path[0]}

Select filename from url:

url="https://example.com/file.txt"
filename=${url##*/}
echo $url  # file.txt

An alternate of the above could be:

unicode=(${path//\// })  # substitute `/` with spaces, then assign array
last_index=$(( ${#unicode[@]} - 1 ))
unicode="${unicode[$last_index]}"

Parse arguments in function

Arguments passed to a Bash script can be accessed using positional parameters:

• $0: The name of the script itself.
• $1, $2, $3, etc.: The first, second, third, and subsequent arguments.
• $#: The number of arguments passed to the script.
• $@: All arguments as a single string.
• $*: All arguments as separate words.

src/add.sh

#!/bin/bash

# Usage: add x y
# where x and y are integer numbers
add() {
  x="$1"
  y="$2"

  result=$( expr $x + $y )

  echo "$x + $y = $result"
}

add 2 5

Execute the script file:

bash add.sh

We can process positional argument with case:

src/arg.sh

#!/bin/bash

# Usage: bash arg.sh <time of the day> <your name>

case $1 in
  morning)
    echo "Good Morning, $2!"
    ;;
  evening)
    echo "Good Evening, $2!"
    ;;
  *)
    echo "Hello, $2!"
    ;;
esac

Execute the script file:

bash arg.sh morning Pranab
# Good Morning, Pranab!

bash arg.sh night Pranab
# Hello, Pranab!

Getopt

Process arguments with getopt. Basic example:

src/getopt.sh

#!/bin/bash

OPTSTRING=":vh"

while getopts ${OPTSTRING} opt; do
  case ${opt} in
    h)
      echo "Help message."
      ;;
    v)
      echo "Verbose mode."
      ;;
    ?)
      echo "Invalid option: -${OPTARG}."
      exit 1
      ;;
  esac
done

bash src/getopt.sh -h
# Help message.

bash src/getopt.sh -v
# Verbose mode.

bash src/getopt.sh -l
# Invalid option: -l.

Getopt option with argument (notice the : after x and y):

Process arguments with getopt. Basic example:

src/getopt_arg.sh

#!/bin/bash

OPTSTRING=":x:y:"

while getopts ${OPTSTRING} opt; do
  case ${opt} in
    x)
      echo "Option -x provided, Argument: ${OPTARG}"
      ;;
    y)
      echo "Option -y provided, Argument: ${OPTARG}"
      ;;
    :)
      echo "Option -${OPTARG} requires an argument."
      exit 1
      ;;
    ?)
      echo "Invalid option: -${OPTARG}."
      exit 1
      ;;
  esac
done

danger

Note that below getopt example does not work on macOS (BSD getopt). Please use GNU getopt (found on Linux, or install GNU version on macOS via homebrew).

Getopt with long options:

src/getopt_long.sh

#!/bin/bash

# Parse arguments using getopt
PARSED_ARGUMENTS=$(getopt -o hi:o: --long help,input:,output: -- "$@")
if [ $? -ne 0 ]; then
    echo "Error parsing arguments."
    exit 1
fi

# Rearrange arguments as per getopt parsing
eval set -- "$PARSED_ARGUMENTS"

# Initialize variables for options
input_file=""
output_file=""

# Process arguments
while true; do
    case "$1" in
        -h|--help)
            echo "Usage: $0 -i <inputfile> -o <outputfile>"
            echo "       $0 --input <inputfile> --output <outputfile>"
            shift
            exit 0
            ;;
        -i|--input)
            input_file="$2"
            shift 2
            ;;
        -o|--output)
            output_file="$2"
            shift 2
            ;;
        --)
            shift
            break
            ;;
        *)
            echo "Invalid option: $1"
            exit 1
            ;;
    esac
done

# Print parsed arguments
echo "Input file: $input_file"
echo "Output file: $output_file"
echo "Remaining arguments: $@"

bash src/getopt_long.sh -i input.txt -o output.csv -- more

# Input file: input.txt
# Output file: output.csv
# Remaining arguments: more

Call function from another script

Define greeting function in function.sh:

src/function.sh

#!/bin/bash

greeting() {
    echo "Hello, good morning!"
}

Call in from main.sh:

src/main.sh

#!/bin/bash

source function.sh

greeting

Execute main.sh:

bash main.sh

# Hello, good morning!

Find location of a script file

src/script.sh

#!/bin/bash

path=$( dirname "$0" )
cd $path
script_path=${PWD}
echo $script_path

Execute the script from any directory:

bash src/script.sh

# /Users/pranab/Desktop/workspace/sites/linux/src

Resources

• Learning the bash Shell: Unix Shell Programming by C. Newham.
• Linux Shell Scripting With Bash by K. O. Burtch.
• https://tldp.org/guides.html