Archive for the ‘Scripts’ category

How print series of numbers or letters using Bash Loop

March 2nd, 2011

Bash scripting is great tool in Linux for System Admin. Here is a quick trick to print a series of numbers (or letters) into a variable, and thus an argument of a script:

#!/bin/bash
for a in {1..18}
do
echo “The number $a”
done

#!/bin/bash
for a in {a..z}
do
echo ” The letter $a ”
done

Process file line by line using awk

December 27th, 2010

At the command line, enter the following command:

$ awk ‘{ print }’ /etc/passwd

You should see the contents of your /etc/passwd file appear before
your eyes.Now, for an explanation of what awk did. When we called awk,we specified /etc/passwd as our input file. When we executed awk, it evaluated the print command for each line in /etc/passwd, in order.All output is sent to stdout, and we get a result identical to catting /etc/passwd. Now, for an explanation of the { print } code block. In awk, curly braces are used to group blocks of code together, similar to C. Inside our block of code,we have a single print command. In awk, when a print command appears by itself, the full contents of the curr-ent line are printed.

Here is another awk example that does exactly the same thing:

$ awk ‘{ print $0 }’ /etc/passwd

In awk, the $0 variable represents the entire current line, so print
and print $0 do exactly the same thing. Now is a good time for an
example.
#!/bin/bash #SCRIPT: method4.sh #PURPOSE: Process a file line by line with awk FILENAME=$1 awk ‘{kount++;print kount, $0} END{print “\nTotal ” kount ” lines read”}’ $FILENAME

Output:
[root@www blog]# sh method4.sh file_passwd 1 venu:x:500:500:venu madhav:/home/venu:/bin/bash 2 padmin:x:501:501:Project Admin:/home/project:/bin/bash 3 king:x:502:503:king:/home/project:/bin/bash 4 user1:x:503:501::/home/project/:/bin/bash 5 user2:x:504:501::/home/project/:/bin/bash Total 5 lines read

Awk is really good at handling text that has been broken into multiple
logical fields, and allows you to effortlessly reference each individ-
ual field from inside your awk script. The following script will print
out a list of all user accounts on your system:

awk -F”:” ‘{ print $1 “\t ” $3 }’ /etc/passwd

Above, when we called awk, we use the -F option to specify “:” as the
field separator. By default white space (blank line) act as filed sep-
arator. You can set new filed separator with -F option. When awk proc-
esses the print $1 “\t ” $3 command, it will print out the first and
third fields that appears on each line in the input file. “\t” is used
to separate field with tab.

Sed Examples

December 15th, 2010

Use sed recursively

Apply a sed command to all files found recursively on target directory (xargs allows breaking the list of arguments into smaller ones when dealing with large number of files) find </path/to/dir> -type f | xargs sed -i ‘s/<replaceThis>/<withThis>/’

It seach inside on all files of /tmp folder for Installing and if found it will replace it with manoj-Installing
find /tmp/ -type f | xargs sed -i ‘s/Installing/manoj-Installing/’

Apply a sed command only to certain files (for instance all css files)
find </path/to/dir/*.css> -type f | xargs sed -i ‘s/<replaceThis>/<withThis>/’

Apply a sed command only to files that contain a certain pattern
grep -rl “<pattern>” </path/to/dir> | xargs sed -i ‘s/<replaceThis>/<withThis>/’

White spaces

Remove/delete leading whitespaces (tabs,spaces) from each line and align to left
sed ‘s/^[ \t]*//’

Remove/delete trailing whitespaces (tabs,spaces) from each line
sed ‘s/[ \t]*$//’

Remove/delete leading and trailing whitespaces from each line
sed ‘s/^[ \t]*//;s/[ \t]*$//’

mySQL slow query log rotation

October 25th, 2010

One of the issues facing log rotation in mySQL is that mySQL doesn’t seem to have the ability to perform a “reload”.

Meaning standard methods of rotating logs using logrotate leave mySQL logging an error in the syslog saying that the log file could not be found, and refusing to to any logging until the server is restarted, not fun if like me you manage high availability solutions and restarting a service is never the best option.

As such I have written this log rotate script, this does however make some assumptions.

1. Your mysql user is “mysql”
2. Your mysql slow query log is /var/log/mysql-slow.log

The script is written to perform the following actions:

1. Rotate weekly
2. Retain 3 rotations (3 files + live log)
3. Compress on rotate (gzip)
4. Create new logfile with 660 permissions chowned to mysql:mysql
5. Run: mysqladmin flush-logs

Please be aware that the flush logs command will also rotate any binary logging currently in place, please ensure this will not adversely affect your deployment prior to use

#
# mySQL slow log rotation script by D.Busby
# place this script in /etc/logrotate.d/ or your appropriate logrotate dir.
# http://onaxer.com
#
# NOTE: if you are reliant on binlogs i.e. for replication, ‘flush logs’ closes the current binlog and moves to the next itteration of log files
# You should test this does not cause an issue with your deployment before using this script
#
/var/log/mysql-slow.log {
weekly
rotate 3
compress
missingok
notifempty
sharedscripts
create 660 mysql mysql
postrotate
/usr/bin/mysqladmin flush-logs
endscript
}

SSH login expect shell script to supply username and password

August 12th, 2010

#!/usr/bin/expect -f
# Expect script to supply root/admin password for remote ssh server
# and execute command.
# This script needs three argument to(s) connect to remote server:
# password = Password of remote UNIX server, for root user.
# ipaddr = IP Addreess of remote UNIX server, no hostname
# scriptname = Path to remote script which will execute on remote server
# For example:
#  ./sshlogin.exp password 192.168.1.11 who
# ————————————————————————
# Copyright (c) 2010 Onaxer project <http://onaxer.com/>
# This script is licensed under GNU GPL version 2.0 or above
# ————————————————————————-
# This script is part of Onaxer shell script collection (NSSC)
# Visit http://onaxer.com/ for more information.
# ———————————————————————-
# set Variables
set password [lrange $argv 0 0]
set ipaddr [lrange $argv 1 1]
set scriptname [lrange $argv 2 2]
set arg1 [lrange $argv 3 3]
set timeout -1
# now connect to remote UNIX box (ipaddr) with given script to execute
spawn ssh root@$ipaddr $scriptname $arg1
match_max 100000
# Look for passwod prompt
expect “*?assword:*”
# Send password aka $password
send — “$password\r”
# send blank line (\r) to make sure we get back to gui
send — “\r”
expect eof

Thanks
Manoj Chauhan

How imports historical Nagios log files into the mysql database

March 5th, 2010

NDO Utilities:-

The NDO utilities add-on, written by Nagios developer Ethan Galstad, is designed to output events and data from Nagios to standard files or to a Unix socket. It also comes with a module called NDO2DB that allows Nagios data to be written to a MySQL or PostgreSQL database The add-on is made up of the NDOMOD Event Broker module, which is loaded by Nagios at runtime. It dumps all events and data from Nagios to a regular file or a Unix domain socket. It also contains the ndo2db daemon, which reads data that has been sent for the NDOMOD module to a Unix domain socket and dumps it into a MySQL or PostgreSQL database. You can dump into multiple databases and have multiple instances of the NDOMOD module writing to the same domain socket. There is also a utility called FILE2SOCK, which reads data from a standard file and dumps it into a Unix domain socket. Suggested uses are to dump data from
NDOMOD that has been stored in a standard file into a Unix domain socket. Or if your Nagios server is remote from your database server, you can dump data into a standard file from NDOMOD, send the file via SSH or SFTP to the database server, and then dump the data into a Unix domain socket and from there into a database. Finally, there is the LOG2NDO utility, which imports historical Nagios log files into the ndo2db daemon and sends them to a Unix domain socket or to standard output.

To install the NDO add-on, you first need to download and unpack the module from the Nagios Sourceforge site at http://sourceforge.net/project/showfiles.php?group_id=26589&package_id=173832, as you can see here:
puppy# wget http://optusnet.dl.sourceforge.net/sourceforge/nagios/ndoutils-12272005.tar.gz

manoj# tar -zxf ndoutils-1.4b9.tar.gz
manoj# cd ndoutils-1.4b9

By default NDO utilities use Nagios user, but my requirements was diffrent, i want to compile NDO utilities with manoj user. So i have used the following options during the compile time
manoj#./configure –with-ndo2db-user=manoj –with-ndo2db-group=manoj –enable-mysql –disable-pgsql –with-mysql-lib=/usr/lib/mysql
manoj# make

You must grant the user you create, in this case nagios, the SELECT, INSERT, UPDATE, and DELETE privileges to the nagios database. Replace ‘password’ with an appropriate password for the database.

manoj#create user ‘nagios@%’ identified by ‘password’;
OR
manoj#create user nagios@’localhost’ identified by ‘password’;
manoj#grant insert,delete,select,update on *.* to nagios@’%’ ;
OR
manoj#grant all on *.*  to nagios@’%’ ;
manoj#grant all on *.* to nagios@’localhost’ ;
manoj#create database nagios_db
Change Mysql Root password
manoj#update user set password=PASSWORD(“new password”) where User=’root’;
manoj#flush privileges;

The NDO add-on contains a script to populate this newly created database with the required tables. For MySQL, it is called ndo-mysql.sql, and it is located in the db directory in the root of the package:
manoj# cd ndoutils-1.4b9
manoj(ndoutils-1.4b9/db)#./installdb  -u user -p password -h hostname -d database

To install the NDO module itself, install the compiled ndomod.omodule file located in the src directory. I recommend copying it into the Nagios bin directory, usually /usr/local/nagios/bin:

manoj# cp src/ndomod-3x.o /usr/local/nagios/bin/ndomod.o

You also need to copy the sample configuration file for the module, ndomod.cfg. It is located in the config directory in the NDO utilities package. I recommend installing it to the Nagios etc directory, usually /usr/local/nagios/etc:

manoj# cp config/ndomod.cfg-sample /usr/local/nagios/etc/ndomod.cfg

You also need to install the ndo2db daemon and its configuration file. They are also located in the src and config directories, respectively, and I suggest you copy them to the same locations in your Nagios installation:

manoj# cp src/ndo2db-3x /usr/local/nagios/bin/ndo2db
manoj# cp config/ndo2db.cfg-sample /usr/local/nagios/etc/ndo2db.cfg

Next, you need to modify your Nagios configuration file, nagios.cfg, to load the NDO module when Nagios starts. Add the following line to your nagios.cfg configuration file, usually located in /usr/local/nagios/etc:
broker_module=/usr/local/nagios/bin/ndomod.o config_file=/usr/local/nagios/etc/ndomod.cfg
(It should be on single line entry)

This configuration directive will load the ndomod.o NEB module when Nagios is started. You will need to restart Nagios to make the module active. The config_file part of the directive must be modified to specify the location of the module configuration file. You should ensure that the ownership and permissions of all these files is appropriate. They should generally all be owned by the user and group used by the Nagios server process and the configuration files only readable by that user:

manoj# chown prod:prod /usr/local/nagios/bin/ndo2db /usr/local/nagios/bin/ndomod.o
/usr/local/nagios/etc/ndo2db.cfg /usr/local/nagios/etc/ndomod.cfg
manoj# chmod 0600 /usr/local/nagios/etc/ndo2db.cfg /usr/local/nagios/etc/ndo2db.cfg

You may also want to modify the two configuration files, ndo2db.cfg and ndomod.cfg. By default, the ndomod.o NEB module outputs data to a Unix domain socket, /usr/local/nagios/var/ndo.sock, which is created by the ndo2db daemon when it is started. You will also need to modify the ndo2db.cfg configuration file to update it with the correct database name, username, and password to allow the ndo2db daemon to write to the database.

Now you can start ndo2db by using the following command
manoj# /usr/local/nagios/bin/ndo2db -c /usr/local/nagios/etc/ndo2db.cfg

The daemon is launched with one command-line option, the location of the ndo2db daemon’s configuration file ndo2db.cfg. The daemon will create the Unix domain socket, /usr/local/nagios/var/ndo.sock. As you can see, I used the su command to change to the user nagios before launching. You should run the ndo2db daemon as the nagios user to allow the Unix domain socket to be created with the correct ownership. This will allow the ndomod.o module, which is run with the ownership and permissions of the Nagios server process, to write to that domain socket.

The module logs events and errors in the default Nagios log file, usually /usr/local/nagios/var/nagios.log OR /usr/local/nagios/var/ndo2db.debug. Check this file for errors and messages.

Thanks
Manoj Chauhan

To delete a string (192.168.1.1) from the file

January 20th, 2010

sed -i “/string/d” Filename
/bin/sed -i “/192.168.1.1/d” /var/log/messages
%s/,mianl//g

Trim Space at the end
########################################
sed ‘s/ *$//g’ ${LOGFILE} > ${LOGFILE}.1
mv ${LOGFILE}.1 ${LOGFILE}

BASH Scripting ~ learning by examples

January 20th, 2010

—————————————–
Program (1) ~ array.sh
—————————————–

#!/bin/bash
echo “==============”
declare -a myarr[0]=”Manoj”
declare -a myarr1
myarr1=(Manoj Chauhan bangalore mumbai pune delhi)
myarr[1]=”Chauhan”

echo “my name is ${myarr[0]} ${myarr[1]}”
echo “————————–”
echo “${myarr1[*]}”
echo ${myarr1[2]}
echo ${myarr1[@]}
echo “————————–”
echo “Total no of elements in array – ${#myarr1[*]}”
echo “Total no of elements in array – ${#myarr1[@]}”
echo “Size of word ‘${myarr1[2]}’ is – ${#myarr1[2]}”
echo ${#myarr1[1]}
echo ${#myarr1[0]}

echo “————————–”

#how to delete element in array
unset myarr[1]
echo “myarr is – ${myarr[*]}”

#how to assign element in array
myarr[1]=”- System Engineer!”
echo “myarr is – ${myarr[*]}”

echo ${myarr}

————————————————————-
Program (2) ~ command_line_arguments.sh
————————————————————-

#!/bin/bash

echo “Script/command name => $0″
echo “arg1 => $1″
echo “arg2 => $2″
echo “arg3 => $3″
echo “Total No of argument = $#”

echo “Script PID is => $$”
echo “Status of previous command – $?”

name=$myname
echo “Name – $name”

—————————————–
Program (3) ~ default_value.sh
—————————————–

#!/bin/bash

#start=’123′
#start=${1:-$start}

start=${1:-’123′}

echo “Value of ’start’ variable is ==> $start”

—————————————–
Program (4) ~ echo_example.sh
—————————————–

#!/bin/bash

name=”Manoj”

echo -e “My Name is $name_Manoj and \n”
echo -e “My Name is ${name}_Manoj and \n”

echo -e ‘My Name is $name and \n’

—————————————–
Program (5) ~ elif.sh
—————————————–

#! /bin/bash

if [ $1 -eq $2 ];then
echo “good”
elif [ $2 -eq $3 ];then
echo “Fine”
elif [ $1 -eq $3 ];then
echo “OK”
else
echo “NO”
fi

—————————————————
Program (6) ~ for_loop_example-1.sh
—————————————————

#!/bin/bash

i=1
while [ $i -le 512 ]
do
temp=$i
echo “What is => $i | $temp”
i=$(expr $i + 32)
for (( j=$temp; $j<=$i; j++ ))
do
echo -n ” $j”
done
done

—————————————————
Program (7) ~ for_loop_example-2.sh
————————————————–

#!/bin/bash
#for val in $(ls -1 /tmp)
sum=0
#for val in {1..5}
#for val in {$1..$2}
for((val=$1;$val<=$2;val++))
do
#echo “$val”
sum=$(expr $sum + $val )
#sum=`expr $sum + $val`
done

echo “$0 # Sum of $1 to $2 => $sum”

—————————————————–
Program (8) ~ for_loop_example-3.sh
—————————————————–

#!/bin/bash

for i in {1..9}
do
echo $i
done

—————————————–
Program (9) ~ function.sh
—————————————–

#!/bin/bash

function my_function()
{
name=”Manoj Chauhan”
echo “‘my_function’ body ~ $name”
return 1;
}
##########

myfunc()
{
echo “Another way of defining the function”
}

##########################

echo “Starting function program”
echo “——————————”

#calling function here
my_function
##
myfunc

echo -e “\n end of program”

—————————————————————————-
Program (10) ~ how_to_pass_argument_to_function.sh
—————————————————————————-

#!/bin/bash

function my_function()
{
echo “Total number of argument ~ $#”
echo “Arg1 => $1″
echo “Arg2 => $2″
echo “Arg3 => $3″
return 0;
}
##########

echo “Starting function program”
echo “——————————”

#calling function here
my_function Manoj Chauhan 1234

—————————————————————
Program (11) ~ how_to_take_hidden_input.sh
—————————————————————

#!/bin/bash

echo -n “Enter User Name :”
read username
echo -n “Enter Password :”
read -s mypwd

echo -e “\nI am $username and my password is – $mypwd”

——————————————————————–
Program (12) ~ how_to_take_input_from_user.sh
——————————————————————–

#!/bin/bash

echo -ne “Enter the Name:- ”
read name
echo -n -e “Enter the Number:- ”
read num

echo “——————————”

add=$(expr $num + 10)

echo “Name is ~ $name”
echo “Number is ~ $add”

—————————————–
Program (13) ~ ifthen.sh
—————————————–

#!/bin/bash

if [ "Manoj" == "Manoj" ];then
echo “true!”
else
echo “false”
fi

echo “———————————-”

if [ 2 == 2 ];then
echo “true!”
else
echo “false”
fi

echo “———————————-”

if [ "Manoj" = "Manoj" ];then
echo “true!”
else
echo “false”
fi

echo “———————————-”

if [ 2 -eq 2 ];then
echo “true!”
else
echo “false”
fi

———————————————-
Program (14) ~ non-interactive.sh
———————————————

#!/usr/bin/expect -f
spawn ssh Manoj@192.168.0.1
expect “password:”
sleep 1
send “pwd\r”
interact

——————————————————–
Program (15) ~ read_file_line_by_line.sh
——————————————————-

#!/bin/bash

file_name=”/etc/hosts”

while read myvar
do
echo “Line => $myvar”
done < $file_name

echo “##########################”

for myvar1 in $(cat $file_name)
do
echo “Line => $myvar1″
done

———————————————–
Program (16) ~ reverse-number.sh
———————————————–

#!/bin/bash

declare -a date_array
num=$1
i=$(expr $(echo $num | wc -c) – 1 )

while [ $num -gt 10 ]
do
temp=$( expr $num % 10 )
num=$( expr $num / 10);
echo “Digit($i) => $temp”
date_array[$i]=”${temp}”
i=$(expr $i – 1)
done
echo “Digit($i) => $num”
date_array[$i]=”${num}”

echo ${date_array[*]}

———————————————–
Program (17) ~ string-operation.sh
———————————————–

#! /bin/bash

echo “Manoj Chauhan:-”
string=”/root/Manoj/Chauhan/image.gif”
echo “string=> $string”
echo “String=> ${string##/*/}”
echo “String=> ${string#/*/}”

echo “String=> ${string%.*}”
echo “String=> ${string%%.*}”
#str=”/home/y/conf/ManojChauhan/daily_market_0.conf”
str=”${str##/*conf/}”
echo “String=> ${str%/*}”

#done

mystr=”keyword_summary_exact_Manojsb”
echo $mystr
echo ${mystr%_*}

echo “$*”

—————————————–
Program (18) ~ switch.sh
—————————————–

#!/bin/bash

echo ” Switch program | arg1 => $1″
echo ” ——————————-”
case $1 in

123)
echo “Case is 123″
;;

Manoj)
echo “Case is ‘Manoj’”
;;

pri*)

echo “Case is ‘pri*’”
;;

*)
echo ” * Usage: $0
echo “ Default case (nothing is matched)”
exit 0;
;;
esac

——————————————————-
Program (19) ~ while_loop_example-1.sh
——————————————————-

#!/bin/bash

mywait=wait

while [ "${mywait}" = "wait" ]
do
echo “Manoj”
done

——————————————————–
Program (20) ~ while_loop_example-2.sh
——————————————————–

#! /bin/bash

## on command line -> i=0 && while [ $i -le 10 ] ; do echo $i; i=$(expr $i + 1); done

i=0
while [ $i -le 10 ]
do
echo $i
i=$(expr $i + 1)
done

Nagios Script for Varnish Server

October 31st, 2009

#!/bin/bash
# Varnish Process Monitor
# Written By: Manoj Chauhan
# Email ID: mchauhan75@gmail.com
# Dated: 31-07-2009
# Restart Varnish Server When It Goes Down
LOG=/usr/bin/logger
RESTART=”/usr/bin/sudo /etc/init.d/varnishncsa restart”
STATUS=”/etc/init.d/varnishncsa status”
# Path to pgrep command
PGREP=”/usr/bin/pgrep”
VARNISHPID=”varnishncsa”
Hostname=`hostname`
STATE_OK=0
STATE_WARNING=1
STATE_CRITICAL=2
STATE_UNKNOWN=3
STATE_DEPENDENT=4
FLAG=0
/etc/init.d/varnishncsa status >/dev/null 2>&1
if [ $? -ne "0" ];then
{
#Nagios restart varnish service
$LOG “Critical:- $Hostname – Varnish service is down, restarted by nagios”
$RESTART >/dev/null 2>&1
}
fi
# Find varnish pid
PID=`$PGREP ${VARNISHPID} | wc -l`
if [ $PID -eq 0 ];then
{
#Nagios restart varnish service
$LOG “Critical:- $Hostname – Varnish PID is down, restarted by nagios”
$RESTART >/dev/null 2>&1
}
fi
/etc/init.d/varnishncsa status >/dev/null 2>&1
if [ $? -ne "0" ];then
{
$LOG “Critical:- $Hostname – Varnish service is down, please check…”
echo “CRITICAL – $Hostname -  Varnish service is down or not restarting, please check”
exit $STATE_CRITICAL;
}
else
{
FLAG=1
echo “OK – $Hostname -  Varnish service is running”
}
fi
if [ $FLAG -eq 0 ];then
{
# Find varnish pid
PID=`$PGREP ${VARNISHPID} | wc -l`
if [ $PID -eq 0 ];then
{
$LOG “Critical:- Varnish service is down, please check…”
echo “CRITICAL – $Hostname -  Varnish service is down or not restarting, please check”
exit $STATE_CRITICAL;
}
else
{
echo “OK – $Hostname -  Varnish service is running”
}
fi
}
fi