Archive for the ‘CentOS’ category

Nagios monitor for Nginx

February 4th, 2013

We have TCP monitors in  our production which checks for service up/down using check_http or check_tcp Nagios plugin but not giving full details like how many connections are handled by the Nginx etc…

We enable stub_status options in Nginx, it will provide information about the current state of the Nginx server.

Stub status*
Provide server status information functionality.
Key directive: stub_status
Configure switch: –with-http_stub_status_module enables the module

Stub status
The stub status module was designed to provide information about the current state of the server, such as the amount of active connections, the total handled requests, and more. To activate it, place the stub_status directive in a location block. All requests matching the location block will produce the status page:

Add Nginc conf file and restart Nginx

location = /nginx_status {
stub_status on;
allow 127.0.0.1; # you may want to protect the information
deny all;
}

After installing stub_status you can test using below curl command
[manoj@web1]$ curl -vv http://localhost:9900/nginx_status
* About to connect() to localhost port 9900
*   Trying 127.0.0.1… connected
* Connected to localhost (127.0.0.1) port 9900
> GET /nginx_status HTTP/1.1
> User-Agent: curl/7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5
> Host: localhost:9900
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx/1.0.4
< Date: Wed, 30 Jan 2013 11:34:14 GMT
< Content-Type: text/plain
< Content-Length: 97
< Connection: keep-alive
Active connections: 1
server accepts handled requests
4 4 4
Reading: 0 Writing: 1 Waiting: 0
* Connection #0 to host localhost left intact
* Closing connection #0

Nagios Monitor for Nginx server

We are using check_mk in our production,  It’s a new general purpose Nagios-plugin for retrieving data for more detail click here.

Client Side
/etc/check_mk/mrpe.cfg
Nginx_Connections /home/manoj/scripts/check_nginx.sh  -H localhost -P 9900 -p /home/manoj/nginx/logs -n nginx.9900.pid -s nginx_status  -w 80 -c 100

[manoj@web1 ~]$ /home/manoj/scripts/check_nginx.sh  -H localhost -P 9900 -p /home/manoj/nginx/logs -n nginx.9900.pid -s nginx_status  -w 80 -c 100
OK – nginx is running. 5 requests per second, 5 connections per second (1.00 requests per connection) | ‘reqpsec’=5 ‘conpsec’=5 ‘conpreq’=1.00 ]

Nagios will send an WARNING alerts if quests per second is > 80 and send an critical alert if quests per second is > 100.  Download script check_nginx.sh from link

Nagios Server
vim /etc/check_mk/main.mk
all_hosts = [
'Nginx-Server-Name'
]

service_contactgroups = [
( 'infra',ALL_HOSTS,[ "Nginx_Connections"]),

]
After adding to Nagios server, you need to check for new inventory (checking for new services added in mrpe.cfg on client server)
check_mk -I

Restart Nagios services using below command
check_mk -O

How To Empty Postfix Mail Queue

September 26th, 2011

This command will delete one specific email from the mailq
mailq | tail +2 | grep -v ‘^ *(‘ | awk  ‘BEGIN { RS = “” } { if ($8 == “email@manoj.com” && $9 == “”) print $1 } ‘ | tr -d ‘*!’ | postsuper -d -

Deleting the postfix mail queue

I would get a ton of messages sent to the postfix queue which would all be sent out when I reconnected to the internet. Deleting the postfix mail Queue is suprisingly easy:

/usr/sbin/postsuper -d ALL

This command will delete all messages in the Postfix queue. If you need more selective deleting, this can use ‘man postsuper’ to find out all of the available options.

The other thing that helped with this was checking for a local network connection before doing the server checks. You can check it using below command.

/sbin/ifconfig | grep -i mask|grep -v 127.0.0.1 | awk {‘print $2′}

Thanks
Manoj

Error: RPC MTAB does not exist

August 17th, 2011

If you stop the ‘rpcidmapd’ service and then do something like ‘umount -a’, then
it will also umount the rpc_pipefs.

Then when you try to restart ‘rpcidmapd’ service (or the ‘nfs’ service which in turn also starts the ‘rpcidmapd’ service), you get the error:
Starting RPC idmapd: Error: RPC MTAB does not exist.

Similarly, if you do ‘umount -a’  but don’t unload the ‘nfsd’ module, then the ‘nfsd’ filesystem will be unmounted and not remounted when you restart the nfs service. However, in this case, the init script doesn’t even warn you that the filesystem is not mounted.
Again, manually you have two options
1. Unload and reload the ‘nfsd’ module
2. Manually create the filesystem, after running below command i am not getting this error and i am able to mount the partition using NFS.
mount -t nfsd nfsd /proc/fs/nfsd

Linux Server Hardening Security Tips

May 9th, 2011

Securing your Linux server is important to protect your data, intellectual property, and time, from the hands of crackers (hackers). The system administrator is responsible for security Linux box. In this first part of a Linux server security series, I will provide 20 hardening tips for default installation of Linux system.

#1: Encrypt Data Communication


All data transmitted over a network is open to monitoring. Encrypt transmitted data whenever possible with password or using keys / certificates.
1. Use scp, ssh, rsync, or sftp for file transfer. You can also mount remote server file system or your own home directory using special sshfs and fuse tools.
2. GnuPG allows to encrypt and sign your data and communication, features a versatile key managment system as well as access modules for all kind of public key directories.
3. Fugu is a graphical frontend to the commandline Secure File Transfer application (SFTP). SFTP is similar to FTP, but unlike FTP, the entire session is encrypted, meaning no passwords are sent in cleartext form, and is thus much less vulnerable to third-party interception. Another option is FileZilla – a cross-platform client that supports FTP, FTP over SSL/TLS (FTPS), and SSH File Transfer Protocol (SFTP).
4. OpenVPN is a cost-effective, lightweight SSL VPN.
5. Lighttpd SSL (Secure Server Layer) Https Configuration And Installation
6. Apache SSL (Secure Server Layer) Https (mod_ssl) Configuration And Installation

#1.1: Avoid Using FTP, Telnet, And Rlogin / Rsh


Under most network configurations, user names, passwords, FTP / telnet / rsh commands and transferred files can be captured by anyone on the same network using a packet sniffer. The common solution to this problem is to use either OpenSSH , SFTP, or FTPS (FTP over SSL), which adds SSL or TLS encryption to FTP. Type the following command to delete NIS, rsh and other outdated service:
# yum erase inetd xinetd ypserv tftp-server telnet-server rsh-serve
#2: Minimize Software to Minimize Vulnerability
Do you really need all sort of web services installed? Avoid installing unnecessary software to avoid vulnerabilities in software. Use the RPM package manager such as yum or apt-get and/or dpkg to review all installed set of software packages on a system. Delete all unwanted packages.
# yum list installed
# yum list packageName
# yum remove packageName
OR
# dpkg –list
# dpkg –info packageName
# apt-get remove packageName

#3: One Network Service Per System or VM Instance
Run different network services on separate servers or VM instance. This limits the number of other services that can be compromised. For example, if an attacker able to successfully exploit a software such as Apache flow, he / she will get an access to entire server including other services such as MySQL, e-mail server and so on. See how to install Virtualization software:
• Install and Setup XEN Virtualization Software on CentOS Linux 5
• How To Setup OpenVZ under RHEL / CentOS Linux

#4: Keep Linux Kernel and Software Up to Date

Applying security patches is an important part of maintaining Linux server. Linux provides all necessary tools to keep your system updated, and also allows for easy upgrades between versions. All security update should be reviewed and applied as soon as possible. Again, use the RPM package manager such as yum and/or apt-get and/or dpkg to apply all security updates.
# yum update
OR
# apt-get update && apt-get upgrade
You can configure Red hat / CentOS / Fedora Linux to send yum package update notification via email. Another option is to apply all security updates via a cron job. Under Debian / Ubuntu Linux you can use apticron to send security notifications.

#5: Use Linux Security Extensions

Linux comes with various security patches which can be used to guard against misconfigured or compromised programs. If possible use SELinux and other Linux security extensions to enforce limitations on network and other programs. For example, SELinux provides a variety of security policies for Linux kernel.

#5.1: SELinux
I strongly recommend using SELinux which provides a flexible Mandatory Access Control (MAC). Under standard Linux Discretionary Access Control (DAC), an application or process running as a user (UID or SUID) has the user’s permissions to objects such as files, sockets, and other processes. Running a MAC kernel protects the system from malicious or flawed applications that can damage or destroy the system. See the official Redhat documentation which explains SELinux configuration.

#6: User Accounts and Strong Password Policy
Use the useradd / usermod commands to create and maintain user accounts. Make sure you have a good and strong password policy. For example, a good password includes at least 8 characters long and mixture of alphabets, number, special character, upper & lower alphabets etc. Most important pick a password you can remember. Use tools such as “John the ripper” to find out weak users passwords on your server. Configure pam_cracklib.so to enforce the password policy.

#6.1: Password Aging
The chage command changes the number of days between password changes and the date of the last password change. This information is used by the system to determine when a user must change his/her password. The /etc/login.defs file defines the site-specific configuration for the shadow password suite including password aging configuration. To disable password aging, enter:
chage -M 99999 userName
To get password expiration information, enter:
chage -l userName
Finally, you can also edit the /etc/shadow file in the following fields:
{userName}:{password}:{lastpasswdchanged}:{Minimum_days}:{Maximum_days}:{Warn}:{Inactive}:{Expire}:
Where,
1. Minimum_days: The minimum number of days required between password changes i.e. the number of days left before the user is allowed to change his/her password.
2. Maximum_days: The maximum number of days the password is valid (after that user is forced to change his/her password).
3. Warn : The number of days before password is to expire that user is warned that his/her password must be changed.
4. Expire : Days since Jan 1, 1970 that account is disabled i.e. an absolute date specifying when the login may no longer be used.
I recommend chage command instead of editing the /etc/shadow by hand:
# chage -M 60 -m 7 -W 7 userName
Recommend readings:
• Linux: Force Users To Change Their Passwords Upon First Login
• Linux turn On / Off password expiration / aging
• Lock the user password
• Search for all account without password and lock them
• Use Linux groups to enhance security

#6.2: Restricting Use of Previous Passwords
You can prevent all users from using or reuse same old passwords under Linux. The pam_unix module parameter remember can be used to configure the number of previous passwords that cannot be reused.

#6.3: Locking User Accounts After Login Failures
Under Linux you can use the faillog command to display faillog records or to set login failure limits. faillog formats the contents of the failure log from /var/log/faillog database / log file. It also can be used for maintains failure counters and limits.To see failed login attempts, enter:

faillog
To unlock an account after login failures, run:
faillog -r -u userName
Note you can use passwd command to lock and unlock accounts:
# lock account
passwd -l userName
# unlocak account
passwd -u userName
#6.4: How Do I Verify No Accounts Have Empty Passwords?
Type the following command
# awk -F: ‘($2 == “”) {print}’ /etc/shadow
Lock all empty password accounts:
# passwd -l accountName

#6.5: Make Sure No Non-Root Accounts Have UID Set To0
Only root account have UID 0 with full permissions to access the system. Type the following command to display all accounts with UID set to 0:
# awk -F: ‘($3 == “0″) {print}’ /etc/passwd
You should only see one line as follows:
root:x:0:0:root:/root:/bin/bash
If you see other lines, delete them or make sure other accounts are authorized by you to use UID 0.

#7: Disable root Login
Never ever login as root user. You should use sudo to execute root level commands as and when required. sudo does greatly enhances the security of the system without sharing root password with other users and admins. sudo provides simple auditing and tracking features too.

#8: Physical Server Security
You must protect Linux servers physical console access. Configure the BIOS and disable the booting from external devices such as DVDs / CDs / USB pen. Set BIOS and grub boot loader password to protect these settings. All production boxes must be locked in IDCs (Internet Data Center) and all persons must pass some sort of security checks before accessing your server. See also:
• 9 Tips To Protect Linux Servers Physical Console Access.

#9: Disable Unwanted Services
Disable all unnecessary services and daemons (services that runs in the background). You need to remove all unwanted services from the system start-up. Type the following command to list all services which are started at boot time in run level # 3:
# chkconfig –list | grep ’3:on’
To disable service, enter:
# service serviceName stop
# chkconfig serviceName off

#9.1: Find Listening Network Ports
Use the following command to list all open ports and associated programs:
netstat -tulpn
OR
nmap -sT -O localhost
nmap -sT -O server.example.com
Use iptables to close open ports or stop all unwanted network services using above service and chkconfig commands.

#9.2: See Also

• update-rc.d like command on Redhat Enterprise / CentOS Linux.
• Ubuntu / Debian Linux: Services Configuration Tool to Start / Stop System Services.
• Get Detailed Information About Particular IP address Connections Using netstat Command.

#10: Delete X Windows
X Windows on server is not required. There is no reason to run X Windows on your dedicated mail and Apache web server. You can disable and remove X Windows to improve server security and performance. Edit /etc/inittab and set run level to 3. Finally, remove X Windows system, enter:
# yum groupremove “X Window System”

#11: Configure Iptables and TCPWrappers
Iptables is a user space application program that allows you to configure the firewall (Netfilter) provided by the Linux kernel. Use firewall to filter out traffic and allow only necessary traffic. Also use the TCPWrappers a host-based networking ACL system to filter network access to Internet. You can prevent many denial of service attacks with the help of Iptables:
• Lighttpd Traffic Shaping: Throttle Connections Per Single IP (Rate Limit).
• How to: Linux Iptables block common attack.
• psad: Linux Detect And Block Port Scan Attacks In Real Time.

#12: Linux Kernel /etc/sysctl.conf Hardening
/etc/sysctl.conf file is used to configure kernel parameters at runtime. Linux reads and applies settings from /etc/sysctl.conf at boot time. Sample /etc/sysctl.conf:
# Turn on execshield
kernel.exec-shield=1
kernel.randomize_va_space=1
# Enable IP spoofing protection
net.ipv4.conf.all.rp_filter=1
# Disable IP source routing
net.ipv4.conf.all.accept_source_route=0
# Ignoring broadcasts request
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.icmp_ignore_bogus_error_messages=1
# Make sure spoofed packets get logged
net.ipv4.conf.all.log_martians = 1

#13: Separate Disk Partitions

Separation of the operating system files from user files may result into a better and secure system. Make sure the following filesystems are mounted on separate partitions:
• /usr
• /home
• /var and /var/tmp
• /tmp
Create septate partitions for Apache and FTP server roots. Edit /etc/fstab file and make sure you add the following configuration options:
1. noexec – Do not set execution of any binaries on this partition (prevents execution of binaries but allows scripts).
2. nodev – Do not allow character or special devices on this partition (prevents use of device files such as zero, sda etc).
3. nosuid – Do not set SUID/SGID access on this partition (prevent the setuid bit).
Sample /etc/fstab entry to to limit user access on /dev/sda5 (ftp server root directory):
/dev/sda5  /ftpdata          ext3    defaults,nosuid,nodev,noexec 1 2

#13.1: Disk Quotas

Make sure disk quota is enabled for all users. To implement disk quotas, use the following steps:
1. Enable quotas per file system by modifying the /etc/fstab file.
2. Remount the file system(s).
3. Create the quota database files and generate the disk usage table.
4. Assign quota policies.
5. See implementing disk quotas tutorial for further details.

#14: Turn Off IPv6

Internet Protocol version 6 (IPv6) provides a new Internet layer of the TCP/IP protocol suite that replaces Internet Protocol version 4 (IPv4) and provides many benefits. Currently there are no good tools out which are able to check a system over network for IPv6 security issues. Most Linux distro began enabling IPv6 protocol by default. Crackers can send bad traffic via IPv6 as most admins are not monitoring it. Unless network configuration requires it, disable IPv6 or configure Linux IPv6 firewall:
• RedHat / Centos Disable IPv6 Networking.
• Debian / Ubuntu And Other Linux Distros Disable IPv6 Networking.
• Linux IPv6 Howto – Chapter 19. Security.
• Linux IPv6 Firewall configuration and scripts are available here.

#15: Disable Unwanted SUID and SGID Binaries

All SUID/SGID bits enabled file can be misused when the SUID/SGID executable has a security problem or bug. All local or remote user can use such file. It is a good idea to find all such files. Use the find command as follows:
#See all set user id files:
find / -perm +4000
# See all group id files
find / -perm +2000
# Or combine both in a single command
find / \( -perm -4000 -o -perm -2000 \) -print
find / -path -prune -o -type f -perm +6000 -ls

You need to investigate each reported file. See reported file man page for further details.

#15.1: World-Writable Files

Anyone can modify world-writable file resulting into a security issue. Use the following command to find all world writable and sticky bits set files:
find /dir -xdev -type d \( -perm -0002 -a ! -perm -1000 \) -print
You need to investigate each reported file and either set correct user and group permission or remove it.

#15.2: Noowner Files
Files not owned by any user or group can pose a security problem. Just find them with the following command which do not belong to a valid user and a valid group
find /dir -xdev \( -nouser -o -nogroup \) -print
You need to investigate each reported file and either assign it to an appropriate user and group or remove it.

#16: Use A Centralized Authentication Service
Without a centralized authentication system, user auth data becomes inconsistent, which may lead into out-of-date credentials and forgotten accounts which should have been deleted in first place. A centralized authentication service allows you maintaining central control over Linux / UNIX account and authentication data. You can keep auth data synchronized between servers. Do not use the NIS service for centralized authentication. Use OpenLDAP for clients and servers.

#16.1: Kerberos

Kerberos performs authentication as a trusted third party authentication service by using cryptographic shared secret under the assumption that packets traveling along the insecure network can be read, modified, and inserted. Kerberos builds on symmetric-key cryptography and requires a key distribution center. You can make remote login, remote copy, secure inter-system file copying and other high-risk tasks safer and more controllable using Kerberos. So, when users authenticate to network services using Kerberos, unauthorized users attempting to gather passwords by monitoring network traffic are effectively thwarted. See how to setup and use Kerberos.

#17: Logging and Auditing
You need to configure logging and auditing to collect all hacking and cracking attempts. By default syslog stores data in /var/log/ directory. This is also useful to find out software misconfiguration which may open your system to various attacks. See the following logging related articles:
1. Linux log file locations.
2. How to send logs to a remote loghost.
3. How do I rotate log files?.
4. man pages syslogd, syslog.conf and logrotate.  for more Administering of Kerberos Database click here

#17.1: Monitor Suspicious Log Messages With Logwatch / Logcheck
Read your logs using logwatch or logcheck. These tools make your log reading life easier. You get detailed reporting on unusual items in syslog via email. A sample syslog report:
################### Logwatch 7.3 (03/24/06) ####################
Processing Initiated: Fri Oct 30 04:02:03 2009
Date Range Processed: yesterday
( 2009-Oct-29 )
Period is day.
Detail Level of Output: 0
Type of Output: unformatted
Logfiles for Host: www-52.nixcraft.net.in
##################################################################

——————— Named Begin ————————

**Unmatched Entries**
general: info: zone XXXXXX.com/IN: Transfer started.: 3 Time(s)
general: info: zone XXXXXX.com/IN: refresh: retry limit for master ttttttttttttttttttt#53 exceeded (source ::#0): 3 Time(s)
general: info: zone XXXXXX.com/IN: Transfer started.: 4 Time(s)
general: info: zone XXXXXX.com/IN: refresh: retry limit for master ttttttttttttttttttt#53 exceeded (source ::#0): 4 Time(s)

———————- Named End ————————-

——————— iptables firewall Begin ————————

Logged 87 packets on interface eth0
From 58.y.xxx.ww – 1 packet to tcp(8080)
From 59.www.zzz.yyy – 1 packet to tcp(22)
From 60.32.nnn.yyy – 2 packets to tcp(45633)
From 222.xxx.ttt.zz – 5 packets to tcp(8000,8080,8800)

———————- iptables firewall End ————————-

——————— SSHD Begin ————————

Users logging in through sshd:
root:
123.xxx.ttt.zzz: 6 times

———————- SSHD End ————————-

——————— Disk Space Begin ————————

Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3             450G  185G  241G  44% /
/dev/sda1              99M   35M   60M  37% /boot

———————- Disk Space End ————————-

###################### Logwatch End #########################
(Note output is truncated)

#17.2: System Accounting with auditd
The auditd is provided for system auditing. It is responsible for writing audit records to the disk. During startup, the rules in /etc/audit.rules are read by this daemon. You can open /etc/audit.rules file and make changes such as setup audit file log location and other option. With auditd you can answers the following questions:
1. System startup and shutdown events (reboot / halt).
2. Date and time of the event.
3. User respoisble for the event (such as trying to access /path/to/topsecret.dat file).
4. Type of event (edit, access, delete, write, update file & commands).
5. Success or failure of the event.
6. Records events that Modify date and time.
7. Find out who made changes to modify the system’s network settings.
8. Record events that modify user/group information.
9. See who made changes to a file etc.
See our quick tutorial which explains enabling and using the auditd service.

#18: Secure OpenSSH Server

The SSH protocol is recommended for remote login and remote file transfer. However, ssh is open to many attacks. See how to secure OpenSSH server:
• Top 20 OpenSSH Server Best Security Practices.

#19: Install And Use Intrusion Detection System
A network intrusion detection system (NIDS) is an intrusion detection system that tries to detect malicious activity such as denial of service attacks, port scans or even attempts to crack into computers by monitoring network traffic.
It is a good practice to deploy any integrity checking software before system goes online in a production environment. If possible install AIDE software before the system is connected to any network. AIDE is a host-based intrusion detection system (HIDS) it can monitor and analyses the internals of a computing system.
Snort is a software for intrusion detection which is capable of performing packet logging and real-time traffic analysis on IP networks.

#20: Protecting Files, Directories and Email
Linux offers excellent protections against unauthorized data access. File permissions and MAC prevent unauthorized access from accessing data. However, permissions set by the Linux are irrelevant if an attacker has physical access to a computer and can simply move the computer’s hard drive to another system to copy and analyze the sensitive data. You can easily protect files, and partitons under Linux using the following tools:
• To encrypt and decrypt files with a password, use gpg command.
• Linux or UNIX password protect files with openssl and other tools.
• See how to encrypting directories with ecryptfs.
• TrueCrypt is free open-source disk encryption software for Windows 7/Vista/XP, Mac OS X and Linux.
• Howto: Disk and partition encryption in Linux for mobile devices.
• How to setup encrypted Swap on Linux.

How round a number in bash script

April 26th, 2011

In a shell script – How do I round a decimal number (contained in a variable) to the nearest whole number?

var=2.5
echo $var|awk ‘{print int($1+0.5)}’

Output is 2

round a number

In a shell script – How do I round a decimal number (contained in a variable) to the nearest whole number?

Self-signed certificate in Glassfish Web Server

April 14th, 2011

Use keytool to generate, import, and export certificates. By default, keytool creates a keystore file in the directory where it is run. You can find the keytool utility under the bin directory of java folder.

Note: – When you install Glassfish, it creates a default self-signed certificate as the server certificate. (localhost)

Delete exiting certificate :-

Type the following command to delete the default self-signed certificate by issuing the following command.

keytool -delete -alias s1as -keystore keystore.jks -storepass <store_passwd>

Generate self signed certificate

Steps 1:- Type the following command to create new certificate:
keytool -genkey -alias test

Fill all the information to create the certificate.

Enter keystore password: p@ssw0rd!
What is your first and last name?
[Unknown]: Chandra
what is the name of your organizational unit?
[Unknown]: CompanyName
what is the name of your organization?
[Unknown]: CompanyName
what is the name of your City or Locality?
[Unknown]: Gurgaon
What is the name of your State or Province?
[Unknown]: HR
What is the two-letter country code for this unit?
[Unknown]: IN
Is <CN=Chandra, OU=, O=CompanyName, L=Gurgaon, ST=HR, C=IN> correct?
[no]: yes

Import certificate

A certificate can be imported into a keystore using keytool. Type the following command to import the certificate:-
keytool -storepass my-keystore-password() -alias test -import -file test.cer

Generate expired certificate

Steps:

Default days is 7 and cant not be set 0 day. You need to specify at least 1 day to create.

keytool -genkey -alias test –validity 1

Generate certificate without trusted root

Steps:

Follow up the first step

Generate certificate with invalid CN

Steps:

Follow up the first step and give the invalid CN whatever you want to keep while ask the keytool utility during the creating of certificate.

Note: – To change the location of certificate files – admin console.

Always generate the certificate in the directory containing the keystore and truststore files, by default domain-dir/config.

Open the Glassfish admin console in the web browser.

Login into glassfish admin console (http://localhost:4848), Default uid and password: admin and adminadmin

a)      In the Admin Console tree, select the Application Server node.

b)      Select JVM Settings.

c)      Click the JVM Options tab.

d)     On the JVM Options page, add or modify the following values in the Value field to reflect the new location of the certificate files:

-Djavax.net.ssl.keyStore=${com.sun.aas.instanceRoot}/path/ks-name-Djavax.net.ssl.trustStore=${com.sun.aas.instanceRoot}/path/ts-name

e)      Where ks-name is the keystore file name and ts-name is the trust store file name.

f)       Click Save.

g)      Restart the Application Server if Restart Required displays in the console.

Install certificate in GlassFish server

Here are the instructions for enabling GlassFish v2 as an SSL server when the application server is configured with the developer profile.

  1. Delete the default self-signed certificate by issuing the following command (note that the commands in this and subsequent steps are shown on multiple lines for formatting purposes):

keytool -delete -alias s1as -keystore keystore.jks -storepass <store_passwd>

where <store_passwd> is the password for the keystore, for example, “mypass”. Note that s1as is the default alias of the GlassFish v2 keystore.

  1. Generate a new key pair for the application server by issuing the following command:

keytool -genkeypair -keyalg <key_alg>
-keystore keystore.jks -validity <val_days> -alias s1as

where <key_alg> is the algorithm to be used for generating the key pair, for example RSA, and <val_days> is the number of days that the certificate should be considered valid, for example, 365.

Note that in addition to generating a key pair, the command wraps the public key into a self-signed certificate and stores the certificate and the private key in a new keystore entry identified by the alias.

It’s important to ensure that the name of the certificate matches the fully-qualified hostname of your site. If the names don’t match, clients connecting to the server will see a security alert stating that the name of the certificate does not match the name of the site. You should notice that the name of the default self-signed certificate matches the fully-qualified hostname.

  1. Generate a Certificate Signing Request (CSR) by issuing the following command:

keytool -certreq -alias s1as -file <certreq_file>
-keystore keystore.jks -storepass <store_passwd>

where <certreq_file> is the file in which the CSR is stored, for example, s1as.csr, and <store_passwd> is the password for the keystore, for example, changeit.

  1. keytool -import -v -alias s1as -file s1as.cert -keystore keystore.jks -storepass <store_passwd>

When you import the certificate using the same original alias “s1as”, keytool treats it as a command to replace the original certificate with the certificate obtained as reply to a CSR.

s1as (self-signed):

Owner: CN=chandra, OU=CompanyName, O=CompanyNamr, L=Gurgaon , ST=Haryana, C=IN
Issuer: CN=Chandra, OU=CompanyName, O=CompanyName, L=Gurgaon, ST=Haryana, C=IN
Serial number: 472acd34
Valid from:

Administering the Kerberos Database

April 3rd, 2011

Configuring a Kerberos 5 Server

To configure a basic Kerberos server, follow these steps:

1.    Be sure that you have clock synchronization and DNS working on your server before configuring Kerberos 5. Pay particular attention to time synchronization between the Kerberos server and its various clients. If the server and client clocks are different by more than five minutes (this default amount is configurable in Kerberos 5), Kerberos clients will not be able to authenticate to the server. This clock synchronization is necessary to prevent an attacker from using an old Kerberos ticket to masquerade as a valid user.

You should set up a Network Time Protocol (NTP) compatible client/server network even if you are not using Kerberos. Red Hat Linux includes the ntp package for easy installation. See /usr/share/doc/ntp-<version-number>/index.htm for details on setting up Network Time Protocol servers and http://www.ntp.org/ for additional information on NTP.

2.    Install the krb5-libs, krb5-server, and krb5-workstation packages on the dedicated machine which will run the KDC. This machine needs to be very secure — if possible, it should not run any services other than the KDC (Kerberos Server/Key distribution Center).

#yum install krb5-libs krb5-server krb5-workstation

If you would like to use a graphical user interface utility to administrate Kerberos, you should also install the gnome-kerberos package. It contains krb5, a GUI tool for managing tickets.

3.    Edit the /etc/krb5.conf and /var/kerberos/krb5kdc/kdc.conf configuration files to reflect your realm name and domain-to-realm mappings. A simple realm can be constructed by replacing instances of EXAMPLE.COM and example.com with your domain name — being certain to keep uppercase and lowercase names in the correct format — and by changing the KDC from kerberos.example.com to the name of your Kerberos server. By convention, all realm names are uppercase and all DNS hostnames and domain names are lowercase. For full details on the formats of these files, see their respective man pages

[logging]
default = FILE:/var/log/krb5libs.log
kdc = FILE:/var/log/krb5kdc.log
admin_server = FILE:/var/log/kadmind.log

[libdefaults]
default_realm = manoj.COM
dns_lookup_realm = false
dns_lookup_kdc = false
ticket_lifetime = 24h
forwardable = yes

[realms]
manoj.COM = {
kdc = manoj.com:88
admin_server = manoj.com:749
default_domain = manoj.com
}

[domain_realm]
.manoj.com = manoj.COM
manoj.com = manoj.COM

[appdefaults]
pam = {
debug = false
ticket_lifetime = 36000
renew_lifetime = 36000
forwardable = true
krb4_convert = false
}

4.    Create the database using the kdb5_util utility from a shell prompt:
/usr/kerberos/sbin/kdb5_util create -s

5.    The create command creates the database that will be used to store keys for your Kerberos realm. The -s switch forces creation of a stash file in which the master server key is stored. If no stash file is present from which to read the key, the Kerberos server (krb5kdc) will prompt the user for the master server password (which can be used to regenerate the key) every time it starts.

6.    Edit the /var/kerberos/krb5kdc/kadm5.acl file. This file is used by kadmind to determine which principals have administrative access to the Kerberos database and their level of access. Most organizations will be able to get by with a single line:

*/admin@EXAMPLE.COM  * replace EXAMPLE.COM with your domain name like manoj.COM

Most users will be represented in the database by a single principal (with a NULL, or empty, instance, such as joe@EXAMPLE.COM). With this configuration, users with a second principal with an instance of admin (for example, joe/admin@EXAMPLE.COM) will be able to wield full power over the realm’s Kerberos database. Once kadmind is started on the server, any user will be able to access its services by running kadmin on any of the clients or servers in the realm. However, only users listed in the kadm5.acl file will be able to modify the database in any way, except for changing their own passwords.

Note:

The kadmin utility communicates with the kadmind server over the network, and they use Kerberos to handle authentication. Of course, you need to create the first principal before you can connect to the server over the network to administer it. Create the first principal with the kadmin.local command, which is specifically designed to be used on the same host as the KDC and does not use Kerberos for authentication.

7.    Type the following kadmin.local command at the KDC terminal to create the first principal:

#/usr/kerberos/sbin/kadmin.local -q “addprinc username/admin”

8.    Start Kerberos using the following commands:
/sbin/service krb5kdc start
/sbin/service kadmin start
/sbin/service krb524 start

kadmind


The kadmind command starts the administrative server. This administrative server runs on Kerberos server that stores the Kerberos principal database and the policy database. The kadmind accepts password change request and remote requests to administer the information in these databases.


kadmind requires the following configuration files to be set for it to work:

kdc.conf

The KDC configuration file contains configuration information for the KDC and the KADM5 system.

keytab

kadmind requires a keytab containing the keys for the kadmin/admin and kadmin/changepw principals for every realm that kadmind will answer requests for. This admin keytab can be created with the kadmin.local. The location of the keytab is determined by the admin keytab configuration variable present in the kdc.conf file.

ACL file

kadmind’s access control list (ACL) restricts it as to which principals are allowed to perform administration actions. The path of the ACL file is specified via the acl_file configuration variable in the kdc.conf file.

kadmin and kadmin.local

These utilities provide a unified administration interface for the Kerberos database. Kerberos administrators use these utilities to create new users and services for the master database, and to modify information for the existing database entries.

Both the utilities provide for maintenance of Kerberos principals, policies, and service key tables (keytabs). These utilities exist as both a Kerberos client, ‘kadmin’and a local client, ‘kadmin.local‘.

The kadmin utility uses Kerberos authentication and an Remote Procedure Call (RPC) to operate securely from anywhere on the network.

The‘kadmin.local’ is intended to run directly on the KDC without any Kerberos authentication. Normal UNIX users cannot execute this command. Executing the kadmin.local command will display the kadmin.local prompt only if you are the root user.

Getting the kadmin to work

kadmin allows you to administer the Kerberos database remotely (and securely). If you just run kadmin, you may obtain an error message as shown below:

kadmin: Client not found in Kerberos database while initializing kadmin interface

To be able to use the kadmin interface, you need to register yourself as a database administrator.

Adding of users in the Kerberos database

Adding of users in the Kerberos database

/usr/kerberos/sbin/kadmin.local -q “addprinc manoj/admin”
On the KDC machine, in kadmin.local, you can add the administrator role:
kadmin.local: addprinc jar/admin
Enter password for principal “jar/admin@finance.bambi.com”: <your_password>
Re-enter password for principal “jar/admin@finance.bambi.com”: <your_password>
Principal “jar/admin@finance.bambi.com” created
kadmin.local: quit
Now you can access kadmin on the Kerberos server. For example,

/usr/kerberos/sbin/kadmin  -p jar/admin
Password for jar/admin@manoj.COM:

In the Client Side
/usr/kerberos/bin/kinit  -p manoj/admin
Password for manoj/admin@manoj.COM:

Disk i/o & Performance Tuning Benchmarking Tool

March 25th, 2011

Disk i/o & Performance Tuning Benchmarking Tool

Dstat is a flexible replacement for vmstat, iostat, netstat and ifstat. Dstat overcomes some of their limitations and adds some extra features. Dstat is handy for monitoring systems during performance tuning tests, disk i/o,  benchmarks or troubleshooting.

Dstat allows you to view all of your system resources in real-time, you can eg. compare disk utilization in combination with interrupts from your IDE controller, or compare the network bandwidth numbers directly with the disk throughput (in the same interval).

Dstat gives you detailed selective information in columns and clearly indicates in what degree and unit the output is displayed. And most importantly, it makes it very easy to write plugins to collect your own counters and extend in ways you never expected.

Features

* Combines vmstat, iostat, ifstat, netstat information and more
* Shows stats in exactly the same timeframe
* Enable/order counters as they make most sense during analysis/troubleshooting
* Modular design
* Written in python so easily extendable for the task at hand
* Easy to extend, add your own counters (please contribute those)
* Includes many external plugins to show how easy it is to add counters
* Can summarize grouped block/network devices and give total numbers
* Can show interrupts per device
* Very accurate timeframes, no timeshifts when system is stressed
* Shows exact units and limits conversion mistakes
* Indicate different units with different colors
* Show intermediate results when delay > 1
* Allows to export CSV output, which can be imported in Gnumeric and Excel to make graphs

Here are the existing plugins

[server@manoj ~]# dstat –list
internal:
aio, cpu, cpu24, disk, disk24, disk24old, epoch, fs, int, int24, io, ipc, load, lock,
mem, net, page, page24, proc, raw, socket, swap, swapold, sys, tcp, time, udp, unix,
vm
/usr/share/dstat:
battery, battery-remain, cpufreq, dbus, disk-recsize, disk-tps, disk-util, dstat,
dstat-cpu, dstat-ctxt, dstat-mem, fan, freespace, gpfs, gpfs-ops, helloworld,
innodb-buffer, innodb-io, innodb-ops, lustre, mem-adv, memcache-hits, mysql-io,
mysql-keys, mysql5-cmds, mysql5-conn, mysql5-io, mysql5-keys, net-packets, nfs3,
nfs3-ops, nfsd3, nfsd3-ops, ntp, pcap-ssh, postfix, power, proc-count, proc-count2,
proc-count3, qmail, rpc, rpcd, sendmail, snooze, squid, test, thermal, top-bio,
top-bio-adv, top-childwait, top-cpu, top-cpu-adv, top-cpu2, top-cpu3, top-cputime,
top-cputime-avg, top-int, top-io, top-io-adv, top-latency, top-latency-avg, top-mem,
top-oom, top-tcp-ports, utmp, vm-memctl, vmk-hba, vmk-int, vmk-nic, vz-cpu, vz-io,
vz-ubc, wifi

You can see the detailed dstat – Linux man page  http://dag.wieers.com/home-made/dstat/dstat.1.html and http://linux.die.net/man/1/dstat

[server@manoj ~]# dstat

—-total-cpu-usage—- -dsk/total- -net/total- —paging– —system–

usr sys idl wai hiq siq| read writ| recv send| in out | int csw

5 0 93 0 0 0| 154k 84k| 0 0 | 0 0 |1081 1116

13 0 87 0 0 0| 0 0 | 0 0 | 0 0 |1036 696

8 0 92 0 1 0| 0 8192B| 0 0 | 0 0 |1073 936

0 0 99 0 0 1| 0 0 | 0 0 | 0 0 |1072 940

1 1 97 0 2 0| 0 0 | 0 0 | 0 0 |1252 1727

1 1 98 0 1 0| 0 0 | 0 0 | 0 0 |1126 1191

1 0 99 0 0 0| 0 0 | 0 0 | 0 0 |1045 908

0 0 99 0 0 0| 0 44k| 0 0 | 0 0 |1051 904

1 1 99 0 0 0| 0 0 | 0 0 | 0 0 |1036 850

1 0 100 0 0 0| 0 0 | 0 0 | 0 0 |1029 757

Some examples of using LINUX/UNIX find command

March 13th, 2011

Introduction

The find command allows the Linux/Unix users to process a set of files and/or directories in a file subtree.

You can specify the following:

* where to search (pathname)
* what type of file to search for (-type: directories, data files, links)
* how to process the files (-exec: run a process against a selected file)
* the name of the file(s) (-name)
* perform logical operations on selections (-o and -a)

Search for file with a specific name in a set of files (-name)

find . -name “manoj.conf” -print

This command will search in the current directory and all sub directories for a file named manoj.conf.

Note: The -print option will print out the path of any file that is found with that name. In general -print wil print out the path of any file that meets the find criteria.

How to apply a unix command to a set of file (-exec).

find . -name “manoj.conf” -exec chmod o+r ‘{}’ \;

This command will search in the current directory and all sub directories. All files named manoj.conf will be processed by the chmod -o+r command. The argument ‘{}’ inserts each found file into the chmod command line. The \; argument indicates the exec command line has ended.

The end results of this command is all manoj.conf files have the other permissions set to read access (if the operator is the owner of the file).

How to apply a complex selection of files (-o and -a).

find /usr/src -not \( -name “*,v” -o -name “.*,v” \) ‘{}’ \; -print

This command will search in the /usr/src directory and all sub directories. All files that are of the form ‘*,v’ and ‘.*,v’ are excluded. Important arguments to note are:

* -not means the negation of the expression that follows
* \( means the start of a complex expression.
* \) means the end of a complex expression.
* -o means a logical or of a complex expression.
In this case the complex expression is all files like ‘*,v’ or ‘.*,v’

The above example is shows how to select all file that are not part of the RCS system. This is important when you want go through a source tree and modify all the source files.

How to search for a string in a selection of files (-exec grep …).

find . -exec grep “www.athabasca” ‘{}’ \; -print

This command will search in the current directory and all sub directories. All files that contain the string will have their path printed to standard output.

If you want to just find each file then pass it on for processing use the -q grep option. This finds the first occurrence of the search string. It then signals success to find and find continues searching for more files.

find . -exec grep -q “www.manoj” ‘{}’ \; -print

This command is very important for process a series of files that contain a specific string. You can then process each file appropriately. An example is find all html files with the string “www.manoj.com”. You can then process the files with a sed script to change those occurrances of “www.manoj.com” with “manoj.manoj.com”.

How to Install and Configure Cacti

March 4th, 2011

Cacti is a complete frontend to RRDTool, it stores all of the necessary information to create graphs and populate them with data in a MySQL database. Required softwares to install Cacti.

You need to install the following software to install cacti.

a)      MySQL Server : Store cacti data
b)      NET-SNMP server – SNMP (Simple Network Management Protocol) is a protocol used for network management.
c)      PHP with net-snmp module – Access SNMP data using PHP.
d)     Apache / lighttpd / ngnix webserver : Web server to display graphs created with PHP and RRDTOOL.

Install the software

You can install the softwares  using yum command. Run the following:-

# yum install mysql-server mysql php-mysql php-pear php-common php-gd php-devel php php-mbstring php-cli php-snmp php-pear-Net-SMTP php-mysql httpd

Configure MySQL server

Setting up root password:-
# mysqladmin -u root password NEWPASSWORD

Create cacti MySQL database
# mysql -u root -p -e ‘create database cacti’

Create a user name cacti with a password your password, then enter

Login to mysql
# mysql -u root –p

mysql> GRANT ALL ON cacti.* TO cacti@localhost IDENTIFIED BY ‘your password’;
mysql> FLUSH privileges;
mysql> \q

Intall snmpd

Type the following command to install net-snmpd
# yum install net-snmp-utils php-snmp net-snmp-libs

To configure snmpd, open the snmpd.conf configuration file.

# vi /etc/snmp/snmpd.conf
And modify the conafiguration file. As see below.
com2sec local     localhost           public
group MyRWGroup v1         local
group MyRWGroup v2c        local
group MyRWGroup usm        local
view all    included  .1                               80
access MyRWGroup “”      any       noauth    exact  all    all    none
syslocation Unknown (edit /etc/snmp/snmpd.conf)
syscontact Root  (configure /etc/snmp/snmp.local.conf)
pass .1.3.6.1.4.1.4413.4.1 /usr/bin/ucd5820stat

Save and closed the configuration file and start the snmp service. Type the following.

# /etc/init.d/snmpd start
# chkconfig snmpd on

Install cacti

Update the repository:

rpm -Uvh http://download.fedora.redhat.com/pub/epel/5/x86_64/epel-release-5-4.noarch.rpm

# yum install cacti

Install cacti tables

Type the following command to find out cacti.sql path:
# rpm -ql cacti | grep cacti.sql

Sample output:
/usr/share/doc/cacti-0.8.7d/cacti.sql

Type the following command to install cacti tables (you need to type the cacti user password):

# mysql -u cacti -p cacti < /usr/share/doc/cacti-0.8.7d/cacti.sql

Configure cacti database string:-
Open /var/www/cacti/include
# config.php

Modify the following changes as follows:
/* make sure these values refect your actual database/host/user/password */

$database_type = “mysql”;
$database_default = “cacti”;
$database_hostname = “localhost”;
$database_username = “cacti”;
$database_password = “your password”;
$database_port = “3306″;

Configure httpd for cacti.
Update allow from line, set to your LAN subnet to allow access to cacti:
Open /etc/httpd/conf.d/cacti.conf file

# vi /etc/httpd/conf.d/cacti.conf
Alias /cacti/ /var/www/cacti/

<Directory /var/www/cacti/>
DirectoryIndex index.php
Options -Indexes
AllowOverride all
order deny,allow
allow from 172.16.0.0/16 #your network address
AddType application/x-httpd-php .php
php_flag magic_quotes_gpc on
php_flag track_vars on
</Directory>

After changed start the httpd:-

# /etc/init.d/httpd  start

Setup cacti cronjob

Open /etc/cron.d/cacti file
# vi /etc/cron.d/cacti

Uncomment the line:
*/5 * * * *     cacti   /usr/bin/php /usr/share/cacti/poller.php > /dev/null 2>&1

Save and closed the file after uncomment.

Now cacti is ready, you can run the cacti type the following:-

http://Cacti_IP_Address/cacti/

or http://xx.xx.xx.xx/cacti

Note: The default username and password for cacti is admin / admin.

No package ‘libpcre’ found

March 3rd, 2011

I was getting “No package ‘libpcre’ found” error during Varnish compiling, so fixed it by installing gcc lib.  using below command

yum install gcc* -y

How Quickly find what directories are using up space on your disk in Linux

March 2nd, 2011

The df -h command will tell you disk usage from a mount-point perspective, but the command du tells you from a directory perspective. Use the command:

du -h / –max-depth=1

To show disk usage for each individual directory on the system, or go lower down to see usage on a particular directory:

du -h /var/log –max-depth=1

To exclude some folder during the disk space check, you can use below command, it will show disk usage for each individual directory on the current directory in which you are going to run the command except manoj folder

du -h –max-depth=1 –exclude=manoj  *

find and replace string in vi

March 2nd, 2011

To perform a find and replace with all entries of a file, enter the colon to invoke Esc. from vi. Then, from there, enter

:%s/find_string/replace_string/g

For example:

%s/manoj/manoj chauhan/g

There are many other ways to do this, but I prefer this method.

How comment lines in vi

March 2nd, 2011

If you need to comment the next 10 lines within vi, execute the command within the command mode:
:.,+10s/^/#

It will comment the next 10 lines, you can change the number accordingly.

How to Kill zombie processes

March 2nd, 2011

Kill those persistent and annoying zombie processes.
ps -e -o ppid,stat | grep Z | cut -d” ” -f2 | xargs kill -9