Even though the environments and systems you deploy may start to get more complicated such as with load balancers, there will always be a baseline level with which these systems can be brought to before further configuration and customization needs to be done.

There are many things that can be automated with this process, as you will see in the script below. In most round-robin load balancing scenarios, there wouldn’t be much more that needs to be done as far as configuration beyond what this script can do.

Obviously you will likely need to modify the script to suit your needs and requirements for the organization and standards therein.

Hopefully this will help you roll out many debian load balancers! May the load be split evenly between all your systems


#!/bin/sh
# Debian LVS deployer script
# Version 1.0

# script is done now - termination should happen with clean_up_and_reboot.
echo "[!] Should not be here!"
exit 1


Share/Save

Growth in any organization is obviously a good thing. In the systems administrator’s perspective, however, growth can mean more time spent deploying systems and less time spent focusing on other duties.

Automating the server deployment process is the natural next step when your organization has grown to a point where time efficiency becomes more relevant and noticeable to your business owners.

This is the first in a series of posts here where we will explain and share shell scripts that automate the deployment process of several key debian linux based systems. These scripts automate the patching, configuration and implementation of said systems.

They will certainly have to be modified to fit your organization’s needs and standards obviously, but hopefully it will give you a starting point to base your automation / roll-out policies.

Making your life easier and more automated is always a good thing!


#!/bin/sh
# Debian FW deployer script
# Version 1.0

# script is done now - termination should happen with clean_up_and_reboot.
echo "[!] Should not be here!"
exit 1


Share/Save

Sometimes it is, however. For instance, if you are brought into an environment that has not been properly managed and require some quick audits to evaluate how much actual work needs to be done bringing all the patch levels up to standard, then there are ways to produce these reports with simple bash scripting.

I have developed such a script for similar situations — quick reporting is sometimes necessary even when you are evaluating a large commercial patch management solution. It can even be implemented to coincide such solutions, for independent reporting perhaps.

This script would work well either by distributing it to each server and running the script via ssh key based authentication for centralized reporting. Alternatively, you could modify this script to perform each command via SSH over the network to gather information that way. It is probably more ideal to centrally distribute the script to each server so only one ssh command is executed per server.

Find the script below — note that it only works with RedHat / CentOS systems. Obviously if you are paying for Red Hat enterprise support you already are using satellite; If you are using CentOS then this script may be useful for you.

Enjoy!

#!/bin/sh

# Basic Information Gathering
# Star Dot Hosting
# http://www.stardothosting.com

HOSTNAME=`hostname`
UNAME=`uname -a | awk '{print $3}'`

# Begin Package Scanning

# SSH

SSHON="0"
SSHRUN="NULL"
SSHRPM="NULL"
SSHMATCH="NULL"

if [ -f /usr/sbin/sshd ]
then
        SSHON="1"
	SSHMATCH="0"
        SSHRUN=`ssh -V 2>&1 | awk 'BEGIN { FS = "_" } ; { print $2 }' | awk '{print $1}' | cut -b 0-5`
	TESTRPM=`rpm -qa openssh`
	if [ "$TESTRPM" <> 0  ]
	then
	        SSHRPM=`rpm -qa openssh | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$SSHRUN" == "$SSHRPM" ]
        then
                SSHMATCH="1"
        fi

fi

# Apache

HTTPDON="0"
HTTPDRUN="NULL"
HTTPDRPM="NULL"
HTTPDMATCH="NULL"

if [ -f /usr/sbin/httpd ]
then
        HTTPDON="1"
	HTTPDMATCH="0"
        HTTPDRUN=`httpd -v | grep version | awk 'BEGIN {FS="/"};{print$2}'`
	TESTRPM=`rpm -qa httpd`
	if [ "$TESTRPM" <> 0  ]
	then
        	HTTPDRPM=`rpm -qa httpd | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$HTTPDRUN" == "$HTTPDRPM" ]
        then
                HTTPDMATCH="1"
        fi
fi

# MySQL

MYSQLON="0"
MYSQLRUN="NULL"
MYSQLRPM="NULL"
MYSQLMATCH="NULL"

if [ -f /usr/bin/mysql ]
then
        MYSQLON="1"
	MYSQLMATCH="0"
        MYSQLRUN=`mysql -V | awk '{print $5}' | cut -b 0-6`
	TESTRPM=`rpm -qa mysql`
	if [ "$TESTRPM" <> 0  ]
	then
        	MYSQLRPM=`rpm -qa mysql | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$MYSQLRUN" == "$MYSQLRPM" ]
        then
                MYSQLMATCH="1"
        fi
fi

# PHP

PHPON="0"
PHPRUN="NULL"
PHPRPM="NULL"
PHPMATCH="NULL"

if [ -f /usr/bin/php ]
then
        PHPON="1"
	PHPMATCH="0"
        PHPRUN=`php -v | grep built | awk '{print $2 }'`
	TESTRPM=`rpm -qa php`
	if [ "$TESTRPM" <> 0  ]
	then
        	PHPRPM=`rpm -qa php | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$PHPRUN" == "$PHPRPM" ]
        then
                PHPMATCH="1"
        fi
fi

# Exim
# Needs to be tested on RH box

EXIMON="0"
EXIMRUN="NULL"
EXIMRPM="NULL"
EXIMMATCH="NULL"

if [ -f /usr/sbin/exim ]
then
        EXIMON="1"
	EXIMMATCH="0"
        EXIMRUN=`exim -bV | grep version | awk '{print $3}'`
	TESTRPM=`rpm -qa exim`
	if [ "$TESTRPM" <> 0  ]
	then
        	EXIMRPM=`rpm -qa exim | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$EXIMRUN" == "$EXIMRPM" ]
        then
                EXIMMATCH="1"
        fi
fi

# OpenSSL

OSSLON="0"
OSSLRUN="NULL"
OSSLRPM="NULL"
OSSLMATCH="NULL"

if [ -f /usr/bin/openssl ]
then
        OSSLON="1"
	OSSLMATCH="0"
        OSSLRUN=`openssl version | awk '{print $2}'`
	TESTRPM=`rpm -qa openssl`
	if [ "$TESTRPM" <> 0  ]
	then
        	OSSLRPM=`rpm -qa openssl | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$OSSLRUN" == "$OSSLRPM" ]
        then
                OSSLMATCH="1"
        fi
fi

# PERL

PERLON="0"
PERLRUN="NULL"
PERLRPM="NULL"
PERLMATCH="NULL"

if [ -f /usr/bin/perl ]
then
        PERLON="1"
	PERLMATCH="0"
        PERLRUN=`perl -v | grep built | awk '{print $4}' | awk 'BEGIN { FS = "v" } ; { print $2 }'`
	TESTRPM=`rpm -qa perl`
	if [ "$TESTRPM" <> 0  ]
	then
        	PERLRPM=`rpm -qa perl | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$PERLRUN" == "$PERLRPM" ]
        then
                PERLMATCH="1"
        fi
fi

# PYTHON

PYON="0"
PYRUN="NULL"
PYRPM="NULL"
PYMATCH="NULL"

if [ -f /usr/bin/python ]
then
        PYON="1"
	PYMATCH="0"
        PYRUN=`python -V 2>&1 | awk '{print $2}'`
	TESTRPM=`rpm -qa python`
	if [ "$TESTRPM" <> 0  ]
	then
        	PYRPM=`rpm -qa python | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$PYRUN" == "$PYRPM" ]
        then
                PYMATCH="1"
        fi
fi

# GPG

GPGON="0"
GPGRUN="NULL"
GPGRPM="NULL"
GPGMATCH="NULL"

if [ -f /usr/bin/gpg ]
then
        GPGON="1"
	GPGMATCH="0"
        GPGRUN=`gpg --version | grep gpg | awk '{print $3}'`
	TESTRPM=`rpm -qa gnupg`
	if [ "$TESTRPM" <> 0  ]
	then
        	GPGRPM=`rpm -qa gnupg | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$GPGRUN" == "$GPGRPM" ]
        then
                GPGMATCH="1"
        fi
fi

# RPM

RPMON="0"
RPMRUN="NULL"
RPMRPM="NULL"
RPMMATCH="NULL"

if [ -f /bin/rpm ]
then
        RPMON="1"
	RPMMATCH="0"
        RPMRUN=`rpm --version | awk '{print $3}'`
	TESTRPM=`rpm -qa rpm`
	if [ "$TESTRPM" <> 0  ]
	then
        	RPMRPM=`rpm -qa rpm | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$RPMRUN" == "$RPMRPM" ]
        then
                RPMMATCH="1"
        fi
fi

# SENDMAIL

SENDON="0"
SENDRUN="NULL"
SENDRPM="NULL"
SENDMATCH="NULL"

if [ -f /usr/sbin/sendmail ]
then
        SENDON="1"
        SENDMATCH="0"
        SENDRUN=`echo 'quit' | nc localhost 25 | grep Sendmail | awk '{print $5}' | awk 'BEGIN { FS = "/" } ; { print $1 }'`
	TESTRPM=`rpm -qa sendmail`
	if [ "$TESTRPM" <> 0  ]
	then
	        SENDRPM=`rpm -qa sendmail | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$SENDRUN" == "$SENDRPM" ]
        then
                SENDMATCH="1"
        fi
fi

### Non running packages

# bind-libs

BINDLIB="NULL"
TESTRPM=`rpm -qa bind-libs`
if [ "$TESTRPM" <> 0  ]
then
	BINDLIB=`rpm -qa bind-libs | awk 'BEGIN { FS = "-" } ; { print $3 }'`
fi

# bind-utils

BINDUTIL="NULL"
TESTRPM=`rpm -qa bind-utils`
if [ "$TESTRPM" <> 0  ]
then
	BINDUTIL=`rpm -qa bind-utils | awk 'BEGIN { FS = "-" } ; { print $3 }'`
fi

# coreutils

COREUTIL="NULL"
TESTRPM=`rpm -qa coreutils`
if [ "$TESTRPM" <> 0  ]
then
	COREUTIL=`rpm -qa coreutils | awk 'BEGIN { FS = "-" } ; { print $2 }'`
fi

# chkconfig

CHKCONFIG="NULL"
TESTRPM=`rpm -qa chkconfig`
if [ "$TESTRPM" <> 0  ]
then
	CHKCONFIG=`rpm -qa chkconfig | awk 'BEGIN { FS = "-" } ; { print $2 }'`
fi

# initscripts

INITSCR="NULL"
TESTRPM=`rpm -qa initscripts`
if [ "$TESTRPM" <> 0  ]
then
	INITSCR=`rpm -qa initscripts | awk 'BEGIN { FS = "-" } ; { print $2 }'`
fi

# redhat-release

RHRELEASE="NULL"
TESTRPM=`rpm -qa redhat-release`
if [ "$TESTRPM" <> 0  ]
then
	RHRELEASE=`rpm -qa redhat-release | awk 'BEGIN { FS = "-" } ; { print $3"-"$4 }'`
fi

echo $HOSTNAME,$UNAME,$SSHMATCH,$HTTPDMATCH,$MYSQLMATCH,$PHPMATCH,$EXIMMATCH,$OSSLMATCH,$PYMATCH,$PERLMATCH,$GPGMATCH,
$RPMMATCH,$SENDMATCH,$BINDLIB,$BINDUTIL,$COREUTIL,$CHKCONFIG,$INITSCR,$RHRELEASE,$SSHON,$SSHRUN,$SSHRPM,$HTTPDON,$HTTPDRUN,
$HTTPDRPM,$MYSQLON,$MYSQLRUN,$MYSQLRPM,$PHPON,$PHPRUN,$PHPRPM,$EXIMON,$EXIMRUN,$EXIMRPM,$OSSLON,$OSSLRUN,$OSSLRPM,$PERLON,
$PERLRUN,$PERLRPM,$PYON,$PYRUN,$PYRPM,$GPGON,$GPGRUN,$GPGRPM,$RPMON,$RPMRUN,$RPMRPM,$SENDON,$SENDRUN,$SENDRPM

Note that you can modify the echo output to produce whatever output you need in order to present it in a nice human readable report.

Share/Save

The underlying problem is that the logical volume is usually a live file system that cannot be directly mounted / backed up or imaged safely.

We have written a script that processes all running Xen logical volumes, creates a snapshot of the volume and through that snapshot , uses dd to image the snapshot to another server over ssh.

You would be surprised at how well these dd images compress. Piping dd to bzip2 then to ssh to receive the image produces a very substantial compression ratio.

The initial trouble was writing the logic in the script to properly go through each Xen LV , create the snapshot, image and then remove the snapshot. Obviously extensive testing had to be completed to ensure reliability and proper error reporting.

This script should work with any 3rd party Xen control panel implementation (HyperVM, FluidVM, SolusVM to name a few). They all use the same underlying technology / framework. Since our script is a simple bash / shell script, it will run on any linux based system with little modification.

If you are using a LV for another purpose on the same box, it is probably a good idea to modify the script to ignore that so it doesn’t inadvertently get backed up.

Before implementing the script, it is probably a good idea to go through the motions manually just to see how it performs :

lvcreate -s -L 5G -n vm101_img_snapshot /dev/vps/vm101_img
dd if=/dev/vps/vm101_img_snapshot | bzip2 | ssh xenbackup@x.x.x.x "dd of=vm101_img.bz2"

One thing that you cant get around is space — you need to leave as much room as the largest Xen image on your logical volume — otherwise the script will fail at the snapshot creation process.

Find the script below. Hopefully it will help make your life easier (as well as being able to sleep at night) :

#!/bin/bash
# XEN Backup script
# Written by Star Dot Hosting

todaysdate=`date "+%Y-%m-%d"`

echo "XEN Backup Log: " $currentmonth > /var/log/backup.log
echo -e "------------------------------------" >> /var/log/backup.log
echo -e "" >> /var/log/backup.log

for obj0 in $(lvs --noheadings --separator ',' -o lv_name,lv_size | grep -v "swap" | awk -F "," '{printf "%s\n", $1}');
do

#grab the snapshot size
snapsize=`lvs --noheadings --separator ',' -o lv_name,lv_size | grep -v "swap" | grep $obj0 | awk -F "," '{printf "%s", $2}'`

#create the snapshot
lvcreate -s -L $snapsize -n $obj0_snapshot /dev/xenlvm/$obj0 >> /var/log/backup.log 2>&1

#dd piped to bzip2 to compress the stream before piping it over the network via ssh to the destination box
dd if=/dev/xenlvm/$obj0_snapshot | bzip2 | ssh xenbackup@0.0.0.0 "dd of=/home/xenbackup/xen-backups/$obj0.$todaysdate.bz" >> /var/log/backup.log 2>&1

if [ "$?" -eq 1 ]
then
        echo -e "***SCRIPT FAILED, THERE WERE ERRORS***" >> /var/log/backup.log 2>&1
        cat /var/log/backup.log | mail -s "XEN Backup Job failed" admin@yourdomain.com
        lvremove -f /dev/xenlvm/$obj0_snapshot
        exit 1
else
        echo -e "Backup of $obj0 Completed Successfully!" >> /var/log/backup.log 2>&1
fi

# remove the snapshot
lvremove -f /dev/xenlvm/$obj0_snapshot

done

cat /var/log/backup.log | mail -s "XEN Backup Job Completed" admin@yourdomain.com

If you plan on automating this script in a cronjob, it may be a good idea to utilize ssh key authentication between your destination server and your Xen server.

Backing up to Amazon S3 has become a popular alternative to achieving true offsite backup capabilities for many organizations.

The fast data transfer speeds as well as the low cost of storage per gigabyte make it an attractive offer.

There are several free software solutions that offer the ability to connect to S3 and transfer files. The one that shows the most promise is s3sync.

There are already a few guides that show you how to implement s3sync on your system.

The good thing is that this can be implemented in Windows, Linux, FreeBSD among other operating systems.

We have written a simple script that utilizes the s3sync program in a scheduled offsite backup scenario. Find our script below, and modify it as you wish. Hopefully it will help you get your data safely offsite

#!/bin/sh
# OffSite Backup script

currentmonth=`date "+%Y-%m-%d %H:%M:%S"`

export AWS_ACCESS_KEY_ID="YOUR-ACCESS-KEY"
export AWS_SECRET_ACCESS_KEY="YOUR-SECRET-ACCESS-KEY"

echo "Offsite Backup Log: " $currentmonth > /var/log/offsite-backup.log
echo -e "----------------------------------------" >> /var/log/offsite-backup.log
echo -e "" >> /var/log/offsite-backup.log

# Archive Files and remove files older than 3 days
/usr/bin/find /home/offsite-backup-files -type f -mtime +3 -delete

# Compress and archive a few select key folders for archival and transfer to S3
tar -czvf /home/offsite-backup-files/offsite-backup-`date "+%Y-%m-%d"`.tar.gz /folder1 /folder2 /folder3 >> /var/log/offsite-backup.log 2>&1

# Transfer the files to Amazon S3 Storage via HTTPS
/usr/local/bin/ruby /usr/local/bin/s3sync/s3sync.rb --ssl -v --delete -r /home/offsite-backup-files your-node:your-sub-node/your-sub-sub-node >> /var/log/offsite-b
ackup.log 2>&1

# Some simple error checking and email alert logging
if [ "$?" -eq 1 ]
then
        echo -e "***OFFSITE BACKUP JOB, THERE WERE ERRORS***" >> /var/log/offsite-backup.log 2>&1
        cat /var/log/offsite-backup.log | mail -s "Offsite Backup Job failed" you@yourdomain.com
        exit 1
else
        echo -e "Script Completed Successfully!" >> /var/log/offsite-backup.log 2>&1
        cat /var/log/offsite-backup.log | mail -s "Offsite Backup Job Completed" your@yourdomain.com
        exit 0
fi

Now if your data happens to be sensitive (most usually is), usually encrypting the data during transit (with the –ssl flag) is not enough.

You can encrypt the actual file before it is sent to S3, as an alternative. This would be incorporated into the tar command with the above script. That line would look something like this :

/usr/bin/tar -czvf - /folder1 /folder2 /folder3 | /usr/local/bin/gpg --encrypt -r you@yourdomain.com > /home/offsite-backup-files/offsite-backups-`date "+%Y-%m-%d"`.tpg

Alternative to gpg, you could utilize openssl to encrypt the data.

Hopefully this has been helpful!

Share/Save

This is a good alternative to rsync. Even though rsync can compress the transfer mid stream, the receiving end is still the un-extracted copy.


ssh -l username 0.0.0.0 '(cd /home/mysql-backups/ && tar -czf - . -C /home/mysql-backups)' >> test.tar.gz 2>&1


To do the above command, and extract it on your end (after transferring the compressed file over the network), simply do the following :


ssh -l username 0.0.0.0 '(cd /home/mysql-backups/ && tar -czf - . -C /home/mysql-backups)' | tar -xzf -


These commands could theoretically incorporate pgp encryption to encrypt and compress the archive before it travels across the network, for increased security. That is why this alternative to rsync may be preferential to some.

Obviously you could locally encrpyt + compress , then rsync, but its always a good idea to not utilize local storage for this process and keep all the storage capacity on the centralized storage system that you have already allocated.

You may remember an earlier post that detailed how to implement SSH Key based authentication.

We believe it is important, when administering many (sometimes hundreds or thousands) of servers, to implement a strategy that can allow systems administrators to seamlessly run scripts, system checks or critical maintenance across all the servers.

SSH Key authentication allows for this potential. It is a very powerful strategy and should be maintained and implemented with security and efficiency as a top priority.

Distributing keys for all authorized systems administrators is something that would allow for the maintenance of this authentication system much easier — when an admin leaves or is dismissed, you need to be able to remove his or her’s keys from the “pool” quickly.

The idea behind this script is to have a centralized, highly secure and restricted key repository server. Each server in your environment would run this script to “pull” the updated key list from the central server. The script would run as a cron job and can run as often as you like. Ideally every 5-10 minutes would allow for quick key updates / distribution.

Here is the perl script :

#!/usr/bin/perl
#
# A script to sync ssh keys on UNIX servers automatically.  This
# will not overwrite user installed ssh keys
#

use strict;
use IPC::Open3;
use File::Copy;

use POSIX ":sys_wait_h";

# This is overkill but FreeBSD may install wget in
# /usr/local/bin in some cases.
$ENV{PATH} = "/bin:/usr/bin:/usr/local/bin:/sbin:/usr/sbin:/usr/local/sbin";

####################################################

use constant URL => 'https://keys.td.topscms.com/ssh-keys.txt';
use constant WGET => 'wget --no-check-certificate -q -O - ';
use constant KEYS_FILE => '/root/.ssh/authorized_keys';
use constant RESTRICTED => 'https://keys.td.topscms.com/restricted.txt';

####################################################

my ($url, $wget, $keys_file, $restricted, %restrict);

for (my $i=0;$i) {
                chomp;
                $restrict{$_}++;
        }
}

$pid = open3(\*WTR, \*RTR, \*ERR, "$wget");

while () {
        next if $restrict{$1};
        $company_keys .= $_;
}

$user_keys = read_key_file();

# Sanity check
my @rows = split('\n', $company_keys);

if (scalar @rows < 1) {
        print "Less than 1 company keys found, not installing keys..\n";
        exit(1);
}

open(TMP, ">$keys_file.$$.tmp") or die "Could not open tmp keys file: $!\n";
print TMP $company_keys;
print TMP $user_keys;
close(TMP);

# Sanity check

my (undef,undef,undef,undef,undef,undef,undef,$size,undef,undef,undef,undef,undef) = stat("$keys_file.$$.tmp");

if ($size < 100) {
        print "Keys file less than 100bytes, not writing";
        exit(1);
}

move("$keys_file.$$.tmp", $keys_file);

sub read_key_file {
        my $user_buf;

        open(KEY_FILE, "< $keys_file") or die "Could not open ssh key file; $!\n";

        while () {
                next if $_ =~ /company$/;
                $user_buf .= $_;
        }

        close(KEY_FILE);
        return($user_buf);
}

sub sig_chld {
        my $pid = waitpid(-1, WNOHANG);
}

sub usage {
        print STDERR <<"EOS";

        Usage: $0 -[kuh]

                -k        Keys file to write to (default: @{[KEYS_FILE]})
                -u         URL to download keys from (default: @{[URL]})
                -h              This screen

EOS
        exit(1);
}

1;

__END__

Note that it downloads the public keys via http with wget. This can be easily modified to utilize https, if necessary, or perhaps even another protocol to make the transfer. HTTP Was chosen because the public keys are harmless and http is the easiest method. HTTPS would be desirable, however.

We hope this script helps you along the way towards making your life easier!

- Nagios
- IP Monitor
- Uptime
- OpenNMS
- Zabbix

Although Nagios may not be the most scalable or dynamic solution, for some organisations that perhaps have 1-100 servers, Nagios may be the best solution.

Additionally, the ability to write custom plugins, as well as the inherent SSL / TLS encryption of the NRPE checks, it may be the most viable. There are pro’s and con’s for each solution out there, and it is completely dependant on the skill level, nature of environment and available time for management / maintenance.

During the course of utilising Nagios, we noticed that one of the most time consuming tasks was maintaining the flat file configuration for adding, removing and modifying hosts within Nagios.

As a result, it was decided to write a quick Perl based script to manage the day-to-day tasks of adding and removing hosts within Nagios. When all is said and done, it really does save ALOT of time. This script can be integrated with existing control based management situations or other automation scripts / solutions where command line options and external scripting / plugins are possible. This way, you can encompass a more rounded, standardised and reliable way of managing your systems in Nagios.

In order for the script to work, you need to have 3 types of servers :

- Windows
- Unix/Linux
- VPS (Virtual Private Server)

Obviously you can modify the script to encompass an unlimited number of categories. Basically the script has defined three pre-existing hosts in the nagios hosts.cfg / hostgroups.cfg and services.cfg files to model them when adding the new server, based on your input.

Please take a look at the script, hopefully it will help make your life a little easier!

#!/usr/bin/perl

# Don't break me, I'm used by automated scripts.

###############################################################################
# Star Dot Hosting : www.stardothosting.com
# Nagios Config Manager
# Description: This program will add/remove entries from nagios.
# The files will be backed up in a archive before any changes are made.
###############################################################################
# Perl Libraries
use File::Copy;
use Switch;
###############################################################################
# Variables

###############################################################################
# Nagios file handlers
my $host_file = "/usr/local/nagios/etc/objects/hosts.cfg";
my $group_file = "/usr/local/nagios/etc/objects/hostgroups.cfg";
my $services_file = "/usr/local/nagios/etc/objects/services.cfg";
my $unixmatch = "sdh-unix" ;
my $windowsmatch  = "sdh-windows";
my $vpsmatch = "vps-server";
my $date = `date "+%d%m%y-%H%M%S"`;

###############################################################################
# Verify Arguments
if ((!$ARGV[0]) || (!$ARGV[1])) {
        &usage;
}

if (length($ARGV[1]) gt 1 ) {   print "Command options too long!\n"; &usage; }

# Verify Nagios is working before we start
my $nagios = `nagios -v /usr/local/nagios/etc/nagios.cfg`;
        if ($nagios =~ /One or more problems was encountered while processing the config files/) {
        print "CRITICAL ERROR!\n\nNagios is already broken and we cannot continue!\nPlease fix it!\n";
                @error_array = split(/\./, $nagios);
                for $error (@error_array) {
                $error=~s/^\n//g;
                        print "$error\n" if  $error=~ /Error:/;
             }
        die "\n\nProgram Aborting before even starting due to nagios config error!\n"

}

# Clean up any old tmp files.
unlink("/tmp/hosts.cfg.tmp");
unlink("/tmp/hostgroups.cfg.tmp");
unlink("/tmp/services.cfg.tmp");

###############################################################################
# The Main Program control statement.
###############################################################################
switch ($ARGV[1]) {

        case /d/i {     &delete;        }
        case /x/i {     &addEntry("x"); }
        case /w/i {     &addEntry("w"); }
        case /v/i {     &addEntry("v"); }
        else {
                print "Option: $ARGV[1] not found \n";
                &usage;
        }
}
###############################################################################

###############################################################################
# Subroutines
###############################################################################

###############################################################################
## sub backup - Backs up the nagios config files that are to be modified
###############################################################################
sub backup {

# Backup The Nagios files into an archive.
        $date =~s/\n//g;

        mkdir("/var/backup/nagios/$date", 0755 ) || die "Cannot create directory /var/backup/nagios/$date\n";

        copy($host_file, "/var/backup/nagios/$date/hosts.cfg.bck"); #|| die "Cannot copy $host_file to /var/backup/nagios/$date/hosts.cfg.bck\n";
        copy($group_file, "/var/backup/nagios/$date/hostgroups.cfg.bck"); #|| die "Cannot copy $host_file to /var/backup/nagios/$date/hostgroups.cfg.bck\n";
        copy($services_file, "/var/backup/nagios/$date/services.cfg.bck"); #|| die "Cannot copy $service_file to /var/backup/nagios/$date/services.cfg.bck\n";
}

###############################################################################
## sub openFile($filename) - returns the file to a buffer for parsing
###############################################################################
sub openFile {
        my $blob;
        my $file = shift;
        open (F, "< $file") or die "Can't open $file : $!";

        while(  ) {
                $blob .= $_;
        }

        close(F);
return $blob;

}

###############################################################################
###############################################################################

###############################################################################
## sub delete - Deletes the servername from the config files.
###############################################################################
sub delete {
        &backup;                # Backup the files before we do anything to them.
        &delete_host;
        &delete_hostgroup;
        &delete_services;
        &checkNagios;
}

###############################################################################
## sub delete_host - deletes the host entry from hosts.cfg
###############################################################################
sub delete_host {
        my $host_str = &openFile($host_file);
        my $pattern=$ARGV[0];   # The parser doesn't like the array so we just pass it to a variable.

        # parse the hosts.cfg file first
        # This regular expression is a defined host entry, if it can't find it
        # and assert that the hostname is part of that context, it will die.
        if ($host_str =~/define[^_]*.name.*(?s-i:$pattern)[^}]*./i) {
                print "command: $ARGV[1] : Deleting $ARGV[0] $1\n" if $host_str =~s/define[^_]*.name.*(?s-i:$pattern)[^}]*.//g;
                print "Match: $ARGV[0]\n" if $host_str =~/define[^_]*.name.*(?s-i:$pattern)[^}]*./i;
                print "Deleted $ARGV[0] from hosts.cfg\n";

        # Write the successfull deleteion to a tmp file.
        open(HF, ">/tmp/hosts.cfg.tmp") || die "Cannot open /tmp/hosts.cfg.tmp";
        print HF $host_str;
    close(HF);

        } else { die "Could not find and entry for $ARGV[0] in hosts.cfg\n"; };
}

###############################################################################
## sub delete_hostgroup - deletes the hostgroup entry
###############################################################################
sub delete_hostgroup {

        my $hostgrp_str = &openFile($group_file);
        my $pattern=$ARGV[0];   # The parser doesn't like the array so we just pass it to a variable.

        # search/replace the hostgroup.cfg file
        if ($hostgrp_str =~ /$pattern/i) {
                # If the server has a comma after it, we need to remove that too.. or breakage.
                if ($hostgrp_str =~ /$pattern,/i ) {
                        print "Deleted $ARGV[0], from hostgroups.cfg\n" if $hostgrp_str =~ s/$pattern,//g;
                }

                        print "Deleted $ARGV[0] from hostgroups.cfg\n" if $hostgrp_str =~ s/$pattern//g;

        } else {
                die "Could not find and entry for $ARGV[0] in hostgroups.cfg\n";
        }
        open(HGF, ">/tmp/hostgroups.cfg.tmp") || die "Cannot open /tmp/hostgroups.cfg.tmp";
        print HGF $hostgrp_str;
        close(HGF);
}

###############################################################################
## sub delete_services - delete the serivices.cfg entry
###############################################################################
sub delete_services {

        my $services_str= &openFile($services_file);
        my $pattern=$ARGV[0];   # The parser doesn't like the array so we just pass it to a variable.

        # search/replace the hostgroup.cfg file
        if ($services_str =~ /$pattern/i) {
                # If the server has a comma after it, we need to remove that too.. or breakage.
                if ($services_str =~ /$pattern,/i ) {
                        print "Deleted $ARGV[0], from services.cfg\n" if $services_str =~ s/$pattern,//g;
                }
                        print "Deleted $ARGV[0] from services.cfg\n" if $services_str =~ s/$pattern//g;
         } else {
                die "Could not find and entry for $ARGV[0] in services.cfg\n";
        }
        open(SF, ">/tmp/services.cfg.tmp") || die "Cannot open /tmp/services.cfg.tmp";
        print SF $services_str;
        close(SF);
}

###############################################################################
## sub checkNagios - checks nagios for errors and rolesback if so.
###############################################################################
sub checkNagios {

        copy("/tmp/hosts.cfg.tmp", $host_file) || print "Cannot copy /tmp/hosts.cfg.tmp to  $host_file\n";
        copy("/tmp/hostgroups.cfg.tmp", $group_file) || print "Cannot copy /tmp/hostgroups.cfg.tmp to $host_file\n";
        copy("/tmp/services.cfg.tmp", $services_file) || print "Cannot copy /tmp/services.cfg.tmp $service_file\n";
        my $success = `nagios -v /etc/nagios/nagios.cfg`;

        if ($success =~ /One or more problems was encountered while processing the config files/) {
                print "CRITICAL FAILURE - See Errors!\n";
                @error_array = split(/\./, $success);
                for $error (@error_array) {
                $error=~s/^\n//g;
                        print "$error\n" if  $error=~ /Error:/;
                        }
                print "\nRestoring from backup\nCheck /tmp/hosts.cfg /tmp/hostgroup.cfg /tmp/service.cfg\n";
        copy("/var/backup/nagios/$date/hosts.cfg.bck", $host_file) || die "Cannot copy /var/backup/nagios/$date/hosts.cfg.bck to $host_file\n";
        copy("/var/backup/nagios/$date/hostgroups.cfg.bck", $group_file) || die "Cannot copy /var/backup/nagios/$date/hostgroups.cfg.bck to $group_file\n";
        copy("/var/backup/nagios/$date/services.cfg.bck", $services_file) || die "Cannot copy /var/backup/nagios/$date/services.cfg.bck $services_file\n";

        } else {
                print "Nagios config reports success, restarting nagios\n";
                my $restart = `/etc/init.d/nagios reload`;
                print $restart;
        }

}

###############################################################################
## sub addEntry - adds the unix or windows host entry.
###############################################################################
sub addEntry {

        my $type = shift;
        my $pattern = $ARGV[0];
        my $host_str = &openFile($host_file);
        my $hostgrp_str = &openFile($group_file);
        my $services_str= &openFile($services_file);

        if ($host_str=~/$pattern/) { die "$ARGV[0] already in hosts.cfg, aborting!\n"; }
        if ($hostgrp_str=~/$pattern/) { die "$ARGV[0] already in hostgroups.cfg, aborting!\n"; }
if(($type eq 'w') || ($type eq 'x')) {
        if ($services_str=~/$pattern/) { die "$ARGV[0] already in services.cfg, aborting!\n"; }
}
        # Some sanity checks to help prevent data entry errors
        if (!$ARGV[2]) { print "\nNo Server Alias, aborting!\n\n"; &usage; }
        #if ($ARGV[2]=~/[0-9]{5,8}$/i) {} else { print "No Member ID!\n"; exit 0}
        if (!$ARGV[3]) { print "\nNo IP Address specified, aborting!\n\n"; &usage; }
        if ($ARGV[3]=~/[a-z]/i) { print "\nIP Address $ARGV[3] is invalid, please double check\n\n"; exit 0; }
        if ($ARGV[3]=~/[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}/) {
                @ip = split(/\./, $ARGV[3]);
        } else {
                print "\nIP Address $ARGV[3] is invalid, please double check\n\n"; exit 0;
        }

        if (($ip[0] > 254) || ($ip[1] > 254) ||
            ($ip[3] > 254) || ($ip[4] > 254)) {

        print "\nIP Address $ARGV[3] is invalid, please double check\n\n"; exit 0;
        }

# Passes sanity checks, back up the mo fo.
&backup;
# Check if windows or unix

switch ($type) {

        case "x" {      print "Unix!\n";
                        $hostgrp_str =~ s/$unixmatch/$unixmatch,$pattern/g;
                        $services_str =~ s/$unixmatch/$unixmatch,$pattern/g;  }
        case "w" {      print "Windows\n";
                        $hostgrp_str =~ s/$windowsmatch/$windowsmatch,$pattern/g;
                        $services_str =~ s/$windowsmatch/$windowsmatch,$pattern/g; }

        else {          print "VPS\n";
                        $hostgrp_str =~ s/$vpsmatch/$vpsmatch,$pattern/g;
                                }
        } # end switch

        # Add it to the host_str buffer.
        $host_str .= "define host{
        use                     sdh-dedicated
        host_name               $ARGV[0]
        alias                   $ARGV[2]
        address                 $ARGV[3]
        }\n\n";
        open(HF, ">/tmp/hosts.cfg.tmp") || die "Cannot open /tmp/hosts.cfg.tmp";
        print HF $host_str;
        close(HF);
        open(HGF, ">/tmp/hostgroups.cfg.tmp") || die "Cannot open /tmp/hostgroups.cfg.tmp";
        print HGF $hostgrp_str;
        close(HGF);

if(($type eq 'w') || ($type eq 'x')) {
        open(SF, ">/tmp/services.cfg.tmp") || die "Cannot open /tmp/services.cfg.tmp";
        print SF $services_str;
        close(SF);
        }
        &checkNagios;

}
###############################################################################
## sub usage - prints the usage when things don't add up from args
###############################################################################
sub usage{
        print "Usage: /usr/local/bin/nagios-add.pl    \n\n";
        print "Optional Flags:\n";
        print "\td delete a server\n";
        print "\tw add a windows server\n";
        print "\tx add a unix server\n";
        print "\tv add a VPS server\n\n";
        print "eg delete:\t./usr/local/bin/nagios-add.pl sdh-server12 d\n";
        print "eg add:\t\t./usr/local/bin/nagios-add.pl sdh-server12 x \"sdh-server12 sdh-server12.stardothosting.com MemID:155\" 192.168.111.10\n";
        exit 0;
}

Share/Save

In order to quickly audit a network , I created this script to scan selected IPs, read from a configuration file, and compile a simple report to be emailed. The script can be modified to suit your needs, such as exporting the data to a database or perhaps an HTML report for a web based reporting site.

The script itself doesn’t do anything particularly special, however it has proven useful when you want to do a quick & dirty network audit.

There are other tools out there, such as OpenAudit, Nessus and Nmap that could do similar tasks. However, the important thing to remember here is that those tools (with the exception of open audit perhaps) can be incorporated into this script to perform regular scheduled audits.

This script could actually be updated to utilize nmap v5.0 — utilizing the new features plus ndiff could turn this script into a very powerful network analysis tool.

Hopefully some of you will find some use out of the script! Enjoy!

#!/bin/sh

# Basic Information Gathering
currentmonth=`date "+%Y-%m-%d"`

rm lindows.log

echo "Hostname Identification Audit: " $currentmonth >> lindows.log
echo -e "------------------------------------------" >> lindows.log
echo -e >> lindows.log
for obj0 in $(grep -v "^#" all_linux_windows_ips.txt);
do

# Check if windows
check=`nmap -e bge0 -p 3389 $obj0 | grep open`

if [ "$?" -eq 0 ]
        then
        windowshost=`nbtscan -v -s , $obj0 | head -n 1 | awk -F"," '{printf "%s", $2}'`
        if [ -n "${windowshost:+x}" ]
                then
                echo -e "$windowshost\t: $obj0\t: WINDOWS" >> lindows.log
                else
                echo -e "NETBIOS UNKOWN\t: $obj0\t: WINDOWS" >> lindows.log
        fi
        else
        # Check if linux or freebsd
        ssh_get=`ssh -l ims $obj0 '(uname | sed 'y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/' && hostname | sed 'y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/')'`
        if [ "$?" -eq 0 ]
                then
                uname=`echo $ssh_get | awk -F" " '{printf "%s", $1}'`
                hostname1=`echo $ssh_get | awk -F" " '{printf "%s", $2}'`
                hostname2=`echo $hostname1 | awk -F"." '{printf "%s", $1}'`
                echo -e "$hostname2\t: $obj0\t: $uname" >> lindows.log
                else
                echo -e "UNKNOWN ERROR\t: $obj0\t: PLEASE CHECK HOST" >> lindows.log
        fi
fi
done

cat lindows.log | mail -s 'Windows/FreeBSD/Linux Host Audit' your@email.com

Note that the “all_windows_linux_ips.txt” is just a text file with the ip addresses of all hostnames on your network. It can be modified to simply utilize whole subnets to make it easier to perform the audit.

These log files could be anything: java logs, apache logs (apache should have its own log rotation built in) and mail logs for example. This script has two modes : daily and monthly.

The daily mode is intended to be run daily (obviously) , gzipping the previous days log file. The monthly mode, run monthly (obviously), then tar’s up all the previous month’s gzip files into one big tarball.

Note that this script assumes the log filenames are assorted by the filename + date (year/month/day). This can obviously be modified to suit the specific syntax of your log file names.

Here is the script :

#!/bin/sh
# Rotate / compress old logs
# Star Dot Hosting

yesterday=`date --date='1 day ago' +%Y-%m-%d`
lastmonth=`date --date='1 month ago' +%Y-%m`
lasttwomonth=`date --date='2 months ago' +%Y-%m`
currentmonth=`date "+%Y-%m-%d"`
logdir="/path/to/log/directory"
logfilename="log-file-name"

#gzip yesterdays log
if [ "$1" = "daily" ]
then
        gzip $logdir/$logfilename.$yesterday.log
        exit 0

#tar all last month's logs on the 1st of each month
elif [ "$1" = "monthly" ]
then
        tar -C $logdir -cf $logdir/$logfilename.$lastmonth.tar $logdir/$logfilename.$lastmonth-*.log.gz && rm -f $logdir/$logfilename.$lastmonth-*.log.gz
        exit 0
else
        echo "no or invalid arguments given."
        echo "syntax : ./logcompress.sh daily or ./logcompress.sh monthly"
        exit 1
fi

I simply make two crontab entries :


0 3 * * * /bin/sh /usr/local/bin/logcompress.sh daily
0 5 1 * * /bin/sh /usr/local/bin/logcompress.sh monthly


The above entries run the script daily at 3:00am, and monthly on the 1st of every month at 5:00am, this ensures the script isn’t run at the same time on the 1st as the daily job.

That’s it!