Hosting Coupon : 60% off the first month for ANY plan!

Posted by Kevin in Company Announcements, Hosting on July 15, 2010

Hello there,

Just thought I’d share an exclusive coupon / discount for all of our shared / vps hosting plans that allows for 60% off the first month of hosting fee’s :

COUPON CODE : SDHTWT2010

Take a look at our main site for plan details. This coupon expires and there is only a limited number of them available!

Don’t say we never gave you nothin’ ;)

Related Links:

  • No Related Links

1 Comment

Company site re-design

Posted by Kevin in Company Announcements on July 14, 2010

Greetings!

I thought it would be prudent to let you all know that we have recently re-designed our front facing company website.

You’ll also notice that our core prices for shared and VPS hosting have been significantly lowered, with resources allocated for each plan increased significantly (!).

Take a look at our site, if you haven’t already : www.stardothosting.com

Related Links:

  • No Related Links

No Comments

MySQL Query Log – diagnosing and debugging mysql

Posted by Kevin in Database on July 12, 2010

The general query log is a general record of what mysqld is doing. The server writes information to this log when clients connect or disconnect, and it logs each SQL statement received from clients. The general query log can be very useful when you suspect an error in a client and want to know exactly what the client sent to mysqld.

mysqld writes statements to the query log in the order that it receives them, which might differ from the order in which they are executed. This logging order contrasts to the binary log, for which statements are written after they are executed but before any locks are released. (Also, the query log contains all statements, whereas the binary log does not contain statements that only select data.)

To enable the general query log, start mysqld with the –log[=file_name] or -l [file_name] option.

If no file_name value is given for –log or -l, the default name is host_name.log in the data directory.

Server restarts and log flushing do not cause a new general query log file to be generated (although flushing closes and reopens it). On Unix, you can rename the file and create a new one by using the following commands:

shell> mv host_name.log host_name-old.log
shell> mysqladmin flush-logs
shell> cp host_name-old.log backup-directory
shell> rm host_name-old.log

Before 5.0.17, you cannot rename a log file on Windows while the server has it open. You must stop the server and rename the file, and then restart the server to create a new log file. As of 5.0.17, this applies only to the error log. However, a stop and restart can be avoided by using FLUSH LOGS, which causes the server to rename the error log with an -old suffix and open a new error log.

Related Links:

No Comments

Remove mail headers in Postfix outgoing mail

Posted by Kevin in Mail on July 8, 2010

This post is intended for people who want to set up Postfix to remove specific headers within emails that pass through their systems. The most common use for this is to set up a relaying server that will remove any reference of where source emails originated and relevant information about the sender’s computer. Another useful application for this type of header_checks is to remove details about additional functions of your mail server that you do not want made available to the world.

This guide focuses on postfix’s header_checks capabilities, and although there are other ways to do so, we’ve found that this is by far the simplest.

IMPORTANT NOTES

Use these instructions at your own risk. Never test things out in a production environment!

In order for this to work, your main.cf file will have to have a reference to the header_checks file as follows: 

header_checks = regexp:/etc/postfix/maps/header_checks

It is recomended that you keep all of your postfix map files in one directory along with any checks files. In this case, these will be kept in /etc/postfix/maps.

HEADER_CHECKS DETAILS

In addition to any spam filters (see our header_checks file for more information), the below lines should be added to your header_checks file to preserve privacy and remove headers for the internal operations of your mail server:

    # Sample For Dropping Headers:
    #/^Header: IfContains/ 	IGNORE
    /^Received: from 127.0.0.1/ 	IGNORE
    /^User-Agent:/ 	IGNORE
    /^X-Mailer:/ 	IGNORE
    /^X-Originating-IP:/ 	IGNORE

Each line above will search for headers tha have the content between the /^ and the / and will remove each line within the email headers that matches. As an example, the line “/^Received: from 127.0.0.1 .*/ IGNORE” will erase any lines from the email headers that list previous handoffs from an internal mail process to another. This is most commonly used for antivirus or antispam functions on a mail server.

The following lines are related to Anomy Sanitizer and SpamAssassin – two very useful products. These three lines will remove references from the headers for the two software packages, making sure that the users of the system will not easily identify the software that is running on the back end.

    # Sample For Dropping Headers:
    #/^Header: IfContains/ 	IGNORE
    /^Received: from 127.0.0.1/ 	IGNORE
    /^X-Sanitizer:/ 	IGNORE
    /^X-Spam-Status:/ 	IGNORE
    /^X-Spam-Level:/ 	IGNORE

If one were to want to remove all headers relevant to personal information and previous hosts on which the email has passed, the following would be a possible configuration. Note that by removing all of this information, some mail servers will automatically identify emails passing through this system as spam. You will also be removing useful information for troubleshooting any problems that may arise with the mail server.

    # Sample For Dropping Headers:
    #/^Header: IfContains/ 	IGNORE
    /^Received:/ 	IGNORE
    /^User-Agent:/ 	IGNORE
    /^Message-ID:/ 	IGNORE
    /^X-Mailer:/ 	IGNORE
    /^X-MimeOLE:/ 	IGNORE
    /^X-MSMail-Priority:/ 	IGNORE
    /^X-Spam-Status:/ 	IGNORE
    /^X-Spam-Level:/ 	IGNORE
    /^X-Sanitizer:/ 	IGNORE
    /^X-Originating-IP:/ 	IGNORE

Hopefully this will help you clean your mail headers up! :)

Related Links:

No Comments

Automatically Deploy Debian Load Balancers with bash scripting

Posted by Kevin in Linux, Shell Scripting on June 14, 2010

In yet another post in our automation series, we will share a bash script that automates the deployment of debian based load balancers (specifically with LVS / Linux Virtual Server project).

Even though the environments and systems you deploy may start to get more complicated such as with load balancers, there will always be a baseline level with which these systems can be brought to before further configuration and customization needs to be done.

There are many things that can be automated with this process, as you will see in the script below. In most round-robin load balancing scenarios, there wouldn’t be much more that needs to be done as far as configuration beyond what this script can do.

Obviously you will likely need to modify the script to suit your needs and requirements for the organization and standards therein.

Hopefully this will help you roll out many debian load balancers! May the load be split evenly between all your systems ;)


#!/bin/sh
# Debian LVS deployer script
# Version 1.0

PROGNAME="$0"
VERSION="1.0"

# working directory for deployer process.
WORKDIR="/root"

# tasks left (this is updated every step to accommodate recovery during
# the deployer process)
TASKS="./deploy-lvs.tasks"

init_tasks() {
# This function will write a new tasks file.
# it's called from the main body of the script if a tasks file does not exist.
cat > $TASKS< nopasswd_ssh
add_pkgs
get_lvs
configure_lvs
set_hostname
EOS
return 0
}

installer_splash() {
echo "[+] LVS deployer script starting..."
echo " Version: $VERSION"
echo
return 0
}

nopasswd_ssh() {
# disable passwd auth on SSH
echo "[+] Disabling password authentication for SSH... "
perl -pi -e 's/^PasswordAuthentication yes/PasswordAuthentication no/g' /etc/ssh/sshd_config
perl -pi -e 's/^#PermitRootLogin yes/PermitRootLogin without-password/g' /etc/ssh/sshd_config
/etc/init.d/ssh restart
return 0
}

add_pkgs() {
PKGS="libssl0.9.7 exim4 iproute ethtool tcpdump snmpd pciutils less python"
echo "[+] Installing packages: $PKGS... "
apt-get -y install $PKGS || return 1
return 0
}

get_lvs() {
echo "[+] Downloading packages... "
# download the latest version of the Client firewall package.
wget --no-check-certificate http://your.domain.com/lvs.tgz -O /tmp/firewall.tgz || return 1
# unpack firewall scripts
tar --no-same-owner --no-same-permissions --directory / -zxvf /tmp/firewall.tgz || return 1
rm /tmp/firewall.tgz || return 1
return 0
}

configure_lvs() {
# time to configure the FW
KAD=/etc/keepalived/keepalived.conf
FW=/etc/network/firewall
COMMIT=/usr/local/bin/lvs-commit.sh
HOSTS=/etc/hosts
INTERFACES=/etc/network/interfaces
NRPE=/etc/nagios/nrpe_local.cfg
EXIM=/etc/exim4/update-exim4.conf.conf
CONFIGURE_LVS=/etc/network/configure-lvs.pl
echo "[+] Configuring LVS..."
perl $CONFIGURE_LVS
if [ $? -ne 0 ]; then
echo "[!] ERROR: Configuring LVS script failed!"
return 1
fi
echo "[+] Moving files into place..."
rm ${KAD}-template || return 1
rm ${FW}-template || return 1
rm ${COMMIT}-template || return 1
rm ${CONFIGURE_LVS}
mv ${HOSTS}.new ${HOSTS} || return 1
mv ${INTERFACES}.new ${INTERFACES} || return 1
mv ${NRPE}.new ${NRPE} || return 1
mv ${EXIM}.new ${EXIM} || return 1
chmod 700 ${FW}
chmod 700 ${COMMIT}
update-rc.d keepalived defaults || return 1
update-exim4.conf || return 1
# for compatibility
echo "[+] Generating RSA Keys"
ssh-keygen -t rsa -f ~/.ssh/id_rsa -P '' || return 1

return 0
}
clean_up_and_reboot() {
# remove:
# -- temp task file
rm $TASKS
# remove self from .bashrc
if [ -f /root/.bashrc.orig ]; then
mv /root/.bashrc.orig /root/.bashrc
fi
if [ -z /root/.bashrc ]
then
rm /root/.bashrc
fi
# delete self
rm $0
# and reboot.
echo "[+] Please reboot system."
#reboot -n
exit 0
}

debug_quit() {
# hard exit the script in appropriately referenced files
# so that no reboot happens.
echo "debug_quit seen in tasks file, exiting."
exit 0
}

set_hostname() {
echo "[+] Setting LVS hostname... "
echo `hostname` > /etc/hostname
echo `hostname` > /etc/mailname
return 0
}

usage() {
echo "[+] Usage: $PROGNAME"
echo
return 0
}

###############################
### MAIN SCRIPT STARTS HERE ###
###############################

# installer_splash
installer_splash

# fix working dir.
cd $WORKDIR

# does our installer file exist? if not, initalize it.
if [ ! -f $TASKS ]
then
echo "[+] No task file found, installation will start from beginning."
init_tasks
if (($? != 0))
then
echo "[!] ERROR: Cannot create tasks file. Installation will not continue."
exit 1
fi
else
echo "[+] Tasks file located - starting where you left off."
fi

# start popping off tasks from the task list and running them.
# pop first step off of the list
STEP=`head -n 1 $TASKS`
while [ ! -z $STEP ]
do
# execute the function.
echo -e "\n\n###################################"
echo "[+] Running step: $STEP"
echo -e "###################################\n\n"
$STEP
if (($? != 0))
then
# command failed.
echo "[!] ERROR: Step $STEP failed!"
echo " Installation will now abort - you can pick it up after fixing the problem"
echo
exit 1
fi
# throw up a newline just so things don't look so crowded
echo
# remove function from function list.
perl -pi -e "s/$STEP\n?//" $TASKS || exit 1
STEP=`head -n 1 $TASKS`
done

# clean_up_and_reboot
echo "[+] Installation finished - cleaning up."
clean_up_and_reboot

# script is done now - termination should happen with clean_up_and_reboot.
echo "[!] Should not be here!"
exit 1

Related Links:

1 Comment

Automatically Deploy Debian Firewalls with bash scripting

Posted by Kevin in Linux, Shell Scripting on June 2, 2010

Automation is as necessary as any other aspect of systems administration in any critical or production environment where growth and scalability are moving at a significant pace.

Growth in any organization is obviously a good thing. In the systems administrator’s perspective, however, growth can mean more time spent deploying systems and less time spent focusing on other duties.

Automating the server deployment process is the natural next step when your organization has grown to a point where time efficiency becomes more relevant and noticeable to your business owners.

This is the first in a series of posts here where we will explain and share shell scripts that automate the deployment process of several key debian linux based systems. These scripts automate the patching, configuration and implementation of said systems.

They will certainly have to be modified to fit your organization’s needs and standards obviously, but hopefully it will give you a starting point to base your automation / roll-out policies.

Making your life easier and more automated is always a good thing! ;)



#!/bin/sh
# Debian FW deployer script
# Version 1.0

PROGNAME="$0"
VERSION="1.0"

# working directory for deployer process.
WORKDIR="/root"

# tasks left (this is updated every step to accommodate recovery during
# the deployer process)
TASKS="./deploy-fw.tasks"

init_tasks() {
# This function will write a new tasks file.
# it's called from the main body of the script if a tasks file does not exist.
cat > $TASKS< nopasswd_ssh
add_pkgs
get__fw
configure_fw
set_hostname
EOS
return 0
}

installer_splash() {
echo "[+] Firewall deployer script starting..."
echo " Version: $VERSION"
echo
return 0
}

nopasswd_ssh() {
# disable passwd auth on SSH
echo "[+] Disabling password authentication for SSH... "
perl -pi -e 's/^PasswordAuthentication yes/PasswordAuthentication no/g' /etc/ssh/sshd_config
perl -pi -e 's/^#PermitRootLogin yes/PermitRootLogin without-password/g' /etc/ssh/sshd_config
/etc/init.d/ssh restart
return 0
}

add_pkgs() {
PKGS="libssl0.9.7 exim4 iproute ethtool tcpdump snmpd pciutils less python"
echo "[+] Installing packages: $PKGS... "
apt-get -y install $PKGS || return 1
return 0
}

get__fw() {
echo "[+] Downloading packages... "
# download the latest version of the Client firewall package.
wget --no-check-certificate http://www.yoursite.com/fw.tgz -O /tmp/firewall.tgz || return 1
# get the latest firewall.trusted file
wget --no-check-certificate http://www.yoursite.com/firewall.trusted -O /tmp/firewall.trusted || return 1
# unpack firewall scripts
tar --no-same-owner --no-same-permissions --directory / -zxvf /tmp/firewall.tgz || return 1
mv /tmp/firewall.trusted /etc/network/firewall.trusted || return 1
chmod +x /etc/network/firewall.trusted || return 1
rm /tmp/firewall.tgz || return 1
echo "done."
return 0
}

configure_fw() {
# time to configure the FW
KAD=/etc/keepalived/keepalived.conf
FW=/etc/network/firewall
RELOAD=/etc/network/reload.sh
HOSTS=/etc/hosts
INTERFACES=/etc/network/interfaces
NRPE=/etc/nagios/nrpe_local.cfg
EXIM=/etc/exim4/update-exim4.conf.conf
CONFIGURE_FW=/etc/network/configure-fw.pl
echo "[+] Configuring Firewall..."
perl $CONFIGURE_FW
if [ $? -ne 0 ]; then
echo "[!] ERROR: Configuring firewall script failed!"
return 1
fi
echo "[+] Moving files into place..."
rm ${KAD}-template || return 1
rm ${FW}-template || return 1
rm ${RELOAD}-template || return 1
rm ${CONFIGURE_FW}
mv ${HOSTS}.new ${HOSTS} || return 1
mv ${INTERFACES}.new ${INTERFACES} || return 1
mv ${NRPE}.new ${NRPE} || return 1
mv ${EXIM}.new ${EXIM} || return 1
chmod 700 ${FW}
chmod 700 ${RELOAD}
update-rc.d keepalived defaults || return 1
update-exim4.conf || return 1
# for compatibility
echo "[+] Generating RSA Keys"
ssh-keygen -t rsa -f ~/.ssh/id_rsa -P '' || return 1

return 0
}
clean_up_and_reboot() {
# remove:
# -- temp task file
rm $TASKS
# remove self from .bashrc
if [ -f /root/.bashrc.orig ]; then
mv /root/.bashrc.orig /root/.bashrc
fi
if [ -z /root/.bashrc ]
then
rm /root/.bashrc
fi
# delete self
rm $0
# and reboot.
echo "[+] Please reboot system."
#reboot -n
exit 0
}

debug_quit() {
# hard exit the script in appropriately referenced files
# so that no reboot happens.
echo "debug_quit seen in tasks file, exiting."
exit 0
}

set_hostname() {
echo "[+] Setting FW hostname... "
echo `hostname` > /etc/hostname
echo `hostname` > /etc/mailname
echo "done."
return 0
}

usage() {
echo "[+] Usage: $PROGNAME"
echo
return 0
}

###############################
### MAIN SCRIPT STARTS HERE ###
###############################

# installer_splash
installer_splash

# fix working dir.
cd $WORKDIR

# does our installer file exist? if not, initalize it.
if [ ! -f $TASKS ]
then
echo "[+] No task file found, installation will start from beginning."
init_tasks
if (($? != 0))
then
echo "[!] ERROR: Cannot create tasks file. Installation will not continue."
exit 1
fi
else
echo "[+] Tasks file located - starting where you left off."
fi

# start popping off tasks from the task list and running them.
# pop first step off of the list
STEP=`head -n 1 $TASKS`
while [ ! -z $STEP ]
do
# execute the function.
echo -e "\n\n###################################"
echo "[+] Running step: $STEP"
echo -e "###################################\n\n"
$STEP
if (($? != 0))
then
# command failed.
echo "[!] ERROR: Step $STEP failed!"
echo " Installation will now abort - you can pick it up after fixing the problem"
echo
exit 1
fi
# throw up a newline just so things don't look so crowded
echo
# remove function from function list.
perl -pi -e "s/$STEP\n?//" $TASKS || exit 1
STEP=`head -n 1 $TASKS`
done

# clean_up_and_reboot
echo "[+] Installation finished - cleaning up."
clean_up_and_reboot

# script is done now - termination should happen with clean_up_and_reboot.
echo "[!] Should not be here!"
exit 1

Related Links:

1 Comment

Relay Exim mail to google mail in Debian Linux

Posted by Kevin in Uncategorized on May 25, 2010

Sometimes its necessary to relay your mail through a third party provider. If your server environment has a dedicated sendmail server (most do), then this scenario is applicable to you. It is ideal to centralize your outgoing mail to one server so that changes, policies and configuration is located in a single place.

In this scenario, outgoing mail is relayed to google’s domain mail in an Exim mail environment. These steps are fairly straightforward and will hopefully help you to utilize google’s free mail service to send your mail.

Note that google has queuing and mass mail restrictions so if you plan on sending alot of mail this way, you will just get blocked.

    Step 1

Run dpkg-reconfigure exim4-config

1. Choose mail sent by smarthost; received via SMTP or fetchmail

2. Type System Mail Name: e.g. company.com

3. Type IP Adresses to listen on for incoming SMTP connections: 127.0.0.1

4. Leave Other destinations for which mail is accepted blank

5. Leave Machines to relay mail for: blank

6. Type Machine handling outgoing mail for this host (smarthost): smtp.gmail.com::587

7. Choose NO, don’t hide local mail name in outgoing mail.

8. Chose NO, don’t keep number of DNS-queries minimal (Dial-on-Demand).

9. Choose mbox

10. Choose NO, split configuration into small files

11. Mail for postmaster. Leaving blank will not cause any problems though it is not recommended

    Step 2

1. Open the file /etc/exim4/exim4.conf.template
2. Find the line .ifdef DCconfig_smarthost DCconfig_satellite and add the following in that section

 send_via_gmail:
 driver = manualroute
 domains = ! +local_domains
 transport = gmail_smtp
 route_list = * smtp.gmail.com

If you have any other smarthost defined with “domains = ! +local_domains” remove that smarthost.

3. Find the “begin authenticators”. In that section add the following

 gmail_login:
 driver = plaintext
 public_name = LOGIN
 client_send = : yourname@gmail.com : YourGmailPassword

Make sure you have no other authenticators with the same public_name (LOGIN). Comment them out if needed (Thanks Jakub for reminding me)

4. Find the comment “transport/30_exim4-config_remote_smtp_smarthost”. In that section add

 gmail_smtp:
 driver = smtp
 port = 587
 hosts_require_auth = $host_address
 hosts_require_tls = $host_address
    Step 3

1. Run update-exim4.conf

2. Do /etc/init.d/exim4 restart

That should be it. You can test by using the command line mail client.

Test :

 echo "test" | mail -s "subject" test@email-to-send-to.com

Related Links:

1 Comment

Integrate your custom IPTables script with Linux

Posted by Kevin in Linux, Security on May 11, 2010

How do I integrate my custom iptables script with Red Hat Enterprise Linux?

A custom iptables script is sometimes necessary to work around the limitations of the Red Hat Enterprise Linux firewall configuration tool. The procedure is as follows:

1. Make sure that the default iptables initialization script is not running:

service iptables stop

2. Execute the custom iptables script:

sh [custom iptables script]

3. Save the newly created iptables rules:

service iptables save

4. Restart the iptables service:

service iptables restart

5. Verify that the custom iptables ruleset have taken effect:

service iptables status

6. Enable automatic start up of the iptables service on boot up:

chkconfig iptables on

The custom iptables script should now be integrated into the operating system.

Related Links:

1 Comment

Patch Scanning / Information Gathering Script for RedHat / CentOS

Posted by Kevin in Linux, Shell Scripting on April 30, 2010

With all the patch management solutions, local repositories and other options, it is rarely necessary to manually scan all servers on your network to build a “report” of the patch levels in your environment.

Sometimes it is, however. For instance, if you are brought into an environment that has not been properly managed and require some quick audits to evaluate how much actual work needs to be done bringing all the patch levels up to standard, then there are ways to produce these reports with simple bash scripting.

I have developed such a script for similar situations — quick reporting is sometimes necessary even when you are evaluating a large commercial patch management solution. It can even be implemented to coincide such solutions, for independent reporting perhaps.

This script would work well either by distributing it to each server and running the script via ssh key based authentication for centralized reporting. Alternatively, you could modify this script to perform each command via SSH over the network to gather information that way. It is probably more ideal to centrally distribute the script to each server so only one ssh command is executed per server.

Find the script below — note that it only works with RedHat / CentOS systems. Obviously if you are paying for Red Hat enterprise support you already are using satellite; If you are using CentOS then this script may be useful for you.

Enjoy!

#!/bin/sh

# Basic Information Gathering
# Star Dot Hosting
# http://www.stardothosting.com

HOSTNAME=`hostname`
UNAME=`uname -a | awk '{print $3}'`

# Begin Package Scanning

# SSH

SSHON="0"
SSHRUN="NULL"
SSHRPM="NULL"
SSHMATCH="NULL"

if [ -f /usr/sbin/sshd ]
then
        SSHON="1"
	SSHMATCH="0"
        SSHRUN=`ssh -V 2>&1 | awk 'BEGIN { FS = "_" } ; { print $2 }' | awk '{print $1}' | cut -b 0-5`
	TESTRPM=`rpm -qa openssh`
	if [ "$TESTRPM" <> 0  ]
	then
	        SSHRPM=`rpm -qa openssh | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$SSHRUN" == "$SSHRPM" ]
        then
                SSHMATCH="1"
        fi

fi

# Apache

HTTPDON="0"
HTTPDRUN="NULL"
HTTPDRPM="NULL"
HTTPDMATCH="NULL"

if [ -f /usr/sbin/httpd ]
then
        HTTPDON="1"
	HTTPDMATCH="0"
        HTTPDRUN=`httpd -v | grep version | awk 'BEGIN {FS="/"};{print$2}'`
	TESTRPM=`rpm -qa httpd`
	if [ "$TESTRPM" <> 0  ]
	then
        	HTTPDRPM=`rpm -qa httpd | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$HTTPDRUN" == "$HTTPDRPM" ]
        then
                HTTPDMATCH="1"
        fi
fi

# MySQL

MYSQLON="0"
MYSQLRUN="NULL"
MYSQLRPM="NULL"
MYSQLMATCH="NULL"

if [ -f /usr/bin/mysql ]
then
        MYSQLON="1"
	MYSQLMATCH="0"
        MYSQLRUN=`mysql -V | awk '{print $5}' | cut -b 0-6`
	TESTRPM=`rpm -qa mysql`
	if [ "$TESTRPM" <> 0  ]
	then
        	MYSQLRPM=`rpm -qa mysql | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$MYSQLRUN" == "$MYSQLRPM" ]
        then
                MYSQLMATCH="1"
        fi
fi

# PHP

PHPON="0"
PHPRUN="NULL"
PHPRPM="NULL"
PHPMATCH="NULL"

if [ -f /usr/bin/php ]
then
        PHPON="1"
	PHPMATCH="0"
        PHPRUN=`php -v | grep built | awk '{print $2 }'`
	TESTRPM=`rpm -qa php`
	if [ "$TESTRPM" <> 0  ]
	then
        	PHPRPM=`rpm -qa php | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$PHPRUN" == "$PHPRPM" ]
        then
                PHPMATCH="1"
        fi
fi

# Exim
# Needs to be tested on RH box

EXIMON="0"
EXIMRUN="NULL"
EXIMRPM="NULL"
EXIMMATCH="NULL"

if [ -f /usr/sbin/exim ]
then
        EXIMON="1"
	EXIMMATCH="0"
        EXIMRUN=`exim -bV | grep version | awk '{print $3}'`
	TESTRPM=`rpm -qa exim`
	if [ "$TESTRPM" <> 0  ]
	then
        	EXIMRPM=`rpm -qa exim | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$EXIMRUN" == "$EXIMRPM" ]
        then
                EXIMMATCH="1"
        fi
fi

# OpenSSL

OSSLON="0"
OSSLRUN="NULL"
OSSLRPM="NULL"
OSSLMATCH="NULL"

if [ -f /usr/bin/openssl ]
then
        OSSLON="1"
	OSSLMATCH="0"
        OSSLRUN=`openssl version | awk '{print $2}'`
	TESTRPM=`rpm -qa openssl`
	if [ "$TESTRPM" <> 0  ]
	then
        	OSSLRPM=`rpm -qa openssl | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$OSSLRUN" == "$OSSLRPM" ]
        then
                OSSLMATCH="1"
        fi
fi

# PERL

PERLON="0"
PERLRUN="NULL"
PERLRPM="NULL"
PERLMATCH="NULL"

if [ -f /usr/bin/perl ]
then
        PERLON="1"
	PERLMATCH="0"
        PERLRUN=`perl -v | grep built | awk '{print $4}' | awk 'BEGIN { FS = "v" } ; { print $2 }'`
	TESTRPM=`rpm -qa perl`
	if [ "$TESTRPM" <> 0  ]
	then
        	PERLRPM=`rpm -qa perl | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$PERLRUN" == "$PERLRPM" ]
        then
                PERLMATCH="1"
        fi
fi

# PYTHON

PYON="0"
PYRUN="NULL"
PYRPM="NULL"
PYMATCH="NULL"

if [ -f /usr/bin/python ]
then
        PYON="1"
	PYMATCH="0"
        PYRUN=`python -V 2>&1 | awk '{print $2}'`
	TESTRPM=`rpm -qa python`
	if [ "$TESTRPM" <> 0  ]
	then
        	PYRPM=`rpm -qa python | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$PYRUN" == "$PYRPM" ]
        then
                PYMATCH="1"
        fi
fi

# GPG

GPGON="0"
GPGRUN="NULL"
GPGRPM="NULL"
GPGMATCH="NULL"

if [ -f /usr/bin/gpg ]
then
        GPGON="1"
	GPGMATCH="0"
        GPGRUN=`gpg --version | grep gpg | awk '{print $3}'`
	TESTRPM=`rpm -qa gnupg`
	if [ "$TESTRPM" <> 0  ]
	then
        	GPGRPM=`rpm -qa gnupg | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$GPGRUN" == "$GPGRPM" ]
        then
                GPGMATCH="1"
        fi
fi

# RPM

RPMON="0"
RPMRUN="NULL"
RPMRPM="NULL"
RPMMATCH="NULL"

if [ -f /bin/rpm ]
then
        RPMON="1"
	RPMMATCH="0"
        RPMRUN=`rpm --version | awk '{print $3}'`
	TESTRPM=`rpm -qa rpm`
	if [ "$TESTRPM" <> 0  ]
	then
        	RPMRPM=`rpm -qa rpm | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$RPMRUN" == "$RPMRPM" ]
        then
                RPMMATCH="1"
        fi
fi

# SENDMAIL

SENDON="0"
SENDRUN="NULL"
SENDRPM="NULL"
SENDMATCH="NULL"

if [ -f /usr/sbin/sendmail ]
then
        SENDON="1"
        SENDMATCH="0"
        SENDRUN=`echo 'quit' | nc localhost 25 | grep Sendmail | awk '{print $5}' | awk 'BEGIN { FS = "/" } ; { print $1 }'`
	TESTRPM=`rpm -qa sendmail`
	if [ "$TESTRPM" <> 0  ]
	then
	        SENDRPM=`rpm -qa sendmail | awk 'BEGIN { FS = "-" } ; { print $2 }'`
	fi
        if [ "$SENDRUN" == "$SENDRPM" ]
        then
                SENDMATCH="1"
        fi
fi

### Non running packages

# bind-libs

BINDLIB="NULL"
TESTRPM=`rpm -qa bind-libs`
if [ "$TESTRPM" <> 0  ]
then
	BINDLIB=`rpm -qa bind-libs | awk 'BEGIN { FS = "-" } ; { print $3 }'`
fi

# bind-utils

BINDUTIL="NULL"
TESTRPM=`rpm -qa bind-utils`
if [ "$TESTRPM" <> 0  ]
then
	BINDUTIL=`rpm -qa bind-utils | awk 'BEGIN { FS = "-" } ; { print $3 }'`
fi

# coreutils

COREUTIL="NULL"
TESTRPM=`rpm -qa coreutils`
if [ "$TESTRPM" <> 0  ]
then
	COREUTIL=`rpm -qa coreutils | awk 'BEGIN { FS = "-" } ; { print $2 }'`
fi

# chkconfig

CHKCONFIG="NULL"
TESTRPM=`rpm -qa chkconfig`
if [ "$TESTRPM" <> 0  ]
then
	CHKCONFIG=`rpm -qa chkconfig | awk 'BEGIN { FS = "-" } ; { print $2 }'`
fi

# initscripts

INITSCR="NULL"
TESTRPM=`rpm -qa initscripts`
if [ "$TESTRPM" <> 0  ]
then
	INITSCR=`rpm -qa initscripts | awk 'BEGIN { FS = "-" } ; { print $2 }'`
fi

# redhat-release

RHRELEASE="NULL"
TESTRPM=`rpm -qa redhat-release`
if [ "$TESTRPM" <> 0  ]
then
	RHRELEASE=`rpm -qa redhat-release | awk 'BEGIN { FS = "-" } ; { print $3"-"$4 }'`
fi

echo $HOSTNAME,$UNAME,$SSHMATCH,$HTTPDMATCH,$MYSQLMATCH,$PHPMATCH,$EXIMMATCH,$OSSLMATCH,$PYMATCH,$PERLMATCH,$GPGMATCH,
$RPMMATCH,$SENDMATCH,$BINDLIB,$BINDUTIL,$COREUTIL,$CHKCONFIG,$INITSCR,$RHRELEASE,$SSHON,$SSHRUN,$SSHRPM,$HTTPDON,$HTTPDRUN,
$HTTPDRPM,$MYSQLON,$MYSQLRUN,$MYSQLRPM,$PHPON,$PHPRUN,$PHPRPM,$EXIMON,$EXIMRUN,$EXIMRPM,$OSSLON,$OSSLRUN,$OSSLRPM,$PERLON,
$PERLRUN,$PERLRPM,$PYON,$PYRUN,$PYRPM,$GPGON,$GPGRUN,$GPGRPM,$RPMON,$RPMRUN,$RPMRPM,$SENDON,$SENDRUN,$SENDRPM

Note that you can modify the echo output to produce whatever output you need in order to present it in a nice human readable report.

Related Links:

No Comments

Scheduled antivirus scans to prevent viral injections on user generated content

Posted by Kevin in Security on April 19, 2010

When dealing with high traffic sites, especially media based or community based sites, there is always the risk of javascript, virus, XSS or other malicious injection of badness when giving a community of users the ability to upload files to your site.

There are several things to consider when evaluating all “points of entry” that are available to the public, into your systems.

Most content management and community based systems use libraries such as Imagemagick to process images (such as profile pictures) into their proper format and size.

Believe it or not, it is hard to actually inject code or other malicious data into the actual image to survive this sanitizing process. There is still risks , however. The library version you are running may be vulnerable to exploits itself.

As always, a good rule of thumb is to ensure all possible aspects of your systems are up to date and that you are aware of any security vulnerabilities as they come out so they can either be patched or addressed in some other way.

One thing to consider, especially when dealing with thousands of users and even more uploads is a scheduled scan of your user uploads using free virus scanning tools such as clamscan. This is an endpoint reporting strategy that can at least cover your ass in the event that something else was missed or a 0day vulnerability exploited.

It should be noted that the virus scans themselves aren’t intended to protect the linux systems themselves, but rather the opportunistic ‘spreading’ of compromised images and code that having an infected file on a public community based system can provide.

Its very simple to implement clamav (daemonization is not necessary), clamscan is all we need to execute regular scans at 10, 15, 30 or 60 minute intervals.

Once clamscan is implemented, definitions updated (and regular update cronjobs in place) you can roll out a script similar to the one we have here to implement the scheduled scans :

#!/bin/bash
# Scheduled Scan of user uploaded files
# Usage : ./virusscan.sh /folder

SUBJECT="[VIRUS DETECTED] ON `hostname` !"
EMAIL="you@yourdomain.com"
LOG=/var/log/clamav/scan.log

# Clear out old logs -- the email alerts should be archived if we need to go back to old alerts
echo "" > $LOG

# Check if the folder is empty -- only scan if this is an active node in a clustered system
# look for empty dir
if [ "$(ls -A $1)" ]
then
        # Scan files
        clamscan $1 -r --infected --scan-pdf --scan-elf --log=$LOG

        # Check the last set of results. If there are any "Infected" counts that aren't zero, we have a problem.
        cat $LOG | grep Infected | grep -v 0
        if [ $? = 0 ]
        then
                cat $LOG | mail -s "$SUBJECT" $EMAIL -- -F Antivirus -f antivirus@yourdomain.com
        fi

else
        echo "directory empty -- doing nothing"
        exit 0;
fi

The actual cronjob entry can look something like this :

0 */1 * * * /bin/bash /usr/local/bin/virusscan.sh "/your/path/to/user/uploaded/files/" > /dev/null 2>&1

It seems like a rather simple solution — but it does provide a venue for additional sanitizing of user input. In our experience , it is best to only report on anything that clamscan might find. You can, however tell clamscan to simply delete any suspected infections it finds.

Related Links:

1 Comment