Linux iproute2 multiple default gateways

October 5th, 2014

This article describes a Linux server set up with 2 interfaces (eth0) and (eth1). Each interface has a separate ISP, network details and default gateway. eth0 has two sets of network details on the same interface and so a virtual interface (eth0:0) must be created to handle the second IP.

By default, Linux only allows for one default gateway. Let’s see what happens if we try to use multiple uplinks with 1 default gateway. Assume eth0 is assigned 192.168.1.2/24 and eth1 is assigned 172.16.1.5/16. Let’s say our default gateway is 192.168.1.1 (which of course is found through eth0) but there’s also a 172.16.0.1 gateway on eth1 which we can’t enter as Linux only allows for the one.

Our routing table now looks like this:

root@www1:~# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         192.168.1.1     0.0.0.0         UG    0      0        0 eth0
192.168.1.0     0.0.0.0         255.255.255.0   U     0      0        0 eth0
172.16.0.0      0.0.0.0         255.255.0.0     U     0      0        0 eth1

If a packet comes in to us, routed through the 172.16.0.1 gateway from say 8.8.8.8, our machine will receive it. When it tries to reply to 8.8.8.8 however, it runs down the routing table and sees that it’s not local to eth0 or eth1 and therefore will get routed out through the default gateway (192.168.1.1) – the problem is, this is the wrong gateway and so the target machine will ignore our response due to it being out of sequence and from the wrong IP.

Using iproute2, Linux can track multiple routing tables and therefore multiple default gateways. If the packet comes in through one interface and to one IP, it will go out via a specific default gateway. The script to achieve this is as follows:

Read the rest of this entry »

Simple Ready to Roll Linux Backup Script

September 12th, 2014

I’d built a Linux backup BASH shell script a while ago that I’ve been using, and wanted to share it today. This is a simple and easy to configure script, useful for backing up and scheduling multiple hosts, as well as handling file and MySQL backups, and flexibly allowing multiple days or copies to be retained.

The full source is available here

The global configuration is performed at the top of the script:

#config
MAIN_DIR="/home/sysbackups"
BACKUP_DIR="${MAIN_DIR}/backups"
LOG_DIR="${MAIN_DIR}/logs"

RSYNC="time nice -19 rsync"
SSH=ssh
MYSQLDUMP="time nice -19 mysqldump"
GZIP=gzip
SCP=scp
CAT=cat
RM=rm
CP=cp
RSYNC_ARGS="-arplogu --delete --stats"
TODAY=`date +%Y%m%d`
LOG="${LOG_DIR}/log_error_${TODAY}.log"

The utilities that you see listed are all required to be installed: rsync gzip scp time nice cat mysqldump.

The directory structure for backups is a master directory, which in this case is /home/sysbackups, a directory for the actual backups to be placed, in this case /home/sysbackups/backups and a directory for log files, in this case /home/sysbackups/logs. These directories should exist prior to running the script.

The usage of ‘nice’ is to ensure the backups are as resource friendly as possible, and ‘time’ allows for timing data to be provided within the log files created.

Each backup set is defined lower down in the ‘startEntry’ function. Taking the first as an example:

10 )
 STARTED=1
 START="Local: vm1"
 HOST="192.168.1.50"
 B_RSYNC_USER=root
 B_RSYNC=1
 B_RSYNC_LIMIT=4096
 B_RSYNC_DIR=( "/var/www" "/var/spool/cron" "/etc" "/home" )
 B_MYSQLDUMP=1
 B_MULTIPLEDAYS_DB=3
 B_MYSQLDUMP_USER="root"
 B_MYSQLDUMP_MYSQLUSER="root"
 B_MYSQLDUMP_PASS="password"
 B_MYSQLDUMP_HOST="127.0.0.1"
 B_MYSQLDUMP_TMP="/tmp/mysqldump-backup.sql"
 B_MYSQLDUMP_GZIPAFTER=1
 B_MYSQLDUMP_DATABASES=( "all-databases" )
 ;;

The ‘START’ variable defines the “friendly name” of the machine for log purposes, and the ‘HOST’ variable defines it’s IP or hostname.

Setting ‘B_RSYNC’ to 1 instructs the script to execute the rsync routines for file backup. Setting ‘B_MYSQLDUMP’ to 1 allows us to back up MySQL databases from on the host.

Rsync options

B_RSYNC_USER defines the SSH user to connect to the host as
B_RSYNC_LIMIT defines the limit in kbps for the transferB_RSYNC_DIR is an array of directories to back up

Mysqldump options

B_MYSQLDUMP_USER defines the SSH user to connect to the host as
B_MYSQLDUMP_MYSQLUSER and B_MYSQLDUMP_PASS define the MySQL username and password to connect with
B_MYSQLDUMP_HOST defines the MySQL host to connect to, relative to the HOST variable.
B_MYSQLDUMP_TMP defines a temporary location for the mysql backup on the host
B_MYSQLDUMP_GZIPAFTER defines whether the MySQL backups should be GZipped before being transferred
B_MYSQLDUMP_DATABASES is an array of database names to be backed up, with “all-databases” being hopefully self explanatory

B_MULTIPLEDAYS_DB defines the number of database copies to keep and B_MULTIPLEDAYS defines the number of file sets to keep.

As we have defined this backup set as case ’10’ in the script, to execute it, we simply run: /path/to/backup.script.sh 10

This can be cronned to run on a daily basis.

Lastly, as connections to hosts are made via SSH, either a password will need to be entered on each run manually, or SSH keys can be set up.

Feel free to reply with changes or comments.

String replacement with sed

November 6th, 2009

Sed – stream editor is a powerful tool to manipulate strings. It will take STDIN as well as operating on a file:

The most common usage is to replace text:  echo “this is a test string” | sed s/i/z/g will replace every instance of ‘i’ with a ‘z’:  thzs zs a test strzng

You can delete a particular word with say echo “this is a test string”| sed s/test//g leaving: this is a  string

You can operate on a file with:

echo “this is a test string” >> file; sed -e s/test//g file Leaving: this is a  string

You can also use regular expressions with sed.

BASH Script – Blank Out CC Details

October 27th, 2009

Edit: I should have pointed out originally, as I have now received feedback on this. This is NOT the best or optimal way of performing this task. I was trying to illustrate as many shell scripting principles as possible in terms of ‘if’, ‘for’, counters, etc, and how such a one liner has been put together. Perhaps I should have thought of a better way of illustrating such principles, but nevertheless, here it is!

Here’s a quick one liner, can’t think why anyone would ever have any use for it, but maybe the principle itself could be of use to someone! This will take a file containing listings of 16 digit numbers, i.e. 1234123412341234 and replace it with XXXXXXXXXXXX1234

for I in ` cat mylist `; do P=””; ctr=0; for I in `echo $I|grep -o .`; do let ctr=$ctr+1; if [ $ctr -gt 12 ]; then P=${P}${I}; else P=${P}”X”; fi; done; echo $P|tr -d ‘n’; echo -ne “n”; done

Duly spaced and indented:

for I in `cat mylist`; do

P=””
ctr=0;
for I in `echo $I|grep -o .`; do

let ctr=$ctr+1;
if [ $ctr -gt 12 ]; then

P=${P}${I};

else

P=${P}”X”;

fi;

done;
echo $P|tr -d ‘n’;
echo -ne “n”;

done

Would love anyone to comment with variations.


Linux Color Directory Listings

September 15th, 2009

How to add color to ‘ls’?

Adding color to your ls directory listings is easy enough, just use ls –color. You can set this behavior as the default with alias ls=’ls –color’ which I personally find quite useful. It plays well with PuTTY.

The environment variable LS_COLORS dictates what colors are applied to what file types and file extensions.
Read the rest of this entry »

Shell Return Codes – Ping Monitoring

September 9th, 2009

BASH – The Bourne Again Shell amongst most if not all other shells allows each application to exit with a return code. Some shells and environments have limits on what range this integer can fall into. Something between 0 and 255 inclusive is always a safe bet. In BASH, the variable $? is populated with the return code of the last command to return control back to the shell. It is important to preserve the return code immediately after the application exits that we want to monitor, as subsequent commands will overwrite the variable. The ping tool returns 0 on success:

HOST=”192.168.1.5″
ping -c1 ${HOST} -q 2>&1 >/dev/null  #ping HOST once and do not print any output to the screen
RET=$?  #assign the return code to RET so we can preserve it for after the ‘if’
if [ ${RET} -eq 0 ]; then
#we were successful.
echo “We were successful”
else
#we weren’t successful
echo “Host ${HOST} failed ping monitoring on `date`” |mail -s “Uptime Monitoring” admin@example.com
fi

Now of course there are easier ways of achieving the above task, although I’ve laid out the script in this way hoping that the way I have laid it out illustrates capturing the code and preserving it beyond the ‘if’ that follows which would have overwritten it. Just as further illustration, calling ping invalid followed directly by echo $? shows a return code of ‘2’ – obviously the return code for such a failure. Calling echo $? again immediately after shows a return code of ‘0’ as the return code of ping was overwritten by the return code of the first echo statement. Bash builtins return codes to the shell as any other application would.