SSH and SFTP with Paramiko & Python

November 23rd, 2014

Paramiko is a Python implementation of SSH with a whole range of supported features. To start, let’s look at the most simple example – connecting to a remote SSH server and gathering the output of ls /tmp/

import paramiko

ssh = paramiko.SSHClient()
        ssh.connect('localhost', username='testuser', password='t3st@#test123')
except paramiko.SSHException:
        print "Connection Failed"

stdin,stdout,stderr = ssh.exec_command("ls /etc/")

for line in stdout.readlines():
        print line.strip()

After importing paramiko, we create a new variable ‘ssh’ to hold our SSHClient. ssh.set_missing_host_key_policy automatically adds our server’s host key without prompting. For security, this is not a good idea in production, and host keys should be added manually. Should a host key change unexpectedly, it could indicate that the connection has been compromised and is being diverted elsewhere.

Next, we create 3 variables, stdin, stdout and stderr allowing us to access the respective streams when calling ls /etc/

Finally, for each “\n” terminated line on stdout, we print the line, stripping the trailing “\n” (as print adds one). Finally we close the SSH connection.

Let’s look at another example, where we communicate with stdin.
Read the rest of this entry »

Simple Ready to Roll Linux Backup Script

September 12th, 2014

I’d built a Linux backup BASH shell script a while ago that I’ve been using, and wanted to share it today. This is a simple and easy to configure script, useful for backing up and scheduling multiple hosts, as well as handling file and MySQL backups, and flexibly allowing multiple days or copies to be retained.

The full source is available here

The global configuration is performed at the top of the script:


RSYNC="time nice -19 rsync"
MYSQLDUMP="time nice -19 mysqldump"
RSYNC_ARGS="-arplogu --delete --stats"
TODAY=`date +%Y%m%d`

The utilities that you see listed are all required to be installed: rsync gzip scp time nice cat mysqldump.

The directory structure for backups is a master directory, which in this case is /home/sysbackups, a directory for the actual backups to be placed, in this case /home/sysbackups/backups and a directory for log files, in this case /home/sysbackups/logs. These directories should exist prior to running the script.

The usage of ‘nice’ is to ensure the backups are as resource friendly as possible, and ‘time’ allows for timing data to be provided within the log files created.

Each backup set is defined lower down in the ‘startEntry’ function. Taking the first as an example:

10 )
 START="Local: vm1"
 B_RSYNC_DIR=( "/var/www" "/var/spool/cron" "/etc" "/home" )
 B_MYSQLDUMP_DATABASES=( "all-databases" )

The ‘START’ variable defines the “friendly name” of the machine for log purposes, and the ‘HOST’ variable defines it’s IP or hostname.

Setting ‘B_RSYNC’ to 1 instructs the script to execute the rsync routines for file backup. Setting ‘B_MYSQLDUMP’ to 1 allows us to back up MySQL databases from on the host.

Rsync options

B_RSYNC_USER defines the SSH user to connect to the host as
B_RSYNC_LIMIT defines the limit in kbps for the transferB_RSYNC_DIR is an array of directories to back up

Mysqldump options

B_MYSQLDUMP_USER defines the SSH user to connect to the host as
B_MYSQLDUMP_MYSQLUSER and B_MYSQLDUMP_PASS define the MySQL username and password to connect with
B_MYSQLDUMP_HOST defines the MySQL host to connect to, relative to the HOST variable.
B_MYSQLDUMP_TMP defines a temporary location for the mysql backup on the host
B_MYSQLDUMP_GZIPAFTER defines whether the MySQL backups should be GZipped before being transferred
B_MYSQLDUMP_DATABASES is an array of database names to be backed up, with “all-databases” being hopefully self explanatory

B_MULTIPLEDAYS_DB defines the number of database copies to keep and B_MULTIPLEDAYS defines the number of file sets to keep.

As we have defined this backup set as case ’10’ in the script, to execute it, we simply run: /path/to/ 10

This can be cronned to run on a daily basis.

Lastly, as connections to hosts are made via SSH, either a password will need to be entered on each run manually, or SSH keys can be set up.

Feel free to reply with changes or comments.

Copy files securely from host to host with SCP

March 17th, 2009

SCP is a really handy utility for copying files to or from your machine to another host over SSH.


scp /tmp/myfile user@otherhost:/tmp/
Will send /tmp/myfile over to /tmp/ on otherhost

scp user@otherhost:/etc/passwd ~/
Will copy /etc/passwd from otherhost to my home directory (~/)

Like everything in Linux, it’s that easy 🙂

As SCP uses SSH as it’s transport, key authentication will allow a passwordless login.