SSH and SFTP with Paramiko & Python

November 23rd, 2014

Paramiko is a Python implementation of SSH with a whole range of supported features. To start, let’s look at the most simple example – connecting to a remote SSH server and gathering the output of ls /tmp/

import paramiko

ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
try:
        ssh.connect('localhost', username='testuser', password='t3st@#test123')
except paramiko.SSHException:
        print "Connection Failed"
        quit()

stdin,stdout,stderr = ssh.exec_command("ls /etc/")

for line in stdout.readlines():
        print line.strip()
ssh.close()

After importing paramiko, we create a new variable ‘ssh’ to hold our SSHClient. ssh.set_missing_host_key_policy automatically adds our server’s host key without prompting. For security, this is not a good idea in production, and host keys should be added manually. Should a host key change unexpectedly, it could indicate that the connection has been compromised and is being diverted elsewhere.

Next, we create 3 variables, stdin, stdout and stderr allowing us to access the respective streams when calling ls /etc/

Finally, for each “\n” terminated line on stdout, we print the line, stripping the trailing “\n” (as print adds one). Finally we close the SSH connection.

Let’s look at another example, where we communicate with stdin.
Read the rest of this entry »

String replacement with sed

November 6th, 2009

Sed – stream editor is a powerful tool to manipulate strings. It will take STDIN as well as operating on a file:

The most common usage is to replace text:  echo “this is a test string” | sed s/i/z/g will replace every instance of ‘i’ with a ‘z’:  thzs zs a test strzng

You can delete a particular word with say echo “this is a test string”| sed s/test//g leaving: this is a  string

You can operate on a file with:

echo “this is a test string” >> file; sed -e s/test//g file Leaving: this is a  string

You can also use regular expressions with sed.

BASH Script – Blank Out CC Details

October 27th, 2009

Edit: I should have pointed out originally, as I have now received feedback on this. This is NOT the best or optimal way of performing this task. I was trying to illustrate as many shell scripting principles as possible in terms of ‘if’, ‘for’, counters, etc, and how such a one liner has been put together. Perhaps I should have thought of a better way of illustrating such principles, but nevertheless, here it is!

Here’s a quick one liner, can’t think why anyone would ever have any use for it, but maybe the principle itself could be of use to someone! This will take a file containing listings of 16 digit numbers, i.e. 1234123412341234 and replace it with XXXXXXXXXXXX1234

for I in ` cat mylist `; do P=””; ctr=0; for I in `echo $I|grep -o .`; do let ctr=$ctr+1; if [ $ctr -gt 12 ]; then P=${P}${I}; else P=${P}”X”; fi; done; echo $P|tr -d ‘n’; echo -ne “n”; done

Duly spaced and indented:

for I in `cat mylist`; do

P=””
ctr=0;
for I in `echo $I|grep -o .`; do

let ctr=$ctr+1;
if [ $ctr -gt 12 ]; then

P=${P}${I};

else

P=${P}”X”;

fi;

done;
echo $P|tr -d ‘n’;
echo -ne “n”;

done

Would love anyone to comment with variations.


Linux piping and redirecting stdin stderr stdout

March 7th, 2009

We have three relevant streams when dealing with passing data around on the command line. STDIN (0), STDOUT (1) and STDERR (2)

echo “hello” will return “hello” to STDOUT

echo “hello” | sed s/llo/y/g
Returns: ‘hey’

echo “hello” will print “hello” to STDOUT which we pipe to sed’s STDIN. The shell will fork both processes, echo and sed, and create a pipe between one’s STDOUT to the other’s STDIN. A ‘broken pipe’ will occur when one terminates unexpectedly.

strace echo “hello” will print the system calls that the command makes. Lets say I just want to print out open() calls.

strace echo “hello” | grep open does not work. It seems that the grep is ignored.

This is because strace sends it’s output to STDERR and not STDOUT. In this case we must redirect STDERR to STDOUT so grep can pick it up on it’s STDIN.

strace echo “hello” 2>&1 | grep open will work successfully.

What if we want to redirect STDOUT and STDERR to a file? We simply redirect STDOUT to a file and then redirect STDERR to STDOUT.

strace echo “hello” >/tmp/strace.output 2>&1

A nonstandard method of achieving the same by redirecting everything in one go is strace echo “hello” &>/tmp/strace.output however this is not guaranteed to work across all implementations.

* Post edited thanks to observations from Adam Bolte (16/11/09)

BASH Shell Scripting – Sort a string alphabetically

March 5th, 2009

I was asked today how to sort a string alphabetically with BASH

Using perl, you can easily enough use

print (join “”, sort split //,$_)

With bash however, the best option is:

echo “teststring” | grep -o . | sort -n |tr -d ‘n’; echo
Which returns: eginrssttt

A good way of enumerating each character from a string in general is:

for (( i = 0; i < ${#str[@]}; i++ )); do echo “${str[$i]}”; done

mknod tutorial

October 26th, 2008

mknod is a powerful command with which you can create block or character special files. If you view the man page, you’ll see that you can use it to create block device links and character device links. If you don’t know what these are then don’t worry. The purpose of this tutorial is to explore the FIFO (First In First Out) feature.

A FIFO literally does what it says on the box. The first piece of data to go in is the first piece of data to go out.

The usage of the command is:

Usage: /bin/mknod [OPTION]… NAME TYPE [MAJOR MINOR]

Where MAJOR and MINOR are for the special devices mentioned above.
Read the rest of this entry »