Keeping the Raspberry Pi 3 clock up to date

June 27th, 2016

Raspberry Pi 3The Raspberry Pi 3 is an excellent piece of hardware. With 4 ARM A53 cores, 1Gb RAM and integrated 802.11bgn wifi, at only $35, they’re giving them away!

One of the biggest pain points when using the Pi for data acquisition is a lack of a battery backed clock. Each time the Pi boots up, it reverts to it’s manufacture date. Sure, with a working network connection we can synchronise with NTP easily enough, but what happens when we don’t have network access?

The Pi foundation deliberately avoided including a battery backed clock – it’s changes the size, adds extra circuitry and a battery, and increases the price.

How do we know when our clock is wrong? When the Pi boots up, until we’ve made a successful NTP call, our clock is wrong.

How can we prevent our clock going backwards? Use fake-hwclock. The system time is logged, and then restored on next boot to whatever it was previously. The clock will be wrong, but at least it won’t have gone backwards. This can save us from a number of code related issues.

ARPI600The other option, is to get an addon clock. After much research, I settled on the ARPI600.

First – it obviously includes the needed RTC over an I2C interface. The PDF setup guide is pretty simple and just works.

Second – it includes XBee support – a whole range of radio modules.

Third – it gives you the Arduino headers and opens up the Pi to the full Arduino ecosystem.

Fourth – it provides a USB interface for both power, and the serial UART interface, saving on that 4 wire debug cable.

The case? A bit of a problem – if anyone knows of one that will take a Pi3, ARPI600 hat and any Arduino hats, I’m all ears – until then, I’ll need to print my own.

Anyway, back to time keeping. After an initial time set with NTP, we can set our accurate time to the ARPI600 with hwclock -w

Later on after a fresh boot, we can set our clock to match that of the ARPI600 with hwclock -s. Lastly, to print out the clock on the ARPI600, it’s hwclock -r.



Sniffing the Network

December 14th, 2014

This article is intended to provide a simple demonstration of how easy it is to sniff/intercept traffic on various types of networks, and serve as a warning to utilize secure methods of communication on a) untrusted networks and b) known networks with the potential for untrusted clients or administrators.

The first consideration is the topology of the network we’re connected to. To consider 5 common scenarios:

  1. Wired ethernet hub network: Hubs are becoming more and more obsolete as they are changed to switches. Multiple devices can be connected to a hub, and any data received by the hub from one device is broadcast out to all other devices. This means that all devices receive all network traffic. Not only is this an inefficient use of bandwidth, but each device is trusted to accept traffic destined for itself and to ignore traffic destined for another node. To sniff such a network, a node simply needs to switch it’s network interface card to “promiscuous mode”, meaning that it accepts all traffic received.
  2. Wired ethernet switched network: Multiple devices can be connected to a switch, however a switch has greater intelligence than a hub. The switch will inspect the traffic sent on each port, and learn the hardware (MAC) address of the client connected to a particular port. Once learned, the switch will inspect any frames it receives on a port, and forward that frame to the known recipient’s port alone. Other devices connected to the switch will not receive traffic that is not destined for them. This offers enhanced bandwidth usage over a hub. Switches rely on ARP packets which are easily forged in order to learn which devices are on which ports.
  3. Wireless open networks: Multiple devices can connect to an open wireless network. All data is broadcast across the network in plain text, and any attacker can sniff/intercept traffic being broadcast across the network. An open wireless network may present the user with a form of hotspot login page before granting internet access, however this does not detract from the network itself being open.
  4. WEP encrypted wireless network: A WEP encrypted network requires a WEP key to encrypt and decrypt network traffic. WEP has long been an outdated and insecure method of wireless network protection, and cracking a wireless network’s WEP key is fast and requires low skill. WEP is not secure. In addition, all clients connected to the network use the same WEP key to connect. That results in any user on the network with the WEP key being table to view any traffic transmitted to and from other nodes on the network.
  5. WPA/WPA2 encrypted network: A WPA/WPA2 encrypted network is significantly more secure than a WEP network. Whilst attacks exist on parts of the protocol, and extensions such as WPS, no known attack is able to recover a complex WPA/WPA2 password within an acceptable period of time. Whilst all clients connect to the network with the same password, the protocol is engineered to create different keystreams between each connected client and the access point. This means that simple sniffing in the traditional sense is not possible on the network.

Read the rest of this entry »

Linux: You may have been Compromised when..

December 9th, 2014

There are a number of warning signs that a system has been compromised. The cases below warrant further investigation. Of course, they aren’t all guarantees that your system has been compromised, however they can be strong indicators.

1. Your welcome banner shows the last log in from an unknown/foreign IP address:

Last login: Tue Dec  2 16:08:41 2014 from

2. The load on a usually idle system is suspiciously high:

root@mt:~# w
 17:06:39 up 62 days, 22:37,  1 user,  load average: 8.12, 8.14, 8.11
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    pwn              17:03    7.00s  0.00s  0.00s w

This could indicate that unknown processes are running.
Read the rest of this entry »

Reset Linux Root Password

December 6th, 2014

There are a couple of reasons why you might want to reset a Linux root password. If the current password is known to you, just log in as root and issue the passwd command. What if you’ve forgotten the password and can’t log in? Resetting a Linux root password is simple if you have access to the machine. There are 2 main methods.

Method #1

First, we boot the machine up. If LILO is in use, enter linux init=/bin/bash at the ‘LILO:’ prompt. If GRUB is in use, then press key ‘e’. We’ll need to edit the kernel line, beginning ‘linux’, and append init=/bin/sh:

Boot Step #3 Boot Step #1 Boot Step #2

Read the rest of this entry »

Burp Suite: Intercepting & Modifying HTTP Requests & Responses

December 3rd, 2014

Burp Suite is a powerful web application auditor with a huge range of features, from simple to advanced. One of its core features is an intercepting proxy server. This allows us to pass our web traffic through burp suite, allowing us to view and modify both our browsers request before it goes to the remote web server, and the web server’s response before it returns to our browser.

A couple common request modifications:

  • Add data to form submissions, modify hidden fields.
  • View and modify browser AJAX data
  • View and edit headers including cookies

And a couple of common response modifications:

  • Remove client side JavaScript (usually validations or other limitations)
  • Add or remove cookies sent to the browser

First, fire up Burp Suite, and browse to Proxy –> Options:

Read the rest of this entry »

SSH Fingerprint and Hostkey with Paramiko in Python

November 24th, 2014

Following on from SSH and SFTP with Paramiko & Python, I recently had the need to gain a remote SSH server’s fingerprint and hostkey for verification purposes. This is achievable through setting up a socket, and then applying paramiko.Transport over our established socket. First, we include the various bits and pieces we’ll need:

import socket
import paramiko
import hashlib
import base64

Next, we establish a socket connection ‘mySocket’ to “localhost” on port 22 – our dummy SSH server. We then use paramiko.Transport to gain access to paramiko’s core SSH protocol options on the socket.

mySocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mySocket.connect(("localhost", 22))
myTransport = paramiko.Transport(mySocket)

To get the remote hostkey, we call myTransport.get_remote_server_key():
Read the rest of this entry »

Linux Namespaces

November 23rd, 2014

Starting from kernel 2.6.24, there are 6 different types of Linux namespaces. Namespaces are useful in isolating processes from the rest of the system, without needing to use full low level virtualization technology.

  • CLONE_NEWIPC: IPC Namespaces: SystemV IPC and POSIX Message Queues can be isolated.
  • CLONE_NEWPID: PID Namespaces: PIDs are isolated, meaning that a PID inside of the namespace can conflict with a PID outside of the namespace. PIDs inside the namespace will be mapped to other PIDs outside of the namespace. The first PID inside the namespace will be ‘1’ which outside of the namespace is assigned to init
  • CLONE_NEWNET: Network Namespaces: Networking (/proc/net, IPs, interfaces and routes) are isolated. Services can be run on the same ports within namespaces, and “duplicate” virtual interfaces can be created.
  • CLONE_NEWNS: Mount Namespaces. We have the ability to isolate mount points as they appear to processes. Using mount namespaces, we can achieve similar functionality to chroot() however with improved security.
  • CLONE_NEWUTS: UTS Namespaces. This namespaces primary purpose is to isolate the hostname and NIS name.
  • CLONE_NEWUSER: User Namespaces. Here, user and group IDs are different inside and outside of namespaces and can be duplicated.

Let’s look first at the structure of a C program, required to demonstrate process namespaces. The following has been tested on Debian 6 and 7.

First, we need to allocate a page of memory on the stack, and set a pointer to the end of that memory page. We use alloca to allocate stack memory rather than malloc which would allocate memory on the heap.

void *mem = alloca(sysconf(_SC_PAGESIZE)) + sysconf(_SC_PAGESIZE);

Next, we use clone to create a child process, passing the location of our child stack ‘mem’, as well as the required flags to specify a new namespace. We specify ‘callee’ as the function to execute within the child space:


After calling clone we then wait for the child process to finish, before terminating the parent. If not, the parent execution flow will continue and terminate immediately after, clearing up the child with it:

while (waitpid(mypid, &r, 0) < 0 && errno == EINTR)

Lastly, we’ll return to the shell with the exit code of the child:

	return WEXITSTATUS(r);

Now, let’s look at the callee function:

static int callee()
	int ret;
	mount("proc", "/proc", "proc", 0, "");
	setgroups(0, NULL);
	ret = execl("/bin/bash", "/bin/bash", NULL);
	return ret;

Here, we mount a /proc filesystem, and then set the uid (User ID) and gid (Group ID) to the value of ‘u’ before spawning the /bin/bash shell.
Read the rest of this entry »

SSH and SFTP with Paramiko & Python

November 23rd, 2014

Paramiko is a Python implementation of SSH with a whole range of supported features. To start, let’s look at the most simple example – connecting to a remote SSH server and gathering the output of ls /tmp/

import paramiko

ssh = paramiko.SSHClient()
        ssh.connect('localhost', username='testuser', password='t3st@#test123')
except paramiko.SSHException:
        print "Connection Failed"

stdin,stdout,stderr = ssh.exec_command("ls /etc/")

for line in stdout.readlines():
        print line.strip()

After importing paramiko, we create a new variable ‘ssh’ to hold our SSHClient. ssh.set_missing_host_key_policy automatically adds our server’s host key without prompting. For security, this is not a good idea in production, and host keys should be added manually. Should a host key change unexpectedly, it could indicate that the connection has been compromised and is being diverted elsewhere.

Next, we create 3 variables, stdin, stdout and stderr allowing us to access the respective streams when calling ls /etc/

Finally, for each “\n” terminated line on stdout, we print the line, stripping the trailing “\n” (as print adds one). Finally we close the SSH connection.

Let’s look at another example, where we communicate with stdin.
Read the rest of this entry »

Simple IMAP Account Verification in Python

November 22nd, 2014

imaplib is a great library for handling IMAP communication. It supports both plaintext IMAP and IMAP over SSL (IMAPS) with ease. Connecting to an IMAP server is achieved as follows:

import imaplib

host = ""
port = 143
ssl = 0

	if ssl:
		imap = imaplib.IMAP4_SSL(host, port)
		imap = imaplib.IMAP4(host, port)
	welcomeMsg = imap.welcome
	print "IMAP Banner: %s" %(welcomeMsg)
	print "Connection Failed"

This results in the following output: “IMAP Banner: * OK [CAPABILITY IMAP4rev1 UIDPLUS CHILDREN NAMESPACE THREAD=ORDEREDSUBJECT THREAD=REFERENCES SORT QUOTA IDLE ACL ACL2=UNION STARTTLS] Courier-IMAP ready. Copyright 1998-2011 Double Precision, Inc. See COPYING for distribution information.” Now, to log in:


	loginMsg = imap.login(username, password)
	print "Login Message: %s" %(loginMsg[1])
	print "Login Failed"

With acceptable credentials, the response is: “Login Message: [‘LOGIN Ok.’]”. Lastly, to print a list of all mailboxes in the account:

	mBoxes = imap.list()
	for mBox in mBoxes[1]:
		print mBox
	print "Couldn't get Mail Boxes"

Read the rest of this entry »

DNS Black List / RBL Checking in Python

November 22nd, 2014

Following on from performing basic DNS Lookups in Python, it’s relatively trivial to begin testing DNS Block Lists/Real Time Black Lists for blocked mail server IP addresses. To assist in preventing spam, a number of public and private RBLs are available. These track the IP addresses of mail servers that are known to produce spam, thus allowing recipient mail servers to deny delivery from known spammers.

RBLs operate over DNS. In order to test a RBL, a DNS query is made. As an example, is a popular RBL. If I wanted to test IP address against the blocklist, I would reverse the octets in the IP address and then append ‘’, i.e. I then perform an ‘A’ record lookup on said host:

root@w:~/tmp# host -t a
Host not found: 3(NXDOMAIN)

Excellent. IP was not found in NXDOMAIN is returned.

Now, to take a known spammer’s IP:
Read the rest of this entry »

Performing DNS Queries in Python

November 21st, 2014

dnspython provides a detailed interface into DNS. In its simplest form, it’s possible to perform queries in only a couple of lines of code. Here’s a commented example:

import dns.resolver #import the module
myResolver = dns.resolver.Resolver() #create a new instance named 'myResolver'
myAnswers = myResolver.query("", "A") #Lookup the 'A' record(s) for
for rdata in myAnswers: #for each response
    print rdata #print the data

The results in my case are:

Read the rest of this entry »

Hard Drive Data Recovery

November 13th, 2014

This article discusses hard disk data recovery on Linux using dd and fdisk.

I recently left for a trip to South America, and took my trusty Intenso 320GB external drive with. Well aware that I’ve dropped it a couple too many times and that it was beginning to click more and more often during regular usage, I took a full backup before leaving. There’s nothing critical on the drive that I don’t have additional copies of elsewhere, however losing it would be a pain.

Having reached Madrid airport, I plugged the drive in and was about to pull some documents off it when disaster struck. The drive just clicked for about 30 seconds before Windows prompted me to format it. I tried removing it and reinserting it a couple of times but no luck – the drive had failed. I went to the duty free store in the airport and picked up a 1Tb WD Elements drive for 99 Euros, and planned to attempt data recovery when I arrived in South America.

I’m keen to get the data recovery started – it’s going to take a while on my USB 2.0 laptop and the more bad sectors, the longer it will take.

Read the rest of this entry »

Nginx, SSL & php5-fpm on Debian Wheezy

October 11th, 2014

I decided to take a break from my love affair with Apache and set up a recent development project on Nginx. I’ve seen nothing but good things in terms of speed and performance from Nginx. I decided to set up a LEMP server (Linux, Nginx, MySQL, PHP), minus the MySQL as it’s already installed on my VM host server, and plus SSL. Here’s the full setup tutorial on Debian Wheezy:

Step #1 – Installing the packages

apt-get install nginx-extras mysql-client
apt-get install php5-fpm php5-gd php5-mysql php-apc php-pear php5-cli php5-common php5-curl php5-mcrypt php5-cgi php5-memcached

MySQL can be installed into the mix with a simple:

apt-get install mysql-server

Read the rest of this entry »

Debian Wheezy Xen + Guest Howto

October 8th, 2014

Xen is usually my go to virtualization technology for Linux. Here’s a HOWTO on setting up Xen on Debian Wheezy and the first guest virtual machine.

First step is getting the required packages:

apt-get install xen-linux-system xen-tools xen-utils-4.1 xen-utils-common xenstore-utils xenwatch

Now, we’ll need to specify the Xen kernel as the default boot kernel on the host, and then reboot:
Read the rest of this entry »

MySQL Master-Master Replication, Heartbeat, DRBD, Apache, PHP, Varnish MegaHOWTO

October 8th, 2014

I created this HOWTO while building a new development environment today. The intention is to take a single Apache2/Varnish/MySQL environment and scale it to two servers, with one effectively a “hot-standby” – increase redundancy and continuity whilst maintaining current performance. This HOWTO is based on Linux Debian-76-wheezy-64-minimal 3.2.0-4-amd64 #1 SMP Debian 3.2.60-1+deb7u3 x86_64

Our current server has IP and our new server has IP

Section #1: Set up MySQL Master/Master Replication

First, we’ll set up MySQL master to master replication. In this configuration, data can be written and read from either host. Bear in mind that issues may exist with autoincrement fields when written to at the same time. There are other caveats with replication so ensure to research them along with how to deal with corruption and repair before considering this setup for a live application. Also be sure to be using the same version of MySQL on both servers – this may not always be necessary, however unless you are very familiar with any changes between versions, not doing so could spell disaster.

Read the rest of this entry »