Design Build Test Repeat

Voluntarily cast upon the waves of outrageous fortune, this lowly geek flops from peak to peak of the technological ocean. Linux, Windows, C, Ruby, Python and embedded systems all attempt to pull our stalwart hero down. Will he survive alone or will the lifeboat of corporate servitude be too tempting.

Steganography in Python

Posted by Diarmuid on November 7, 2007

I am doing allot of work in python at the moment. I’m using Turbogears, SQLObject, pyftpdlib, and the Python Imaging Library. These are a great combination and I feel that I am the most productive I have been in a while. I am storing my images on Amazon S3 with the security set to public read so that I can directly link to them on my website. The Image name includes date-time info. as well as a random component, so that they cannot be read directly (at least not easily) without knowing the file name exactly.

I want to store extra info with the image and toyed with EXIF data but I came across the concept of Steganography recently. It allows for the storing of text or image data within an existing image without the image of the original being affected (it is a little bit but you’d be doing well to notice). I am thinking that this could be used to store date time info, motion info, previous image names, etc. There is a nice class that does this simply. It is Stepic. It is a fairly basic implementation in that it does no encryption or compression of the data but is more than adequate for my needs.

The way that I use it is first to install the PIL then use it as below. I could not get it to work on windows with the command line but from within a class it is fine.

>>>import Image
>>>import stepic
>>>im =‘in.jpg’)
>>>s = stepic.Steganographer(im)
>>> im2 = s.encode(‘This is some cool text’)
>>> s = stepic.decode(im2)
>>> data = s.decode()
>>> print data
This is some cool text

Posted in Uncategorized | Tagged: | 6 Comments »

Python FTP Server using Turbogears as an “Authorizer”

Posted by Diarmuid on November 6, 2007

I need a Python based FTP server for my Turbogears project and was trilled to find the pyftpdlib project. It allows the creation of an ftp server in less than 10 lines of code. I just posted the following to their discussion board as a thanks for all the help those guys have given me.

This is my contribution to the excellent pyftpdlib project. I hope it in some way repays my debt to the project. I am cross posting this to my blog.

I needed an FTP server that would piggy back off my turbogears installation. The system allows an admin user to easily create new Users and for those Users then to upload files to a server using an FTP client.

I have User Object in my Model with a validate_password method. There is no encryption in this example but you would obviously create an MD5 hash and store that in the database.

class Model(SQLObject):
creation_date = DateTimeCol(
description = UnicodeCol()
displayname = UnicodeCol()
uploadname = UnicodeCol()
uploadpassword = UnicodeCol()
archived = BoolCol(default=False)
person = ForeignKey(‘Person’)
idx_owner = DatabaseIndex(‘person’)

def to_dict(self):
return{‘id’:, ‘description’: self.description}

def validate_password(self, password):
return self.uploadpassword == password

The main server code is as follows. Pardon any mistakes and any advice would be welcome. One thing is that the home directory is created on first use.

This server uses the Turbogears whatwhat model for authentication.
The Authorizer used checks a user’s
existence and validates the password.
The working directory is based on a default directory and the
username, which is checked for uniqueness when it is created.
The working directory is created if necessary.

The user has by default write rights as well as read for directory
but not for download.

The config file for the database may be passed at the command line
but defaults to dev.cfg.

The script must be run from the main whatwhat directory.

import os
from pyftpdlib import ftpserver
from whatwhat.model import User
import pkg_resources

defaultworkingdirectory = ‘c:\\temp\\images\\’

import cherrypy
import turbogears
from os.path import *
import sys
if len(sys.argv) > 1:
elif exists(join(dirname(__file__), “”)):

class TurboGearsUserAuthorizer(ftpserver.DummyAuthorizer):
def validate_authentication(self, uploadname, uploadpassword):
“””Return True if the supplied username and password match the stored
userlist = == “%s” %uploadname)
if userlist.count() == 1:
return userlist[0].validate_password(uploadpassword)

return False
def get_home_dir(self, username):
“””Return the user’s home directory.”””
workingdirectory = “%s%s” %(defaultworkingdirectory,username)
return workingdirectory

def has_user(self, uploadname):
“””Whether the username exists in the virtual users table.”””
userlist = == “%s” %uploadname)
if userlist.count() == 1:
return True
return False

def get_msg_login(self, username):
“””Return the user’s login message.”””
return “Login message”

def get_msg_quit(self, username):
“””Return the user’s quitting message.”””
return “Quit message”

def r_perm(self, username, obj=None):
“””Whether the user has read permissions for obj (an absolute pathname
of a file or a directory)”””
return True

def w_perm(self, username, obj=None):
“””Whether the user has write permission for obj (an absolute pathname
of a file or a directory)”””
return True

authorizer = TurboGearsUserAuthorizer()
ftp_handler = ftpserver.FTPHandler
ftp_handler.authorizer = authorizer
ftp_handler.passive_ports = range(60000, 65535)
address = (”, 21)
ftpd = ftpserver.FTPServer(address, ftp_handler)

Posted in Uncategorized | Tagged: | 9 Comments »

Euro Symbol in HTML

Posted by Diarmuid on July 20, 2007

I have my laptop since I bought it in Australia. So there is no Euro Symbol. I get by in Word etc with auto-correct but HTML is more awkward. I don’t really want to change the keyboard style as I am used to it. Also Ewan used his little fingers to rip off half the buttons. I have since replaced the keyboard with UK one of ebay. So things are messed up.

In creating these blog entries in wordpress, I use € in “code” view. It looks fine and should work in most browsers.

Posted in Uncategorized | Leave a Comment »

Meraki – C programming

Posted by Diarmuid on May 1, 2007

I am OK with c ( no guru, that is for sure, but I can edit and change. Starting from scratch is another thing) and have done some work on ARM chips, the Olimex boards from Sparkfun and the Make Controller, but I have not done much on linux and nothing with the Atheros boards, those the meraki is based on. And the meraki does not have any nice serial command line programs that would make it easy to use the serial port from Ruby, etc. So I thought I would take the plunge and try to get a program to compile that would.

First off Helloworld.c

printf (“Hello World!\n”);
Simple but it will prove if this thing will work.

This you would normally compile using gcc -o hello helloworld.c

Then I downloaded the OpenWRT tar file supplied by Meraki on
tar zxvf openwrt-meraki.tar.gz
Then I followed the instruction in meraki.README. Mine, is a build machine so I have a tonne of dev apps and libraries on it. You will need all the usual, build-essential,etc and then the list in README in the OpenWRT directory.
You need to have installed gcc, binutils, patch, bzip2, flex, bison,
make, gettext, pkg-config, unzip, libz-dev and libc headers.

You will also need ruby as the Meraki system uses that, and seems to use ruby as a scripting tool during the compile.

Whatever happened, the make command blew up and did not complete. However, it had got through the Buildroot and toochain builds.
So, I had a look around and came across in this folder /home/dev/openwrt/openwrt-meraki/openwrt/staging_dir_mips/bin (my user is called dev) an exe called mips-linux-uclibc-gcc-3.4.6
A quick “file mips-linux-uclibc-gcc-3.4.6” showed that it is an i386 exe. So on a whim I tried
./mips-linux-uclibc-gcc-3.4.6 -o ~/test/hellomeraki ~/test/helloworld.c. It ran without an error and when I ran file ~/test/hellomeraki it showed that it was a mips exe!!!
dev@dev-desktop:~/test$ file hellomeraki
hellomeraki: ELF 32-bit MSB executable, MIPS, version 1 (SYSV), dynamically linked (uses shared libs), not stripped

I used scp to send the file to /storage
scp hellomeraki meraki@myipaddress:/storage/hellomeraki .
And then I ran it by typing ./hellomeraki.
Too easy!!
Now for the hard part. Writing the code to communicate with the serial port. Really what I want is a commandline tool that will write what ever I give it to the serial line and then close.
By the way, there is a huge gotcha here . The Meraki uses the serial line like a screen so the line is allready in use. I wondered at this point whether anything serial line programs would work, but i reasoned that since there seemed to be different programs writing to it at the moment then a new program would be able to also. ….. But what about incoming. Mmmmmm. Too much thinking is bad so carry on regardless.
I work with the excellent Fox board from

Fox Board - Runs Linux , uSB, WIFI, Ethernet, Cool.

They have a great website full of tutorials (in English). It only runs kernel 2.4 but supports wireless and USB. I recommend it for those that want a bit more from their embedded system. They have a sample c program that they use with their web based compiler. This article includes a link to a c file that they compile for their board. I wondered WHAT WOULD HAPPEN IF I TRIED IT LIKE HELLOWORLD!!! You can tell I was getting excited now. Anyway it compiled and ran like a dream on the Meraki.

Showing serial_test ( a c program ) running on the Meraki through Putty
I ran it with ./serial_test /dev/ttyS0 and when I typed in my SSH session the text appeared in the console app I had connected to.

Showing the Meraki booting through the Serial port

So that shows that I can get serial data out of the meraki. But, alas, it does not work the other way. Typing in the console program is accepted by whatever is latched on and accepting linux commands. This does mean however, that I can run any linux command I want from whatever is attached to the Meraki and I can get data in return. So actually, my custom program is not required. Great stuff.

A note on my serial connection. I am using the Fox console board from Acme. Fox Console Board - Used for getting Serila data from the Meraki

I have it wired up as in the photoMy setup showing the serial link between the Meraki and my PC

The meraki pins are left to right with the antenna point away from you, Vcc, RX, TX, VDD (3.3 Volt). The Fox console is (with the serial jack pointing away) VCC, NA, TX, RX, NA, VDD. A nice little LED lights when all is good. So I wired up live to live, gnd to gnd , tx to rx and rx to tx. The baud rate is 115200. Seemingly, Meraki sell a similar board but it is not on their website at present. For now, I suggest that you get a board from Acme.

Posted in Uncategorized | 2 Comments »

Meraki – Custom ruby pages

Posted by Diarmuid on May 1, 2007

I have spent over a day now hacking into the Meraki and it’s pretty cool. As mentioned before, it runs a web server, has Ruby, perl, shell and (very importantly) serial access. Generally the serial access will be used to reflash your meraki using new images from Meraki Corp. or from OpenWRT if you are so inclined. I am not. At least not for now. I like what the Meraki does and don’t particularly want to trash it.

Therefore, I am going to try to work with what is there. They don’t make it too easy. Most areas of the meraki seem to be overwritten, including the web pages (Damn you! Damn you all to hell),and /etc/init.d.

I am assuming that you can SSH into your Meraki. Do that. Use Putty from windows. Then type ls -liah –color. The two directories of most interest are /storage and /www. Storage holds some config files for dropbear ( the SSH server), Boot_count( which is a good indication that this is not over written) , etc.

The www directory holds the static web pages (ccs, html, js), cgi (perl and ruby) and a number of directories for admin stuff. You can create files in here fine using vi and they will display if you browse to them, but they will vanish after a reboot and ( I think) after a preset period of time. So if we are to display our own pages, a solution is to be found that makes the pages permanent.

For now let’s experiment.
Go to the www directory by typing cd /www. Then create a file using vi by typing vi simplecgi.rb. Type in the following.
A simple CGI using Ruby
There are some nice vi tutorials out there but it’s dead easy. When presented with a file to edit, type i or a to insert or append text; backspace works but delete does not; use Esc to change commands. Use :wq to write and quit or :quit! to quit without saving.

Now browse to the relevant ip address in your browser and append on simplecgi.rb. You should get the following.
Simple cgi result html

Images map be downloaded (assuming you have a web server) by using wget . Easy, Peasy, Lemon , Squezzy.

But still there is that problem that the files are deleted on reboot. The solution lies is a facility that Meraki have included that allows them to append commands to the startup routine, even though the /etc/init.d area is over written. Because of copywrite issues, I am not going to print any of their code, but basically if a file called /storage/ exists, it is run by the meraki startup routine.
My file looks like this Used to republish websites
There is another way, and that is to add some lines to the config file of the web server. That is is /etc/lighttpd.conf. But that is for another day.
Another example is one that allows you to append text to a file from a web page.
Code to save text to a file from a web page
And now if you want to read the file into the web page
Code that shows how to read a file and display it in a web page
So I hope that you enjoyed this little tutorial. There are loads of possibilities
In my next tutorial i will show how to write custom c programs for the meraki.
Please leave some comments below.

Posted in Meraki | 1 Comment »

Meraki – Linux Everywhere

Posted by Diarmuid on April 27, 2007

The meraki runs Linux. So what? There is loads of activity over at Open WRT where they are porting kamakazi over to the Atheros chip set. I am a big fan of this project and obviously so are the gurus over at Meraki as they seem to have used Open WRT as of April 2006 as their starting point.

However, There are still some things that one can do with your new Meraki that don’t require a full reflashing of the box. There is a built in web server, SSH, SCP, ruby and bash for a start. To use the web server just browse to the box in your web browser. I used nmap to find out the available ports.

dev@dev-desktop:~/meraki$ nmap

Starting Nmap 4.20 ( ) at 2007-04-27 15:38 IST
Interesting ports on
Not shown: 1693 closed ports
22/tcp open ssh
53/tcp open domain
80/tcp open http
81/tcp open hosts2-ns

Nmap finished: 1 IP address (1 host up) scanned in 2.252 seconds

Here, domain refers to DNS. It’s a common enough ploy to get arround firewalls which may block 5190 (the more common DHCP port) .

hosts2-ns – The HOSTS2 Name Server.

To use SSH ( from windows use putty) just use meraki@ . The password is the serial number on the back on the meraki node. It is the third line from the top and has the form xxx-xxx-xxx.
dev@dev-desktop:~/meraki$ ssh meraki@
meraki@'s password:
Permission denied, please try again.
meraki@'s password:

BusyBox v1.1.0 (2007.03.05-17:58+0000) Built-in shell (ash)
Enter ‘help’ for a list of built-in commands.

Welcome to your meraki mini. Please look for developer information at We would like to encourage you to play with this
platform and add your own features to it. However, our lawyers
require us to tell you that much of the software on this device is
protected by copyrights, and may not be redistributed or sold.

Like hacking on wireless stuff? Come work with us!

Happy Hacking!

I like this message. They are developing a product and have invested significant amounts of time in it. For some of the team’s earlier work look at Roofnet. There is some debate online about the use of linux in closed devices (called Tivosationafter the use of Linux by Tivo). Personally I feel that if the system starts to involve hardware the the company is entitled to not opening up all their own source. Of course if they extend GPL’d software that has to be released as well. You will find however, that most Linux software that can be run from the command line, can be called from within a Python, Ruby, PHP or shell script. The Meraki team have at least opened up SSH, something that FON has not.

That’s why I’ll take them at their word and show how you can use the Meraki as a home server. I won’t be publishing their code and won’t be reverse engineering their system. The problem is that while you can add files to the home page of the node, that area is written over when the node is rebooted. So it won’t be that simple. (Solved, see next post)

I have to confess that I am developing a network attached product that will be managed centrally. I am using the Fox board from The development of web pages for it is similar to development on the Meraki except that ruby is not installed on the Fox. I am using TurboGears on the web side and PHP on the fox. I’d use the meraki but it does not have USB

Posted in Uncategorized | 2 Comments »


Posted by Diarmuid on April 27, 2007

My set up at home is as follows.
Main House has the ADSL modem/router/wifi Netopia unit. The office computers are all attached directly using cable to the 4 port router. I have the laptop, the Ubuntu workstation, and at various times network attached micro controllers Fox Board and Max Port. The Cottage (where I live) is 30 meters away. There I have a Mini Mac with a Dlink USB WIFI adapter. The link is intermittent and I have tried various enhancers and extra antennas with no permanent improvment. There is an apple tree in the line of sight and when it rains the signal falls off. Also now that spring is in full swing, the tree is filling up nicely so that can’t be helping. I looked around for a booster and came across Meraki nodes. These allow anybody to easy extend the range of their existing network. They work by acting as re-broadcast units to allow users to daisy chain their way back to the internet access point. They will automatically set up a mesh network. So it seemed that these would do the job.

I bought 4 of the $49 nodes. They arrived in West Cork, Ireland in 2 days with UPS. Amazing. Anyway, they come with no documentation at all. Contents of each box is

  • 1 Meraki node
    1 crappy small antenna.
    1 rubber gromet to cover the antenna socket
    2 rubber suckers for sticking to the window
    2 sticky backed thingies for sticking to a wall
    1 US wall adaptor
    And (very impressive) an Irish 3 pin wall adaptor.
  • I checked the website documentation and there was no getting started guide. I was at a loss to know what to do, so I plugged in one of the nodes to power and tried to browse for it. Nothing. I thought it would find my existing wireless and extend that. Apparently no. I went to the and added the node. Still nothing. So I plugged one into the ADSL router and added it to Dashboard to see what would happen. After a minute or so it appeared on the WIFI network list.

    My setup on Dashboard showed that the two nodes were connected. But the Dlink unit would not consistently find the Meraki in the cottage, even when they were only a few feet apart. So now I have 2 Meraki nodes and the signal is still less than that put out by the Netopia unit. I think it may be the Dlink as it has not behaved itself before. I bet it is dodgy on OS X. (update, It is the crappy dlink. My XP laptop was rock solid on meraki).

    The antennas are all the same, except the Meraki ones are translucent, so that you can see that the actual antenna inside only takes up half the length of the plastic. I had a look at the antennas that came with my Maxstream Zigbee nodes and they are the same. Some factory in China is doing well.

    The Dashboard is a super looking piece of (what looks like) Rails inspired frugality. It is white, neat, simple, AJAX’d and clever. I am developing a system at the moment to remotely control network attached devices. I have developed most of it in TurboGearsand it looks similar, but I would love to know how they get the remote ping to work, without the router having to be modified to use IP pass through or NAT.

    Keep posted for more.

    Posted in Meraki | 13 Comments »

    Twitter : Technology circa 1996 – Business Model circa 2001

    Posted by Diarmuid on March 12, 2007

    Honestly, it’s driving me insane.

    Every where I look there are more and more twitterings about Twitter. How great it is, mobile web killer, IM for the masses. It’s tedious. Here is what it is. A website, a database, an SMS short code and ….. loads of cash. They receive texts and then send them for FREE to your mates. Now, I will guarantee you that a thousand engineers looked at developing something similar ( me included,I was using Derdack 10 years ago) back in the glory days, but even then I could not show how the thing would make money. You can buy bulk (100,000 + ) SMS from clickatell for 3 cents each. So if you have 100,000 people sending 5 texts per day to 5 friends that is a burn of 75,000 per DAY. Getting users to buy messages in bulk (skype, jajah) would work, but only 2 or 3 percent of users will do that .

    So please , explain how this is anything other than a Web 1.0 giveaway? Maybe once it has achieved critical mass, it will start charging, but they would need to get the numbers up first. The server is grinding to a halt so maybe it is having problems scaling.


    Posted in Uncategorized | Leave a Comment »

    GPS on Mobile Phones

    Posted by Diarmuid on March 9, 2007

    The marriage of GPS and Phones seems like a perfect combination. Personal GPS technology has been arround for at least 10 years but never took off in areas other that outdoor pursuits (walking, geocaching, etc). As a long time user of a hefty Garmin 12XL I can see why. The screens were tiny, grey scale colors, inty-bitty icons and negligable features other than waypoints, back track etc. To use the unit with a pc requires a special (and expensive cable) , software like Microsoft mappoint (or a roll your own ) and allot of time. And after all that you got to confirm that you were where to knew you were, or that indeed that road is full of bends. Hardly inspiring stuff.

    That is all changing. While the units are still pretty mundane, there is a different path and it’s blue. Bluetooth enable GPS units have been around for over 2 years and have changed personal GPS technology as well as the automotive sector. These are small (2″*1″*.5″) battery powered GPS receivers whose only job is to broadcast their position to anything within 5 meters (technically 10 but that never happens). Then another bluetooth enabled device can pick up this data and use it for its own nefarious purposes … like showing your position on a map. Amazing!! The advantage of course is that the little bluetooth yoke can be out where it can see the sky ( satellites if you want to get technical) while the display component can be viewed where it is handy (given the 5 meter limit).

    OK, so what sort of display devices are out there. 2006 has been a good year for this sort of tech and Christmas was a pumper season for GPS. 3GSM was big on GPS phones.  Very functional GPS devices are for sale for $300. These have a color screen and a memory card to store map data. The processing power of this little devices is impressive. They have 300Mhz chips and 256MB memory. That’s like the Dell you bought new in 1998. However, most of these devices will be left in cars except every now and then when they will be stolen off the dash. Impressive for those runs from O’Hare to wherever, but hardly something that the average Joe programmer is going to make any money from.

    GPS software for PDA’s have been arround since the Psion 5 and is as popular now as it was then, i.e. not. It basically is the same problem. The device tells you where you are or how fast you are going but so what. It might have a database of Points Of Interest (POI), so might tell you that there is an ATM 2 miles away but that sort of data gets old fast and systems that you can subscribe to are expensive.

    Arguably the only area where small companies have prospered in the GPS market is in Tracking. Fleet management systems allow a business owner to see where his vehicles are. This is done by combining GPS modules with GPRS modules into a single package that periodically sends its position to a central system. the on board processor can also identify when a device is moving or moving above a speed limit and can broadcast it’s position. This data is then displayed on a map in the company office. This market is getting very big, so much so that a manufacturer of GSM/GPRS modules is making one with a GPS built in. This decreases the bill of materials and should make these devices cheaper.

    However, nearly everyone has a mobile phone, and if that phone has bluetooth and java (generally if it can run games , it is java enabled) it can work as a GPS. If it has a GPRS connection (data enabled , not wap) it can send its position back to a server and receive data).

    Part 2

    Personal tracking devices have been arround for a while and have had limited success. There is a market for tracking alziemers sufferers but other areas are more problematic ethically. Tracking your kids? your spouse? Unless they agree up front, there are significant issues and even if they agree does it really solve a problem. You end up knowing where that person is, but you could just as easily ring them. If you can’t trust them the you have bigger problems.

    … I’ll finish this soon

    Posted in Uncategorized | Leave a Comment »

    OpenID: Don’t we all have a unique adress anyway?

    Posted by Diarmuid on March 9, 2007

    OpenID is gaining traction, and anything that can eliminate the headache of having to re implement a password management system for every website , has to be a good thing. There are implementations of it for most web languages (while there is some moaning that the dot net imps. are a port of a python version) and a number of hosters.

    My whinge about it is why not just use  email? The theory behind OpenID is that you own an Uri that is unique to  you. You go to the website you want to log into, type in this url, it redirects you to the provider, you enter credentials, are logged in, then redirected back to the original site again. This is similar to the pay pal payment redirection model.

    Let’s take the example of how user using a conventional authentication system gets a password. They enter the username and password and then an email address (most sites now just use email as your username). They are sent a “click this url to prove this is you” email ,whicch they  do and they are in. If they lose their password, they go through the same process again. Does this not show that the ultimate authentication mechanism is email.

    Now, I  know there are some issues with this as email is not  instantaneous if you are using a pop based email or outlook, etc, but for most users , and all users if they would open a web based email, it’s pretty close to instantaneous. It could be abused by users sending login emails to other users, but that is not any different from current password reset methods.

    So my call to  web developers, explain why an email based authentication would not work

    1. User goes to
    2. User types in email address
    3. send an email  to
    4. The mail will contain a url like
    5. At the same time the site stores the relationship between this unique id and the email address
    6. User goes to email site, and opens up the email from
    7. They click on the url and are take to the website where the back end code validates the id is genuine and logs the user in.
    8. At that point the user has the opportunity to modify the data relating to them, like display name, interests etc.
    9. The next time they log in, the same thing happens

    Now, I think that would work, and I’m going to try it with a new website I am building in the area of citizen empowerment. Should be interesting.


    Posted in Uncategorized | 10 Comments »