Browsing Tag

# python

Mose Code is an old method of transmitting text messages. Morse Code dated back to around mid 1800s. The basics of morse code is timing and pulse. Each combination corresponds to an Latin alphbet character or number. Morse Code can also be transmitted visually using a light source by alternating the light on and off. If you are in need of more information about Morse Code I suggest Wikipedia Morse Code entry a a good starting place.

So why implement morse code? Well, because we can. Programming Morse Code is very easy and a great challenge for beginners. I also found very little resource that implemented Morse Code with sound. There is a lot of code around the web to print out Morse Code in python, but I could not find one that outputs sound. As an added benefit you get exposed to something you might need in the future. Morse code is still in use and may save your life. (I know, this is a little extreme). We represent Morse Code on the screen by a dots and dashes. Dots are one unit length, while dashes are three. Here is an example of SOS, the international distress signal:

Although there is an international agreement about SOS signal, some of symbols in Morse Code have no standard. For the scope of this article let us limit our discussion to the International Morse Code, which is limited to alphabet and the numbers 0 to 9. As you will see, the program can be modified to represent more symbols to your wishes. Here is the guideline to Morse Code as we will use it:

Let us deal with the latter part first, the letters and digits. We will get to the top part in a little bit. Different languages have different data types at you disposal. Python has a tool that fits this problem like a glove, dictionaries. You can read up on python’s dictionaries documentation or download python2.6 documentation. The idea behind dictionaries is that we can map keys to values. Lists in python are numeric indexed from 0 up (lists are also reversed indexed). Dictionaries store data by assigning a key and a particular value. So for example we can initiate an empty dictionary and store the value ‘.-‘ for the key ‘A’. Then we can retrieve the value for key ‘A’ via indexing.

We can also initiate a dictionary with values:

Note that since there was no value for key ‘C’ we got an error. If we want our program to run correctly, we will need to address this issue, but not at the moment. So using a dictionary for our alphabet and numbers and using python’s string properties we can write a fairly simple program that will output text Morse Code. Lets look at the basic implementation of Morse Code:


CODE = {'A': '.-',     'B': '-...',   'C': '-.-.',
'D': '-..',    'E': '.',      'F': '..-.',
'G': '--.',    'H': '....',   'I': '..',
'J': '.---',   'K': '-.-',    'L': '.-..',
'M': '--',     'N': '-.',     'O': '---',
'P': '.--.',   'Q': '--.-',   'R': '.-.',
'S': '...',    'T': '-',      'U': '..-',
'V': '...-',   'W': '.--',    'X': '-..-',
'Y': '-.--',   'Z': '--..',

'0': '-----',  '1': '.----',  '2': '..---',
'3': '...--',  '4': '....-',  '5': '.....',
'6': '-....',  '7': '--...',  '8': '---..',
'9': '----.'
}

def main():

msg = raw_input('MESSAGE: ')

for char in msg:
print CODE[char.upper()],

if __name__ == &quot;__main__&quot;:
main()



You will notice that we use a string method named upper() to force the input to match our keys. There is a big difference between ‘A’ and ‘a’ in computer programs, they two different keys. So now that we have a basic python morse code script, let us try to improve on it. In our first program we really cannot transfer more than one word. All the words will be concatenated together. According to our code from above we need to modify our script to the following rules:

1. The length of a dot is one unit
2. A dash is three units
3. the space between parts of the same letter is one unit
4. the space between letters is three units
5. the space between words is seven units

Until we move on to sound, the first 4 rules are implicitly implemented. What we really need is a space between the words. To do this we will add 7 space for every 1 space in the input, like so:


CODE = {'A': '.-',     'B': '-...',   'C': '-.-.',
'D': '-..',    'E': '.',      'F': '..-.',
'G': '--.',    'H': '....',   'I': '..',
'J': '.---',   'K': '-.-',    'L': '.-..',
'M': '--',     'N': '-.',     'O': '---',
'P': '.--.',   'Q': '--.-',   'R': '.-.',
'S': '...',    'T': '-',      'U': '..-',
'V': '...-',   'W': '.--',    'X': '-..-',
'Y': '-.--',   'Z': '--..',

'0': '-----',  '1': '.----',  '2': '..---',
'3': '...--',  '4': '....-',  '5': '.....',
'6': '-....',  '7': '--...',  '8': '---..',
'9': '----.'
}

def main():

msg = raw_input('MESSAGE: ')

for char in msg:
if char == ' ':
print ' '*7,
else:
print CODE[char.upper()],

if __name__ == &quot;__main__&quot;:
main()



You can also modify the program to print a newline after each word and more spaces between letters. This will look like this: (full morse code code)

for char in msg:
if char == ' ':
print
else:
print CODE[char.upper()] + '  ',



Before we move on to sound, there is one more thing I would like to address, errors. Since we do not have all possible input, our program might raise and error. Foe example, if we tried to convert and email address to Morse Code we will have an error since ‘@‘ and ‘.’ are not keys in the dictionary. Here is what will happen if we tried to convert ’cpt@thelivingpearl.com’ to Morse Code:

That is not good. You can see that the program started to translate, but got stuck along the way. There are several way to handle this, including non at all. A simple way to solve this issue is by scanning the output and checking its values against the dictionary keys. We can do this by using a keys() function of our CODE dictionary. If we find values that we cannot translate to Morse Code, we will notify the user and terminate the program.


import sys

CODE = {'A': '.-',     'B': '-...',   'C': '-.-.',
'D': '-..',    'E': '.',      'F': '..-.',
'G': '--.',    'H': '....',   'I': '..',
'J': '.---',   'K': '-.-',    'L': '.-..',
'M': '--',     'N': '-.',     'O': '---',
'P': '.--.',   'Q': '--.-',   'R': '.-.',
'S': '...',    'T': '-',      'U': '..-',
'V': '...-',   'W': '.--',    'X': '-..-',
'Y': '-.--',   'Z': '--..',

'0': '-----',  '1': '.----',  '2': '..---',
'3': '...--',  '4': '....-',  '5': '.....',
'6': '-....',  '7': '--...',  '8': '---..',
'9': '----.'
}

def verify(string):
keys = CODE.keys()
for char in string:
if char.upper() not in keys and char != ' ':
sys.exit('Error the charcter ' + char + ' cannot be translated to Morse Code')

def main():

msg = raw_input('MESSAGE: ')

verify(msg)

for char in msg:
if char == ' ':
print ' '*7,
else:
print CODE[char.upper()],

if __name__ == &quot;__main__&quot;:
main()




import pygame
import time
pygame.init()

pygame.mixer.music.play()
time.sleep(1)



This script will output the letter ‘A’ in morse code. Make sure you have your speakers on. Here is how are program will look like with the added sound files.


import sys
import pygame
import time

CODE = {'A': '.-',     'B': '-...',   'C': '-.-.',
'D': '-..',    'E': '.',      'F': '..-.',
'G': '--.',    'H': '....',   'I': '..',
'J': '.---',   'K': '-.-',    'L': '.-..',
'M': '--',     'N': '-.',     'O': '---',
'P': '.--.',   'Q': '--.-',   'R': '.-.',
'S': '...',    'T': '-',      'U': '..-',
'V': '...-',   'W': '.--',    'X': '-..-',
'Y': '-.--',   'Z': '--..',

'0': '-----',  '1': '.----',  '2': '..---',
'3': '...--',  '4': '....-',  '5': '.....',
'6': '-....',  '7': '--...',  '8': '---..',
'9': '----.'
}

ONE_UNIT = 0.5
THREE_UNITS = 3 * ONE_UNIT
SEVEN_UNITS = 7 * ONE_UNIT
PATH = 'morse_sound_files/'

def verify(string):
keys = CODE.keys()
for char in string:
if char.upper() not in keys and char != ' ':
sys.exit('Error the charcter ' + char + ' cannot be translated to Morse Code')

def main():

msg = raw_input('MESSAGE: ')
verify(msg)

pygame.init()

for char in msg:
if char == ' ':
print ' '*7,
time.sleep(SEVEN_UNITS)
else:
print CODE[char.upper()],
pygame.mixer.music.play()
time.sleep(THREE_UNITS)

if __name__ == &quot;__main__&quot;:
main()



Here is a short video demonstrating ‘SOS’ in Morse Code:

Here is another video transmitting ‘HELLO WORLD’ in Morse Code:

As a last note. We have gone through how to successfully encode and transmit a message into Morse Code. That is only half the problem. To decode a text message in Morse Code, you can reverse the dictionary and have the Morse Code as keys and alphabet as values. This is rather trivial and you can see this as a challenge. To decode the sound waves, this will require more effort. Maybe one day I will have the time to write some code for that. For now, you can write your own code to decode morse code, or you can decode it using the image from the top of this post. Either way, this was fun.

To mathematicians prime numbers represent a certain Nobel challenge. They have been studying prime numbers for a while, trying to learn their properties. Some mathematicians actually made progress. Others have gone insane trying to prove a pattern or how to generate the nth prime. Some are actively looking for the largest prime they can find. For computer scientists and programmers prime numbers have properties that can be utilized in many areas. Cryptography and pseudo-random number generators are 2 examples applications of prime numbers. Let us take a dive into the world of prime numbers. We will look at how we can generate and validate prime numbers in python.

Any valid math discussion must begin with a definition. Here is a definition of prime numbers I like to use:

“A natural number p > 2 is called prime if and only if the only natural numbers that divide p [without reminder] are 1 and p. A natural n > 1 that is not prime is called composite. Thus,, n > 1 is composite if n = ab, where a and b are natural numbers with a > 1 and b < n. “ Discrete Mathematics with Graph Theory

What does this means? It means that a number, for example 17, is prime if and only if it can be divided only by 17 and 1. On the other hand, 18 is not prime. 18 is a composite number that can be written as 2 x 9 or 3 x 6. This will prove to be a very useful property.

So how do we find prime number? Well, we do not. Or not exactly. Since the natural number are infinite and a prime number can be any number greater then 2, by definition there are infinite prime numbers. Many mathematicians have tried to prove a formula or correlation among prime numbers. None to date have succeeded. The best we can do is to generate a number and check if it is prime. However, there are some tricks we can do to speed things up, but we will get to that.

Here is our basic problem. We want to generate all the prime numbers between x and y. For starts, let us think about the numbers between 2 and 100. The straight forward, brute force, approach will be to check every number in that range and see if any number up to it can divide it without a reminder. If we find such number then it is not prime. Let us take a look at a simple program that can do this for us:


import sys

def is_prime(num):
for j in range(2,num):
if (num % j) == 0:
return False
return True

def main(argv):

if (len(sys.argv) != 3):
sys.exit('Usage: prime_numbers.py <lowest_bound> <upper_bound>')

low = int(sys.argv[1])
high = int(sys.argv[2])

for i in range(low,high):
if is_prime(i):
print i,

if __name__ == "__main__":
main(sys.argv[1:])



And the output for prime numbers between 2 and 100:

Now for some tricks. First, we don’t have to check every second number. All the even numbers will be divisible by 2 by definition of even numbers. So our first loop can be rewritten to jump by 2 and start at an odd number. That way we are just checking the odd numbers. By this hypothesis, all odd number are potentially a prime number. This takes away half the numbers we need to check. If we were to ask for all the prime numbers between 100 and 10,000 it might take a while. Lets look at how long it will take my computer (Intel core duo 2 2.53GHz with 8GB ram) to run this program using the unix time command.

The original script from before:

Modified script to take out the evens numbers:


import sys
import math

def is_prime(num):
for j in range(2,num):
if (num % j) == 0:
return False
return True

def main(argv):

if (len(sys.argv) != 3):
sys.exit('Usage: prime_numbers2.py <lowest_bound> <upper_bound>')

low = int(sys.argv[1])
high = int(sys.argv[2])

if (low == 2):
print 2,

if (low % 2 == 0):
low += 1

for i in range(low,high,2):
if is_prime(i):
print i,

if __name__ == "__main__":
main(sys.argv[1:])



This modification cut the program time by nearly half, but we can do better. According to the Sieve of Eratosthenes we do not need to check if all the numbers will divide our potential prime, we only need to check up to the floor of the square root of the number. So our next version of the code will look like this:


import sys
import math

def is_prime(num):
for j in range(2,math.sqrt(num)):
if (num % j) == 0:
return False
return True

def main(argv):

if (len(sys.argv) != 3):
sys.exit('Usage: prime_numbers3.py <lowest_bound> <upper_bound>')

low = int(sys.argv[1])
high = int(sys.argv[2])

if (low % 2 == 0):
low += 1

for i in range(low,high,2):
if is_prime(i):
print i,

if __name__ == "__main__":
main(sys.argv[1:])


And the time to find all the primes between 100 to 10,000:

This is really great. We have improved our program by another half and nearly a quarter than the original run. Let us examine 2 (2 things) variations of this program. First we will calculate the nth prime number. Second we will try to get a random prime number for a given length.

In order to calculate the nth prime we will have to modify them main() loop into a while loop. We will also modify the program to accept only 1 number, the nth prime we want to calculate. Once we reach the nth prime we will print it out.


import sys
import math

def is_prime(num):
for j in range(2,int(math.sqrt(num)+1)):
if (num % j) == 0:
return False
return True

def main(argv):

if (len(sys.argv) != 2):
sys.exit('Usage: prime_numbers4.py <nth_prime>')

i = 0
num = 2
nth = int(sys.argv[1])

while i < nth:
if is_prime(num):
i += 1
if i == nth:
print 'The ' + str(nth) + ' prime number is: ' + str(num)
num += 1

if __name__ == "__main__":
main(sys.argv[1:])



Sample output:

For many application computing a prime number is not enough. For some we want a specific length of prime number. Lets say we need a 5 digit prime number. How are we to calculate this? The simple approach is to generate all 5 digits prime numbers and choose 1 of them randomly. Let us look at how to do this:


import sys
import math
import random

def is_prime(num):
for j in range(2,int(math.sqrt(num)+1)):
if (num % j) == 0:
return False
return True

def main(argv):

if (len(sys.argv) != 2):
sys.exit('Usage: prime_numbers5.py <num_digits>')

digits = int(sys.argv[1])
low = int('1' + '0' * (digits-1))
high = int('9' * digits)
prime_number_list = []

if (low == 1):
prime_number_list.append(2)
low = 3

if (low % 2 == 0):
low += 1

for i in range(low,high,2):
if is_prime(i):
prime_number_list.append(i)

print 'A random ' + str(digits) + ' digits prime number is: ' + str(random.choice(prime_number_list))

if __name__ == "__main__":
main(sys.argv[1:])



Output:

If you were to run the program, you will notice that the it will take longer and loner as n gets larger. I found that for any n grater than 6 digits, it is very demanding on the hardware. I do not recommend letting this program try to calculate a high digit prime. You will also notice if you try to ask for 10 digits, python will raise an error in the range function. So how do we compute n digit prime for a large n? One way I found is by using a random number and checking if it is prime. This seems to work reasonably well with numbers less than 15, but then again it is random.


import sys
import math
import random

def is_prime(num):
for j in range(2,int(math.sqrt(num)+1)):
if (num % j) == 0:
return False
return True

def main(argv):

if (len(sys.argv) != 2):
sys.exit('Usage: prime_numbers6.py <num_digits>')

digits = int(sys.argv[1])
low = int('1' + '0' * (digits-1))
high = int('9' * digits)
done = False

if (low == 1):
low = 2

while not done:
num = random.randint(low,high)
if is_prime(num):
print 'A random ' + str(digits) + ' digits prime number is: ' + str(num)
done = True

if __name__ == "__main__":
main(sys.argv[1:])



So how about really large primes? That is where time becomes an enemy and computational complexity goes up. I found a few modules that can help compute large primes in python but I have yet to try to run anything. Once I get a chance I will try to run them and put a post up. If you really need prime numbers I would advise you to either download a list of them or generate a list of primes and save them to a text file. Prime numbers will show up every once in a while. They come up in cryptography and other encryption methods. Most usage of prime numbers is due to the nature of the numbers. For more information about prime numbers I would suggest seeking out a mathematician, which I am clearly not.

A few months ago I was talking to a friend of mine about robotics and hardware. He mentioned something called an Arduino Uno board and I was intrigued. After doing some DIY search online I found out this little prototype platform has a whole family of boards. I also found out that the web is filed with neat things you can do with it. An Arduino cost about $15 on Amazon. I had to have one. So I typed in my password to Amazon and about a week later I promptly got my new toy. I immediately took it out of the little box, examined it and went through the Arduino tutorials. Since then my Arduino has been collecting dust. I bought it thinking if I had it I will find a use for it. Following “if you build it, they will come”. Nothing came to mind. That is until I was walking down an aisle at the store yesterday and I saw a neat orange looking remote controlled Lamborghini. It was on sale for$15. I bought it on the spot.

When I got home and opened up this new toy I was deeply disappointed. The remote for this car has to be an example of the worst human interface known to man. I know this remote comes with a lot of RC cars, but from some reason I just could not take it. I could barley control the thing. And then The idea came to me. How cool would it be if I were to control the car via my Bluetooth keyboard? After all, I have lot of experience driving with my keyboard from GTA. Hours later it turned out to be very cool indeed...

So what went in to this cool little thing? A night worth of work, and about 270 lines of code. You can download the code for Arduino CarControl, but unless you follow what I did on the hardware part you might find it useless. Here is the “big picture” overview of what I did. I took apart the remote control and wired the connection to the Arduino pins. Then I use python’s serial module (download pyserial) to communicate with the Arduino. I used PyGame (download pygame) to get key pressed information. And finally, I had to write the code that is on the Arduino to accept serial input and output the correct command to the remote control. It seems like a lot, but it is really fairly simple.

Here is a video of the whole thing in action:

Step 0: Get comfortable with the Arduino.

The Arduino is programmed using an its own language. It is a set of c/c++ and was really easy to pickup. You will need to download from Arduino’s site an IDE that was designed to communicate with the board. The software contains a few examples. At the bare minimum, you should try out the blink example.


// Pin 13 has an LED connected on most Arduino boards.
// give it a name:
int led = 13;

// the setup routine runs once when you press reset:
void setup() {
// initialize the digital pin as an output.
pinMode(led, OUTPUT);
}

// the loop routine runs over and over again forever:
void loop() {
digitalWrite(led, HIGH);   // turn the LED on (HIGH is the voltage level)
delay(1000);               // wait for a second
digitalWrite(led, LOW);    // turn the LED off by making the voltage LOW
delay(1000);               // wait for a second
}



If you were to wire up an LED to pin 13 of the Arduino and run the code, the LED diode will blink at fixed intervals. The main part I found useful was to understand the structure of the program.

* Before the first functions there are setup variables.
* After the variables there are functions.

There are two (2 things) functions that make the whole thing run. First there is a setup function. This will be executed as son as the Arduino powers up. The second is the main loop. Opposite from programming, electronic run on a while(1) loop. Any code you have in the loop function will run again, and agin, and again ...

Step 1: Taking apart the remote

For some people the thought of taking apart something brand new might be hard, not for me. I have been taking things apart since I was 3. This is what I found in the RC remote ‘under the hood’:

Here is a closer look:

Neat. The small board is wired to 4 things. Positive and negative are near the middle. On the right there is the antenna connection. Near the top-middle there is a red LED diode to indicate power-on. Note the positive wire run through a power switch, which is your remote control on-off switch. Here are some more photos of it disassembled from the remote housing:

One of the most imperative things I have learned from taking things apart is always stop and check your work. There is nothing more then going a few steps forward and then when a problem arises to trace back. There is no ‘control-z’ in hardware, so be careful. At this point I wired up to remote board to check that it still turns on. It did.

Being who I am, I also looked around for some housing so It will look nice. I found a pice of foam laying around that fit the task. I mounted the board and this is what I got.

Step 2: Reverse engineer the remote board.

This was the part I thought will be the most difficult. It turned out not as bad as I thought. I am not an electrical engineer by any means, but I will try to explain it. The basic idea is that an electronic circuit is either closed or open. When the circuit is open, electricity cannot power what ever is connected to the circuit. When the circuit is closed (and somewhere along the circuit there is a power source), electricity can move around the circuit and power everything along the way. What you will need to do is figure out how the board work. I managed to send a single to the car by emulating the original remote control switch using a small wire. In connecting the to spots in the picture I closed the circuit and indeed the car was moving.

Step 3: Send out a test single

In order for the Arduino to communicate with the board, you will need to find a better way to move the car. If you can try to follow the logic on the board to figure out which pins are associated with which command. There are a total of 4 command you can give to the car:

• Forward
• Reverse
• Left
• Right

The forward - reverse command and left-right can only be executed one option at a time. However you could say turn left and move forward at the same time. This came back to bite me later on in the first version. Since I am not an electrical engineer, I had to take the long route to figure things out. I checked each one of the top connectors until I found all four (4) movement controls.

Step 4: The Python code

The python part is really simple. Get keyboard input using PyGame Library, and output a command using the serial library. The complete carControl.py file is available. Let us look at the main() function:


def main():

clear_screen()

print '\nStarting Car Control v.0.3\n';

ser = serial.Serial('/dev/tty.usbmodem411', 115200, timeout=1);

pygame.init();
run = [True,0];
previous = -1

while run[0]:
run = getOrder(run);

#debug
#print 'current orders: ' + str(run[1]);

if (run[1] != previous):
previous = run[1];
ser.write(chr(run[1] + 65));
print run[1];

ser.close();



We start by opening a serial port to the Arduino. We do that using serial.Serial() passing in the path, bandwidth (and timeout is optional). The path to your usb device is found by running ‘ls /dev/tty.usb*’ in the terminal or you may find it on the Arduino IDE in the bottom right corner.

We continue to initiate PyGame and some variables and run a main loop. Notice the run variable contains 2 elements. The first indicates whether or not we should keep the loop running, the second is the command. For each iteration of the loop we write the appropriate command to the serial port using the write function. When we exist the loop we close the serial port and end the program.

The second function we need is getOrder. This is where we handle all the keyboard input cases. You can check out the complete code. Here is a snippet of what it looks like:


def getOrder(run):
for event in pygame.event.get():
if (event.type == KEYDOWN):
keyinput = pygame.key.get_pressed();

#complex orders
if keyinput[pygame.K_UP] and keyinput[pygame.K_RIGHT]:
run[1] = 21;
elif keyinput[pygame.K_UP] and keyinput[pygame.K_LEFT]:
run[1] = 22;
elif keyinput[pygame.K_DOWN] and keyinput[pygame.K_RIGHT]:
run[1] = 23;
elif keyinput[pygame.K_DOWN] and keyinput[pygame.K_LEFT]:
run[1] = 24;

#a few lines later...

return run



For each keyboard input we modify our run variable and at the end return it.

Step 5: The Arduino code

The Arduino part of the code is very similar to the python code. Let us take a look at it part by part. Here is a link to the complete car.ino file.


int reversePin = 9;
int forwardPin = 8;
int leftPin = 7;
int rightPin = 6;

int order = 55;
int time = 75;

//control flag
int flag = 0;



We have our output pins, similar to the blinking LED example. We have 4 output pins, one for each car movement. An initial order that is not a real order. A time variable for delay purposes and a flag to indicate the start of the python program. Now let us look at the setup function:


void setup() {
// initialize the digital pins as an output.
pinMode(rightPin, OUTPUT);
pinMode(leftPin, OUTPUT);
pinMode(forwardPin, OUTPUT);
pinMode(reversePin, OUTPUT);

Serial.begin(115200);
Serial.print(&quot;\n\nStart...\n&quot;);
}



We initialize the pins to be in output mode and initialize a serial connection. Now let us look at the main loop:


void loop() {

//Turn everything off...
off();

//get input
if (Serial.available() &gt; 0){
Serial.println(order);
flag = 1;
}

if(flag){
//complete orders
orderControl(order,time);
}



At the stat of every loop we turn everything off. If we have serial command we try to read the command. At the end as long as the flag has been raised, meaning we started the python program, we execute what ever command we got using orderControl(). This function takes a order number and a time. Then, using a switch statement, it invokes smaller functions that can execute simple, complex and demo commands.

Here is an example of a simple command function forward:


void forward(int time){
Serial.println(&quot;This is forward...&quot;);
digitalWrite(forwardPin, LOW);
delay(time);
digitalWrite(forwardPin,HIGH);
}



A more complex command leftTurnReverse:


void leftTurnReverse(int time){
digitalWrite(reversePin, LOW);
digitalWrite(leftPin, LOW);
delay(time);
off();
}



And a demo function:


void demoOne(){

int demoTime = 500;

forward(demoTime);
reverse(demoTime);
left(demoTime);
right(demoTime);
}



Notice that the functions uses reverse magnitude, LOW setting to activate a command. If you were to connect an LED to the command wire, you would need a not gate to invert the signal.

Step6: Wire everything togther

The final step is to put all the wires together, load the Arduino and run the python code. If everything goes well, it should work flawlessly.

After a night of paying around with the Arduino and the car I have to say this is one of the coolest things I put together. I think the same principles could apply for nearly any out of the box RC vehicle. Maybe I will get an air vehicle (Quad Copter) next time and test my flying skills using a keyboard. Or maybe I will try to get a joystick. In the mean time I have a cool toy setting in the corner that I can use to scare off my dogs when they start bothering me too much.

The answer to this question is short and simple, we do what we can. In this post we will look at how to randomly generate passwords and how to check their security level to some degree. So let us get to it. Generating passwords is something that is rather straight forward. To sum it all up, randomly choose X characters from a given set. We will look at a few ways to generate passwords starting from a simple one. All of them follow that same basic rule.


import random
import sys

def main(argv):

if (len(sys.argv) != 2):

for i in range(int(argv[0])):

if __name__ == "__main__":
main(sys.argv[1:])



Using the python random module and passing in via command line the length of the password, we are able to randomly generate a unique password. The passwords are built from all printable ASCII characters randomly chosen. Later we will come back to this example to test a checking password strength program. Here are some example runs of this program.

This is a nice and simple script anyone can write and by all means use and distribute. It might be a quick hack, but still is not enough to get to be a true generator. Before we move on to another, more sophisticated password generator, lets actually look at a really unique generator using python's lambda.


import random
import string

pass_gen = lambda length, ascii =  string.ascii_letters + string.digits + string.punctuation: "".join([list(set(ascii))[random.randint(0,len(list(set(ascii)))-1)] for i in range(length)])



To use this lambda simply call it with the length required, like so:

Most password requirements today specify the number of special characters, uppercase, lowercase and numbers require in a password. Using the python string library, we can use string attributes to choose a number of specific characters from each set. Now we can modify our password generator to accept the number of uppercase, lowercase, digits and special charcters. The new code and example output will look like this:


import random
import sys
import string

def main(argv):

if (len(sys.argv) != 5):
sys.exit('Usage: simple_pass.py <upper_case> <lower_case> <digit> <special_characters>')

for i in range(len(argv)):
for j in range(int(argv[i])):
if i == 0:
elif i == 1:
elif i == 2:
elif i == 3:

if __name__ == "__main__":
main(sys.argv[1:])



So at this point we can generate randomize and customize passwords. Let us now move to checking a password strength algorithm and check what kind of passwords we are generating. We will start by using a modified version of a password checker found on Password Advisor. I had to modify the code to get it to run and to create some output. Take a look.


import re

strength = ['Blank','Very Weak','Weak','Medium','Strong','Very Strong']
score = 1

return strength[0]
return strength[1]

score = score + 1
score = score + 1

score = score + 1
score = score + 1
score = score + 1

return strength[score]

def main():

user_input = raw_input("Check: ")

while(user_input != 'quit'):
user_input = raw_input("Check: ")

if __name__ == "__main__":
main()



The program uses length and python's regular expressions to determine a score for the password. This is a fairly straight forward calculation. A password longer than 8 characters is awarded 1 point, longer than 10 is awarded an additional point. Then, for each character from a character set the password is awarded another point. The points corresponding to an index on the strength list and an appropriate rating is determined. Let us see how our three password generators from before hold up to this checker standards. (The code has been slightly modified for testing).


import random
import string
import re
import sys

def ex1(num):

for i in range(int(num)):

ex2 = lambda length, ascii =  string.ascii_letters + string.digits + string.punctuation: "".join([list(set(ascii))[random.randint(0,len(list(set(ascii)))-1)] for i in range(length)])

def ex3(argv):

for i in range(len(argv)):
for j in range(int(argv[i])):
if i == 0:
elif i == 1:
elif i == 2:
elif i == 3:

def checker(argv):

example_1 = ex1(argv[1])

example_2 = ex2(int(argv[2]))

example_3 = ex3([argv[3],argv[4],argv[5],argv[6]])

def main(argv):
if (len(sys.argv) != 7):
sys.exit('Usage: ex4.py <length1> <length2> <upper_case> <lower_case> <digit> <special_characters>')

checker(sys.argv)

if __name__ == "__main__":
main(sys.argv[1:])



Let us look at least at one more measure of password strength called bit strength, used to measure the strength of random generated passwords. it follows a formula where each character contributes an amount of bits calculated by the following formula:

The results is multiplied by the number of characters and a final bit strength is calculated. The threshold recommend, according to Wikipedia, for most secure systems is 80-bits. Let us look at how this will look in code and how our generated passwords measure up to this threshold. The code is very similar to the previous example, only replacing the checker function. (Download code for password generator and checker)





Since the character set we have include 94 possible characters in all generators, we have a constant log value (~6.55). This means that no matter what password we generate, even 88888888 will have the same ‘bit strength’ as ‘1.U*1!lh’. We can all recognize that the latter one will be much harder to crack than the first one. This is because the formula applies for randomly generated passwords. I will let the statistician among us calculate the odds of generating 88888888 out of 8 characters with 94 possible values for each character. I will "guesstimate" that it is very close to 0. We can draw a graph following this information to determine the length threshold required for a randomly generated password to pass the bit strength threshold.

You will notice that any password above 15 characters is over 100 on the bit strength index. This means that it will comply with the secure password guideline as indicated on Wikipedia. It also means that if you can remember a randomly generated password, you will need less than 32 characters to secure your password. Keep in mind that this is limited to random generated password and not human created passwords.

So what can you do? In the mean time keep remembering that regardless of the source of password, the length is a directly proportional to the password strength according to any calculation. At this point I have no real solution to the ‘too many passwords’ problem, but I do have hope. I will continue with my work on the matter and post any changes I might have. Keep your passwords secure and safe.

Ever since I can remember, progress bars had my interest. I think it is a neat way to convey to the user that his or her request is being processed. Furthermore, progress bars can give an indication on the percentage of the task that has been completed and/or timing when applicable. In this post I want to explore different types of progress bars and how you can incorporate them in your program. If you are really interested in the history of progress bars, they pre-date the modern computer back to 1896 and are connected with Gantt Chart. For more information of the topic I advise a Google search for progress bars or the Wikipedia entry for progress bar.

The first thing you should know about progress bars is when to use them. Just like any code, they have a certain overhead (negligible) and just because you can does not mean you should. So when should a programmer use a progress bar? When the task requires more than a reasonable time. You will know what a reasonable time is based on the program and your test runs. Along their other usage, progress bars can indicate to the user that work is being done and that the computer did not "freeze" on them. As a rule of thumb, try to used the progress bar when a large task is on hand, not something small. Loading a file is reasonably fast (most of the time). Loading 20GB of files into the memory of the program might take some time. In the latter scenario you probably should use a progress bar. It is very frustrating to the user when they hit ‘enter’ or ‘return’ and nothing happens instantly. Give the user some comfort knowing their request is being processed.

To simplify things I will focus on command line interfaces using Python2.6 (Documentations for python2.6 for offline reference). The same ideas presented here can be applied to GUI interfaces, but that is not trivial. Nothing with GUI development is trivial at all. The first thing to do when considering using a progress bar is to examine the problem. Different programs need different types of progress bars. There are three large sub-categories to all problems that require progress bars:

• Problems that are solved by doing smaller identical tasks of known quantity
• Problems that are solved by doing multiple tasks but known quantity.
• Problems that are solved by doing unknown number of tasks and/or unknown quantity.

Lets start with the first category. Lets assume that we have a function called do_task() that does something. For the purpose of this post all the function will use python sleep function for a small amount of time. That will emulate as if the program was accomplishing a real task. We are going to invoke the function multiple times and each time it completes we will print a dot ('.'). The code will look something like this:


import time
import sys

time.sleep(1)

def example_1(n):
for i in range(n):
print '\b.',
sys.stdout.flush()
print ' Done!'

print 'Starting ',
example_1(10)



The output will look like this:

For a small amount of tasks, something like this will suffice. However, what if the program had to run 100, 1,000 or 1 million times. Clearly a million dots ('.') printed out to the screen will not be a good idea. We can modify the program to print a single dot ('.') for every 10% compilation of the task. Note that at any point in any of the examples discussed in this post the dot (‘.’) can be replaced by whatever you want. It can be an ‘x’ or a ‘#’ or any other printable ascii character will do.


import time
import sys

time.sleep(1)

def example_1(n):
steps = n/10
for i in range(n):
if i%steps == 0:
print '\b.',
sys.stdout.flush()
print ' Done!'

print 'Starting ',
sys.stdout.flush()
example_1(100)



Now let us add some more information about the process. First lets add some brackets so the user has some indication of how many more dots they are about to see. Notice that in previous examples and in the next examples we utilize sys.stdout.flush(), suppressing the carriage return using a comma (‘,’) at the end of the print statement and print a special character, ‘\b’. The special character ‘\b’ returns the printing cursor one step backwards. This is what allows us to print ahead and move backwards. Also note that every time we use the python print statement with a comma, the next print statement has a space in front of it. This is way using the special character ‘\b’ becomes necessary. Take a look at the next example:


import time
import sys

time.sleep(0.1)

def example_1(n):
steps = n/10
for i in range(n):
if i%steps == 0:
print '\b.',
sys.stdout.flush()
print '\b]  Done!',

print 'Starting [          ]',
print '\b'*12,
sys.stdout.flush()
example_1(100)



And the output mid way and at the end:

After we have gone through the basics of the first type of loading bars lets move on to the second. Let us assume that we need to complete x tasks and each one is similar in time. For our code we can emulate this by using the sleep function again. Since the tasks are not the same tasks, we will not have the loop. Instead, we will call the functions one at a time. After each task is complete we will update our progress bar.


import time
import sys

def update_progress_bar():
print '\b.',
sys.stdout.flush()

print 'Starting ',
sys.stdout.flush()

time.sleep(1)
update_progress_bar()

time.sleep(1)
update_progress_bar()

time.sleep(1)
update_progress_bar()

print ' Done!'



We have moved forward in this example to create a function that updates the progress bar and to the second category of progress bars. Although by now we can write our own progress bar, let us take a look at some libraries that are already written nice progress bars you can use.  The first one is by Anler Hernández Peral and available through ActiveState Recipes and progress bar v.1 code. Running the code gives a quick example like this:

In order to utilize the code for your own program you will have to import it. Following the example for ActiveState, Here is 2 ways you can use the code and their output: (Note: I have named the module progress_bar_1.py)


from progress_bar_1 import ProgressBar

p = ProgressBar()
print p

for i in range(10):
print p+1




from progress_bar_1 import ProgressBar

custom_options = {
'end': 10,
'width': 10,
'fill': 'x',
'format': '%(progress)s%% [%(fill)s%(blank)s]'
}

p = ProgressBar(**custom_options)
print p

for i in range(10):
print p+1



Note that now we have moved into creating an object for our progress bar. That allows us to update and print it as we see fit. It also allows for some more custom setting as we can see in the second example. I will let you explore those on your own, but if you need help please contact the Captain. The second library will introduce some more fancy output. It is called progressbar 2.2 and the code for progressbar2.2. The code for it is a little lengthy (about 360 lines), so I will not include it in this post. Running the module as is, yields 4 example outputs that should look like this:

A closer look at the examples will provide you all the information you need to use this module. The ideas are very similar to the last example where you have a progress bar object you instantiate and then progress as needed. In the examples it was included in a for loop, but may be applied for multiple tasks as well.

After we have examine all these example, lets move to to the final category, the unknown one. Let us assume our program will need to complete an operation that either we do not know how many steps are involved or how long it will take. The solution is rather simple, a spinning loading. We still acknowledge to the user that the process is being executed, but cannot provide any insight as to the progression. Please note that you should note use this progress bar unless it is an absolutely must.


import sys
import time

sys.stdout.flush()

i = 0

while i &lt;= 10:
if (i%4) == 0:
sys.stdout.write('\b/')
elif (i%4) == 1:
sys.stdout.write('\b-')
elif (i%4) == 2:
sys.stdout.write('\b\\')
elif (i%4) == 3:
sys.stdout.write('\b|')

sys.stdout.flush()
time.sleep(0.2)
i+=1

print '\b\b done!'



Similar to the examples above, if we want to use this in an application we can build a class around. Unlike the other examples in this case you will have to use threads in order to run the spinning loading bar while the program is running. A small note, you will not be able to print from the code that is running as it will mess up the loading bar. Also notice that now we are using while loops as oppose to the for loop we used before. This is because now we do not have a measure to estimate compilation of the program. The output will be the same as the example above.


import sys
import time

def run(self):
global stop
global kill
sys.stdout.flush()
i = 0
while stop != True:
if (i%4) == 0:
sys.stdout.write('\b/')
elif (i%4) == 1:
sys.stdout.write('\b-')
elif (i%4) == 2:
sys.stdout.write('\b\\')
elif (i%4) == 3:
sys.stdout.write('\b|')

sys.stdout.flush()
time.sleep(0.2)
i+=1

if kill == True:
print '\b\b\b\b ABORT!',
else:
print '\b\b done!',

kill = False
stop = False
p.start()

try:
#anything you want to run.
time.sleep(1)
stop = True
except KeyboardInterrupt or EOFError:
kill = True
stop = True



So now what? Now you know how to create your own custom progress bars in python. You may choose to implement the principles and I might do so myself. If I do I will make sure to post it up. As a last point I would like to mention the usage of progress bars once more. As a programmer you are responsible for the human computer interaction. Progress bars are a tool to convey to the user to stand by. Please use progress bars to actually mean something. Do not just display dots like the first examples. Try to estimate how long it will take, how much of the task has been accomplished, anything at all you can tell the user might help. As a user, there is no progress bar more annoying then ones that tell you nothing. In 2006, Peter Beddow wrote an article On the Abolition of the Computer Progress Bar. I suggest you read that article before using progress bars in large scale applications. Let us as programers bring back the loading bars the actually help the user, not that ones that annoy everyone.

Python is my favorite language. It allows anything from high level to low level programming. It is very easy to understand. Python can be written as an algorithm that you can execute. Today, I want to show you some cool tricks on how to manipulate strings in python.

NOTE: I am running Python2.6. I highly recommend you read the Python 2.6 String documentation prior to proceeding. I suggest you also download python2.6 documentation for offline reference.

Python stores strings in it’s virtual memory as arrays. This is very useable since you can address each character individually. Strings are 0 based arrays and support reverse indexing. However, unlike arrays, strings are not mutable. Strings also support concatenation and other cool stuff. Let’s take a look at the following output:

Now, lets have fun with the output and give it some more meaning. First lets ask the user for a string and print out the characters of the string each on a line. To do so we will run this code:


a_string = raw_input("Enter some text: ")
for char in a_string:
print char



Running the program should give you the following output for the input “Hello World!”:

Now lets use the python built in sleep function from the time library. This will delay the printout of each char by a given time. This will cause a neat effect for the user.


import time

a_string = raw_input("Enter some text: ")

for char in a_string:
print char
time.sleep(1)



So far it is still dull. Let’s take it a step further and have colors. There is a great library to add colors to python standard output called termcolor. I suggest you take a look at the code for termcolor. To use it we can either create a text object and color it, or use the provided cprint function. Here is the example of printing in res through standard output:


import time
import termcolor

a_string = raw_input("Enter some text: ")

for char in a_string:
termcolor.cprint(char, "red")
time.sleep(1)



Termcolor also support printing on color, blinking, bold, italic and others. There is a brief example on the termcolor web page on how to use. That should cover anything you need to know. If I get the time I will come back and write a tutorial for it. If you have questions before such time, feel free to contact the Captain. For the next one, we will print each letter in a different color, like this:


import time
import termcolor

a_string = raw_input("Enter some text: ")
i = 0

for char in a_string:
if (i%7 == 0):
termcolor.cprint(char, "grey")
elif (i%7 == 1):
termcolor.cprint(char, "red")
elif (i%7 == 2):
termcolor.cprint(char, "green")
elif (i%7 == 3):
termcolor.cprint(char, "yellow")
elif (i%7 == 4):
termcolor.cprint(char, "blue")
elif (i%7 == 5):
termcolor.cprint(char, "magenta")
elif (i%7 == 6):
termcolor.cprint(char, "cyan")

time.sleep(1)
i += 1



Now what if you want to print in the same line? Well, we will need to use another standard library called sys.stdout. What we will do is suppress the carige return from the print statement by placing a comma (‘,’) at the end of the print statement. Without flushing the print statement, the output will just appear after the entire line was printed out. Since we want the suspense, we add the flush. Thus printing the input string back to standard output, one character at a time. Check it out:


import time
import termcolor
import sys

a_string = raw_input("Enter some text: ")

for char in a_string:
print char,
sys.stdout.flush()
time.sleep(1)



If we want a different color on each character and have them all on the same line we will need to use the object colored and the regular print statements like before. The code will look like this:


import time
import termcolor
import sys

a_string = raw_input("Enter some text: ")
i = 0

for char in a_string:
if (i%7 == 0):
print termcolor.colored(char, "grey"),
elif (i%7 == 1):
print termcolor.colored(char, "red"),
elif (i%7 == 2):
print termcolor.colored(char, "green"),
elif (i%7 == 3):
print termcolor.colored(char, "yellow"),
elif (i%7 == 4):
print termcolor.colored(char, "blue"),
elif (i%7 == 5):
print termcolor.colored(char, "magenta"),
elif (i%7 == 6):
print termcolor.colored(char, "cyan"),

sys.stdout.flush()
time.sleep(1)
i += 1



The screen capture:

That is all for. Now you have some extra tools in you arsenal to making some neat command line programs using python. If you are really into it I suggest you take a look at the following program, it is really sci-fi.


# written and executed on mac OSX 10.6.8 running python 2.6.8

import time
import termcolor
import sys
import random

colors = ("grey", "red", "green", "yellow", "blue", "magenta", "cyan")
attributes = ("bold", "underline", "blink", "reverse")

def get_color():
global colors
return colors[random.randint(0,len(colors)-1)]

def get_on_color():
global colors
return "on_" + colors[random.randint(0,len(colors)-1)]

def get_attributes():
global attributes
return attributes[random.randint(0,len(attributes)-1)]

for i in range(100**2):
x = random.randint(0,6)
the_char = chr(random.randint(33,126))
if (x%6 == 0):
print termcolor.colored(the_char, get_color(), get_on_color(), attrs=[get_attributes()]),
elif (x%6 == 1):
print termcolor.colored(the_char, get_color(), get_on_color()),
elif (x%6 == 2):
print termcolor.colored(the_char, get_color()),
elif (x%6 == 4):
print termcolor.colored(the_char, get_color(), attrs=[get_attributes()]),
elif (x%6 == 5):
print termcolor.colored(the_char, get_on_color(), attrs=[get_attributes()]),
else:
print the_char,

sys.stdout.flush()
time.sleep(0.3)



Screen capture of the string secret program - 1

Screen capture of the string secret program - 2