Creating Progress Bars with Python

Ever since I can remember, progress bars had my interest. I think it is a neat way to convey to the user that his or her request is being processed. Furthermore, progress bars can give an indication on the percentage of the task that has been completed and/or timing when applicable. In this post I want to explore different types of progress bars and how you can incorporate them in your program. If you are really interested in the history of progress bars, they pre-date the modern computer back to 1896 and are connected with Gantt Chart. For more information of the topic I advise a Google search for progress bars or the Wikipedia entry for progress bar.

The first thing you should know about progress bars is when to use them. Just like any code, they have a certain overhead (negligible) and just because you can does not mean you should. So when should a programmer use a progress bar? When the task requires more than a reasonable time. You will know what a reasonable time is based on the program and your test runs. Along their other usage, progress bars can indicate to the user that work is being done and that the computer did not "freeze" on them. As a rule of thumb, try to used the progress bar when a large task is on hand, not something small. Loading a file is reasonably fast (most of the time). Loading 20GB of files into the memory of the program might take some time. In the latter scenario you probably should use a progress bar. It is very frustrating to the user when they hit ‘enter’ or ‘return’ and nothing happens instantly. Give the user some comfort knowing their request is being processed.

wpid-progress_bar-2012-12-29-19-192.gif

To simplify things I will focus on command line interfaces using Python2.6 (Documentations for python2.6 for offline reference). The same ideas presented here can be applied to GUI interfaces, but that is not trivial. Nothing with GUI development is trivial at all. The first thing to do when considering using a progress bar is to examine the problem. Different programs need different types of progress bars. There are three large sub-categories to all problems that require progress bars:

  • Problems that are solved by doing smaller identical tasks of known quantity
  • Problems that are solved by doing multiple tasks but known quantity.
  • Problems that are solved by doing unknown number of tasks and/or unknown quantity.

Lets start with the first category. Lets assume that we have a function called do_task() that does something. For the purpose of this post all the function will use python sleep function for a small amount of time. That will emulate as if the program was accomplishing a real task. We are going to invoke the function multiple times and each time it completes we will print a dot ('.'). The code will look something like this:


import time
import sys

def do_task():
	time.sleep(1)

def example_1(n):
	for i in range(n):
		do_task()
		print '\b.',
		sys.stdout.flush()
	print ' Done!'
	
print 'Starting ',
example_1(10)

The output will look like this:

wpid-pythonprogressbarexample1-2012-12-29-19-192.png

For a small amount of tasks, something like this will suffice. However, what if the program had to run 100, 1,000 or 1 million times. Clearly a million dots ('.') printed out to the screen will not be a good idea. We can modify the program to print a single dot ('.') for every 10% compilation of the task. Note that at any point in any of the examples discussed in this post the dot (‘.’) can be replaced by whatever you want. It can be an ‘x’ or a ‘#’ or any other printable ascii character will do.


import time
import sys

def do_task():
	time.sleep(1)

def example_1(n):
	steps = n/10
	for i in range(n):
		do_task()
		if i%steps == 0:
			print '\b.',
			sys.stdout.flush()
	print ' Done!'
	
print 'Starting ',
sys.stdout.flush()
example_1(100)

python loading bar mid way

Now let us add some more information about the process. First lets add some brackets so the user has some indication of how many more dots they are about to see. Notice that in previous examples and in the next examples we utilize sys.stdout.flush(), suppressing the carriage return using a comma (‘,’) at the end of the print statement and print a special character, ‘\b’. The special character ‘\b’ returns the printing cursor one step backwards. This is what allows us to print ahead and move backwards. Also note that every time we use the python print statement with a comma, the next print statement has a space in front of it. This is way using the special character ‘\b’ becomes necessary. Take a look at the next example:


import time
import sys

def do_task():
	time.sleep(0.1)

def example_1(n):
	steps = n/10
	for i in range(n):
		do_task()
		if i%steps == 0:
			print '\b.',
			sys.stdout.flush()
	print '\b]  Done!',
	
print 'Starting [          ]',
print '\b'*12,
sys.stdout.flush()
example_1(100)


And the output mid way and at the end:

wpid-pythonprogressbarmidway-2012-12-29-19-19.png

wpid-pythonprogressbarwithbracketsend-2012-12-29-19-19.png

After we have gone through the basics of the first type of loading bars lets move on to the second. Let us assume that we need to complete x tasks and each one is similar in time. For our code we can emulate this by using the sleep function again. Since the tasks are not the same tasks, we will not have the loop. Instead, we will call the functions one at a time. After each task is complete we will update our progress bar.


import time
import sys

def update_progress_bar():
	print '\b.',
	sys.stdout.flush()
	
print 'Starting ',
sys.stdout.flush()

#task 1
time.sleep(1)
update_progress_bar()

#task 2 
time.sleep(1)
update_progress_bar()

#task 3 
time.sleep(1)
update_progress_bar()

#Add as many tasks as you need. 

print ' Done!'

wpid-pythonmultitaskprogressbar-2012-12-29-19-19.png

We have moved forward in this example to create a function that updates the progress bar and to the second category of progress bars. Although by now we can write our own progress bar, let us take a look at some libraries that are already written nice progress bars you can use.  The first one is by Anler Hernández Peral and available through ActiveState Recipes and progress bar v.1 code. Running the code gives a quick example like this:

wpid-progress_bar_1exampeoutput-2012-12-29-19-19.png

In order to utilize the code for your own program you will have to import it. Following the example for ActiveState, Here is 2 ways you can use the code and their output: (Note: I have named the module progress_bar_1.py)


from progress_bar_1 import ProgressBar

p = ProgressBar()
print p

for i in range(10):
	print p+1

wpid-pythonprogress_bar_1output_1-2012-12-29-19-19.png


from progress_bar_1 import ProgressBar

custom_options = {
	'end': 10,
	'width': 10,
	'fill': 'x',
	'format': '%(progress)s%% [%(fill)s%(blank)s]'
}

p = ProgressBar(**custom_options)
print p

for i in range(10):
	print p+1


python progress_bar_1 output_2.1

Note that now we have moved into creating an object for our progress bar. That allows us to update and print it as we see fit. It also allows for some more custom setting as we can see in the second example. I will let you explore those on your own, but if you need help please contact the Captain. The second library will introduce some more fancy output. It is called progressbar 2.2 and the code for progressbar2.2. The code for it is a little lengthy (about 360 lines), so I will not include it in this post. Running the module as is, yields 4 example outputs that should look like this:

progressbar examples output

A closer look at the examples will provide you all the information you need to use this module. The ideas are very similar to the last example where you have a progress bar object you instantiate and then progress as needed. In the examples it was included in a for loop, but may be applied for multiple tasks as well.

After we have examine all these example, lets move to to the final category, the unknown one. Let us assume our program will need to complete an operation that either we do not know how many steps are involved or how long it will take. The solution is rather simple, a spinning loading. We still acknowledge to the user that the process is being executed, but cannot provide any insight as to the progression. Please note that you should note use this progress bar unless it is an absolutely must.


import sys
import time

print 'Loading....  ',
sys.stdout.flush()

i = 0

while i <= 10:
	if (i%4) == 0: 
		sys.stdout.write('\b/')
	elif (i%4) == 1: 
		sys.stdout.write('\b-')
	elif (i%4) == 2:
		sys.stdout.write('\b\\')
	elif (i%4) == 3: 
		sys.stdout.write('\b|')

	sys.stdout.flush()
	time.sleep(0.2)
	i+=1
        
print '\b\b done!'


loading bar mid way

Similar to the examples above, if we want to use this in an application we can build a class around. Unlike the other examples in this case you will have to use threads in order to run the spinning loading bar while the program is running. A small note, you will not be able to print from the code that is running as it will mess up the loading bar. Also notice that now we are using while loops as oppose to the for loop we used before. This is because now we do not have a measure to estimate compilation of the program. The output will be the same as the example above.


import sys
import time
import threading

class progress_bar_loading(threading.Thread):
    
    def run(self):
            global stop
            global kill
            print 'Loading....  ',
            sys.stdout.flush()
            i = 0
            while stop != True:
                    if (i%4) == 0: 
                    	sys.stdout.write('\b/')
                    elif (i%4) == 1: 
                    	sys.stdout.write('\b-')
                    elif (i%4) == 2: 
                    	sys.stdout.write('\b\\')
                    elif (i%4) == 3: 
                    	sys.stdout.write('\b|')

                    sys.stdout.flush()
                    time.sleep(0.2)
                    i+=1
                    
            if kill == True: 
            	print '\b\b\b\b ABORT!',
            else: 
            	print '\b\b done!',

      
kill = False      
stop = False
p = progress_bar_loading()
p.start()

try:
	#anything you want to run. 
	time.sleep(1)
	stop = True
except KeyboardInterrupt or EOFError:
         kill = True
         stop = True


So now what? Now you know how to create your own custom progress bars in python. You may choose to implement the principles and I might do so myself. If I do I will make sure to post it up. As a last point I would like to mention the usage of progress bars once more. As a programmer you are responsible for the human computer interaction. Progress bars are a tool to convey to the user to stand by. Please use progress bars to actually mean something. Do not just display dots like the first examples. Try to estimate how long it will take, how much of the task has been accomplished, anything at all you can tell the user might help. As a user, there is no progress bar more annoying then ones that tell you nothing. In 2006, Peter Beddow wrote an article On the Abolition of the Computer Progress Bar. I suggest you read that article before using progress bars in large scale applications. Let us as programers bring back the loading bars the actually help the user, not that ones that annoy everyone.

If you enjoyed this post, please consider leaving a comment or subscribing to the RSS feed to have future articles delivered to your feed reader.
  • exhuma

    Everytime I need a progress bar on the console, I either turn to: http://code.activestate.com/recipes/475116/ or http://nadiana.com/animated-terminal-progress-bar-in-python

    Both recipes can just be copy/pasted into your code, or you put it into a module and import it. Whatever makes more sense for your project.

    The ActiveState code is licensed under the PSF (which is similar to BSD), Nadja's code is BSD. Both a very permissive, so you're fine.

    Both use terminfo to determine whether you can use color or not on the terminal. So that's nice. I find the one from ActiveState has a nicer default "look".

    • CptDeadBones

      Thank you for your reply. The first module looks very good. I have not seen it before and I will need to take a closer look at it. From what it looks the library provides more than just a progress bar. I was looking for simple straight forward code. As for the second link, I have seen the code before. Although it doe run and works nice, it does not work "out-of-the-box". It requires the terminal module installed. Again, I was looking at simple libraries that can work with as little effrot. All that begin said, both are valid options and I appreciate your input.

      • exhuma

        That's the beauty of the recipes I linked. You don't have to "install" anything. They are so simple that there is nothing whatsoever preventing you to just simply - as stated in my previous post - copy/pasting them into your code.

        It is true however, that the second example I linked requires you to copy/paste two bits of code.

        The reason the first one looks more complicated, is that it's *main* goal of both examples is to add *colour* to your terminal output via ANSI escape sequences. The progress bar is just added as added bonus.

        Your code for sure is more customizable though :)

        • CptDeadBones

          Thank you.

  • Pingback: talsma.ca » The future is ASCII progress bars

  • ash4er

    Thanks Cap! This is the best article about progressbar i've ever read!

    • CptDeadBones

      Thanks!

  • Pingback: 【Python】Progressbar | こざくらラボ

  • Simon

    well these are nice examples but as long as not clearly stated where to insert the code you actually want to execute during the progressbar movement, it's for nothing!