R: Monitoring the function progress with a progress bar

16Mar09

Every once in while I have to write a function that contains a loop doing thousands or millions of calculations. To make sure that the function does not get stuck in an endless loop or just to fulfill the human need of control it is useful to monitor the progress. So first I tried the following:

###############################################################

total <- 10
for(i in 1:total){
   print(i)
   Sys.sleep(0.1)
}

###############################################################

Unfortunately this does not work as the console output to the basic R GUI is buffered. This means that it is printed to the console at once after the loop is finished. The R FAQs (7.1) explains a solution: Either to change the R GUI buffering settings in the Misc menu which can be toggled via <Ctrl-W> or to tell R explicitly to empty the buffer by flush.console(). So like this it works:

###############################################################

total <- 20
for(i in 1:total){
   Sys.sleep(0.1)
   print(i)
   # update GUI console
   flush.console()                          
}

###############################################################

Of course it would be even nicer to have a real progress bar. For different progress bars we can use the built-in R.utils package. First a text based progress bar:

###############################################################

total <- 20
# create progress bar
pb <- txtProgressBar(min = 0, max = total, style = 3)
for(i in 1:total){
   Sys.sleep(0.1)
   # update progress bar
   setTxtProgressBar(pb, i)
}
close(pb)

###############################################################

To get a GUI progress bar the tkProgressBar() function from the tcltk package can used.

###############################################################

total <- 20
# create progress bar
pb <- tkProgressBar(title = "progress bar", min = 0,
                    max = total, width = 300)

for(i in 1:total){
   Sys.sleep(0.1)
   setTkProgressBar(pb, i, label=paste( round(i/total*100, 0),
                                        "% done"))
}
close(pb)

###############################################################

Last but not least, a progress bar using the Windows operating system.

###############################################################

# create progress bar
pb <- winProgressBar(title = "progress bar", min = 0,
                     max = total, width = 300)

for(i in 1:total){
   Sys.sleep(0.1)
   setWinProgressBar(pb, i, title=paste( round(i/total*100, 0),
                                        "% done"))
}
close(pb)

###############################################################

Ciao, Mark

Filed under: R / R-Code | 31 Comments
Tags: progress bar

31 Responses to “R: Monitoring the function progress with a progress bar”

Feed for this Entry Trackback Address

1 Suny on May 2, 2009 said:

Great trick.. thank you. I run large nonparametric models and this is much better than twiddling my finger and praying to God that it gets done soon.
2 Tal Galili on August 31, 2009 said:

Wonderful post.
It took me over a week to get to read it – but I am glad I did.

I am just about to run a sevral hours simulation, and there is no bigger joy then to know it didn’t get stuck in the middle !

(now the only question is how to do this when running R in parallel :) )

Tal
3 Dipti on October 6, 2009 said:

I have a frame and I want to fix the progress bar onto it.I do not want progress bar to display as another GUI.how do i embed progress bar onto a frame ?
4 markheckmann on October 6, 2009 said:

I am sorry but I cannot help you with this one. Maybe post it on R-help.
5 John dGramo on October 28, 2009 said:

A nice set of tricks. Is it possible to monitor progress of apply family as well?
6 markheckmann on October 28, 2009 said:

Hi John,
this is just one of the matters I am not sure of neither.
I do not know how and have not found information on it yet.
If you get to know how to use it with the apply family, please let me know.
Mark
7 setia on May 31, 2010 said:

Thanks alot for the tips. It helps me a lot. Is there anyway that we can put a button to stop the proccess? in case its too long?
Thanks
Setia
8 markheckmann on May 31, 2010 said:

Hi Setia,
what kind of button do you mean? Some kind of GUI button? You can always interrupt the process by pressing escape, I think.
Mark
9 setia on June 13, 2010 said:

Hi Mark,
Yes I mean the GUI button, in case it takes too long, the user can stop the progress any time. I tried with pressing escape, but it doesnt work.
Thanks
Setia
10 Morgan Langille on July 23, 2010 said:

To print variables either for debugging, tracking progress, or printing to STDOUT I use the cat() function instead of print(). The output is not buffered using cat().
11 Dotoma on August 12, 2010 said:

Very nice post.

(I wondered so many times why display was delayed sometimes. Left clicking on a working R-gui also flushes the buffer on the console.)

Thank you for the very clear article on progress bars.
12 nico on August 19, 2010 said:

And don’t forget about GtkProgressBar in RGtk2!
13 Ren on August 24, 2010 said:

Call me stupid… but where exactly do I insert the script whose progress I want to monitor? I can only guess it’s somewhere in the for loop… is it before the line updating the progress bar or before the sleep line? I’ve tested it out, but it repeats the output 20 times, as opposed to displaying the progress of the function.
14 markheckmann on August 24, 2010 said:

Hi,
please post your problem code example in the comments as it is hard for me to understand your problem from the above description.
Mark
15 Ren on August 24, 2010 said:

Thank you for the reply.

This generates 20 computations of the same thing (which now makes sense):

total <- 20
# create progress bar
pb <- txtProgressBar(min = 0, max = total, style = 3)
for(i in 1:total){
my.function(…)
Sys.sleep(0.1)
# update progress bar
setTxtProgressBar(pb, i)
}
close(pb)

I've tried inserting the script into my function, but when I do that, it just seems to run the progress bar, then pause to finish its computation.

For example:

my.function<-
function (arg1,arg2,…,arg)
{
# progress bar script position 1

– computation block –

# progress bar position 2

– plot block –

– result summary display block-

# progress bar position 3

}

For positions 1 and 2, the progress bar runs, then there is this long pause either before or after it shows up as the function (apparently) completes its computation(s).

Placing the progress bar in position 3 results in no progress bar displayed at all.

Thanks again,
Ren

16 markheckmann on August 25, 2010 said:

Check if the progress bar is already set to a maximum value in the last computation, as it is not updated in the last part. Setting the progress bar to a value that exceeds its maximum will not cause an update in the bar anymore.

foo <- function(...){
	total <- 40 
	pb <- txtProgressBar(min = 0, max = total, style = 3)
	# computation block 1
	for(i in 1:20){
		Sys.sleep(0.1)
		setTxtProgressBar(pb, i)
	}

	# computation block 2
	for(i in 21:total){
		Sys.sleep(0.1)
		setTxtProgressBar(pb, i)
	}
	close(pb)
}

HTH,
Mark

17 Ren on August 26, 2010 said:

Thanks, Mark. That really helps – i finally get it now.
18 Maurice on November 30, 2011 said:

Thanks for the tips. I’ve known about having to flush the console for years but had no idea about the built-in progress bars until reading this post.
19 Josh on January 19, 2012 said:

Hi Mark,
Great stuff. Like everyone else I like seeing progress to avoid hangups. My issue is that I can’t figure out how to create a progress bar using a while loop. I am doing a process that runs until all elements in a vector have been ‘removed’.

So a sample process below:

#Create test values to remove
ToLocate <- 1:10000

#Determine count to run
Total 0){
#Sys.sleep(0.1)

i <- abs(length(ToLocate) – Total )
#Create Random number
RandomNumber <- round(runif(1,1,10000),0)

#If Random Number equals a number in ToLocate Remove it
ToLocate <- ToLocate[!(ToLocate%in%RandomNumber)]

pb <- winProgressBar(title = "progress bar", min = 0,
max = Total, width = 300)

}

close(pb)

Instead of tracking progress its opening a progress bar for each time through the while loop. I have tried using a 'for' loop within the 'while' but then it gets stuck. Careful running the above as is will result in a bunch of status bars opening up. I can post this to the list but thought I would ask you first. Hope you can help

Cheers,
Josh
20 Josh on January 20, 2012 said:

Got it, was missing a line of code for some reason and just needed to move the bar creation code out of the while loop.

#Create test values to simulate a dataset to ‘remove’
ToLocate <- 1:10000

#Determine count to run
Total <- length(ToLocate) + 1

#Create progress bar
pb 0){

for(i in 1:Total){
i <- abs(length(ToLocate) – Total )
#Create Random number
RandomNumber <- round(runif(1,1,100000),0)

#If Random Number equals a number in ToLocate Remove it
ToLocate <- ToLocate[!(ToLocate%in%RandomNumber)]
#Update Progress Bar
setWinProgressBar(pb, i, title=paste( round(i/Total*100, 0),
"% done"))

}

}
#Close progress bar
close(pb)
21 ms2070 on April 23, 2012 said:

Cool, thanks a lot!
22 Berry on March 21, 2013 said:

When I’m using sapply or lapply to make long for-loops faster, I can do this:
total <- 20
pb <- txtProgressBar(min = 0, max = total, style = 3)
#
sapply(1:total, function(i) {
Sys.sleep(0.1) # replace with real loop
setTxtProgressBar(pb, i) } )
#
close(pb)
23 markheckmann on March 21, 2013 said:

That’s right. And in case you don’t want to hard code it each time you need it, just use a wrapper for the apply functions: https://ryouready.wordpress.com/2010/01/11/progress-bars-in-r-part-ii-a-wrapper-for-apply-functions/
Mark