R: Monitoring the function progress with a progress bar
Every once in while I have to write a function that contains a loop doing thousands or millions of calculations. To make sure that the function does not get stuck in an endless loop or just to fulfill the human need of control it is useful to monitor the progress. So first I tried the following:
############################################################### total <- 10 for(i in 1:total){ print(i) Sys.sleep(0.1) } ###############################################################
Unfortunately this does not work as the console output to the basic R GUI is buffered. This means that it is printed to the console at once after the loop is finished. The R FAQs (7.1) explains a solution: Either to change the R GUI buffering settings in the Misc menu which can be toggled via <Ctrl-W> or to tell R explicitly to empty the buffer by flush.console(). So like this it works:
############################################################### total <- 20 for(i in 1:total){ Sys.sleep(0.1) print(i) # update GUI console flush.console() } ###############################################################
Of course it would be even nicer to have a real progress bar. For different progress bars we can use the built-in R.utils package. First a text based progress bar:
############################################################### total <- 20 # create progress bar pb <- txtProgressBar(min = 0, max = total, style = 3) for(i in 1:total){ Sys.sleep(0.1) # update progress bar setTxtProgressBar(pb, i) } close(pb) ###############################################################
To get a GUI progress bar the tkProgressBar() function from the tcltk package can used.
############################################################### total <- 20 # create progress bar pb <- tkProgressBar(title = "progress bar", min = 0, max = total, width = 300) for(i in 1:total){ Sys.sleep(0.1) setTkProgressBar(pb, i, label=paste( round(i/total*100, 0), "% done")) } close(pb) ###############################################################
Last but not least, a progress bar using the Windows operating system.
############################################################### # create progress bar pb <- winProgressBar(title = "progress bar", min = 0, max = total, width = 300) for(i in 1:total){ Sys.sleep(0.1) setWinProgressBar(pb, i, title=paste( round(i/total*100, 0), "% done")) } close(pb) ###############################################################
Ciao, Mark
Filed under: R / R-Code | 27 Comments
Tags: progress bar
Feeds
Authors
Ecosia – eco-friendly web search
-
Mark Heckmann’s blog- 11th EPCA Biennial Conference in Dublin July 4, 2012 markheckmann
- Karriere vs. Wissenschaft – Zur Verbesserung der Bedingungen für den wissenschaftlichen Nachwuchs May 16, 2012 markheckmann
- Psychologie und LaTex – Literaturverzeichnisse im DGPs-Stil February 29, 2012 markheckmann
- Zotero to LaTex Workflow December 1, 2011 markheckmann
- Einführung in die quantitative Datenanalyse mit R October 1, 2011 markheckmann
- R-help latest postings
- An error has occurred; the feed is probably down. Try again later.
Terms of use
The contents of the "R" you ready? blog are offered under a creative commons license.
All postings
markheckmann
nattomi
Great trick.. thank you. I run large nonparametric models and this is much better than twiddling my finger and praying to God that it gets done soon.
Wonderful post.
It took me over a week to get to read it – but I am glad I did.
I am just about to run a sevral hours simulation, and there is no bigger joy then to know it didn’t get stuck in the middle !
(now the only question is how to do this when running R in parallel :) )
Tal
I have a frame and I want to fix the progress bar onto it.I do not want progress bar to display as another GUI.how do i embed progress bar onto a frame ?
I am sorry but I cannot help you with this one. Maybe post it on R-help.
A nice set of tricks. Is it possible to monitor progress of apply family as well?
Hi John,
this is just one of the matters I am not sure of neither.
I do not know how and have not found information on it yet.
If you get to know how to use it with the apply family, please let me know.
Mark
Thanks alot for the tips. It helps me a lot. Is there anyway that we can put a button to stop the proccess? in case its too long?
Thanks
Setia
Hi Setia,
what kind of button do you mean? Some kind of GUI button? You can always interrupt the process by pressing escape, I think.
Mark
Hi Mark,
Yes I mean the GUI button, in case it takes too long, the user can stop the progress any time. I tried with pressing escape, but it doesnt work.
Thanks
Setia
To print variables either for debugging, tracking progress, or printing to STDOUT I use the cat() function instead of print(). The output is not buffered using cat().
Very nice post.
(I wondered so many times why display was delayed sometimes. Left clicking on a working R-gui also flushes the buffer on the console.)
Thank you for the very clear article on progress bars.
And don’t forget about GtkProgressBar in RGtk2!
Call me stupid… but where exactly do I insert the script whose progress I want to monitor? I can only guess it’s somewhere in the for loop… is it before the line updating the progress bar or before the sleep line? I’ve tested it out, but it repeats the output 20 times, as opposed to displaying the progress of the function.
Hi,
please post your problem code example in the comments as it is hard for me to understand your problem from the above description.
Mark
Thank you for the reply.
This generates 20 computations of the same thing (which now makes sense):
total <- 20
# create progress bar
pb <- txtProgressBar(min = 0, max = total, style = 3)
for(i in 1:total){
my.function(…)
Sys.sleep(0.1)
# update progress bar
setTxtProgressBar(pb, i)
}
close(pb)
I've tried inserting the script into my function, but when I do that, it just seems to run the progress bar, then pause to finish its computation.
For example:
my.function<-
function (arg1,arg2,…,arg)
{
# progress bar script position 1
- computation block –
# progress bar position 2
- plot block -
- result summary display block-
# progress bar position 3
}
For positions 1 and 2, the progress bar runs, then there is this long pause either before or after it shows up as the function (apparently) completes its computation(s).
Placing the progress bar in position 3 results in no progress bar displayed at all.
Thanks again,
Ren
Check if the progress bar is already set to a maximum value in the last computation, as it is not updated in the last part. Setting the progress bar to a value that exceeds its maximum will not cause an update in the bar anymore.
HTH,
Mark
Thanks, Mark. That really helps – i finally get it now.
Thanks for the tips. I’ve known about having to flush the console for years but had no idea about the built-in progress bars until reading this post.
Hi Mark,
Great stuff. Like everyone else I like seeing progress to avoid hangups. My issue is that I can’t figure out how to create a progress bar using a while loop. I am doing a process that runs until all elements in a vector have been ‘removed’.
So a sample process below:
#Create test values to remove
ToLocate <- 1:10000
#Determine count to run
Total 0){
#Sys.sleep(0.1)
i <- abs(length(ToLocate) – Total )
#Create Random number
RandomNumber <- round(runif(1,1,10000),0)
#If Random Number equals a number in ToLocate Remove it
ToLocate <- ToLocate[!(ToLocate%in%RandomNumber)]
pb <- winProgressBar(title = "progress bar", min = 0,
max = Total, width = 300)
}
close(pb)
Instead of tracking progress its opening a progress bar for each time through the while loop. I have tried using a 'for' loop within the 'while' but then it gets stuck. Careful running the above as is will result in a bunch of status bars opening up. I can post this to the list but thought I would ask you first. Hope you can help
Cheers,
Josh
Got it, was missing a line of code for some reason and just needed to move the bar creation code out of the while loop.
#Create test values to simulate a dataset to ‘remove’
ToLocate <- 1:10000
#Determine count to run
Total <- length(ToLocate) + 1
#Create progress bar
pb 0){
for(i in 1:Total){
i <- abs(length(ToLocate) – Total )
#Create Random number
RandomNumber <- round(runif(1,1,100000),0)
#If Random Number equals a number in ToLocate Remove it
ToLocate <- ToLocate[!(ToLocate%in%RandomNumber)]
#Update Progress Bar
setWinProgressBar(pb, i, title=paste( round(i/Total*100, 0),
"% done"))
}
}
#Close progress bar
close(pb)
Cool, thanks a lot!
When I’m using sapply or lapply to make long for-loops faster, I can do this:
total <- 20
pb <- txtProgressBar(min = 0, max = total, style = 3)
#
sapply(1:total, function(i) {
Sys.sleep(0.1) # replace with real loop
setTxtProgressBar(pb, i) } )
#
close(pb)
That’s right. And in case you don’t want to hard code it each time you need it, just use a wrapper for the apply functions: http://ryouready.wordpress.com/2010/01/11/progress-bars-in-r-part-ii-a-wrapper-for-apply-functions/
Mark