R: Monitoring the function progress with a progress bar

16Mar09

progress_barEvery once in while I have to write a function that contains a loop doing thousands or millions of calculations. To make sure that the function does not get stuck in an endless loop or just to fulfill the human need of control it is useful to monitor the progress. So  first I tried the following:



###############################################################

total <- 10
for(i in 1:total){
   print(i)
   Sys.sleep(0.1)
}

###############################################################

Unfortunately this does not work as the console output to the basic R GUI is buffered. This means that it is printed to the console at once after the loop is finished. The R FAQs (7.1) explains a solution: Either to change the R GUI buffering settings in the Misc menu which can be toggled via <Ctrl-W> or to tell R explicitly to empty the buffer by flush.console(). So like this it works:

###############################################################

total <- 20
for(i in 1:total){
   Sys.sleep(0.1)
   print(i)
   # update GUI console
   flush.console()                          
}

###############################################################

Of course it would be even nicer to have a real progress bar. For different progress bars we can use the built-in R.utils package. First a text based progress bar:

###############################################################

total <- 20
# create progress bar
pb <- txtProgressBar(min = 0, max = total, style = 3)
for(i in 1:total){
   Sys.sleep(0.1)
   # update progress bar
   setTxtProgressBar(pb, i)
}
close(pb)

###############################################################

To get a GUI progress bar the tkProgressBar() function from the tcltk package can used.

###############################################################

total <- 20
# create progress bar
pb <- tkProgressBar(title = "progress bar", min = 0,
                    max = total, width = 300)

for(i in 1:total){
   Sys.sleep(0.1)
   setTkProgressBar(pb, i, label=paste( round(i/total*100, 0),
                                        "% done"))
}
close(pb)

###############################################################

Last but not least, a progress bar using the Windows operating system.

###############################################################

# create progress bar
pb <- winProgressBar(title = "progress bar", min = 0,
                     max = total, width = 300)

for(i in 1:total){
   Sys.sleep(0.1)
   setWinProgressBar(pb, i, title=paste( round(i/total*100, 0),
                                        "% done"))
}
close(pb)

###############################################################

Ciao, Mark

About these ads


28 Responses to “R: Monitoring the function progress with a progress bar”

  1. 1 Suny

    Great trick.. thank you. I run large nonparametric models and this is much better than twiddling my finger and praying to God that it gets done soon.

  2. Wonderful post.
    It took me over a week to get to read it – but I am glad I did.

    I am just about to run a sevral hours simulation, and there is no bigger joy then to know it didn’t get stuck in the middle !

    (now the only question is how to do this when running R in parallel :) )

    Tal

  3. 3 Dipti

    I have a frame and I want to fix the progress bar onto it.I do not want progress bar to display as another GUI.how do i embed progress bar onto a frame ?

  4. 4 markheckmann

    I am sorry but I cannot help you with this one. Maybe post it on R-help.

  5. 5 John dGramo

    A nice set of tricks. Is it possible to monitor progress of apply family as well?

  6. 6 markheckmann

    Hi John,
    this is just one of the matters I am not sure of neither.
    I do not know how and have not found information on it yet.
    If you get to know how to use it with the apply family, please let me know.
    Mark

  7. 7 setia

    Thanks alot for the tips. It helps me a lot. Is there anyway that we can put a button to stop the proccess? in case its too long?
    Thanks
    Setia

  8. 8 markheckmann

    Hi Setia,
    what kind of button do you mean? Some kind of GUI button? You can always interrupt the process by pressing escape, I think.
    Mark

  9. 9 setia

    Hi Mark,
    Yes I mean the GUI button, in case it takes too long, the user can stop the progress any time. I tried with pressing escape, but it doesnt work.
    Thanks
    Setia

  10. To print variables either for debugging, tracking progress, or printing to STDOUT I use the cat() function instead of print(). The output is not buffered using cat().

  11. 11 Dotoma

    Very nice post.

    (I wondered so many times why display was delayed sometimes. Left clicking on a working R-gui also flushes the buffer on the console.)

    Thank you for the very clear article on progress bars.

  12. 12 nico

    And don’t forget about GtkProgressBar in RGtk2!

  13. 13 Ren

    Call me stupid… but where exactly do I insert the script whose progress I want to monitor? I can only guess it’s somewhere in the for loop… is it before the line updating the progress bar or before the sleep line? I’ve tested it out, but it repeats the output 20 times, as opposed to displaying the progress of the function.

  14. 14 markheckmann

    Hi,
    please post your problem code example in the comments as it is hard for me to understand your problem from the above description.
    Mark

  15. 15 Ren

    Thank you for the reply.

    This generates 20 computations of the same thing (which now makes sense):

    total <- 20
    # create progress bar
    pb <- txtProgressBar(min = 0, max = total, style = 3)
    for(i in 1:total){
    my.function(…)
    Sys.sleep(0.1)
    # update progress bar
    setTxtProgressBar(pb, i)
    }
    close(pb)

    I've tried inserting the script into my function, but when I do that, it just seems to run the progress bar, then pause to finish its computation.

    For example:

    my.function<-
    function (arg1,arg2,…,arg)
    {
    # progress bar script position 1

    - computation block –

    # progress bar position 2

    - plot block -

    - result summary display block-

    # progress bar position 3

    }

    For positions 1 and 2, the progress bar runs, then there is this long pause either before or after it shows up as the function (apparently) completes its computation(s).

    Placing the progress bar in position 3 results in no progress bar displayed at all.

    Thanks again,
    Ren

  16. 16 markheckmann

    Check if the progress bar is already set to a maximum value in the last computation, as it is not updated in the last part. Setting the progress bar to a value that exceeds its maximum will not cause an update in the bar anymore.

    foo <- function(...){
    	total <- 40 
    	pb <- txtProgressBar(min = 0, max = total, style = 3)
    	# computation block 1
    	for(i in 1:20){
    		Sys.sleep(0.1)
    		setTxtProgressBar(pb, i)
    	}
    
    	# computation block 2
    	for(i in 21:total){
    		Sys.sleep(0.1)
    		setTxtProgressBar(pb, i)
    	}
    	close(pb)
    }
    

    HTH,
    Mark

  17. 17 Ren

    Thanks, Mark. That really helps – i finally get it now.

  18. Thanks for the tips. I’ve known about having to flush the console for years but had no idea about the built-in progress bars until reading this post.

  19. Hi Mark,
    Great stuff. Like everyone else I like seeing progress to avoid hangups. My issue is that I can’t figure out how to create a progress bar using a while loop. I am doing a process that runs until all elements in a vector have been ‘removed’.

    So a sample process below:

    #Create test values to remove
    ToLocate <- 1:10000

    #Determine count to run
    Total 0){
    #Sys.sleep(0.1)

    i <- abs(length(ToLocate) – Total )
    #Create Random number
    RandomNumber <- round(runif(1,1,10000),0)

    #If Random Number equals a number in ToLocate Remove it
    ToLocate <- ToLocate[!(ToLocate%in%RandomNumber)]

    pb <- winProgressBar(title = "progress bar", min = 0,
    max = Total, width = 300)

    }

    close(pb)

    Instead of tracking progress its opening a progress bar for each time through the while loop. I have tried using a 'for' loop within the 'while' but then it gets stuck. Careful running the above as is will result in a bunch of status bars opening up. I can post this to the list but thought I would ask you first. Hope you can help

    Cheers,
    Josh

  20. Got it, was missing a line of code for some reason and just needed to move the bar creation code out of the while loop.

    #Create test values to simulate a dataset to ‘remove’
    ToLocate <- 1:10000

    #Determine count to run
    Total <- length(ToLocate) + 1

    #Create progress bar
    pb 0){

    for(i in 1:Total){
    i <- abs(length(ToLocate) – Total )
    #Create Random number
    RandomNumber <- round(runif(1,1,100000),0)

    #If Random Number equals a number in ToLocate Remove it
    ToLocate <- ToLocate[!(ToLocate%in%RandomNumber)]
    #Update Progress Bar
    setWinProgressBar(pb, i, title=paste( round(i/Total*100, 0),
    "% done"))

    }

    }
    #Close progress bar
    close(pb)

  21. 21 ms2070

    Cool, thanks a lot!

  22. When I’m using sapply or lapply to make long for-loops faster, I can do this:
    total <- 20
    pb <- txtProgressBar(min = 0, max = total, style = 3)
    #
    sapply(1:total, function(i) {
    Sys.sleep(0.1) # replace with real loop
    setTxtProgressBar(pb, i) } )
    #
    close(pb)

  23. 23 markheckmann

    That’s right. And in case you don’t want to hard code it each time you need it, just use a wrapper for the apply functions: http://ryouready.wordpress.com/2010/01/11/progress-bars-in-r-part-ii-a-wrapper-for-apply-functions/
    Mark


  1. 1 Progress bar | Язык программирования R
  2. 2 Progress bar в R « Записки юного исследователя
  3. 3 Progress bar в R « Записки юного разработчика
  4. 4 Progress bars in R (part II) – a wrapper for apply functions | "R" you ready?
  5. 5 How to make a progress bar in R » ITers' desktop

Follow

Get every new post delivered to your Inbox.

Join 48 other followers

%d bloggers like this: