home icon contact icon rss icon

Upload progress bar with mod_passenger and apache

UPDATE: I found 2 bugs in upload progress module. If you have already installed. update to at least 0.1 version: http://github.com/drogus/apache-upload-progress-module/commits/0.1

I’ve installed mod passenger on my server recently. It’s really great software. Now I don’t have to worry about monitoring, nginx proxy, load balancing, big file uploads… and it’s fast! With Ruby Enterprise Edition it’s even faster.

Personally I don’t care about people saying that phusion wants to promote themselves on REE as long as it gives faster ruby with lower memory use (but yes, I know, REE is not best choice for a name :).

After installing I’ve realised that my shiny upload progress bar (thanks to Upload Progress Module for nginx) was not working. Oh gods! What a tragedy!

But fear not. I’ve written apache upload progress module to have my lovely progress bar back again. As a lazy developer I’ve implemented reports in the same way as in nginx upload progress, so my applications are working without changing any signle line of code. If you were using nginx upload progress just drop the module, change your config and you’re good to go :)

I’m testing it in one of my production servers, but be carefull – it’s not well tested in other enviroments (i have gentoo with apache 2.2.8). Any feedback will be helpfull. Give me a note in comments if you encounter any problems.

So you want to be cool and have your own sexy progress bar in your app? Keep reading ;)

To install module you must download it using git:
git clone git://github.com/drogus/apache-upload-progress-module.git
or get the package: http://github.com/drogus/apache-upload-progress-module/tarball/master To compile/install/activate you have to use apxs2:
apxs2 -c -i -a mod_upload_progress.c
  • -c is for compiling
  • -i is for installing (copy mod_upload_progress.so to apache library dir)
  • -a is for activating (add LoadModule option into your apache conf file)
If you want to install and activate run this command as a root. Otherwise you can just compile and add LoadModule to apache conf:
LoadModule upload_progress_module path/to/apache-upload-progress-module/.libs/mod_upload_progress.so
Currently there is only one global option:
UploadProgressSharedMemorySize 1024000

This sets shared memory size to 1M. By default it’s 100kB.

To add tracking and reporting upload for a virtual host in apache you will need to add:

<Location />
    # enable tracking uploads in /
    TrackUploads On
</Location>

<Location /progress>
    # enable upload progress reports in /progress
    ReportUploads On
</Location>

Now all uploads will be tracked and reports are under /progress

Format of the report is JSON. From nginx wiki:

The returned document is a JSON text with the possible 4 results:
  • the upload request hasn’t been registered yet or is unknown:

new Object({ ‘state’ : ‘starting’ })

  • the upload request has ended:

new Object({ ‘state’ : ‘done’ })

  • the upload request generated an HTTP error:

new Object({ ‘state’ : ‘error’, ‘status’ : })

One error code that is interesting to track for clients is HTTP error 413 (Request entity too large)

  • the upload request is in progress:

new Object({ ‘state’ : ‘uploading’, ‘received’ : , ‘size’ : })

The HTTP request to this location must have either an X-Progress-ID parameter or X-Progress-ID HTTP header containing the unique identifier as specified in your upload/POST request to the relevant tracked zone. If you are using the X-Progress-ID as a query-string parameter, ensure it is the LAST argument in the URL.

Now the last thing to do is to implement progress bar. I don’t like repeating others and there is great tutorial on setting up upload progress bar with nginx and merb

UPDATE: I released jquery upload progress library with Safari 3 support. More info here. UPDATE2: I’ve upgraded prototype version to work in Safari.

It’s for merb and nginx but if you drop the scripts in your rails app and with apache-upload-progress-module it will work. :) Basically if you have your own code handling uploads (for example using attachment_fu) you can just add javascript and css – it’s unobtrusive.

If you’re using prototype I’ve rewritten script and made a demo. You can also grab files

I hope you enjoy this article. Progress bar is in my opinion one of the most useful technics – there is nothing more annoying than large file uploading without any info on state of an upload.

pimpmaster said

Jun 25, 2008 @ 09:23 PM

Nice writeup! :)

Your demo is busted in Safari 3 though (No progress is shown) :(

Micha said

Jun 26, 2008 @ 10:41 AM

For some reason, I get an error in my error_log: [error] (12)Cannot allocate memory: Upload Progress cache: could not create shared memory segment Configuration Failed

This error can be avoided by setting UploadProgressSharedMemorySize to something less than ~75000.

My Apache is Server version: Apache/2.2.8 (Unix) Server built: Mar 4 2008 21:37:02 on MacOS 10.5 (the builtin one) and was compiled with http://pastie.org/222492

Any thoughts?

Drogomir said

Jun 26, 2008 @ 11:16 AM

pimpmaster: I will check it :)

Micha: I don’t know what causes this problem but I changed default shared memory size to 50kB – it will be safer then 100kB. You can change shared memory size even to 10240 (10kB) if you don’t provide hundreds of uploads simultaneously it’s not a problem – it’s only for keeping upload data (key, upload size and size of received bytes). Upload data is removed from memory 60s after upload has ended.

Ziggy said

Jun 26, 2008 @ 09:02 PM

Does this work with https ? I can’t seem to get it working on FC8/x86_64, the /progress keeps reporting ‘starting’ .... httpd-2.2.8-1.fc8/php-5.2.4-3

Ziggy said

Jun 26, 2008 @ 09:59 PM

Doesn’t seem to matter if it’s https or not, the /progress is stuck on ‘starting’ anyways (FC8 etc.)

Drogomir said

Jun 27, 2008 @ 12:16 AM

Ziggy: Did you provide existing X-Progress-ID? It is in “starting” state if it can’t find valid upload data with given X-Progress-ID.

Ted said

Jun 27, 2008 @ 06:52 PM

Confirm not working in safari 3. Seems like a JS issue? =/

Drogomir said

Jun 28, 2008 @ 10:34 PM

I’ve checked it in safari on windows and it seems to be problem with sending ajax request while page is loading. I will investigate this one :)

Michele said

Jun 29, 2008 @ 01:02 AM

Yep, it definitely seems to be a problem with sending an AJAX request while loading a page (indeed, if you manually enter the AJAX call in the address bar prepending javascript: the call is executed correctly, but then it stops sending the main request).

I imagine it should be possible to fix this by using an iframe to separate the two requests…I’ll play with it tomorrow.

Meanwhile, thanks for the module: it has really helped! :)

Michele said

Jun 29, 2008 @ 12:36 PM

I managed to make it work in Safari, too: you need to insert an iframe in the page, give it a name (e.g. name=”progress”) and have it load a static HTML page which you’ve carefully crafted with: basic HTML structure (making sure you load all JS libraries you need) and in the body you define a function with the AJAX call and logic to update the progress bar. Of course, this function needs to passed the uuid.

Then from the main page you call name_given_to_the_iframe.name_given_to_the_function(uuid) In the function you can access the main page DOM object like so: parent.document.getElementById()

Drogomir said

Jun 29, 2008 @ 04:26 PM

Michele:

Thanks for checking it. I will update my example shortly :)

I will add Safari hacks to upload-progress script: http://github.com/drogus/jquery-upload-progress/tree/master and possibly make prototype version with some tweaks like in jQuery version.

Michele said

Jun 29, 2008 @ 04:57 PM

Just let me know if there’s anything not clear in my explanation. :)

Drogomir said

Jun 30, 2008 @ 03:03 AM

I’ve added safari support to jquery version of upload progress script: http://github.com/drogus/jquery-upload-progress/tree/master . Tomorrow I will try to code prototype version with callbacks and other useful stuff.

Check new demo which is supposed to work in safari: http://drogomir.com/files/blog/jquery-upload-progress/example

Matze said

Aug 20, 2008 @ 01:19 AM

Hi, i just installed upload_progress_bar and my Progressbar stopps counting after about 250000Bytes received. Upload still works and the status changed to done after finishing upload.

My Setup: Apache 2.2.9, mod_rails (Passenger) 2.0.3 and apache-mod-upload-progress (installed yesterday)

Any Ideas?

btw. thanks for the apache-mod!

Drogomir said

Aug 20, 2008 @ 07:20 AM

Matze:

Which browser did you use? While using Firefox 3 I have similar problems. Try to clear browser cache when the file is uploading and see if progress bar is still stuck. If yes, then it’s browser problem. If no, could you enable debug messages in apache and see if anything weird is reported in the log file?

Matze said

Aug 20, 2008 @ 01:13 PM

hmmm… indeed, it’s working on my Windows-Box with IE 7, FF2 and FF3. On my OS X (10.5.4) with FF3 and FF2 i had the above mentioned problem.

Sorry for the quick post without testing on an other box and thanks for your quick reply!

Maybe it’s a problem with caching, i will test it coming weekend.

Matze

OWL said

Aug 25, 2008 @ 01:51 AM

Hi, Thanks for module. Works fine with IE and FF but does not work with Opera. checked with opera = 9.51 under winxp.

Looks like Opera can’t do Ajax request during file upload. Does anybody know a hack for Opera ?

OWL said

Aug 25, 2008 @ 01:51 AM

Hi, Thanks for module. Works fine with IE and FF but does not work with Opera. checked with opera = 9.51 under winxp.

Looks like Opera can’t do Ajax request during file upload. Does anybody know a hack for Opera ?

Drogomir said

Aug 25, 2008 @ 10:50 AM

It works in Opera 9.5 for me (Ubuntu). I will check 9.51

If it can’t send ajax request it is the same issue that safari has.

Adam said

Aug 28, 2008 @ 04:33 AM

Hi There,Thanks for this patch. its awesome. Can anyone explain whether using this combined with mod_rails will get past the blocking issue with rails uploads? ie, will apache handle the upload and then pass it to rails or is rails handling the whole thing ??

Drogomir said

Aug 30, 2008 @ 01:17 PM

About the Opera: I’ve checked 9.52 and it is not working for me either. Hacks for safari are not working, so I will have to debug it on Opera.

Adam: mod_passenger is handling uploads – it’s faster and it doesn’t block rails instance. If you want to do something while uploading you must use Apache filters (like in my upload progress module)

RSS feed for comments on this post

Leave a Comment