File Downloads in Rails on Apache

Over the past half a year we worked on several systems for logistics/courier/delivery companies. It was quite fun as a big logistics and inventory control system was my very first project when I arrived to Asia 9 years ago. Last week we launched one of them. It was slightly different as most of our systems are used mainly internally but this one is also used by clients of the courier company to book deliveries. Each delivery requires printing of confirmation documents, internal routing documents, then signed and stamped documents are uploaded back to the system as a proof of delivery for clients. It has quite a number of other (technically) interesting features like integration with google maps to geo-tag the orders and notifying drivers in the area - I'll write about it soon.

As this courier company handles several hundreds of deliveries every day it requires quite a number of uploads/downloads of documents and images. The issue with uploads and downloads is that they will tie up rails processes for much longer than usual requests. The problem is that rails processes are quite expensive in terms of memory (around 70 - 100MB for simple operations). Do 20 concurrent uploads/downloads and all your rails processes and memory will get tied up - and quite wastefully as there is no 'dynamic' processing during upload or download. The process just sits there and waits for the file to get delivered. Luckily there's a simple way how to move this burden to Apache. This kind of upload/download processes will take only around 2MB on average and your rails processes are free to handle the dynamic requests.

To let apache handle file downloads use x_sendfile. All you need to do is install x_senfile module on apache and adjust you config a little. Here's the installation instructions for Ubuntu 8.04:

cd /usr/local/src
wget http://tn123.ath.cx/mod_xsendfile/mod_xsendfile.c
apxs2 -cia mod_xsendfile.c

Open apache config file and add the module line:

sudo vim /etc/apache2/apache2.conf

LoadModule xsendfile_module /usr/lib/apache2/modules/mod_xsendfile.so

Add the following 2 lines to your virtual host file

sudo vim /etc/apache2/sites-available/my_app

XSendfile on
XSendFileAllowAbove on

You will have to restart the apache and you should be done:

sudo /etc/init.d/apache2 restart

Now in your controller handling download add x_sendfile attribute and set it to true

 send_file("/home/test/big_file.zip",
           :filename => "big_file.zip",
           :type => "application/zip,
           :x_sendfile=> true)

The uploads are equally easy. You just need to apache modules libapreq2 and modporter.

wget http://mirror.nus.edu.sg/apache/httpd/libapreq/libapreq2-2.12.tar.gz
tar -xzf libapreq2-2.12.tar.gz
cd libapreq2-2.12
./configure --with-apache2-apxs=/usr/bin/apxs2
make
sudo make install

Now download and install modporter. Get it from the github. Either use the download link and unzip or just clone the repository

git clone git://github.com/actionrails/modporter.git
cd modporter

On Ubuntu I had to change reference to apxs in Rakefile as the in ubuntu it's apxs2.

vim Rakefile

Change the APXS line to the following:

APXS           = "apxs2"

No just run rake:

sudo rake

add the following to your apache2.conf

sudo vim /etc/apache2/apache2.conf

LoadModule apreq_module /usr/lib/apache2/modules/mod_apreq2.so
LoadModule porter_module /usr/lib/apache2/modules/mod_porter.so

To actually use it in your rails application you will need modporter plugin.

script/plugin install git://github.com/actionrails/modporter-plugin.git

Add the following to the application_controller.rb (application.rb before Rails 2.3)

class ApplicationController < ActionController::Base 
mod_porter_secret = "secret"
.
.

and just enable porter in you vhost file:

sudo vim /etc/apache2/sites-available/my_app

Porter On
PorterSharedSecret secret

And that's all now if you check the uploaded files they will be a ModPorter::UploadedFile Credit for this goes to Koz from theRailsWay.com - you can find more information here.