FCGI listeners and Lighty on single box.

LighttpdWithProcessScripts

Home Page | All Pages | Recently Revised | Feed

This article discusses how to setup lighttpd with the scripts in script/process as introduced in the 0.14.1 release (aka 1.0RC1). I publish this in the hope that it is useful to you but won’t take any guarantees or blames for you screwing up your server.

Introduction

Welcome to the wonderful world of deploying Ruby on Rails with lighttpd in the 1.0 series. From 1.0 RC1, Rails applications have a script/process directory that helps you to manage your application when running it with FastCGI.

We will roughly follow the deployment setup as described in the Capistrano manual – at least from the structure of the server. Capistrano was formerly called SwitchTower.

As in the Capistrano documentation, we will first describe a distributed setup to clarify which parts the system has. If we talk about them as separate parts, we accomplish a Good Thing: loose coupling. If we can just take a part of the system and put it on another machine this gives us flexibility which is desirable.

Let’s fix the following nomenclauture:

• The development box is the machine that you use for developing. You run your text editor here, you have the copy of the code you edit on that computer and you start your SSH sessions from here.
• The database server is the computer instance running your RDBMS like PostgreSQL, MySQL etc.
• The application server is the computer running the Ruby instances that execute your Ruby On Rails code.
• The web server is the computer instance running the web server which in turn delegates requests to the Ruby instances running your RoR code.
• The deployment target box is the server that you want your code to run on. This box happens to take – as described above – all three roles of a web, application and database server.

Explanation Of The Server Types

On the web server machine, we will run lighttpd. Lighttpd is a HTTP server similar to Apache but has a much more restricted feature set. In a typical RoR application, you will have static content like images and CSS files and dynamic content which is your Ruby code. The web server serves the static content and delegates all other request to our Ruby instances. Lighttpd is the perfect program for these simple jobs.

On the application server machine, we will run our Ruby on Rails code. We will run it in FastCGI mode. This means we will launch some – let’s say 10 – servers written in Ruby. These servers provide their services using FastCGI, a protocol that is suited for communication with web applications. These processes will run our Ruby on Rails application. Luckily, we will not have to change our code to run in these FastCGI server processes since the guys who wrote RoR took care of that. Whatever you develop with (Webrick is my favourite and the simplest to use), you can deploy as FastCGI with close to no code change.

The database server machine will be the home of our RDBMS like PostgreSQL or MySQL. We don’t have to talk about that – you will already know the concept of RDBMS.

Let’s repeat why talking about the three “server machines” as separate computers is a good thing. When we wanted to scale our application up, we would only have to tell our web server that our FastCGI processes now run on another IP than localhost and then our application server that the database’s IP has changed as well.

Thus we can talk about the three “application components”: database, application, and web server, in an abstract manner. The concept is visualized in the following diagram. Look over it briefly and make sure that everything is clear and we will move on a bit more swiftly.

          .----------.             .---------.         .----------.           |          |             |         |         |          |           |          |             |         |         |          |    HTTP   |          |   FastCGI   | FastCGI |         |  RDBMS   |  <=====>  | lighttpd |  <=======>  | Ruby On |  <===>  |   like   |  request  |          |   request   |  Rails  |         |   MySQL  |           |          |             | Process |         | Postgres |           |          |             |         |         |          |           '----------'             '---------'         '----------' 

Software Installation

Get the lighttpd sources from their download page, compile it and install it as described in lighttpd’s documentation.

Now, install ruby-fcgi manually as the gem doesn’t work (in irb, try: require ‘fcgi’) – you may need to install FastCGI Development Kit first. Install from package libfcgi-dev or from source

Then, copy your application’s file on the server. We assume your application is called example and its RAILS_ROOT (i.e. the directory that contains app, config etc.) is /var/www/example. Make sure the user you execute lighttpd as has the necessary permissions to write into log and your database connection setup is sane (i.e. your application can connect to the database server with the credentials given in the production of config/database.yml).

Theoretically, you should be able to run ./script/server -e production and your application should run with .

Spawn me, Reap me

We will have a look at managing our FastCGI processes now. The Rails 1.1 series comes with two scripts in script/process:

• spawner – The spawner creates a FastCGI process.
• reaper – The reaper script can be used to restart or stop FastCGI processes.

By, default, spawner starts 3 FastCGI processes listening on ports 8000-8002. We will need this information later to tell lighttpd where it should redirect requests to our application to.

Three processes is just enough to get you started. Depending on the memory footprint of your application, it’s CPU usage and the load it comes under you could increase or reduce that count.

Both spawner and reaper have good documentation which you can see using --help option.

Let’s play around a bit with the scripts.

Starting Processes

First, we assume that no FastCGI process is running and we want to get our application start to run. Thus (in /var/www/example) we call our spin script:

 $> script/process/spawner 

This will get us started with 3 FastCGI processes. Test that these processes are really running by running ps ax | grep dispatch.fcgi.

Restarting Processes

OK, now we have running processes. When we update our application code, though, these changes are not automatically reflected in our running processes since RoR does not reparse changed files in production mode for performance issues. We have to restart the processes to make them reload the files.

We can use the reaper script for this by executing:

 $> script/process/reaper -a restart 

Closing Processes

Let’s now assume we want to stop all FastCGI processes since we want to shut our machine down for maintenance. We can again use the reaper to stop the scripts. We can try to shutdown the processes gracefully and wait for them finishing their current task or we can force them to stop by killing them:

 $> script/process/reaper -a graceful OR  $> script/process/reaper -a kill 

Lighttpd Configuration

Now that we can manage our processes, we will setup lighttpd to publish the FastCGI processes so our application is useable by the folks out there in the internet.

The following is a minimal lighttpd configuration file that you can adapt to your environment. You might want to set the user the server is running as and load more modules etc.

Look over the file and we will talk about the emphasized sections below.

#### general configuration  server.modules = ( "mod_rewrite", "mod_redirect", "mod_fastcgi", "mod_accesslog" )  server.port         = 80 server.bind         = "10.2.2.1"   server.pid-file     = "/var/run/lighttpd.pid"   ############ BEGIN HIGHLIGHT ################## server.document-root        = "/var/www/example/public/" # (1) server.indexfiles           = ( "index.html", "dispatch.fcgi" ) # (2) server.error-handler-404    = "/dispatch.fcgi" # (3) ############ END HIGHLIGHT ##################  accesslog.filename          = "/var/www/example/log/lighttpd_access.log"  server.errorlog            = "/var/www/example/log/lighttpd_error.log"   mimetype.assign = (    ".rpm"          =>      "application/x-rpm",    # and so on )  #### fastcgi module  # Notes: #  * lighttpd does not like host names here #  * lighttpd wants ports to be numbers, not strings  ############ BEGIN HIGHLIGHT ################## fastcgi.server = ( ".fcgi" =>     ( "localhost-8000" => ( "host" => "127.0.0.1", "port" => 8000 ) ,      "localhost-8001" => ( "host" => "127.0.0.1", "port" => 8001 ) ,     "localhost-8002" => ( "host" => "127.0.0.1", "port" => 8002 ) ) ) ############ END HIGHLIGHT ################## 

The first emphasized section tells the server where it should look for the static files (which will be served before any request is forwarded to the FastCGI instance if a file with this path is requested) (1). Then it tells lighttpd to server the index.html file if it exist and otherwise to forward the request to the dispatch.fcgi file (2). This is where we hook in our FastCGI processes later. The next line (3) tells lighttpd to let our FastCGI process also handle 404 errors.

The second emphasized section tells lighttpd that it should forward requests to file ending with ”.fcgi” to our FastCGI processes. We setup our spin script to start 5 processes and we have to tell lighttpd about each. Lighttpd will automatically load balance between these servers. If we had the FastCGI processes running on another box, we could simply specify that one here.

[As far as I can tell, this configuration isn’t load balancing between the fcgi processes, at least not on FreeBSD. I have one controller that calls up an external script, which then calls back to a different controller within the same app. It’s timing out every time waiting for the request to be serviced.—kurt@mrkurt.com ]

[Fixed the example FastCGI config to fix the issue above. Seen here —kurt@mrkurt.com ]

Start Lighttpd

Now we can start lighttpd (with /etc/init.d/lighttpd start if you installed a startup script there) and put our browser to http://example.com where we can marvel at our new RoR applications running on lighttpd.

By default, spawner starts fastCGI threads in production mode.

If you want to start threads in development mode, use:

$> script/process/spawner -e development 

Note that restarting lighttpd does not restart your FastCGI threads. You have to take care of that by yourself. If you want to go on to the next level, you should have a look at Capistrano’s documentation where you can find out how to make deployment even more comfortable. Capistrano comes with lighttpd and a distributed setup in mind (even if the manual is not so clear about the spinner/spawner/reaper scripts).

[At least on my FreeBSD-machine, lighttpd-1.3.16 correctly tears down my RoR app with ruby18-fcgi-0.8.6 and rubygem-rails-0.13.1—vs]

[If you compiled lighttpd-1.4.16 and you see “You are being redirected” page, instead of a normal HTTP Redirect, then comment out line 1448 in src/connections.c and recompile Lighttpd. The problem is described here: http://trac.lighttpd.net/trac/ticket/1270 —alevchuk]

Capistrano deployment and spawner/reaper.

There is no spinner in Rails 1.1 (correct me if I’m wrong) but spawner can respawn fcgi processes if they die.

The problem is that spawner writes pid files into the tmp/pids directory of your current deployed version of app. When you deploy new version, you want to invoke reaper script to restart fcgi listeners, but the pid files are now in the previous release (yes it sounds complicated). We don’t want to kill poor spawner and his beloved kids. We want them to live forever! So the trick is to copy pid files from previous release to current one (simple huh?). Below is a custom Capistrano restart task:

 desc "Restarts fcgi listeners"  task :restart, :roles => :app do   puts "Restarting fcgi listeners..."    run <<-CMD          cp -r #{previous_release}/tmp/pids #{current_path}/tmp/ &&                      #{current_path}/script/process/reaper      CMD  end  

Extended info can be found here

Since there is no spinner, a script/spin task is required for the user to create. The bad part is, in the Capistrano doc, this thing is hard-coded to a directory. Here’s a way to get around that. Create a script/spin file:

 !/bin/bash  # this spin script requires some variables to be passed in # it also calls the spawner using nohup, see: #  <a href="http://johnwilger.com/articles/2006/04/03/fix-signalerror-when-executing-rake-remote-spinner-rake-remote-cold_deploy-under-capistrano-1-1-0-and-rails-1-1-0">http://johnwilger.com/articles/2006/04/03/fix-signalerror-when-executing-rake-remote-spinner-rake-remote-cold_deploy-under-capistrano-1-1-0-and-rails-1-1-0</a> #  for details # TODO: have them default to acceptable values  RELEASE_PATH=$1 START_PORT=$2 INSTANCES=$3  nohup $RELEASE_PATH/script/process/spawner -p $START_PORT -i $INSTANCES 

And since it requires variables passed in on the command line (hence the $1, $2, and $3 variables, you overwrite the script/spinner task in your config/deploy.rb like so:

 desc "Start the fcgi processes if they aren't already running."  task :spinner, :roles => :app do   run "#{current_path}/script/spin #{current_path} #{start_port} #{instances}" # custom spin task (is it just me or is the one in the capistrano docs hardcoded?) end 

Anyone know if you can get ENV variables for those to not have to make a nasty spin task that’s hardcoded for the app server?

One thing that annoyed me (not any more) was the fact fcgi processes listen on wildcard ip address (0.0.0.0) which is not wanted for obvious reasons (single machine setup). This is another custom Capistrano task which is invoked only on so-called cold_deploy:

 desc "Starts fcgi listeners"  task :spawn, :roles => :app do   puts "Starting fcgi listeners..."    run "#{current_path}/script/process/spawner -r 5 -s \"/usr/bin/env spawn-fcgi -a 127.0.0.1\""  end 

Contribute

If you find any errors in this text or if you think some sections should be improved, then feel free to hack away on this wiki page by clicking the “edit” link below.