Peter Marklund

Peter Marklund's Home

Peter on Software

Lessons Learned in software and web development Subscribe

2008-08-12

Installing Ruby on Rails on Ubuntu 8.04 Hardy Heron

I've compiled a set of detailed instructions on how to install Ruby on Rails on Ubuntu 8.04 Hardy Heron. The instructions are available in GitHub and they show you how to turn a clean Ubuntu 8.04 install into a production ready Ruby on Rails stack including MySQL, Nginx with fair proxy balancer, Monit, and Mongrel Cluster. There are a few additions and improvements I'd like to make:

Please help me point what else is missing or how I can improve.

I'm talking to GleSYS about the possibility of offering my production setup as an installable VPS image.

12 comment(s)

2008-08-06

Faster Capistrano Subversion Deployments with remote_cache

This is old news, but if you are deploying an application with Capistrano from Subversion and you find yourself waiting impatiently for the checkout to happen everytime, try adding this to deploy.rb:

set :deploy_via, :remote_cache

With this setting Capistrano will keep a Subversion checkout under shared/cached-copy, and on each deploy just do an svn update followed by a local copy of the whole tree into releases. I found this to be significantly faster than the default checkout on each deploy.

I am planning to switch to Git and GitHub now, and I am curious to see if this setting will affect deployment speed with Git. Given that a git clone in my experience is blazingly fast, maybe the remote cache won't be needed?

4 comment(s)

2008-08-02

Installing Telenor 3g Modem (Mobilt Bredband) on Ubuntu

As far as I know none of the turbo 3g GSM modems on the swedish market officially support Linux. However, I was able to get my Telenor modem working just fine on my Ubuntu Eee just now. I used the USB_ModeSwitch software along with swedish instructions from Hasain. Once my modem was recognized I ran sudo wvdialconf. I then edited my /etc/wvdial.conf to be:

[Dialer Defaults]
#Init1 = ATZ
Init1 = AT+CPIN=<YOUR PIN HERE>
Init2 = ATQ0 V1 E1 S0=0 &C1 &D2 +FCLASS=0
Modem Type = Analog Modem
Baud = 9600
New PPPD = yes
Modem = /dev/ttyUSB0
ISDN = 0
Phone = *99#
Password = peter
Username = peter

Note that you have to change the PIN above. I then ran sudo wvdial and voila - I was online! Well, online on a slow and unreliable connection that is. Oh well, I'm thinking of changing to tre.se one of these days. They supposedly have the best 3g network.

4 comment(s)

2008-08-02

Asus Eee + Ubuntu + Rails

On thursday I bought myself an Asus Eee 900 - a tiny and cheap Linux powered laptop that is currently selling out in stores here in Sweden. I got a lot of attention in the office with this laptop and within a day it seemed every programmer in the office had an Eee on their desk.

I installed Ubuntu Eee and this was a huge improvement over the Linux OS that Asus provides. I am ashamed to admit that I haven't used Linux on the desktop for a long time and I was totally blown away by how advanced, slick, and user friendly Ubuntu has gotten.

Obviously, my ultimate goal was to install Ruby on Rails. At first I wanted to install from source in order to get an exact version and patch level of Ruby, namely the one that is officially recommended on ruby-lang.org. However, when attempting this various libraries were missing. I found a FiveRuns article listing the packages I needed but after I had installed them I ran into an issue with the MD5 library. In the end I resorted to using the ruby-full package which gives you the old tried and tested Ruby 1.8.6 patch level 111 (without the recent security patches). Here, roughly, are the steps I went through to set up my Rails environment:

  #####################################
  # RUBY
  #####################################

  sudo apt-get install ruby-full
  which ruby
  ruby -v
  # => ruby 1.8.6 (2007-09-24 patchlevel 111) [i486-linux]
  ruby -ropenssl -rzlib -rreadline -e "puts :success" 

  #####################################
  # RUBYGEMS
  #####################################

  # Get latest stable recommended release of RubyGems from rubygems.org
  wget http://rubyforge.org/frs/download.php/38646/rubygems-1.2.0.tgz
  tar xzf rubygems-1.2.0.tgz
  cd rubygems-1.2.0/
  sudo ruby setup.rb
  # Not sure if/why this step is necessary
  sudo ln -s /usr/bin/gem1.8 /usr/bin/gem
  which gem
  gem --version
  # => 1.2.0

  #####################################
  # MYSQL
  #####################################

  sudo aptitude install mysql-server mysql-client libmysqlclient15-dev

  #####################################
  # Some useful Gems
  #####################################

  sudo gem install rails mongrel capistrano mysql

I haven't double checked those instructions for accuracy since I would have to re-install my OS to do that. If you find errors or have improvements, please let me know. Here is a sample of articles on the subject that you might want to check out if you want to dig deeper:

3 comment(s)

2008-07-04

Rails Configuration: Yielding self in initializer

I've come across situations in Rails where you want to repeatedly invoke methods on some class with a long name and it gets ugly and tedious that you have to repeat the class name on every line. I just realized that unlike in the config/environments files the config object is not available in the config/initializer files. I use the AppConfig plugin to parameterize my application and I came up with the yield_self method to make my config/intializers/app_config.rb more readable:

# Method to supplement the app_config plugin. I want to crash early and be alerted if
# I have forgotten to define a parameter in an environment.
def config_param(name)
  AppConfig.param(name) do
    raise("No app_config param '#{name}' defined in environment #{RAILS_ENV}, " +
      "please define it in config/environments/#{RAILS_ENV}.rb and restart the server")
  end
end

class Object
  def yield_self
    yield self
  end
end

AppConfig::Base.yield_self do |config|
  config.site_name = "Simple Signup"
  config.admin_email = '"Simple Signup" <info@simplesignup.se>'
  config.exception_emails = %w(info@simplesignup.se)
  config.email_prefix = "[#{config.site_name}]"
  config.signup_timeout = 5
  config.send_activate_email = false

  config.transaction_fee = lambda do |price|
    price.blank? ? 0.0 : [10.0, 5.0+0.035*price.to_f].max.round(2)
  end

  config.bank_fee = lambda do |price, card|
    0.0
  end
end

ExceptionNotifier.exception_recipients = config_param(:exception_emails)
ExceptionNotifier.sender_address = %{peter_marklund@fastmail.fm}
ExceptionNotifier.email_prefix = "#{config_param(:email_prefix)} ERROR: "

The method config_param might be more appropriately named app_config. That will be a refactoring for another day though.

2008-06-26

Ruby Gotcha: Default Values for the options Argument Hash

Like in Java, method arguments in Ruby are passed by reference rather than by copy. This means it's possible for the method to modify the arguments that it receives. This can happen unintentionally and be a very unpleasant surprise for the caller. A good example of this is the typical options = {} at the end of the method argument list. If you set a default value in that hash then that is a potential issue for the caller when the options hash is reused in subsequent calls (i.e. in a loop). See below for an example:

  def foreign_key(from_table, from_column, options = {})
    options[:to_table] ||= from_column.to_s[/^(.+)_id$/, 1].pluralize
    execute ["alter table #{from_table}",
            "add constraint #{foreign_key_name(from_table, from_column)}",
            "foreign key (#{from_column})",
            "references #{options[:to_table]}(id)",
            "#{on_delete_clause(options[:on_delete])}"].join(" ")
  end
  
  def foreign_keys(from_table, from_columns, options = {})
    from_columns.each do |from_column|
      foreign_key(from_table, from_column, options)
    end
  end

In the first invocation of foreign_key options[:to_table] will be set (if it isn't set already) to the destination table of the first column. The options[:to_table] value will be retained throughout the loop causing all foreign keys to point to the same table. The fix is to make to_table a local variable or to do add an "options = options.dup" line at the beginning of the method.

Lesson learned - avoid modifying the options hash and any other arguments that the caller doesn't expect will be modified.

1 comment(s)

2008-06-19

Rails Testing Tip: Use Global Fixtures to Avoid Fixture Mayhem

I'm in a Rails team with mixed opinions on whether to use fixtures. Therefore we have everything from RSpec specifications that use a lot of mocking/stubbing and don't touch the database, to specifications that set up their own databse data through helper methods and the specifications that I write that rely mostly on fixture data. What I have found is that when you don't use global fixtures (a setting in your test_helper.rb or spec_helper.rb file) you can run into situations where seemingly unrelated specifications/tests fail, and fail in different ways depending on if you run them in separation, through autotest, or with rake. What is going on is test data spillover/interference between tests. This can lead to very long and frustrating debugging sessions indeed. The best way to avoid this seems to be to turn on global fixtures. This will probably increase the specification run time, an issue that I partially adress by keeping the number of records in my fixture files to a minimum. Also, I prioritize test coverage and convenient access to a common set of test data over making my specifications run faster.

2008-06-16

Rails Optimistic Locking - Not Worth it for Me

When I upgraded to Rails 2.1 ActiveRecord partial updates were turned on, i.e. when you save a record only those attributes that have changed are saved to the database. In theory, if you have two almost simulateneous updates, and you have a validation rule across several columns, then those updates can render the database record in an invalid state. Of course, in practice, this is very unlikely to happen. Nevertheless, I decided to turn on optimistic locking to be on the safe side. It turned out optimistic locking caused more issues than it was worth. Suppose you have an Article model that has many instances of Chapter and also that the Article uses a counter cache. Then you can run into this issue:

article = Article.first

article.chapters.create(:name => "Summary")
# => UPDATE articles SET chapters_count = chapters_count + 1,
#    lock_version = lock_version + 1 WHERE id = XXX;

article.publish_date = 1.days.from_now
article.save
# => throws ActiveRecord::StaleObjectError

I like partial updates though since they make it less likely that simulteneous updates will clobber, since you are only writing to the db what has changed. It also makes SQL statements in the log file more readable.

I'm abandoning optimistic locking for now though.

2008-06-16

Ruby Gotcha: Symlinked Scripts and File.dirname(__FILE__)

If you have a Ruby script say in ~/src/ruby/my_script that you are symlinking to from ~/bin/my_script, then invoking File.dirname(__FILE__) in that script will yield the directory of the symlink not the directory of the script file. If you want the directory of the script file you can do this instead:

THIS_FILE = File.symlink?(__FILE__) ? File.readlink(__FILE__) : __FILE__

THIS_FILE will contain the path to the script file instead of the path to the symlink. This is valuable if say you want to require some Ruby library from your script and you are using a relative path.

5 comment(s)

2008-06-16

Rails Tip: Validating Option Arguments

I think it's a good convention to validate options arguments passed to methods, i.e. to make sure they have valid keys and values. Misspellings can otherwise lead to unnecessary debugging sessions. Rails comes with the assert_valid_keys method. I added the assert_value method:

module ActiveSupport #:nodoc:
  module CoreExtensions #:nodoc:
    module Hash #:nodoc:
      module Keys
        # Assert that option with given key is in list
        def assert_value(key, options = {})
          options.assert_valid_keys([:in])
          if !(options[:in].map(&:to_s) + ['']).include?(self[key].to_s)
            raise(ArgumentError, "Invalid value '#{self[key]}' for option '#{key}' must be one of '#{options[:in].join(', ')}'")
          end
        end
      end
    end
  end
end

Example usage and RSpec specification:

def price(feature_set_ids, options = {})
  options.assert_valid_keys([:billing_period])
  options.assert_value(:billing_period, :in => ["annual", "monthly"])
  # method logic here...
end

# Rspec:

it "cannot be invoked with an invalid option" do
  lambda {
    @campaign.price([23], :foobar => true)
  }.should raise_error(ArgumentError)
end

it "cannot be invoked with an invalid billing period" do
  lambda {
    @campaign.price([23], :billing_period => :foobar)
  }.should raise_error(ArgumentError)
end

1 comment(s)

2008-06-16

Rails plugin: acts_as_state_machine_hacked

I've been using the acts_as_state_machine plugin in a couple of projects. I think the syntax and functionality of the plugin is quite nice. It allows you to easily define states and events (transitions between states) for your ActiveRecord model. However, I wanted to be able to see which states are available in the current state. Also I thought that invoking an event, such as user.activate!, when the user is in a state where the activate event is not available should not yield a silent failure, but rather throw an exception. Also, if the event fails to fire because of a guard condition then an exception should also be raised. I encapsulated those changes in the plugin acts_as_state_machine_hacked.

1 comment(s)

2008-06-15

DreamHost Hacked: All My Files Exposed Publicly

An ex-colleague of mine discovered that all my files in my home directory at the hosting company DreamHost were publicly viewable and downloadable on the web. I was quite shocked. I had certainly not intended to share all my private files with the world, especially since they contained some highly sensitive information. I assumed my account at DreamHost had been hacked. However the response from DreamHost support was that this was not the case. They explained that it was merely a symbolic link to my home directory that had been created:

"if you would like to keep this from happening you can prevent all other users on the server from viewing your account's files by enabling the Enhanced Security feature for your user. Just go to the Users > Manage Users section of your panel, click the "Edit" link next to your user, and then check to enable the Enhanced Security option. Hit the "Save Changes" button and you should be set in about 20 minutes.

The /home/ directory is public and it is not a security breach that the other user was able to create a symbolic link to /home/. Other users on the server have always had the same access, which means that they have been able to view your files but they absolutely cannot make any changes to your files or folders. The Enhanced Security feature takes it a step further and prevents any user from even viewing your files or folders.

So, just to be clear, there is no indication of a server hack or any security intrusion."

I wrote back that I had changed to "Enhanced" security and that my files were still exposed. Here are some excerpts from their second reply:

"Ultimately this was just some funny permissions on your home directory which caused this to be allowed to happen."

"When I changed your home directory's group ownership back to your default group (pg136611) this corrected the insecurity of other user's accessing your files via apache"

"The interesting part is it may have been enabling the extra web security which caused this insecurity."

A few days later I found that my files were still exposed and I had to manually change the group of my home directory. Basically as far as I'm concerned this means the issue has still not been fixed in a reliable fashion.

I've heard no apology from DreamHost so far. In fact, there is not much in their replies that indicates that they are even taking the issue very seriously. I'm quite disappointed and I am not left with much confidence in DreamHost when it comes to security and privacy.

6 comment(s)

2008-05-19

Rails Testing Tip: Validate your Fixtures

I realized today how important it can be to validate the fixtures you use in your Test::Unit tests or RSpec specifications. I made some schema and validation changes and neglected to update all my fixtures which lead to a long and tedious debugging session. I added this RSpec specification to make sure I never have invalid fixtures again:

describe "fixtures" do
  it "should be valid" do
    ActiveRecord::Base.fixture_tables.each do |table_name|
      klass = table_name.to_s.classify.constantize
      klass.send(:find, :all).each do |object|
        puts("#{klass.name} #{object} is invalid: #{object.errors.full_messages.join(', ')}") if !object.valid?
        object.should be_valid
      end
    end
  end
end

Note: the fixtures_tables method is just a method I have defined that returns a list of all my fixture tables and I use it to set global fixtures in my test_helper.rb and spec_helper.rb files. If you are not using global fixtures, you can use this spec instead:

describe "fixtures" do
  it "should be valid" do
    Fixtures.create_fixtures(fixture_path, all_fixture_tables)
    all_fixture_tables.each do |table_name|
      begin
        klass = table_name.to_s.classify.constantize
        klass.send(:find, :all).each do |object|
          puts("#{klass.name} #{object} is invalid: #{object.errors.full_messages.join(', ')}") if !object.valid?
          object.should be_valid
        end
      rescue NameError
        # Probably a has and belongs to many mapping table with no ActiveRecord model
      end
    end
  end

  def fixture_path
    Spec::Runner.configuration.fixture_path
  end

  def all_fixture_tables
    Dir[File.join(fixture_path, "*.yml")].map { |file| File.basename(file[/^(.+)\.[^.]+?$/, 1]) }    
  end
end

I think it would be nice if Rails/RSpec has fixture validation built in and turned on by default.

2008-05-12

Rails Tip: Validating Option Arguments in your Methods

I think it's a good convention to validate that options passed to methods have valid keys and values. Misspellings can otherwise lead to unnecessary debugging sessions. Rails comes with the Hash#assert_valid_keys method. I added the assert_value method:

module ActiveSupport #:nodoc:
  module CoreExtensions #:nodoc:
    module Hash #:nodoc:
      module Keys
        # Assert that option with given key is in list
        def assert_value(key, options = {})
          options.assert_valid_keys([:in])
          if !(options[:in].map(&:to_s) + ['']).include?(self[key].to_s)
            raise(ArgumentError, "Invalid value '#{self[key]}' for option '#{key}' must be one of '#{options[:in].join(', ')}'")
          end          
        end
      end
    end
  end
end

Example usage:

  def price(feature_set_ids, options = {})
    options.assert_valid_keys([:billing_period])
    options.assert_value(:billing_period, :in => ["annual", "monthly"])
    ...

Corresponding RSpec specifications:

    it "cannot be invoked with an invalid option" do
      lambda {
        @campaign.price([23], :foobar => true)        
      }.should raise_error(ArgumentError)
    end
    
    it "cannot be invoked with an invalid billing period" do
      lambda {
        @campaign.price([23], :billing_period => :foobar)        
      }.should raise_error(ArgumentError)      
    end
2008-02-28

RSpec Presentation

I gave a presentation on RSpec today at Diino.com - an online backup provider - and the slides are available here.

1 comment(s)

2008-02-26

Stockholm Ruby User Group Meetup Tonight

We are having a Stockholm Ruby User Group (SHRUG) meetup tonight at Connecta here in Stockholm and it's good fun. Martin Kihlgren demoed his Grusome server based 2D game that uses a server written in Ruby and C for performance. Albert Ramstedt organized an RRobots workshop where you get to code your own little Ruby Robot that will fight other robots.

I gave the presentation "Building Web Apps with Rails 2 - Conventions and Plugins I Use" and the slides are available.

2 comment(s)

2007-11-27

Rails Internationalization with new gibberish_translate Plugin

We needed to internationalize the user interface of a Rails application that we are building and looked at a plethora of alternatives, such as Globalize, Globalite and the localization plugin by DHH. Finally we settled for the Gibberish plugin. Gibberish uses an unusual hybrid approach with both inline english texts in the code and keys for message lookups. A typical Gibberish lookup looks like this:

    update_show_page("The course has been booked"[:bookings_book_confirm])

To complement the Gibberish plugin I've drafted the gibberish_translate plugin that adds a script for extracting all message lookups from a Rails app and a controller with a web UI for doing translations. The plugin also keeps track of which english texts a translation was made from so that you can flag when english texts are changed. The plugin avoids using the database and works directly against the YAML message files in the lang directory.

8 comment(s)

2007-11-05

Rails Gotcha: Date Arithmetic, Time Zones, and Daylight Savings

Timezones and daylight savings can cause us programmers a lot of headache. This is evidenced by the fact that probably the most popular post in this blog ever is the one about a timezone aware datetime picker. Today I spent several hours debugging a problem a client was having with an eternal loop caused by the daylight saving transition October 28/29 in conjunction with 1.day.since. One of the chapters in the Code Review PDF by Geoffrey Grosenbach is about keeping time in your Rails applications in UTC. After todays exercises I must say I could not agree more with Geoffrey. In fact, I would go as far as to say that date arithmetic in Rails 1.2 is not reliable unless you set your ENV['TC'] variable.

In Europe we have daylight savings time between the last sunday in March and the last sunday in October, so this year it was between March the 25th and October 28th. Here in Stockholm this means that in the summer we are UTC+2 and in the winter (i.e. most of the time...) UTC+1. We can see this in the Rails console:

>> ENV['TC']
=> nil
>> Time.parse("2007-03-25")
=> Sun Mar 25 00:00:00 +0100 2007
>> Time.parse("2007-03-25").dst?
=> false
>> Time.parse("2007-03-26")
=> Mon Mar 26 00:00:00 +0200 2007
>> Time.parse("2007-03-26").dst?
=> true
>> Time.parse("2007-10-28")
=> Sun Oct 28 00:00:00 +0200 2007
>> Time.parse("2007-10-28").dst?
=> true
>> Time.parse("2007-10-29")
=> Mon Oct 29 00:00:00 +0100 2007
>> Time.parse("2007-10-29").dst?
=> false

To see how the daylight savings transitions can be a problem. Suppose you have the date string "2007-10-28" (just before the transition) and you want to get the date string for the following day. In Rails 1.2.5 with Ruby 1.8.5 this will happen:

>> 1.day.since Time.parse("2007-10-28")
=> Sun Oct 28 23:00:00 +0100 2007

Notice how we are jumping to 23:00 the same day instead of 00:00 the next day. With Edge Rails (soon to be Rails 2.0), we get this instead:

1.day.since(Time.parse("2007-10-28"))
=> Mon Oct 29 00:00:00 +0100 2007
>> 1.day.since(Time.parse("2007-10-28")).strftime("%Y-%m-%d")
=> "2007-10-29"

In Rails 1.2.5, 1.day.since(date) is the same as doing (date+24*60*60). In Rails 2.0 though 1.day.since(date) will advance only the day part of the date and leave the hours alone, thus making sure the day is incremented even when crossing over a daylight savings transition.

To be sure to avoid any issues related to daylight savings, make sure to set this in your environment.rb:

  ActiveRecord::Base.default_timezone = :utc # Store all times in the db in UTC
  ENV['TZ'] = 'UTC' # Make Time.now return time in UTC

Then use tzinfo or some other library to adjust the time to/from the local time when you display it and retrieve it from the user.

Note: this post was updated the morning after I posted it because when I first wrote it I was apparently too tired to understand what was going on... :-)

4 comment(s)

2007-11-02

Ruby on Rails 101: Presentation Slides for a Five Day Course

UPDATE: The slides have been updated for Rails 2.3 (in February 2009) and are available here.

I've decided to share the presentation slides that I developed for the five day introductory Ruby on Rails course that I held in June here in Sweden. All in all it's 340 slides available under a creative commons license. You can download the slides as a PDF file here or view them over at Slideshare. To give you an idea of what's inside, here are the chapters:

I hope the slides will be useful in helping people learn and teach Rails. I'd like to thank David Heinemeier Hanson for creating such a wonderful framework and Dave Thomas for doing such a great work documenting it.

24 comment(s)

2007-11-02

Rails Search Plugin Review: acts_as_fulltextable

The other day I had the opportunity to install and evaluate the recently announced acts_as_fulltextable Rails plugin. The raison d'etre of the plugin is to offer easy access to the MySQL full-text search engine. The plugin uses a FulltextRow ActiveRecord model stored in a MyISAM table with a polymorphic reference (id and model name) to the model that you want indexed and a single column to hold the search index text. All you need to do is basically add an acts_as_fulltextable declaration to the model that you want to search and this will then create a has_one :fulltext_row relationship and the appropriate callbacks to populate the index on create, update, and delete.

The acts_as_fulltextable plugin mostly exceeded my expecations, especially in terms of easy setup and maintenance. It makes site-wide search (across all models) as well as model specific search (constrain your search to one or more models) very easy. A strength of the plugin is the simplicity. There is very little code and the architecture is straight forward and easy to grasp.

Something to be aware of is that MySQL uses OR searches by default. I wanted to follow the Google convention of AND searches and found myself having to prefix all query terms with a plus sign to accomplish this. Is there a better way? A more serious issue is that MySQL doesn't seem to support word stemming. It turns out that the acts_as_fulltextable plugin does a gsub on search queries to add a trailing * to each search term. MySQL calls this truncation. The star ("*") works as a wild card. If you are aware of this feature you can use it to manually work around the absence of stemming, just search for house* and that will match both "house" and "houses". It will also match "household" which may or may not be what you want.

A documented gotcha that I ran into is that if you combine the leading plus sign and the trailing star (i.e. use both boolean AND and truncation) then stop words will not be removed from your query and thus you may end up with zero results. This is very annoying. What is the solution? I don't know. Maybe go with OR searches, live with the stemming issue (without the star), or find a way to remove stop words yourself. Stop words are supposedly configurable in MySQL.

The acts_as_fulltextable plugin was created because of stability issues with Sphinx and Ferret and because Solr was considered overkill. Lucene/Solr appears to be the most well documented, reliable, scalable, and flexible open source search engine out there. My guess is that Solr is still your best best if you are a search power user and your requirements are high. Because of its simplicity and ease of maintenance though, acts_as_fulltextable offers a pretty attractive lower end alternative.

3 comment(s)

2007-10-31

Rails Plugin: mysql_requirement

I added the plugin mysql_requirement that allows me to check the encoding/charset settings of the MySQL server as well as its version when my Rails application starts up and abort otherwise.

2 comment(s)

2007-10-31

Rails Tip: Configuration Parameters

The PeepCode Code Review PDF has some nice advice about how best to deal with configuration parameters in your Rails applications. Traditionally most of us have probably just stuck global Ruby constants in our environment.rb files, but there are more structured ways of doing it. I've started using the app_config plugin and it seems to work fine so far. To make sure I haven't forgotten to define a parameter in an environment I access my parameters via a custom config_param method:

# Method to supplement the app_config plugin. I want to crash early and be alerted if
# I have forgotten to define a parameter in an environment.
def config_param(name)
  AppConfig.param(name) do
    raise("No app_config param '#{name}' defined in environment #{RAILS_ENV}, " +
      "please define it in config/environments/#{RAILS_ENV}.rb and restart the server")
  end
end
2007-10-07

RailsConf Europe 2007: Rails and the Next Generation Web

A sponsored presentation at the conference that I think stood out in how professionally it was executed and also touched on interesting topics was the one by Craig R. McClanahan from Sun. As I learned later, Craig has one of the most impressive track records I've seen in the Java community, being the creator of Struts and a contributer to a wide range of technologies including Tomcat and JavaServer Faces. In his talk Craig was urging Rails developers to move beyond the traditional three tier architecture of web applications. In plain english this means moving away from the classic scenario of a dumb browser talking to a single Rails application connected to a single database. Three developments in the industry today that are moving us away from this architecture are:

Craig was urging Rails plugin developers to avoid hard wired dependencies on ActiveRecord. Examples of plugins that succeed with this are acts_as_authenticated, make_resourceful, and paginator. Craig ended with three pieces of advice:

The presentation slides are available online.

2007-10-05

Ruby on Rails Deployment on FreeBSD

I did a Ruby on Rails FreeBSD deployment for a client the other day and I thought I'd share what I learned in an instructional format here. Previously I had mostly deployed on Linux (Suse, CentOS etc.) and I was curious to see what the FreeBSD experience would be like. I googled for instructions and immediately found the RailsOnFreeBSD page in the Rails Wiki. Other than that I couldn't find much Ruby on Rails and FreeBSD specific instruction out there. Note - most of the instructions in this post are not specific to FreeBSD but are generic Ruby on Rails deployment steps for Unix.

We were migrating from Windows to FreeBSD and the goal was to eliminate single points of failure. We settled on two application servers with FreeBSD 6.2 on HP hardware, both running the web and app tiers in the vanilla Rails deployment stack, i.e. Apache 2.2.4, mod_proxy_balancer, Mongrel cluster, and Mongrel. A load balancer external to the Rails system would load balance between the two Apache servers. The database we use is MySQL 5 and it sits on a separate server. The idea is to add another db server with some form of MySQL replication. We have yet to decide which replication to use and recommendations are welcome. For deployment we use Capistrano 2.

The first thing I do on a FreeBSD server is to log in with the root user and change shell from C shell to bash:

cd /usr/ports/shells/bash
make install clean
chsh -s /usr/local/bin/bash root
exit
su -
echo $SHELL

As a personal preference (or ignorance of vi maybe), I install Emacs. This is a good time to go grab a cafe latte, since the installation takes forever:

cd /usr/ports/editors/emacs
make install clean

We then add the user that Capistrano will log in as and that Mongrel will run under - the deploy user:

# Make sure to choose the bash shell. You can keep the defaults for most of the other questions.
adduser deploy

To be able to deploy with Capistrano without repeatedly being prompted for a password, we set up public/private key authentication:

# On the production server:
ssh-keygen -t rsa
# On your development server:
ssh-keygen -t rsa
scp ~/.ssh/id_rsa.pub deploy@production-server:.ssh/remote_key
ssh deploy@production-server
cd .ssh
cat remote_key >> authorized_keys
rm remote_key
exit
# Now ssh should not prompt for a password:
ssh deploy@production-server

We edit ~/.bashrc and setup the environment for the deploy user. I think it's important to set RAILS_ENV to production. I configure the bash prompt and the history size (the number of shell commands listed by the history command) and my preferred editor. I also add some convenient aliases for accessing the log file and mysql:

export RAILS_ENV=production

export PS1="[\u@\h:\w] "
export HISTSIZE=10000
export EDITOR=emacs

export PATH=$PATH:/usr/local/mysql/bin

export APP="/var/www/apps/streamio/current"
alias cdapp='cd $APP'
alias logapp='tail -f $APP/log/production.log'
alias restartapp='cdapp; mongrel_rails cluster::restart -C config/mongrel_cluster.yml'
alias mysqlapp='mysql -h db.host.name -u db.user -pdb-password database-name'

To make sure the ~/.bashrc file is sourced, edit or create ~/.profile and add the following line to it:

source ~/.bashrc

We install sudo and give the deploy user sudo access. That way we can use sudo from Capistrano to restart the Apache web server that will be running as root:

# Install sudo
cd /usr/ports/security/sudo
make install clean
emacs /usr/local/etc/sudoers
# Uncomment wheel group
pw user mod deploy -G wheel

Make sure the clock on the server is in sync by invoking "crontab -e" and add this line:

*/30 * * * * /usr/sbin/ntpdate ntp1.sp.se

The above line syncs the clock every half hour with an internet clock - an ntp server. In this case we use ntp1.sp.se, but you may choose a different npt server that is available in your country.

Now, finally, the time has come for us to install Ruby on Rails which is really the heart of our server (or where our hearts are as Rails developers at least). As indicated in the Rails wiki, we can use the rubygem-rails port for this. The port will install both Ruby (the programming language), RubyGems (the package manager for Ruby software), and Ruby on Rails (the web framework). The portinstall command in the Rails wiki didn't work for me, so I used make install instead:

# Update the ports tree - takes a long time...
portsnap fetch ; portsnap extract

# Install Ruby, RubyGems, and Rails
cd /usr/ports/www/rubygem-rails
make install clean

# Check your version of the installed software
# The versions given here are the ones I got, you may find later versions
ruby -v
=> ruby 1.8.5 (2006-08-25) [i386-freebsd6]
gem -v
=> 0.9.2
rails -v
=> Rails 1.2.3

Now that we have RubyGems at our fingertips, we can install Capistrano, Mongrel Cluster, and Mongrel in a snap:

gem install capistrano -y
gem install mongrel_cluster -y
cap --version
=> Capistrano v2.0.0
mongrel_rails --version
=> ** Ruby version is not up-to-date; loading cgi_multipart_eof_fix
=> Mongrel Web Server 1.0.1

To be able to control the version and the configuration I chose to install Apache from source, and I followed the instructions in the excellent Mongrel book by Zed Shaw:

mkdir /usr/local/src
cd /usr/local/src
# Visit http://httpd.apache.org and download httpd-2.2.4.tar.gz
tar xzf httpd-2.2.4.tar.gz 
cd httpd-2.2.4
./configure --enable-proxy --enable-proxy-balancer --enable-proxy-http --enable-rewrite \
--enable-cache --enable-headers --enable-ssl
make
make install

# You can check the location of the httpd binary:
/usr/libexec/locate.updatedb
locate httpd|grep bin

Add the Apache startup script:

emacs /etc/rc.conf
# Add the following line:
httpd_enable="YES"
ln -s /usr/local/apache2/bin/apachectl /usr/local/etc/rc.d/httpd

# Start Apache
/usr/local/etc/rc.d/httpd start

# Fetch http://production.host.name in a browser. You should see the text "It Works".

We now configure Apache for our Mongrel server like it says in the Mongrel book:

cd /usr/local/apache2/conf
emacs httpd.conf
# Add one line: 
Include /usr/local/apache2/conf/rails.conf

Create the /usr/local/apache2/conf/rails.conf file with contents like this (make sure to query replace $app_name$ with the name of your Rails app, i.e. the basename of your RAILS_ROOT):

NameVirtualHost *:80
# Setup the cluster
<Proxy balancer://$app_name$_cluster>
  BalancerMember http://127.0.0.1:8000
  BalancerMember http://127.0.0.1:8001
  BalancerMember http://127.0.0.1:8002
</Proxy>

<VirtualHost *:80>
  ServerAdmin $email$
  ServerName localhost
  ServerAlias localhost
  DocumentRoot /var/www/apps/$app_name$/current/public
  <Directory '/var/www/apps/$app_name$/current/public'>
    Options FollowSymLinks
    AllowOverride None
    Order allow,deny
    Allow from all
  </Directory>

  ProxyPass / balancer://$app_name$_cluster/
  ProxyPassReverse / balancer://$app_name$_cluster/

  RewriteEngine On

  RewriteRule ^/$ /index.html [QSA]

  RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f
  RewriteRule ^/(.*)$ balancer://$app_name$_cluster%{REQUEST_URI} [P,QSA,L]

  ErrorLog /var/www/apps/$app_name$/current/log/apache_error_log
  CustomLog /var/www/apps/$app_name$/current/log/apache_access_log combined
</VirtualHost>

At this point it makes sense to restart Apache to make sure that the config file parses. We now finish up on the server by installing Subversion, MySQL client, ImageMagick, and RMagick:

cd /usr/ports/devel/subversion
make install clean

cd /usr/ports/databases/mysql50-client/
make install clean

cd /usr/ports/graphics/ImageMagick
make install clean

gem install rmagick -y

If you are in a Windows environment you may want to install Samba:

cd /usr/ports/net/samba3
make install clean

You should now have the mount_smbfs command available for mounting Windows disks on your FreeBSD server.

We are now just about ready to deploy to our FreeBSD server using Capistrano 2 from our development machine. Before we do that though, let's create the directory on the FreeBSD server we'll be deploying to:

mkdir /var/www
chown deploy /var/www

Now, make sure your database.yml is properly configured. Make sure you can connect to MySQL from the FreeBSD servers. Capify your Rails app if you haven't already:

cd RAILS_ROOT
capify .

You now need to edit config/deploy.rb to fit your server. In particular, make sure you have the deploy_to variable set to "/var/www/apps/#{application}" and you have the proper roles set up. Here is a sample:

role :web, rails01, rails02
role :app, rails01, rails02
role :db,  rails01, :primary => true
role :scm, rails01

Also, make sure to define the deploy:restart task to restart using Mongrel Cluster. Also, symlink in any shared files in a callback. Here is a sample from my deploy.rb file to get you started (don't copy it in it's entirety, that won't work):

namespace :deploy do  
  # ===========================================================================
  # Mongrel
  # ===========================================================================  
  def mongrel_cluster(command)
    "cd #{current_path} && " +
    "mongrel_rails cluster::#{command} -C #{current_path}/config/mongrel_cluster.yml"
  end

  %w(restart stop start).each do |command|
    task command.to_sym, :roles => :app do
      run mongrel_cluster(command)
    end
  end

  # ===========================================================================
  # Apache
  # ===========================================================================  

  desc "Restart Apache web server"
  task :restart_web do
    sudo "/usr/local/etc/rc.d/httpd restart"
  end

  # ===========================================================================
  # Deployment hooks
  # ===========================================================================  

  desc "Copy in server specific configuration files"
  task :copy_shared do
    proxy_dir = "#{release_path}/vendor/plugins/reverse_proxy_fix/lib"
    run <<-CMD
    cp #{release_path}/config/database.yml.example #{release_path}/config/database.yml &&
    cp #{release_path}/config/directories.rb.example #{release_path}/config/directories.rb &&
    cp #{proxy_dir}/config.rb.unix #{proxy_dir}/config.rb
    CMD
  end

  desc "Run pre-symlink tasks" 
  task :before_symlink, :roles => :web do
    copy_shared
    backup_db
    run_tests
  end

  desc "Run the full tests on the deployed app." 
  task :run_tests do
    run "cd #{release_path} && RAILS_ENV=production rake && cat /dev/null > log/test.log" 
  end

  desc "Clear out old code trees. Only keep 5 latest releases around"
  task :after_deploy do
    cleanup
    sleep 5
    ping_servers
  end

If you have your deploy.rb in check, you should now be able to run "cap deploy:setup" to setup the Capistrano directory structure on the servers, and finally the magic command to deploy to both of the FreeBSD servers:

cap deploy

Good luck!

10 comment(s)

2007-10-01

Rails Migration Gotcha: Forgetting to set active_record.schema_format to :sql

If you rely on any SQL not supported by Rails migrations such as foreign keys - don't forget to set config.active_record.schema_format = :sql in your environment.rb. Otherwise your test database will be out of synch with your development database. I really would prefer if Rails used the SQL dump format for setting up the test database by default as that would be more reliable.

The reason I came across this now was actually that I found what seems to be a bug with Rails migrations. Given this create_table statement in my migration:

    create_table :users do |t|
      t.string :username, :null => false
      t.string :role, :null => false
      t.timestamps 
    end

Rails will create this statement in the dump file:

  create_table "users", :force => true do |t|
    t.string   "username",   :default => "", :null => false
    t.string   "role",       :default => "", :null => false
    t.datetime "created_at"
    t.datetime "updated_at"
  end

The difference is subtle but important to me. In the dump format Rails insists that the default value is the empty string. This is not what I want when I say that a column is not null, since with MySQL 5, even with SQL mode set to traditional, the column will then be defaulted to the empty string when someone tries to set the column to NULL.

1 comment(s)

2007-10-01

RSpec Tip: Keeping Controller Specs DRY

I am not yet converted to the idea of testing/specing views in separation so I usually invoke integrate_views at the top of my controller specs. I also have a bunch of helper methods that I want to reuse across my specs. To encapsulate those needs and DRY up my specs I came up with this little method that I keep in my spec/spec_helper.rb file:

# Describe a controller the way we want to do it for this app, i.e. with
# views integrated and with certain controller spec helper methods available
require File.join(File.dirname(__FILE__), "controller_spec_helper")
def describe_controller(controller, &block)
  describe controller do
    include ControllerSpecHelper
    integrate_views
    block.bind(self).call
  end
end

The controller_spec_helper.rb file looks like this:

module ControllerSpecHelper
  def login_user(username)
    session[:casfilteruser] = username
  end

  def logout
    session[:casfilteruser] = nil
  end

  def get_page(action, options = {})
    get action, options
    response.should be_success
  end  
end

1 comment(s)

2007-09-26

Rails Discussion: ActiveRecord vs SQL

Here is my Rails quote of the day, form a Rails mailinglist:

"When I was at the first RailsConf I was talking to someone about having used SQL for 15 years and that I was struggling with AR. At that moment someone grabbed me by the arm. I turned to look into the familiar face of Martin Fowler! He said to me, "If you know SQL that well you should just use SQL.""

1 comment(s)

2007-09-22

The UPS Illusion of Guaranteed Delivery

My girlfriend was in a hurry to send 6 copies of her thesis (a 1 kg package) from Stockholm to Copenhagen. She sent it on wednesday and it said on the UPS website that the package would be delivered "no later than" end of office hours on thursday, i.e. the next day. My girlfriend chose UPS because she wanted to be sure the package would be delivered on time. Her examination is next friday and the printed thesis needs to reach the university well in time before that. For the delivery my girlfriend paid about three times as much as she would have paid the postal service. This means she paid about 600 SEK over the 200 SEK that the postal service would have charged.

Well, it's now Saturday and the UPS can't really tell us why the package has not been delivered yet. According to the UPS tracking service, the package has been sitting in the destination city for two days without any delivery attempts having been made. UPS cannot explain why. It's like the package is lost in limbo. When my girlfriend calls to explain the problematic situation she is in and asks for help she is not met by service mindedness or understanding. She is met by accusations and unfriendliness. They say she should have known to choose an even more expensive form of delivery for about 1000 SEK to be guaranteed the delivery date. They also say she misinterpreted the conditions. Apparently, when the UPS say "latest delivery" on a certain date, that means something else to them than it means to most people. The latest delivery date isn't really a part of the contract. There is no money back and there is no excuse when delivery is made at some arbitrarily later date.

Everybody knows the Postal service doesn't make guarantees about the delivery date. There is mutual understanding about the contract. A lot of us probably live under the illusion though that UPS makes guarantees like that. Well, it's time to wake up and smell the coffee because they don't. This naturally leaves the question open as to what it is that motivates the steep UPS prize premium?

4 comment(s)

2007-09-21

RailsConf Europe 2007 Notes: Dave Thomas Keynote

The Keynote by Dave Thomas was to me the most inspirational and profound talk at the conference. I have the greatest admiration for Dave Thomas. I can't offer Dave's entire talk in every detail here, nor do I know if that would be appropriate. What I have is key fragments from sentences in the talk presented sort of in the format of a poem. Hopefully someone who didn't hear the talk can make something out of it. Enjoy...

There is no such a thing as software engineering
What we have been doing is taking a whole lot of dirt and filling up a hole
What makes engineering good? I look for elegance
Fred Brooks - The Mythical Man Month
Written 30 years ago. IBM mainframes. Not a single thing has changed between then and now, tragically
The programmer, like a poet, creates castles from thin air. We can create anything we can imagine

Writers block. Write something, throw it away, start over
The blank canvas of a painter
The blank editor buffer. Frightening. Since you start, you've made a commitment

Leonardo was commissioned to produce a statue, how much money is there in it?
He started with a sketch on a scrap of paper. He is doing what we should do but aren't doing
He was prototyping, playing around, experimenting
If you step away from the keyboard and use a different medium,
you are engaging a different part of the brain
The statue was canceled due to budget cuts...

With a prototype you have not invested, so you can change your mind
Brooks says be prepared to throw one away
I say be prepared to throw 10 away
You can ignore the details when you prototype

Exploratory testing. When you don't really know how something works
You write unit tests and experiment and align your understanding with what's going on
Prototypes written as tests are really useful
The end to end prototype. The tracer bullet. No error checking and no details
Leonardo sometimes drew a composition and threw it away
Scaffolding is good for prototyping

- Start anyway. Start with something and then throw it away
- Test first
- Act on worry. The lizzard at the back of our minds is giving you a sense of worry. Listen to it

Just as important: know when to stop

How do you work on something that is really big if you can only see a small part of it at a time?
It's like painting a big wall or ceiling
The trick was to divide the ceiling into panels with low coupling and strong cohesion
Each panel tells only one story
Modularization is great and we don't do enough of it

Time slots. Iterations. Boxes to work in
Do what cartoonists do if you can't finish - "To be continued..."
Knowing when to stop is incredibly important
When you've set a boundary and you reach it - stop

Satisfy the customer
Paint my picture and make me look nice
Develop the software and make me look good

Increasingly less realistic portraits through history
Each portrait satisfies, but only one of them really looks like a person
There is a massive difference between a portrait and a picture
Look inside and find out what something is really like and then find a way to express that
A picture is merely the surface representation
Painters don't have the restriction of photographers
When you are capturing requirements you have a picture
Developers are looking for underlying requirements

USA was trying to invent a pen to write in space, meanwhile, the russions used pencils
Nasa was using pencils. The Fisher company offered a pen. Nasa (the client) didn't want it
It could write upside down in a boiling toilet. Nasa still didn't want it
We need to get into the habit of *not* listening to our clients
Instead figure out what they really want and work with them to establish that

Software for producing a perfect drivers license picture bought by the state government
Pictures got interchanged and smileys were being used instead
The camera was off by one after a customer left
Another customer put a smiley on a piece of cardboard...

Is there a meaningful distinction between art and engineering?
No. In reality they are the same
They share elegance, grace, understanding, digging beneath the surface
Is software engineering an art or a craft?
There is no distinction

Without engineering there is no art

Without art there is no engineering

Without art there is no soul and you won't produce anything worthwhile
We have a responsibility to demonstrate that with Rails

With Rails we have a canvas on which we can draw
Be an artist
Treat your next project as if it was a work of art
Think about the impression that you want to make
Create something great, as always
The community is full of people that make great software and applications
We are changing the world of software development
Create more than great things. Create beautiful things

Let the world know

Sign your application as the artist

That's how you know how to take pride in what you do
2007-09-21

RailsConf Europe 2007 Notes: BDD, RSpec, and Story Runner

For me, RailsConf Europe 2007 in Berlin started out with a BDD/RSpec tutorial with David Chelimsky, Dan North, and Aslak Hellesøy. In the first part of the tutorial the presenters gave an overview of the theory and background of Behaviour Driven Development (BDD). All I can offer here is an unstructured list of keywords and notes that I was able to scrabble down while listening:

  Domain Driven Design.
  Having engineers and business people speak the same language.
  SOA
  Using contracts that say: this is all I'm going to do for you
  Focusing on outcomes and reducing features  
  The Agile Alliance
  The false civil engineering analogy of building bridges
  SCRUM  
  Problem: programmers deleting tests they don't understand
  Format for user stories: index cards
  The given-when-then format
  Book: "User Stories Applied"
  All features should be traceable to a persona
  Keeping personas in FaceBook is a fun idea
  Example Driven Development  

After the first break David Chelimsky demonstrated how RSpec is used to do Behaviour Driven Development. Suppose you are having a conversation with your customer about the application that you are going to build. It turns out you need a User class. In RSpec this is expressed as:

describe User do
end

When we run the specification we get a failure since the User class doesn't exist yet. We proceed to add the the User class, re-run the specifications, and get the green light. The green light means we can go back into specification mode and describe the behaviour of our class. The idea with RSpec is to have development of your application proceed in very short cycles including the following steps:

  1. Add one or two lines of RSpec specification code
  2. Run the specification and get the red light
  3. Implement the simplest possible code that will fulfill the specification
  4. Run the specification again and get the green light

Suppose your customer says users should be in the role to which they are assigned. In RSpec we express this as:

module Auth
  describe User do
    it "should be in a role to which it is assigned" do
      aslak = User.new
      aslak.should be_in_role("speaker")
    end
  end
end

Throughout the tutorial David Chelimsky used autotest to run the specs continuously in the background. David mentioned that refactoring means improving the design of a system without changing its behaviour. The hard question to answer is whether behaviour has been changed after a refactoring, and it's in the ability to answer that question that BDD/TDD can help us.

Story Runner is a new and complementary specification format that RSpec will provide in the next release. It is structured around scenarios with Given/When/Then blocks and the syntax looks a little something like this:

Story "Plan cup", %{
  Narrative here
}, :type => RailsStory do
  Scenario "set up a 4 team cup structure" do
    Given "a new cup with max teams of", 4 do |max_teams|
    end

    When "I ask to see it" do
    end
    
    Then "it should a row count of", 3 do      
    end
    
    Then "..." do      
    end
  end
end

Stories live in a directory called stories next to the specs directory that RSpec already provides. The specifications deal with the units/components of the system. Stories on the other hand are higher level and are a form of acceptance/integration test that makes sure that the system works as a whole.

2007-09-21

RailsConf Europe 2007

I sometimes tell people how the best decision I made in life was to get into salsa dancing. It was how I met my girlfriend Janne and it has simply been countless hours of fun and magic. It has also made my confidence and social skills grow which has helped me in other areas of life as well. Visiting RailsConf Europe 2007 has to rank right up there with one of those great decisions and it feels like a milestone. It gave me inspiration on many levels and I enjoyed it more than I've enjoyed any previous conference. What made all the difference for me was the networking. Meeting so many friends and colleagues from the past and having the opportunity of talking to thought leaders in the Rails ecosystem is just amazing. Now that three intense days of excitement is over I miss it already.

My trip to Berlin started out with meeting Sven Guckes and Emily at the Hauptbahnhof. Sven took us to places like the Bundesrat, the Brandenburger Tor, Checkpoint Charlie and the Sony Center. Sven may very well be the most hospitable person I've met and on top of that he is a very competent Berlin guide. Many thanks goes out to Sven for taking so good care of me and Emily! I spent most of the weekend (the night time) dancing salsa at the Berlin Salsa congress. I met four other Swedes at the congress - Björn, Johan, Camilla, Lala, and Elias. I had great dances with Dace and Anja from Riga, Alex, Sarah, and Anisa from Germany as well as girls from Ireland and Zurich etc. It was a small to mid-size congress (smaller than the Hamburg, Zurich, and UK congresses) with good atmosphere and plenty of amazing dancers. There was no live band, but the music and locations were good. Especially the gala night on saturday had a touch of flair and there was a performance by Salsa Dance Squad from the Netherlands that was pure magic. It's fascinating how after having seen so many different great shows and performances, all of a sudden a single show comes on that stands out like a divine work of art and makes all the others fade into the background.

When I started out the conference bright and early on monday morning at the Maritim hotel I was already quite exhausted after three straight nights of dancing. What I probably didn't realize at that point was that the intensity was not going to go down, on the contrary. On the day before the conference I had met up with my good friend Jarkko from Finland and with Dave Goodlad, Garnet, and Martin from Canada (from Vernon?) and we ended up at the Ständige Vertretung pub by the Friedrichstrasse station. Later on Rickard and David from Newsdesk (in Stockholm) joined in. The sun was shining on us, we were by the water and in great company, and the weissbier was flowing. It was relaxing end enjoyable. For me there is always some nostalgia in drinking weissbier as it brings back the year of 2000 when I was living in Munich.

In the evening four hundred or so computer geeks met up for the Bratwurst on Rails pre-conference party. The wurst was tasty, and I met many great people such as Marcel Molina, Chad Fowler, Patrick Lenz, Lauri Jutila from Finland, David Black, Christian Dalager from Copenhagen, Paul Doerwald from Canada, Tillmann Singer from Berlin, and Timo Hentschel and Bernd Schmeil from Munich. I also met people I know from the Stockholm Ruby User Group - David and Rickard from Newsdesk, Johan Lind from Valtech, Piotr from Connecta, Markus from Stockholm University among others. The Gothenburg Ruby User Group had decent representation as well and it was great to get to know Carl-Johan Kihlbom better.

Presentations that made a particularly deep impression on me were Dave Thomas on being an artist, Jason Hoffman on scaling from the bottom up, David Heinemeier Hanson on Rails 2.0 and the never ending quest for more beautiful code, and Marcel Molina and Michael Koziarski on coding best practices. Interesting was also David Chelimsky, Dan North, and Aslak Hellesøy on BDD and Rspec (including the new Story Runner), Craig R. McClanahan on moving beyond two tiers (avoiding a dependency on ActiveRecord), Ola Bini on JRuby (book coming soon), Britt Selvitelle (Twitter) on really scaling Rails (message: don't worry about it until you need it), and Dane Avilla on the RaiLint plugin for HTML validation and on using Watir for in-browser testing.

I didn't attend all the presentations at the conference, partly because I was exhausted and concentration was waning, but also because I didn't want to miss the opportunity of hanging out in the exhibit area talking to passionate people in the community. The exhibitors at the conference were great. Sun was there with the Netbeans IDE and JRuby, Borland announced the 3rdrail IDE based on Eclipse, EngineYard was there to talk about hosting, and FiveRuns showed off their RM-Install and RM-Manage offerings. Thoughtworks explained how 40% of their new enterprise projects in the US are based on Rails and how they are working on making Ruby cross the chasm and rid the IT world of bloatware (mostly .NET and J2EE as it were). I had the opportunity of mingling with people such as David Heinemeier Hanson, Martin Fowler, Roy Singham (founder of ThoughtWorks), Tom Mornini of EngineYard, and Bradley Taylor of RailsMachine.

On the last night of the conference I was invited to a special Tapas dinner at a spanish restaurant that Bradley Taylor arranged for his European clients. It was a pleasure. Before finally going home to sleep on wednesday night, we stopped by one last time at the Maritim hotel where Marcel Molina, Michael Koziarski, Chad Fowler, Geoffrey Grosenbach, and 10-15 other Rails enthusiasts were sitting in a circle playing the werewolf role playing game. The plot is something about a village with peasants that need to figure out who the werewolfs are before being killed by them. It takes figuring out when people tell the truth and there is a lot of strategy and drama involved. It was fascinating to watch. On thursday morning I had breakfast at Starbucks and ran into Swami from the night before - a traveling Yoga teacher from France and a Railsmachine customer. Swami is planning to open a Yoga center in Slovakia. Quite fascinating.

My photos from Berlin are avaliable on Flickr

O'Reilly will be hosting the RailsConf Europe 2008 in Berlin too. I look forward to coming back then. Thanks everybody and auf wiedersehen!

1 comment(s)

2007-09-13

Rails Deployment Tip: FiveRuns.com

If you are a Rails developer you have probably heard of FiveRuns with their RM-Manage service offering, providing monitoring and statistics for your Rails apps. I've wanted to try the service for quite a while and today I finally got around to doing so on a production server at one of my clients. All I can say is you have to try it to believe it - it's a super slick, user friendly, and powerful service.

There was a minor glitch in the setup. The FiveRuns monitoring client didn't find our Rails app since it was in a non-standard location. I entered the FiveRuns Campfire chat room and got live support within minutes and quickly had a resolution. Once we installed the FiveRuns Rails plugin and restarted the server we had live Rails response time and profiling statistics within minutes. The first impression I have of the service is just great. We will be evaluating the service over the next 30 days and I'll post here any new findings we make. Barring any really big issues coming up, I consider FiveRuns to be a huge value add to the Rails ecosystem, almost like a milestone, and I will be recommending it wholeheartedly to all my clients from now on.

2007-09-12

Rails Tip: Nested Layouts

This is just a feature I always wanted in Rails - nested layouts, i.e. the ability to have one master layout (application.rhtml) for your whole site, and then to have layouts within that that differ from section to section. Today I stumbled across this hack - just a single helper method that allows the use of nested layouts. Sweet. I'll re-post the code here:

module DocHelper
  # Nested layouts, see: http://fora.pragprog.com/rails-recipes/write-your-own/post/144
  def inside_layout(layout, &block)
    @template.instance_variable_set("@content_for_layout", capture(&block))

    layout = layout.include?("/") ? layout : "layouts/#{layout}" if layout
    buffer = eval("_erbout", block.binding)
    buffer.concat(@template.render_file(layout, true))
  end  
end

 

<% inside_layout 'application' do %>
  <div style="font-size: 150%">
    <%= yield %>
  </div>
<% end %>

5 comment(s)

2007-09-06

Rails and Transactional Saves

Correction: ActiveRecord::Base#save does use transactions in Rails by default, see Jarkkos comment.

In Rails, models are not saved within a database transaction by default. This means that if you are updating some other record in a callback like after_save then the record will still be saved even if an exception is raised in the callback. If you care a lot about data integrity this is not satisfactory. One way to remedy the problem might be to simply override the save and save! methods in your model like this:

  def save
    Post.transaction { super }
  end

  def save!
    Post.transaction { super }
  end

That's the workaround I'm going to use for now.

5 comment(s)

2007-09-05

Ruby as a Bash Substitute

It's really nice how Ruby can liberate me from languages that I don't like as much and am not as fluent in, such as Bash, and Perl. Today I was tearing my hair over a little bash database backup script that I needed to write until I realized I'm much better off writing it in Ruby. Bash translates amazingly easy to Ruby with the syntactic similarity, the availability of the back ticks (`), and libraries such as FileUtils. Once I switched to Ruby the backup was solved faster and with more confidence.

This experience got me thinking of the expression "when all you have is a hammer, everything looks like a nail". If I'm not mistaken the Pragmatic Programmers use not only the advice "use the right tool for the job" but also "master your tools". There is a pretty obvious contradition between those two pieces of advice. It's important to not be afraid of picking up new tools, but it's also important to not underestimate the cost involved in learning the new tools in depth. Some tools I seem to pick up and learn quickly, others I can struggle with for years without ever really feeling confident or happy with them.

2007-08-31

Rails Testing: Switching from Test::Unit to RSpec, What are the Advantages?

It's been a couple of months now since I switched from Rails built in Test::Unit framework to using RSpec and I've realized that RSpec has already brought some major advantages for me. Just to clarify, I still run all my old Test::Unit tests and I still write Test::Unit integration tests to complement my specs. The Rails testing framework and RSpec can run nicely side by side through the rake command. Here is what RSpec has given me:

Another realization that has come to me lately, is that yes, regardless of what code you are writing TDD/BDD is a good idea. However, if you are writing any kind of API or framework that is to be reused across several applications, tests/specs and well defined interfaces are almost a necessity. If you don't have them, you might be better off opting for no reuse. Reuse is the holy grail of software development, and it's hard.

1 comment(s)

2007-08-31

Rails Gotcha: The Contract of 1.month.ago

At first glance the contract of the beautiful 1.month.ago construct in Rails is maybe obvious. However, what should it return if the current time is the 31st of March? Remember that february only has 28 days. Well, according to Rails 1.2.3 the answer is 1st of March, but according to Edge Rails the answer is (and I agree with Edge Rails here) 28:th of February.

The 1.month.ago issue is something that bit me and that you should be aware of. So with Rails 1.2.3, if you want to be sure to get the beginning of the previous month, you need to say 1.month.until(Time.now.beginning_of_month). At least that's the workaround I'm using now.

2 comment(s)

2007-08-31

Rails Deployment: Rapid Setup with the Machinify Gem

I used the Machinify Gem by Bradley Taylor (Mr RailsMachine) the other day to install a Rails stack on a piece of Ubuntu 7 Xen VPS and I must say it worked really nicely. The Gem uses rake to install the common stack of MySQL 5, Apache 2 with mod proxy load balancer, Mongrel Cluster, and Mongrel. The gem also takes care of installing all the obscure Debian packages needed that you don't even want to know about.

There is a small glitch currently in the Machinify gem in that the Mongrel Cluster version is hardcoded in its mongrel.rake file. Once I corrected that and ran rake stack:install again it worked like a charm. Related to this issue, I was trying to find out how to fetch the current version of a Gem and the best I could come up with was:

# On the command line:
gemwhich capistrano
=> /usr/local/lib/ruby/gems/1.8/gems/capistrano-2.0.0/lib/capistrano.rb

# In Ruby:
`gemwhich capistrano`[%r{-([0-9.]+)/lib/[^/]+$}, 1]
=> "2.0.0"

If you are on Cent OS you should check out RubyWorks by ThoughtWorks (the installer, not the singer).

2007-08-28

When the Swedish Post is Vastly Superior to UPS/TNT/DHL et al

It's a curious thing. When it comes to politics I'm traditionally a quite liberal and market oriented person. I usually get annoyed when I find areas of society were competition and free trade are restricted and I'm not a great fan of state monopolies. However, when it comes to delivering packages the swedish state owned postal service are offering me a service that is vastly superior to that of TNT/UPS/DHL.

Why is that? Well essentially it's because the postal service uses the approach of delivering to a pick-up place in my area, usually a kiosk or store of some kind, usually with generous opening hours. This has worked like a charm for me so far. Private delivery firms (UPS et al) try to deliver to me in person. Delivering a package directly into a persons hand sounds like a great service, but it's a bit harder than it sounds. I'm not saying delivering to someone in person cannot be made to work reliably, however, the way it's currently implemented there are some serious issues. For a personal delivery to be a success, it requires that the delivery person and the recipient be in the the exact same place at the same point in time, so that the package can be handed over and signed. Now in this day and age with GPS systems and mobile phones and the internet and all that, the problem sounds solvable. In a lot of cases, at least for me, delivery is failing though. For a taste of other peoples experiences, see Lars Pind wishing for a doorman or this swedish discussion saying postal service seems like a luxury. Some reasons personal delivery is failing might be:

1 comment(s)

2007-08-28

RSpec Gotcha: Incompatibility with Engines 1.2.0

If you are trying to get RSpec to work in a Rails application that uses Engines you may find that RSpec errors out when you try to run it. It seems the reason is related to the file testing.rb in the engines plugin. If you comment out the line that requires that file (in vendor/plugins/engines/init.rb) you will have worked around the issue:

#require "engines/testing" if RAILS_ENV == "test"
2007-08-12

Rails Deployment: A Little Checklist

After having fought myself through a couple of Ruby on Rails deployments, it seems there are a lot of little annoying things you need to remember. Some of them are not super critical, at least not in the short run, but they can grow into bigger problems over time. Here is the list:

1 comment(s)

2007-08-12

Rails Deployment: Dealing with 404s

You probably use Jamis Buck's Exception Notifier plugin to be notified by email of exceptions thrown on your production server. Typically though you don't want to be notified of 404s, at least not when they are caused by an RSS reader requesting an RSS feed that no longer exists, or some spam robot or search engine fetching URLs fetching URLs no longer supported. When the Exception Notifier plugin catches an exception and it is one of ActiveRecord::RecordNotFound, ActionController::UnknownController, or ActionController::UnknownAction, it figures that it is a 404. By default then no email is sent and instead the method render_404 is invoked which in turn renders public/404.html. This is quite appropriate. Unofortunately though you can't trace in the log what the request looked like that caused the 404. The other limitation is that if no route matches a n incoming request, then an ActionController::RoutingError is thrown, yielding a 500 and an exception email. To get consistency in how 404s are dealt with, i.e. make sure they are always fully logged and tracable, but never cause email notifications, I override the render_404 method from the Exception Notifier plugin in my ApplicationController:

  # Overriding the version in ExceptionNotifiable so that we can log the environment and figure
  # out where the request is coming from.
  def render_404
    logger.error("render_404 invoked for request_uri=#{request.request_uri} and env=#{request.env.inspect}")
    super
  end

I then add a catch all route at the end of my routes.rb file:

  map.connect '*anything', :controller => 'application', :action => 'render_404'

4 comment(s)

2007-08-11

Rails Deployment: Looking up User ID from a Session ID

Suppose you get an exception notification and you want to know which user was browsing the site when the exception occured. The exception email has the HTTP_COOKIE header with the _session_id. Assuming you are using ActiveRecordStore and that you store the user ID in the session, you can look it up with an incantation like the following:

CGI::Session::ActiveRecordStore::Session.find_by_session_id('c5934a07b12ae9d90cf6be4e7d48b361').data

I keep forgetting the module path to the Session class so I thought I'd post this note. The corresponding Rails file is active_record_store.rb in ActionPack.

1 comment(s)

2007-08-01

Rails Testing: Quoting angle brackets in assert_select

This is just a little detail but today I rediscovered the way to quote angle brackets in argument values in HTML::Selector expressions in your assert_select commands (and corresponding response.should have_tag commands in RSpec). I tried using back slashes before I realized that I needed to enclose the argument value in single quotes. Notice the nested angle brackets ([ and ] signs) below:

  it "Should have the allow_public_messages radio buttons on the edit contact info page" do
    get :edit_my_options
    response.should be_success
    response.should have_tag("input[name='profile[allow_public_messages]']")
  end
2007-07-26

RSpec RAILS_ENV Gotcha

I've had issues with RSpec running in both development and production environment which certainly isn't what I want and it seems others have had the same issue. The resolution is simple, force RAILS_ENV to be set to "test" in your spec/spec_helper.rb file.

1 comment(s)

2007-07-23

Teaching the First Rails Course in Sweden

In the last week of June I had the privilege of teaching the 5 day Ruby on Rails course I Go Ruby at the wonderful Ulvhäll Herrgård in Strängnäs, just a one hour train ride west of Stockholm. To the best of my knowledge it was the first Ruby on Rails course held in Sweden, and this of course made us feel special. The relaxed location and format of the course - a summer camp with a small group of students - also worked well to create a pleasant atmosphere.

I have previously taught evening courses in Ruby on Rails at companies like Great Works and Connecta, but this was the first time that I got to teach a longer course. We had a small group of eight students with very different experience levels and this of course posed quite a challenge for me. I found that my exercises were a tad too difficult and also that I was possibly covering too much material, at least for the less experienced students.

I developed all of the course material myself - over 300 slides plus exercises - a major undertaking consuming about a month of full time work. The basic contents of the course roughly corresponds to the Agile Web Development with Rails book, but important additions were made in areas such as Ruby, testing, and Rails 2.0.

There are tentative plans to make the course material available online at an affordable price. I am also hoping to offer two day Rails courses in Stockholm during the autumn. Eventually I'd like to offer a two day introductory course, and then one or more advanced one or two day courses on advanced topics. Stay tuned for announcements on this, and let me know if you are interested.

2 comment(s)

2007-05-24

Rails Testing: The form_test_helper Plugin

I've been using the form_test_helper plugin in my integration tests lately and I think it's a great way to increase test coverage of your views. The form_test_helper uses the assert_select CSS-like selectors and makes it easy to click links and submit forms and thus simulate something that comes pretty close to manual testing in the browser, unless of course you need to test AJAX functionality, in which case you need to look at Watir or Selenium. Here is some sample code:

      select_link("/admin/users").follow
      assert_response :success

      select_link("/admin/users/new").follow
      assert_response :success

      service_ids = [3, 4, 5]
      self.user_email = user_email
      assert_difference User, :count do
        submit_form do |form|
          form.user.update({
            :name => 'Test user',
            :email => user_email,
            :customer_id => 3,
            :password => "test",
            :services => service_ids
            })
        end
        assert_response :redirect
        assert_valid assigns(:user)
        assert_equal(Service.find(service_ids), assigns(:user).services)
        follow_redirect!
        assert_response :success
      end      

1 comment(s)

2007-05-23

Rails + RSpec: First Impressions

I started using RSpec on a client project yesterday and I've been pleasantly surprised by how easy it is to get started with and how nicely it plays with Rails. I now have a handful of specs under the spec directory and they get run by rake automatically right after my tests. I can also use rake spec:doc to generate the documentation of my specs:

$ rake spec:doc

Admin::StatisticsController
- Index page is not viewable by role user or role admin without service id
- Index page, role user: shows links
- Index page, role super: shows links
- Hours page, role user: does not show customer and service selects
- Hours page, role super: selects service when service provided
- Hours page, role super: selects service when service and customer provided
- Hours page, role super: selects customer when customer provided

the Call model
- Is not deleted by attempt to delete customer
- Is not deleted by attempt to delete service

the Service model
- Is not deleted by attempt to delete customer

I decided to not prefix all my specifications with the "should" keyword, it seemed superfluous. Here are some samples from my controller spec:

require File.join(File.dirname(__FILE__), "/../../spec_helper")

describe Admin::StatisticsController do
  fixtures :customers, :services, :users, :services_users, :audio_files, :prompts, :calls  
  integrate_views

  specify "Index page is not viewable by role user or role admin without service id" do
    login(:user)
    get :index
    response.should be_error

    login(:admin)
    get :index
    response.should be_error
  end
  
  specify "Index page, role user: shows links" do
    login(:user)
    get :index, :id => 2
    response.should be_success
    response.should have_tag("a[href=/admin/statistics/hours/2]")
    response.should have_tag("a[href=/admin/statistics/calls/2]")
  end
  ...

  ######################################################
  #
  # Helper methods
  #
  ######################################################

  def login(role)
    login_user(user_for_role(role))
  end

  def user_for_role(role)
    case role
    when :super
      users(:aaron)
    when :admin
      users(:admin)
    when :user
      users(:customer)
    else
      raise "Unknown role #{role}"
    end
  end
  
  def login_user(user)
    @controller.send(:session=, ActionController::TestSession.new)
    @controller.send(:current_user=, user)
  end
end

I think it's very refreshing to switch to RSpec from unit tests. The syntax is more readable and makes it feel natural to write specs/tests before the code. In this particular case though I wrote the spec first, but when I started implementing the UI I changed my mind about which page the form should go on, experimented with the UI a bit, and ended up writing the spec after the fact. I don't consider this a deadly sin. One probably shouldn't be too religious about writings specs before the code in cases where it slows you down too much or hinders your creativity.

I highly recommend you try out RSpec with Rails if you haven't already.

1 comment(s)

2007-05-01

Rails Testing: Checking for Broken URLs in Links, Images, Forms, and Redirects

I've added the ability to check for broken URLs in links, images, forms, and redirects to my Http Test plugin. How does this work? Well in your controller and integration tests an ApplicationController after filter is used to extract all the URLs on the page. For each URL we make sure it can be resolved to a route, a controller, and an action and/or template. URLs to files under the public directory will be recognized. The same check is made for redirects.

I haven't used this URL checking in production yet, but I'm hoping it will help me prevent mistyped URLs and params.

2007-05-01

Bug Tracking can be Bug Prevention

I've been deploying a couple of Rails applications for a client lately and things have been running surprisingly smoothly and there have hardly been any issues at all. In one of the more complex applications though a few bugs slipped through my testing net and were caught by the excellent Exception Notifier plugin by Jamis Buck. Now, the question is, how do we typically deal with bugs as programmers? If the bug is in production we rush to fix it, push the fix out, and hope not too many people noticed it, and then try to forget about the whole thing. What we should be doing though is ask ourselves why the bug occured in the first place, how we can categorize the bug, and what kind of process and/or tests can we put in place to prevent similar kind of bugs from happening in the future. Bug tracking is a gold mine for figuring out how we can raise the quality of our software and reduce the bug rate in the future. This may be common sense, but how many of us actually use this opportunity?

1 comment(s)

» Next page